Caveat: I do not have a degree in international relations.
After reading Foreign Policy Analysis by Chris Adlen and Amnon Aran, I believe, in a nutshell, an Institutional State is a formalized entity encompassing a people and/or territory.
The United States is an Institutional State. Germany is an Institutional State.
A Clustered State is a formalized entity encompassing a set of .... States, Beliefs, Needs, what have you.
While Germany is an Institutional State, and as such it can and does set foreign policy, monetary policy, or other things that are part and parsel of International Relations, it is also a member of a Clustered State, the European Union. It, and the other members of the EU, set, and have set, Foreign Policy, to the betterment of the whole Clustered State.
Kuwait is an Institutional State. It has International Relations. OPEC is a Clustered State, which has International Relations and of which, Kuwait also benefits from those relations.
Of course, a lot of the words, let alone concepts, used in the paper are beyond me, so I may be totally off base.
NOTE: I downloaded their paper from academia.edu and read it.