I am interested in the efficacy of superstates and the apparent rarity of them. The closest things that are related, are the European Union and the East African Federation (which is still theoretical). The USSR is the best example.
This did not actually pan out as World Revolution did not happen. Stalin has switched USSR to "Socialism in one country". After that transition, USSR never wanted to be a superstate anymore. It has semi-declined Bulgaria and Mongolia from joining as constituent Socialist Republics. Even after WWII where USSR had opportunity to become a superstate, it was mostly forfeited: There were some superstates happening, such as Yugoslavia and Czechoslovakia, but not USSR, which mainly consisted out of remnants of Russian Empire, including Baltic states. There were a few tiny exceptions, such as incorporating Western Ukraine or Tuva. But, even as USSR denied it, it was actually a direct descendant of Russian Empire and did not deviate much from its actual borders.
I believe that Empires are not superstates. Russian Empire formed from multiple Eastern Slavic principalities; German Empire formed from smaller German states; Italy also had its Risorgimento. But those were always formed top-down: people were told that they are now subjects of the new, larger state, and they should be happy about that but they will not have terribly much say about it. Cultural homogenuity helped a bit but was neither necessary nor did it guarantee success.
I treat federalized states, including USSR, as transition states instead of a distinct end result. The constituent federation subjects will either try to run apart (USSR, Spanish troubles with separatism, prospective Scotland exit) or they will become more homogenous as to only keep federalism on paper (USA and Switzerland actually pretty slow on this, while federalized Germany is not much different from unitary France these days).
The closest to the (vaguely defined) superstates are empires and federal states.
An empire is a political unit made up of several territories and peoples, "usually created by conquest, and divided between a dominant center and subordinate peripheries".
Historically an empire was often several kingdoms united under the rule of a single king, who would then style themselves as an Emperor, Caesar, The King of the Kings, etc. Russian Empire was very much the case, as can be seen from the full title of the Emperor, enumerating various principalities under his rule:
By the Grace of God, We, NN, Emperor and Autocrat of All the Russias, Moscow, Kiev, Vladimir, Novgorod; Tsar of Kazan, Tsar of Astrakhan, Tsar of Poland, Tsar of Siberia, Tsar of Chersonese Taurian, Tsar of Georgia; Lord of Pskov and Grand Prince of Smolensk, Lithuania, Volhynia, Podolia, Finland; Prince of Estland, Livland, Courland, Semigalia, Samogitia, Belostok, Karelia, Tver, Yugra, Perm, Vyatka, Bolgar and others; Lord and Grand Prince of Nizhny Novgorod, Chernigov, Ryazan, Polotsk, Rostov, Yaroslavl, Beloozero, Udoria, Obdoria, Kondia, Vitebsk, Mstislav, and all of the northern countries Master; and Lord of Iberia, Kartli, and Kabardia lands and Armenian provinces; hereditary Sovereign and ruler of the Circassian and Mountainous Princes and of others; Lord of Turkestan; Heir of Norway; Duke of Schleswig-Holstein, Stormarn, Dithmarschen, and Oldenburg, and others, and others, and others.
Note Schleswig-Holstein and other territories removed from Russia, which came to the crown via marriages to European nobility.
Interestingly, term Tsar/Czar itself is distorted form of Ceasar, adopted by Ivan III (Ivan the Great) after the unification of the Russian principalities, following the end of the Mongol rule. However, by the time of Peter I (Peter the Great) the term was devalued in its meaning and began to be used in sense of King: Czar David/Solomon=King David/Solomon. This resulted in diplomatic problems, where the protocols of the European emperors (Kings of Kings) did not allow to treat Peter as equal. Hence adoption of Emperor as the official title.
Later empires, like British or French one understood union were governed by democratic governments, but still understood as a union of quasi-independent territories. They partially survive in the form of the Commonwealth of Nations and Francophonie.
federated state (which may also be referred to as a state, a province, a region, a canton, a land, a governorate, an oblast, an emirate, or a country) is a territorial and constitutional community forming part of a federation.1 Such states differ from fully sovereign states, in that they do not have full sovereign powers, as the sovereign powers have been divided between the federated states and the central or federal government.
Federal states are again not uncommon - Switzerland and Belgium are federal states for a long time, as well as the United States, Germany, The United Kingdom (England, Scotland, Wales and Northern Ireland), etc.
The USSR was technically a federal state as well (literally the Union of the Soviet Socialist Republics.) However, in practice it was dominated by Russia, and could be seen as a successor of the Russian Empire.
Short answer: logistics.
Long answer: The extent of a state (both historically and pragmatically) is the range over which it can effectively project both military and constabulary force. A state is defined by its ability to control, defend, and regulate territory, and the larger the territory, the more that logistical problems start to rear their heads. Most large states expand through decentralization: creating fiefs, smaller state units, provinces, etc that self-govern but show allegiance to the central government, but self-governing substates lay the groundwork for future secessions and revolutions.
Technological improvements in production, transit, and communication have extended the 'reach' of a state dramatically over the millennia, but even in the modern era it can still take months to field a proper military force at any distance. That means there are still limits on how extensive a state can be before it succumbs to instabilities.