What are the commonalities between these countries' institutions which
helped them reduce perception of corruption?
Frame challenge: Institutions are not terribly important. What really matters is the legal culture of a country.
All of the top ten countries are from a contiguous region of Europe, except New Zealand, which was a colony of another country in the same region that mostly overwhelmed the indigenous Maori population, and Singapore, which was also a British colony and adopted its legal institutions with a particular emphasis of following the rules imposed by the government.
Also, note that not all of the countries are small. Germany has 80 million people give or take. The United Kingdom, with about 60 million people, is #11. Both Germany and the United Kingdom are large federal states (and Switzerland is a small federal state), rather than small homogeneous unitary states. Three of the next four are also former British colonies: Hong Kong, Canada, and Ireland, and Iceland is derived from immigrants from Norway, which is in the top ten.
Singapore and Hong Kong aren't linguistically or ethnically the same as their European counterparts but share their legal culture and legal heritage. Finland's namesake language is not Germanic and the same is true of some of the Swiss languages.
The legal systems of Iceland, Germany, and Switzerland are very different from the common law systems of the former British empire and hence institutionally very different.
But all of these countries have a legal culture that spans different legal institutions and embraces the rule of law and a lack of tolerance for corruption.
Institutionally, lots of the countries with high corruption were established with the same governmental institutions, but those institutions didn't work without the underlying legal culture upon which they were premised.
For example, Sudan started out with British legal institutions when it became independent, but had not absorbed the British legal culture very deeply at the time it became independent, and the result was a corrupt disaster.
This is reinforced by studies in development economics in Africa that find that economic development rates more strongly tracks the ethnic boundaries of populations that share common cultures, than the artificial, colonially imposed boundaries of the countries in Africa (with each country having its own institutions).
For example, regions that are predominantly ethnically Igbo people (mostly in Nigeria) have tended to experience more economic development than adjacent regions in the same political subdivisions where other ethnicities are predominant.
In the same vein, while Japan's civil code comes from a foundation that was almost a verbatim translation of the German civil code, the way that the legal systems of the two countries work in practice is very different, because the legal culture of the two countries is very different.
Culture matters more than institutions when it comes to rates of economic development and some of those cultural matters relate to perceptions of corruption.
Put another way, what matters is the people in the system who operate it, not the system itself.
It would be ideal for the answers to include practical actions that
can be replicated in other countries.
Lack of corruption due to an established legal culture is very hard to replicate, since cultural leanings run deep and take centuries, or at least, many decades, to establish.
There is strong evidence, for example, that the political and legal cultures of different regions of the United States have been largely stable since the colonial era more than two centuries ago and can be traced back even further to the historical experiences of the immigrant populations in these regions. You can find similar evidence from regions within Germany.
One of the famous examples of a reformer leveraging the recognition that it was culture and not institutions that was most important was Mustafa Kemal Atatürk, whose radical reforms including dictates on how people in Turkey should dress and whether they should wear beards, in addition to many less obvious matters, because it was hard to know what parts of a culture mattered to a successful transition to modernity, and which didn't. His extreme cultural reforms, many of which were later rolled back, are ridiculed to a great extent today. But as a result of his approach, the legal and political culture in Turkey is closer to the West than almost any other predominantly Islamic country in the world (although still, obviously, far away from countries that have had a Western style legal culture for a longer period of time).
But, the focus of practical actions to improve the situation should be on how civil servants, legal professionals, and business people in the society are socialized (ideally starting at a young age), not on the substantive legal rules and institutions involved.