The important thing to note is that each country – and often even regions of a country – develops at different timescales due to their individual history, geographic situation and other factors. Within Germany, for example, the state of Bavaria was still rather poorly developed in the 1950's with a very agrarian society but managed to develop rather well and rapidly becoming one of the richest German states by economic power by the 1990's; meanwhile, other regions' developments were slower, sometimes even stalled or even went backwards due to overreliance on specific industries that became obsolete. (This is not even counting the former GDR which went on an entirely different path.) Therefore, trying to find general trends based on only a few indicators is often difficult or even misleading to impossible.
Taking the six countries you have mentioned and ranking them by economic power or HDI gives some somewhat distinct groups:
- Russia and Turkey with a GDP of around $30,000 and an HDI of 0.8something;
- Brazil and China have a GDP upwards of $15,000 and an HDI in the upper 0.7;
- South Africa has a GDP in the lower $10,000 with an HDI of around 0.7; and finally
- India is at the bottom of the list with a GDP below $10,000 and an HDI of around 0.65.
If I was going to look for indicators that explained this, I would look to history first.
Russia and Turkey
Both these nations have been independent since quite a while. Furthermore, they have a high degree of cultural and ethnic homogenity – for Russia, I'm especially considering the European core. The dominant demographic has been in charge of ruling the country in some way or another since, well, as long as they exist. Prior to World War I, they both ruled empires, although the Russian Empire was noted among the European powers for being somewhat delayed in its development, and the Ottoman Empire might be considered an early modern holdover rather than an empire of its age.
Both nations took to modernising in after the First World War, albeit on different path: Turkey chose a path of aggresive westernisation including introducing some form of democracy, while Russia followed the teachings of Marx and Lenin and decided to modernise under a socialist regime. Despite the differences, both regimes were successful in both the industrial modernisation and increasing the other scores that form the HDI such as access to education.
Towards the end of the 20th century, the Soviet Union famously collapsed and a new, democratic state took its place in Russia. While this did impact the economy note that before the state's collapse the economy was large so there was enough to fall back on, if you will.
In recent years, these two state's democracy indexes probably trended downwards but as the economies were already relatively large they stayed there. In general, less democracy need not burden an already large economy.
Brazil and China
I want to say that these two are on a similar level with respect to HDI and GDP by chance rather than by similarity. While both were colonised by European powers (in China's case: to some extent), Brazil gained its independence in the early 19th century while China went through a turbulent history in the first half of the 20th century until it was finally reunited as a modern state in 1949 when the Communist Party won the Chinese Civil War.
In the latter half of the 20th century, China began an aggressive economic development while maintaining a relatively tight control over civil liberties, resulting in both high economic index scores and low democracy index ones.
Brazil has a more complex and less easily summarised economic history but also one additional confounding factor: while the population of China is culturally and ethnically mostly homogenous (while minorities are being suppressed they do only consitute a relatively small percentage of the total population; and across the eastern half I believe most people would agree that they are more or less homogenously Chinese), Brazil was a major destination of European immigration resulting in a country that is 48 % white. As in most other places of the globe, the white elite believed in racist ideas until well into the 20th century resulting in a plethora of problems non-white Brazilians have to deal with and setting back the economic and HDI development. However, on a democracy scale Brazil ranks equivalent to the other two countries on your list.
South Africa
In a lot of ways, the history of South Africa parallels that of Brazil but being a century late. It was a colony of the British Empire until it gained independence in 1910. Unlike Brazil, the society remained deeply segregated in a system known as Apartheid until the 1990's which essentially held the development of the Black majority (80 % of the population) as low as possible while benefiting the ruling white minority. Of course, this as well as the sudden changes that came about when Apartheid ended affected the economic outlook as well as the HDI score negatively. However, like Brazil the country scores well in democracy which might affect its placement relative to other nations with similar history.
India
India was a colony of the British Empire, later Commonwealth until after the Second World War, tying it with China for latest period of being subjected to foreign rule. While historically, India was an economic and trade powerhouse until the British conquest, it has had a much harder time rebounding from its colonial period.
Like Russia, Turkey and China, the vast majority of the population descends from those who lived there a century ago (but I hesitate using homogenous in India's case even more than I do in China's case as the country is home to many different cultures, ethnic groups and languages).
India started off its post-colonial phase as a very underdeveloped economy; but at the same time China had already enjoyed a couple of decades of independence (albeit not as a unified state) so it had a development head-start which it could use to its economic advantage after World War II. India, in general, took a slower economic path than China while focussing more on democracy, resulting in its high democracy index. Since the GDP heavily influences the HDI, its lower GDP puts it at a disadvantage even when other aspects of the HDI such as education might look much better.
Conclusion
As I said in the beginning, it is difficult to compare different countries with very different histories and societies. Within the list of six, Turkey and Russia are probably the two with the most similarities in their history, resulting in a similar HDI score – although note the very different paths they took throughout the 20th century. On the other hand, one might see South Africa as very similar to Brazil yet a couple of decades behind; that would explain why Brazil is doing better than South Africa in HDI and GDP despite the latter having a better democracy index.
Finally, India is probably better compared to other former colonies that did not become independent until after the Second World War, which have a mostly autochthon population yet a diversity of ethnicities and languages. For example, take Nigeria (independence 1960~63; diverse population, mostly locally originating), which did not have a 70-year history of democracy since its independence. Its HDI is 0.539, its GDP (PPP) per capita is around $5000 and its democracy index is around 4. India has clearly outpaced Nigeria in its economic development far more than Brazil did South Africa despite India only getting just over a decade of head start.
