Are the political compass and similar political tests scientific?

Question

It seems to be increasingly accepted that the traditional left-right political spectrum is insufficient. Attempts have been made to create alternatives, such as the Political Compass, which adds and authoritarian/libertarian axis, as well as others, like 8 Values which add even more.

The field seems to have a lot of overlap with personality testing. I understand that most personality tests are regarded as unscientific. The personality test with the most scientific acceptance is the Big Five model, which was derived statistically via factor analysis.

To what extent are the popular political classifications accepted academically, and have there been any attempts to apply the same sort of statistical analysis to understand political tendencies?

Answer 1

Big Five was derived through EFA (exploratory factor analysis) and then tested through many a CFA (confirmatory factor analysis) studies, so I'm guessing you're looking for something similar in the realms of political spectrum.

Any such test is only as relevant as the questions it fathoms to ask, i.e. you may entirely miss a dimension if it didn't occur to you ask anything about it. There were early attempts to base political spectrum questionnaires based on actual political statements found in the press, e.g. Eysenck's--compare to basing Big Five on adjectives existent in various languages--this was one of the two avenues for its development.)

Hans Eysenck began researching political attitudes in Great Britain. He believed that there was something essentially similar about the National Socialists (Nazis) on the one hand and the communists on the other, despite their opposite positions on the left–right axis. As Hans Eysenck described in his 1956 book Sense and Nonsense in Psychology, Eysenck compiled a list of political statements found in newspapers and political tracts and asked subjects to rate their agreement or disagreement with each. Submitting this value questionnaire to the same process of factor analysis used by Ferguson, Eysenck drew out two factors, which he named "Radicalism" (R-factor) and "Tender-Mindedess" (T-factor). [...]

Eysenck's dimensions of R and T were found by factor analyses of values in Germany and Sweden, France and Japan.

[...]

In further research, Hans J. Eysenck refined his methodology to include more questions on economic issues. Doing this, he revealed a split in the left–right axis between social policy and economic policy, with a previously undiscovered dimension of socialism-capitalism (S-factor).

While factorially distinct from Eysenck's previous R factor, the S-factor did positively correlate with the R-factor, indicating that a basic left–right or right–left tendency underlies both social values and economic values, although S tapped more into items discussing economic inequality and big business, while R relates more to the treatment of criminals and to sexual issues and military issues.

So the methods used for developing political factors were no so unlike although surveying the press is likely a less stable method (temporally) than surveying languages.

On the other hand, quite a few political science researchers have presupposed some factors theoretically and just tested them. Or not even tested them. I suspect the rather popular Nolan chart on which a lot of the "compasses" out there on the web are based on wasn't subject to much validation. Actually in order to speak of validation, one needs questionnaire and not merely a chart (or set of dimensions).

Another important issue is to ask what do you want to use these factors for. As Philipp's comment rightly points out, the factors one may posit or even identify via EFA may or may not be relevant or salient to some actual politics issue, like some particular election outcome.

Also, as it's pointed out e.g. in Big Five research (which also has finer-grained facets), broad dimensions may not satisfactorily explanatory for some particular issue (e.g. like fit for a specific job, in the case of personality). Depending how sensitive you want your analysis to be on some dependant variable, you may have consider more [sub]factors. E.g., in the case of Irish politics:

In the Irish case, using data from the 2002 Irish National Election Study (INES), Marsh et al. (2008: 41) found that fully six dimensions were required to adequately capture the variation in voters’ issue preferences. The issues that constitute a single cultural dimension in some other contexts were found to form three separate dimensions in the Irish case: a secular-liberalism versus a religious-conservatism dimension; an environmental dimension and a dimension related to European integration. Furthermore, they found that positions on economic issues were not constrained by a single left–right dimension, but rather formed two separate dimensions: one relating to the role of the state in the economy, and the other relating to redistribution. The sixth dimension identified was a distinctly Irish one related to the question of Northern Ireland.

[...]

Four separate dimensions can adequately account for voters’ opinions on a diverse range of policy issues that were salient in the 2016 general election. These are the economic, cultural, religious and austerity dimensions. The first three of these ideological dimensions relate to more or less perennial issues and are likely to have continued relevance in Irish politics in the years to come. The long-term importance of the austerity dimension is less certain.

But for some other purposes as single dimension may explain a lot e.g. in the case of Taiwan parties:

Moral foundations theory provides a framework for understanding the traditional liberal–conservative dichotomy in political factions. Typically, factions on the liberal side are more concerned with individualizing foundations—including care/harm and fairness/cheating—for the protection of individual rights and welfare whereas factions on the conservative side are concerned with both individualizing and binding foundations—including loyalty/betrayal, authority/subversion, and sanctity/degradation—for the maintenance of existing social ethics. Our research extended this framework to the analysis of Taiwanese political factions, which are not distributed conspicuously along the liberal–conservative line but instead on whether Taiwan should become a legally independent state or unify with the People's Republic of China (Mainland China). Our results indicate that despite the scarce use of the terms liberal or left and conservative or right in common communication, a liberal–conservative dimension underlies the Taiwanese political spectrum. Specifically, supporters of Taiwan independence exhibit liberal‐like moral concerns whereas supporters of China unification and the status quo demonstrate conservative‐like moral concerns. Moreover, indirect effects exist through moral foundations from political factions to stances on social issues; this is especially prevalent in the case of Taiwan independence camp's clear support for the legalization of same‐sex marriage, a stance resulting from anti‐authoritarian moral and political characteristics.

So the question whether some classification is "scientific" doesn't have a single answer. It's more useful to ask if a classification is explanatory for some stated purpose.

Interestingly enough, I did find one paper that finds the social dimension mostly irrelevant for party identification in the US, with the [Nolan-style] economic dimension dominating.

One dimension is the long-standing economic dimension, which concerns government spending, taxes, and redistribution policies. A second dimension, which has become salient only in more recent decades, focuses on social and cultural issues such as abortion and gay rights. [...]

Thus while social issues have clearly become a more salient dimension of American politics in recent decades—as seen in their growing effect on voters’ ideological identifications — when it comes to partisanship, traditional concerns about the role of government and economic issues loom supreme.

They do find some differences on elections turnout based on the social dimension though.

Answer 2

Most of them are "unscientific" in that they presuppose a categorisation and then find statements to fit it.

A friend of mine was involved in an attempt to do a more scientific version. This worked by deciding on the questions first and then using principal components analysis on the results to determine a classification.

Answer 3

Yes and no. It's wrong to think of science as only having good models, correct data, right answers, and total consensus. The beauty of science is that its methods better allow us cope with, (and improve upon, and profit from), bad models, incorrect data, wrong answers, and controversies.

Some models that today seem quite wrong, were "good enough" for people to get by with for thousands of years. Some of the sailors who got by with Ptolemaic Astronomy were able to find their way -- but we owe more to the sailors who got lost and the people that tried to figure out why.

Tacking a bunch of received opinion political contraries on opposing sides of a line, (or an n-space graph), seems to have worked out for some people, (some of the time), for quite a while now. Naturally those who've had good luck with current political models provide the most funding, (i.e. "mainstream" political science), so most political research uses the models those sponsors like best. But those happy satisfied few are less likely to experience puzzlement sufficient to sponsor or inspire better models for posterity.

In which case it's the side-streams where the lasting work will be done, (perhaps right now, or ten years ago, or someday...), but a general consensus might wait for decades or centuries -- until the time that such new models become so useful that those who ignore them become history.