Measuring the state of democracy across the world helps us understand the extent to which people have political rights and freedoms.
But measuring democracy comes with many challenges. People do not always agree on what characteristics define a democracy. These characteristics — such as whether an election was free and fair — are difficult to define and assess. The judgment of experts is to some degree subjective. They may disagree about a specific characteristic or how something as complex as a political system can be reduced into a single measure.
How do researchers address these challenges and measure democracy?
What is the Regimes of the World data?
In some of our work on democracy, we rely on the Regimes of the World (RoW) data by political scientists Anna Lührmann, Marcus Tannenberg, https://www.shravskitchen.com/ and Staffan Lindberg1, published by the Varieties of Democracy (V-Dem) project.2
The project is managed by the V-Dem Institute, based at the University of Gothenburg in Sweden. It spans seven more regional centers around the world and is run by five principal investigators, dozens of project and regional managers, and more than 100 country coordinators.
V-Dem is funded through grants and donations by government agencies and private foundations, such as the Swedish Research Council, the European Commission, and the Marcus and Marianne Wallenberg Foundation.
How does RoW characterize democracy?
Regimes of the World distinguishes four types of political systems: closed autocracies, electoral autocracies, electoral democracies, and liberal democracies.
- Closed autocracy: citizens do not have the right to choose either the chief executive of the government or the legislature through multi-party elections
- Electoral autocracy: citizens have the right to choose the chief executive and the legislature through multi-party elections; but they lack some freedoms, such as the freedoms of association or expression that make the elections meaningful, free, and fair
- Electoral democracy: citizens have the right to choose the chief executive and the legislature in meaningful, free and fair, and multi-party elections
- Liberal democracy: electoral democracy and citizens enjoy individual and minority rights, are equal before the law, and the actions of the executive are constrained by the legislative and the courts
You can find data on the more specific characteristics and derived measures in our Democracy Data Explorer.
How is democracy scored?
Regimes of the World treats democracy as a binary, by classifying a country as either a democracy or not.
This scoring thereby differs from other approaches such as Varieties of Democracy’s electoral democracy index and other projects, which classify countries as a spectrum, with some being scored as more democratic than others.
What years and countries are covered?
As of version 13 of the dataset, V-Dem covers 202 countries, going back in time as far as 1789. Many countries have been covered since 1900, including before they became independent from their colonial powers.
RoW covers countries and years since 1900. But we expand the years and countries covered and refine the coding rules, as detailed below.
How is democracy measured?
How does RoW work to make its assessments valid?
To measure what it wants to capture, RoW uses data from the Varieties of Democracy project, which assesses the characteristics of democracy mostly through evaluations by experts.3
These anonymous experts are primarily academics and members of the media and civil society. They are also often nationals or residents of the country they assess, and therefore know its political system well and can evaluate aspects that are difficult to observe.
V-Dem’s own team of researchers supplements the expert evaluations. They code some easier-to-observe rules and laws of the political system, such as whether the legislature has a lower and upper house.
How does RoW work to make its assessments precise and reliable?
V-Dem uses several experts per country, year, and topic, to make its assessments less subjective. In total, around 3,500 country experts fill out surveys for V-Dem every year.
While there are fewer experts for small countries and for the time before 1900, they rely typically on 25 experts per country and 5 experts per topic.
How does RoW work to make its assessments comparable?
V-Dem also works to make their coders’ assessments comparable across countries and time.
The surveys ask the experts to answer very specific questions on completely explained scales about sub-characteristics of political systems — such as the presence or absence of election fraud — instead of making them rely on their broad impressions.
The surveys are available in English, Arabic, French, Portuguese, Russian, and Spanish to reduce misunderstandings.
Experts further evaluate hypothetical countries, many coded several countries, and they denote their own uncertainty and personal demographic information.
V-Dem then uses this information to investigate expert biases, which they have found to be limited: they only find that experts from a country tend to be stricter in their assessments. 4
How are the remaining differences in the data dealt with?
V-Dem uses a statistical model to address any remaining differences between coders.5
The model combines the experts’ ratings of actual countries and hypothetical countries, as well as the experts’ stated uncertainties and personal demographics to produce best, upper-, and lower-bound estimates of many characteristics.6
V-Dem provides these different estimates for all of its main and supplementary indices, including the Electoral Democracy Index and the subindices for free and fair elections, freedom of association, and freedom of expression.
With the different estimates, V-Dem explicitly acknowledges that its coders can be uncertain or make errors in their measurement.
In addition to its main classification, RoW provides an expanded version that identifies countries that may fit better into the next-higher or -lower main categories. You can find the data in our Democracy Data Explorer.
The overall classification is the result of evaluating whether necessary characteristics are present or not. If the experts consider a country’s elections to have been both multi-party and free and fair, and the country as having had minimal features of an electoral democracy in general7, RoW classifies it as a democracy.
A country is classified as a liberal democracy if the experts consider the country’s laws to have been transparent; the men and women there as having had access to the justice system; and the country as having had broad features of a liberal democracy overall.8 If it does not meet one of these conditions, the country is classified as an electoral democracy.
A country is classified as an autocracy if it does not meet the above criteria of meaningful, free and fair, multi-party elections. It is classified as an electoral autocracy if the experts consider the elections for the legislature and chief executive — the most powerful politician — to have been multi-party. It is classified as a closed autocracy if either the legislature or chief executive has not been chosen in multi-party elections.
How is the data made accessible and transparent?
V-Dem, which publishes the RoW data, releases its data publicly and makes it straightforward to download and use.
It publishes the overall scores, the underlying subindices, and several hundred specific questions by country-year, country-date, and coder.
V-Dem also releases descriptions of how RoW measures democracy, as well as the questions and coding procedures that guide the experts and researchers.
How do we change the data?
In our work, we expand the years covered by RoW further.
While RoW covers the years since 1900, we use V-Dem’s historical data from 1789 to 1899 to expand the classification’s coverage back in time.
To expand the time coverage of today’s countries and include more of the period when they were still non-sovereign territories, we identified the historical entity they were a part of and used that regime’s data whenever available.9
Finally, we make some additional minor changes to the coding rules.10
Our code and data are available on GitHub and record our revisions in detail.
Join The Discussion