INSTITUTE - V-Dem / Varieties of Democracy...3 This document outlines the methodological...
Transcript of INSTITUTE - V-Dem / Varieties of Democracy...3 This document outlines the methodological...
I N S T I T U T E
Methodology
Copyright © University of Gothenburg, V-Dem Institute,
University of Notre Dame, Kellogg Institute. All rights reserved.
Version 7 - May 2017
1
Suggestedcitation:Coppedge, Michael, John Gerring, Staffan I. Lindberg, Svend-Erik Skaaning, Jan Teorell, JoshuaKrusell,KyleL.Marquardt,ValeriyaMechkova,DanielPemstein,JosefinePernes,LauraSaxer,NataliaStepanova,EitanTzelgov,Yi-tingWangandStevenWilson.2017.“V-DemMethodologyv7.”VarietiesofDemocracy(V-Dem)Project.
Authors Collaborators• MichaelCoppedge–U.ofNotreDame • DavidAltman–PontificiaU.CatólicadeChile• JohnGerring–BostonUniversity • MichaelBernhard–UniversityofFlorida• StaffanI.Lindberg–U.ofGothenburg • M.StevenFish–UCBerkeley• Svend-ErikSkaaning–AarhusUniversity • AdamGlynn–EmoryUniversity• JanTeorell–LundUniversity • AllenHicken–UniversityofMichigan
• JoshuaKrusell–U.ofGothenburg• KyleL.Marquardt–U.ofGothenburg• ValeriyaMechkova–U.ofGothenburg• DanielPemstein–NorthDakotaStateU.• JosefinePernes–U.ofGothenburg• LauraSaxer-U.ofGothenburg• NataliaStepanova–U.ofGothenburg• EitanTzelgov–U.ofEastAnglia• Yi-tingWang–NationalChengKungU• StevenWilson–UniversityofGothenburg
• CarlHenrikKnutsen–UniversityofOslo• KellyMcMann–CaseWesternReserve
PamelaPaxton–U.ofTexas• BrigitteSeim–U.ofNorthCarolina• JeffreyStaton–EmoryUniversity
2
TableofContents
1. CONCEPTUALSCHEME 4PRINCIPLES–MEASUREDBYV-DEM’SDEMOCRACYINDICES 4COMPONENTS 10INDICATORS 11SUMMARY 12
2. DATACOLLECTION 14
HISTORYOFPOLITIES 14CODINGTYPES 17EXPERTRECRUITMENT 18EXPERTCODINGPROCESS 21BRIDGE-ANDLATERALCODING 25PHASES 26
3. MEASUREMENT 27
THEQUESTIONNAIRE 27IDENTIFYING,CORRECTING,ANDQUANTIFYINGMEASUREMENTERROR 28
MeasurementModels 28CorrectingErrors 35
VERSIONSOFC-VARIABLES 37ADDITIONALPOSSIBILITIESFORIDENTIFYINGSOURCESOFMEASUREMENTERRORINTHEFUTURE 40
REFERENCES 43
APPENDIXA:V-DEMINDICES,COMPONENTS,ANDINDICATORS 47
3
Thisdocumentoutlinesthemethodologicalconsiderations,choices,andproceduresguiding
the development of the Varieties of Democracy (V-Dem) project. Part I sets forth the
conceptual scheme.Part IIdiscusses theprocessofdatacollection.Part IIIdescribes the
measurementmodelalongwitheffortstoidentifyandcorrecterrors.
Wecontinuallyreviewourmethodology—andoccasionallyadjustit—withthegoal
ofimprovingthequalityofV-Demindicatorsandindices.Wethereforeissueanewversion
ofthisdocumentwitheachnewversionofthedataset.
Additional project documents complement this one.V-Dem Codebook includes a
comprehensive list of indicators, response-categories, sources, and brief information
regarding theconstructionof indices.V-DemCountryCodingUnits explainshowcountry
unitsaredefinedandlistseachcountryincludedinthedataset,withnotespertainingtothe
yearscoveredandspecialcircumstancesthatmayapply.V-Dem:ComparisonsandContrastssurveysthefieldofdemocracyindicatorsandsituatestheV-Demprojectinrelationtothese
efforts.V-DemOrganizationandManagement introducestheprojectteam,thewebsite,
outreachtotheinternationalcommunity,funding,progresstodate,andsustainability.
Versioningofthecurrentdocument,V-DemCodebook,V-DemCountryCodingUnits
andV-DemOrganization&Managementdocumentsaresynchronizedwiththereleaseof
eachnewdataset,whileV-Dem:ComparisonsandContrastsisnot.
Severalconfigurationsof theV-Demdatasetareavailable, includingcountry-year,
country-date,andcoder-leveldatasets.Foradditionaldocumentationandguidance,users
shouldrefertotheHowtoCitefilethatisappendedtoeachdatadownload.
IntheV-DemWorkingPaperSeries,userscanfindamoretechnicaldiscussionofthe
measurementmodelweusetoaggregatecoder-leveldatatopointestimatesforcountry-
years(Pemsteinetal.2015,WP#21,seealsoWP#41onIRTmodels),whileworkingPaper
#6 introduces the democracy indices, their components, indicators, and rules for
aggregation.WorkingPaper#25discussesthedetailsoftheElectoralDemocracy/Polyarchy
index.WorkingPaper#22describestheindexofEgalitarianDemocracy.Additionalworking
papers provide in-depth treatments of more specialized indices such as the Female
Empowerment Index (#19), the Core Civil Society Index (#13), Measuring Subnational
Democracy (#26), theCorruption Index (#23), andordinal versionsof theV-Dem indices
(#20).TheV-DemWorkingPaperSeriesisavailablefordownloadontheV-Demwebsite(v-
4
dem.net).
V-Dem isamassive,global collaborativeeffort.Anup-to-date listingofourmany
collaborators,withoutwhomthisprojectwouldnotbepossible,isalsoavailableontheweb
site.Collaborators includeProgramManagers,RegionalManagers, InternationalAdvisory
Boardmembers,theV-DemInstitutestaff(Director,Program-,Operations-,DataProcessing
and Data Managers, Assistant Researchers, and Post-Doctoral Fellows and Associate
Researchers), Research Assistants, and Country Coordinators. We are also especially
indebted to some 2,800 Country Experts, whose identitiesmust remain anonymous for
ethicalreasons.
The website serves as the repository for other information about the project,
includingCountryandThematicReports,BriefingPapers,publications,grantandfellowship
opportunities, and the data itself. Data for all 173 countries included in the first public
release (V-Dem Dataset v5) is also available for exploration with online analysis tools
(countryandvariablegraphs,motioncharts,and–soon–globalmaps).
1. ConceptualScheme
Anymeasurementschemerestsonconcepts. Inthissection,wesetforththeconceptual
schemethatinformstheV-Demproject–beginningwith“democracy”andproceedingto
thepropertiesandsub-propertiesofthatfar-flungconcept.Bywayofconclusion,weissue
severalclarificationsandcaveatsconcerningtheconceptualscheme.V-Dem:Comparisons
andContrastsprovidesamoredetaileddiscussion,butwerecaptheessentialpointshere.
Principles–MeasuredbyV-Dem’sDemocracyIndices
Thereisnoconsensusonwhatdemocracywrit-largemeansbeyondavaguenotionofrule
bythepeople.Politicaltheoristshaveemphasizedthispointforsometime,andempiricists
woulddowelltotakethelessontoheart(Gallie1956;Held2006;Shapiro2003:10–34).At
thesametime,interpretationsofdemocracydonothaveanunlimitedscope.
A thorough search of the literature on this protean concept reveals seven key
principlesthatinformmuchofourthinkingaboutdemocracy:electoral,liberal,majoritarian,
consensual,participatory,deliberative,andegalitarian.Eachoftheseprinciplesrepresents
5
adifferentwayofunderstanding“rulebythepeople.”Theheartofthedifferencesbetween
theseprinciplesisinthefactthatalternateschoolsofthoughtprioritizedifferentdemocratic
values.Thus,whilenosingleprincipleembodiesallthemeaningsofdemocracy,theseseven
principles, taken together, offer a fairly comprehensive accounting of the concept as
employedtoday.1
TheV-Demprojecthassetouttomeasuretheseprinciples,andthecorevalueswhich
underliethem.Wesummarizetheprinciplesbelow.
• Theelectoralprincipleofdemocracyembodiesthecorevalueofmaking
rulers responsive to citizens through periodic elections, as captured by
Dahl’s (1971, 1989) conceptualization of “polyarchy.” Our measure for
electoraldemocracyiscalledthe“V-DemElectoralDemocracyIndex.”We
considerthismeasurefundamentaltoallothermeasuresofdemocracy:
wewouldnotcallaregimewithoutelections“democratic”inanysense.
• The liberal principle of democracy embodies the intrinsic value of
protectingindividualandminorityrightsagainstapotential“tyrannyofthe
majority” and state repression. This principle is achieved through
constitutionally-protectedcivil liberties,strongruleof law,andeffective
checksandbalancesthatlimittheuseofexecutivepower.
• Theparticipatoryprincipleembodiesthevaluesofdirectruleandactive
participation by citizens in all political processes.While participation in
elections counts toward this principle, it also emphasizes nonelectoral
formsofpoliticalparticipation,suchascivilsocietyorganizationsandother
formsofbothnonelectoralandelectoralmechanismsofdirectdemocracy.
• Thedeliberativeprincipleenshrinesthecorevaluethatpoliticaldecisions
inpursuitofthepublicgoodshouldbeinformedbyaprocesscharacterized
1Thisconsensusonlyholds insofarasmostscholarswouldagreethatsomepermutationoraggregationoftheseprinciplesunderlieconceptionsofdemocracy.Forexample,scholarscanreasonablyarguethatthelistcould consist of seven, six, or five principles; our “principles”may be “properties” or “dimensions;” and“majoritarian”and“consensual”areactuallyoppositepolesofasingledimension.Asaresult,weintendforthisdiscussiontoassureconsumersofthedataofthecomprehensivenatureofourinventoryofcorevaluesofdemocracy:namely,thatitincludesalmostalltheattributesthatanyuserwouldwanttohavemeasured.
6
by respectful and reason-based dialogue at all levels, rather than by
emotionalappeals,solidaryattachments,parochialinterests,orcoercion.
• Theegalitarianprincipleholdsthatmaterialand immaterial inequalities
inhibit the actual use of formal political (electoral) rights and liberties.
Ideally,allgroupsshouldenjoyequaldejureanddefactocapabilitiesto
participate;toserve inpositionsofpoliticalpower;toput issuesonthe
agenda; and to influence policymaking. Following the literature in this
tradition, gross inequalities of health, education, or income are
understood to inhibit the exercise of political power and the de facto
enjoymentofpoliticalrights.
Theconceptualschemepresentedabovedoesnotcaptureallthetheoreticaldistinctionsat
playinthecomplexconceptofdemocracy.Wehavechosentofocusonthecorevaluesand
institutionsthattheotherprinciplesemphasizeintheircritiqueoftheelectoralconception
asastand-alonesystem.Eachoftheseprinciplesislogicallydistinctand—atleastforsome
theorists—independently valuable. Moreover, we suspect that there is a considerable
divergenceintherealizationofthepropertiesassociatedwiththesesevenprinciplesamong
the world’s polities. Some countries will be particularly strong on electoral democracy;
otherswillbestrongontheegalitarianproperty,andsoforth.
AggregationProcedures
Atthispoint,V-Demoffersseparateindicesoffivevarietiesofdemocracy:electoral,liberal,
participatory, deliberative, and egalitarian. We anticipate providing indices for the
remaining two principles – majoritarian and consensual – in the near future.2 V-Dem
CodebookcontainstheaggregationrulesforeachindexandseveralV-DemWorkingPapers
(presentandforthcoming) layoutjustificationsforthechoicesmadeineachaggregation
scheme.Thehigh-levelindices,measuringcoreprinciplesofdemocracy,arereferredtoas
democracyindices.
2 Themajoritarian principle of democracy (reflecting the belief that a majority of the people must becapacitatedtoruleandimplementtheirwillintermsofpolicy);andtheconsensualprincipleofdemocracy(emphasizingthatamajoritymustnotdisregardpoliticalminoritiesandthatthereisaninherentvalueintherepresentationofgroupswithdivergentinterestsandview).
7
Sartori held that every defining attribute is necessary for the concept. This logic
requiresmultiplyingtheattributessothateachofthemaffectstheindexonlytothedegree
that theothersarepresent. Family resemblancedefinitionsallowsubstitutability: ahigh
valueononeattributecancompensateforalowvalueonanother.Thislogiccorresponds
toanadditiveaggregationformula.Therearesoundjustificationsfortreatingallofthese
attributesasnecessary,ormutuallyreinforcing.Forexample,ifoppositioncandidatesare
notallowedtorunforelectionortheelectionsarefraudulent,thefactthatalladultshave
votingrightsdoesnotmattermuchforthelevelofelectoraldemocracy.Buttherearealso
goodreasonstoregardtheseattributesassubstitutable.Wherethesuffrageisrestricted,
the situation is less undemocratic if the disenfranchised are still free to participate in
associations, tostrikeandprotest,andtoaccess independentmedia (Switzerlandbefore
1971)thaniftheylacktheseopportunities(ItalyunderMussolini).Evenwheretheexecutive
isnotelected,citizenscanfeelthattheyliveinafairlydemocraticenvironmentaslongas
theyarefreetoorganizeandexpressthemselves,asinLiechtensteinbefore2003.
Becausewebelieveboththenecessaryconditionsandfamilyresemblancelogicsare
validforconceptsofdemocracy,ouraggregationformulasincludeboth;becausewehave
nostrongreasontoprefertheadditivetermstothemultiplicativeterm,wegivethemequal
weight.Theindexisformedbytakingtheaverageof,ontheonehand,theweightedaverage
oftheindicesmeasuringfreedomofassociation(thick)(v2x_frassoc_thick),cleanelections
(v2xel_frefair), freedom of expression (v2x_freexp_thick), elected executive (v2x_accex),
andsuffrage(v2x_suffr)and,ontheother,thefive-waymultiplicativeinteractionbetween
thoseindices.Thisishalfwaybetweenastraightaverageandstrictmultiplication,meaning
the average of the two. It is thus a compromise between the two most well-known
aggregationformulasintheliterature,bothallowing(partial)"compensation"inonesub-
componentforlackofpolyarchyintheothers,butalsopunishingcountriesnotstronginone
sub-componentaccordingtothe"weakestlink"argument.Theaggregationisdoneatthe
levelofDahl’ssub-components(withtheoneexceptionofthenon-electoralcomponent).
Theindexisaggregatedusingthisformula:
v2x_polyarchy=.5MPI+0.5API
=.5(v2x_accex*v2xel_frefair*v2x_frassoc_thick*v2x_suffr*v2x_freexp_thick)
8
+.5(1/8v2x_accex+1/4v2xel_frefair+1/4v2x_frassoc_thick+1/8v2x_suffr+
1/4v2x_freexp_thick)
Thesumof theweightsof theadditive termsequals theweightof the interaction
term.Theadditivepartoftheformulaletsthetwocomponentsthatcanachievehighscores
based on the fulfillment of formal-institutional criteria (elected officials and suffrage)
togetherweighhalfasmuchastheothercomponentsthatenjoyastrongerindependent
standingintermsofrespectfordemocraticrights(cleanelections,freedomoforganization
and expression).3 In any event, because most of the variables are strongly correlated,
differentaggregationformulasyieldverysimilarindexvalues.Theofficialformulapresented
herecorrelatesat.94to.99withapurelymultiplicativeformula,apurelyadditiveformula,
one thatweights the additive terms twice asmuch as themultiplicative term, one that
weightsthemultiplicativetermtwiceasmuchastheadditiveterms,andonethatweights
suffragesixtimesasmuchastheotheradditiveterms.Themaindifferenceacrossthese
formulasisintheirmeanvalues,withsomebeingclosertooneandothers(i.e.themore
multiplicativeformulas)beingclosertozero.
The Electoral Democracy Index also serves as the foundation for the other four
indices.Therecanbenodemocracywithoutelectionsbut,followingthecanonineachof
thetraditionsthatarguesthatelectoraldemocracy is insufficientforatruerealizationof
“rulebythepeople,”thereismoretodemocracythanjustelections.Wethereforecombine
the scores for our Electoral Democracy Index (v2x_polyarchy) with the scores for the
components measuring deliberation, equalitarianism, participation, and liberal
constitutionalism, respectively. This is not an easy task. Imagine two components,
P=Polyarchy and HPC=High Principle Component (liberal, egalitarian, participatory, or
deliberative),4thatwewanttoaggregateintomoregeneraldemocracyindices,whichwe
will call DI (DeliberativeDemocracy Index, EgalitarianDemocracy Index, and so on). For
convenience,bothPandHPCarescaledtoacontinuous0-1interval.Basedonextensive
3OnecouldarguethatthesuffragedeservesgreaterweightbecauseitliesonadifferentdimensionthantheothersandisthekeycomponentofoneofDahl’stwodimensionsofpolyarchy(Dahl1971;Coppedgeetal.2008).However,ourformulaallowsarestrictedsuffragetolowertheElectoralDemocracyIndexconsiderablybecauseitdiscountsalltheothervariablesinthemultiplicativeterm.
4TheHPCsare indicesbasedon theaggregationofa largenumberof indicators (liberal=23,egalitarian=8,participatory=21,deliberative=5).
9
deliberations among the authors and othermembers of the V-Dem research group,we
tentativelyarrivedatthefollowingaggregationformula:
DI=.25*P1.6+.25*HPC+.5*P1.6*HPC
TheunderlyingrationaleforthisformulaforallfourDIsisthesameasthatforthe
ElectoralDemocracy Index: equalweighting of the additive terms and themultiplicative
term in order to respect both the Sartorian necessary conditions logic and a family
resemblance logic. For example, the degree of deliberation stillmatters for deliberative
democracy even when there is no electoral democracy, and electoral democracy still
mattersevenwhenthereisnodeliberation;butthehighestlevelofdeliberativedemocracy
canbeattainedonlywhenthereisahighlevelofbothelectoraldemocracyanddeliberation.
Themoreacountryapproximatespolyarchy,themoreitscombinedDIscoreshould
reflect the unique component. This perspective is a continuous version of theoretical
arguments presented in the literature saying that polyarchy or electoral democracy
conditions should be satisfied to a reasonable extent before the other democracy
componentgreatlycontributestothehighlevelindexvalues.Atthesametime,itreflects
theviewintheliteraturethat,whenacertainlevelofpolyarchyisreached,whatmattersin
termsof,say,participatorydemocracyishowmuchoftheparticipatorypropertyisrealized.
This argument also resembles the widespread perspective in the quality of democracy
literatureemphasizingthatthefulfillmentofsomebaselinedemocracycriteriaisnecessary
beforeitmakessensetoassessthequalityofdemocracy.5Giventhisbodyofliterature,it
becomesnecessarytospecifytherateatwhichacomponentshouldinfluenceaDIscore.
Wedosobyraisingthevalueofacomponentby1.6.Weidentifythisnumericvalueby
definingananchorpoint:whenacountryhasapolyarchyscoreof.5(inpractice,thisisa
thresholdontheElectoralDemocracyIndexbeyondwhichcountriestendtobeconsidered
electoraldemocraciesinaminimalsense)anditsHPCisatitsmaximum(1),thehighlevel
indexscoreshouldbe.5.6
Takentogether,theseindicesofferafairlycomprehensiveaccountingof“varieties
of democracy.” The five (soon to be seven) democracy indices constitute a first step in
5Foranoverview,seeMunck(2016).6Define theexponentasp. SettingPolyarchy=.5,HPC=1,andHLI=.5,andsolving forDI=.25*Polyarchy^p+.25*HPC+.5*Polyarchy^p*HPC,p=log(base0.5)of.25/.75≈1.6.
10
disaggregatingtheconceptofdemocracy.Thenextstepisthecomponents.
Components
Themain democracy components, already included in the discussion above, specify the
distinct properties associatedwith the principles. TheV-DemElectoralDemocracy Index
consistsoffivesub-components(eachofthesesub-componentsbeingindicesthemselves
built from a number of indicators) that together capture Dahl’s seven institutions of
polyarchy:freedomofassociation,suffrage,cleanelections,electedexecutive,andfreedom
ofexpression.Thecomponentindicesmeasuringtheliberal,deliberative,participatory,and
egalitarianpropertiesof democracy (majoritarian and consensualwill be released in the
near future) follow the principles of democracy described in the previous section – but
withoutthecoreunifyingelementofelectoraldemocracy.Theycaptureonlywhatisunique
foreachoftheprinciples.Assuch,thesecomponentsaremutuallyexclusive,ororthogonal
toeachother.
These main democracy components typically have several sub-components. For
example, the liberal democracy component consists of three sub-components, each
capturedwithitsownindex:theEqualitybeforethelawandindividuallibertyindex;the
Judicialconstraintsontheexecutiveindex;andtheLegislativeconstraintsontheexecutive
index.
Inadditiontothecomponentandsub-componentindicesthatarepartoftheV-Dem
democracy indicesconceptual scheme,membersof theV-Demteamhaveconstructeda
series of indices of lower-level concepts such as civil society, party institutionalization,
corruption,andwomen’spoliticalempowerment.Wealsolisttheseindicesintheappendix.
In total, V-Dem offers 5 indices of components, 14 sub- components, and 12 related
conceptswith19sub-concepts.TheV-Demdatasetincludesalloftheseindices.Published
V-Demworkingpapersalreadydetailmanyoftheseindices(e.g.papers#6,#13,#17-20).
Additionalworkingpaperswillprovidefurtherdetailsonotherindices.
Weusetwotechniqueswhenaggregatingintodemocracyindices,components,and
sub- components, as well as related concepts’ indices. For the first step, going from
indicatorsto(sub-)components,weaggregatethelatentfactorscoresfrommeasurement
model (MM) output. More specifically, we use relevant theoretical distinctions in the
11
literature togroup interval-levelMMoutput into setsof variables that sharea common
underlying concept.We then randomly select 100 draws from each variable’s posterior
distribution(seedetailsunder“MeasurementModels”below),anduseaunidimensional
Bayesian factor analysis (BFA) to measure this latent concept sequentially for each
randomly-selected draw in each grouping of variables. We then combine the posterior
distributions of the latent factor scores in each variable group to yield the latent factor
scores.Inallanalyses,thevariablesgenerallyloadhighlyontheunderlyingfactor.
For the next level in the hierarchy –another subcomponent, a component, or a
democracyindexdependingonthecomplexityoftheconceptualstructure(seeAppendixA)
– we take the latent factor scores from the separate BFAs and use in combination in
constructingthe“HigherLevelIndices”(HLIs).HLIsarethuscompositemeasuresthatallow
thestructureoftheunderlyingdatatopromulgatethroughthehierarchyinthesameway
astheBFAsdo–andcriticallycarryoverthefullinformationaboutuncertaintytothenext
levelinordertoavoidallowingtheaggregationtechniqueartificiallyincreasetheestimated
confidence – while being faithful to the theoretically informed aggregation formula.
FollowingtheformulaofeachHLI(seetheV-DemCodebook),wetakeaveragesorproducts
ofeachoftherelevantBFAfactorscoreposteriordistributions,andthencalculatethepoint
estimates(means)andconfidenceintervalsacrosstheresultingmatrixtogeneratetheHLI
estimates. For example, the liberal component of democracy index comprises three
elements: equality before the law and individual liberties, judicial constraints on the
executive,andlegislativeconstraintsontheexecutive.Webelievethesethreeelementsare
substitutiveandthereforetaketheaverageofthesethreeelementstoconstructtheliberal
componentindex.FortheDIs,weusetheequationsdiscussedabovetoassignweightsto
thecombinations.
Indicators
Thefinalstepindisaggregationistheidentificationofindicators.Inidentifyingindicatorswe
lookforfeaturesthat(a)arerelatedtoat leastonepropertyofdemocracy;(b)bringthe
political process into closer alignmentwith the coremeaningof democracy (rule by the
people);and(c)aremeasurableacrosspolitiesandtime.
Indicatorstaketheformofnominal(classifications,text,dates),ordinal(e.g.,Likert-
12
stylescales),orintervalscales.Somerefertodejureaspectsofapolity–rulesthatstatute
orconstitutionallaw(includingtheunwrittenconstitutionofstatesliketheUnitedKingdom)
stipulate.Othersrefertodefactoaspectsofapolity–thewaythingsareinpractice.
Thereareover350uniquedemocracyindicatorsintheV-Demdataset.Welisteach
indicator, along with its response-type, in the V-Dem Codebook. We discuss coding
proceduresingreaterdetailinthenextsection.TheV-Demdatasetcontainsmanyindicators
thatwedonotincludeinthecomponentanddemocracyindicesdiscussedabove,though
theyarerelatedtodemocracy.Theirabsencereflectsthefactthatwehavesoughttomake
thecomponent-anddemocracyindicesasorthogonalaspossibletoeachother,andalsoas
parsimoniousaspossible.Furthermore,wheneverwehavemeasuresofboththede jure
andthedefactosituationinastate,ourindicesbuildprimarilyonthedefactoindicators
becausewewant themeasures to portray the “real situation on the ground” as far as
possible.
Summary
Tosummarize,theV-Demconceptualschemerecognizesseverallevelsofaggregation:
●Coreconcept(1)●DemocracyIndices(5,soontobe7)
●DemocracyComponents(5)●Subcomponents,andrelatedconcepts(46)
●Indicators(≈350)
As anappendix to this document,weattacha tablewith a completehierarchyof
democracyindices,democracycomponentindices,democracysub-componentindices,and
indicators,aswellasthehierarchyofrelatedconceptindices.
Several important clarifications apply to this taxonomy. First, our attempt to
operationalizedemocracydoesnotattempttoincorporatethecausesofdemocracy(except
insofarassomeattributesofourfar-flungconceptmightaffectotherattributes).Regime-
typesmaybeaffectedbyeconomicdevelopment(Epsteinetal.2006),colonialexperiences
(Bernhard et al. 2004), or attitudes and political cultures (Almond & Verba 1963/1989;
Hadenius& Teorell 2005;Welzel 2007). However,we do not regard these attributes as
constitutiveofdemocracy.
13
Second,ourquesttoconceptualizeandmeasuredemocracyshouldnotbeconfused
with the quest to conceptualize and measure governance.7 Of course, there is overlap
betweenthesetwoconcepts,sincescholarsmayconsidermanyattributesofdemocracyto
beattributesofgoodgovernance.
Third,werecognizethatsomeindicatorsandcomponents(listedintheCodebook)are
moreimportantinguaranteeingapolity’soverall levelofdemocracythanothers,though
thepreciseweightingparametersdependuponone’smodelofdemocracy.
Fourth,aspectsofdifferentideasofdemocracysometimesconflictwithoneanother.
Atthelevelofprinciples,thereisanobviousconflictbetweenmajoritarianandconsensual
norms,whichadoptcontraryperspectivesonmostinstitutionalcomponents.Forexample,
protectingindividuallibertiescanimposelimitsonthewillofthemajority.Likewise,strong
civilsocietyorganizationscanhavetheeffectofpressuringgovernmenttorestrictthecivil
libertiesenjoyedbymarginalgroups(Isaacn.d.).Furthermore,thesameinstitutionmaybe
differentlyviewedaccordingtodifferentprinciplesofdemocracy.Forexample,thecommon
practiceofmandatoryvotingisclearlycontrarytotheliberalmodel(whereindividualrights
aresacrosanctandincludetherightnottovote),buttheparticipatorymodelsupportsthis
practice,sinceithasademonstratedeffectinboostingturnoutwhereversanctionsaremore
thannominal.
Suchcontradictionsareimplicitindemocracy’smultidimensionalcharacter.Nowide-
rangingempirical investigationcanavoidconflictsamongdemocracy’sdiverseattributes.
However, with separate indicators representing these different facets of democracy it
shouldbepossibletoexaminepotentialtradeoffsempirically.
Fifth,ourproposedsetofdemocracyindices,components,andindicators,whilefairly
comprehensive,isbynomeansexhaustive.Theproteannatureofdemocracyresistsclosure;
there are always potentially new properties/components/indicators that, from one
perspectiveoranother,maybeassociatedwiththisessentiallycontestedterm.Moreover,
7SeeRose-Ackerman(1999)andThomas(2010).Inglehart&Welzel(2005)arguethateffectivedemocracy–as opposed to purely formal or institutional democracy – is linked to rule of law: a formally democraticcountrythatisnotcharacterizedbytheruleoflawisnotdemocraticinthefullsenseoftheterm.InordertorepresentthisthickconceptofdemocracytheymultiplytheFreedomHouseindicesbyindicesofcorruption(drawnfromTransparencyInternationalortheWorldBank),producinganindexofeffectivedemocracy.SeeHadenius&Teorell(2005)andKnutsen(2010)forcriticaldiscussions.
14
someconceptionsofdemocracyaredifficulttocaptureempirically;thisdifficultyincreases
whenanalyzingtheseconceptionsovertimeandacrosscountriesonaglobalscale.Thisfact
limitsthescopeofanyempiricalendeavor.
Sixth,principlesandcomponents,whilemucheasiertodefinethandemocracy (at-
large), are still resistant to authoritative conceptualization. Our objective has been to
identifythemostessentialanddistinctiveattributesassociatedwiththeseconcepts.Even
so,wearekeenlyawarethatothersmightmakedifferentchoices,andthatdifferenttasks
require different choices. The goal of the proposed conceptual framework is to provide
guidance,notto legislate inanauthoritativefashion.Theschemademonstrateshowthe
various elements of V-Dem hang together, according to a particular set of inter-
relationships. We expect other writers will assemble and dis-assemble these parts in
whateverfashionsuitstheirneedsandobjectives.Inthisrespect,V-Demhasthemodular
qualitiesofaLegoset.
Finally,asshouldbeobvious,thissectionapproachesthesubjectfromaconceptual
angle.Elsewhere(e.g.,intheV-DemCodebookandinV-DemComparisonsandContrasts,as
wellasinworkingpapersfoundontheV-Demwebsite),wedescribetechnicalaspectsof
indexconstructioninmoredetail.
2. DataCollection
Theviabilityofanydatasethingescriticallyonitsmethodofdatacollection.V-Demaimsto
achievetransparency,precision,andrealisticestimatesofuncertaintywithrespecttoeach
(evaluativeandindex)datapoint.
HistoryofPolities
Ourprincipalconcerniswiththeoperationofpoliticalinstitutionsthatexistwithinlargeand
fairlywell-defined political units andwhich enjoy amodicumof sovereignty or serve as
operationalunitsofgovernance(e.g.,coloniesofoverseasempires).Werefertotheseunits
aspolitiesorcountries.8
8 We are not measuring democracy within very small communities (e.g., neighborhoods, school boards,
15
Wearenotconcernedmerelywiththepresentandrecentpastofthesepolities.In
our view, understanding the present – not tomention the future – requires a rigorous
analysisofhistory.Theregimesthatexisttoday,andthosethatwillemergetomorrow,are
theproductofcomplexprocessesthatunfoldoverdecades,perhapscenturies.Although
regimechangesaresometimessudden,likeearthquakes,thesedramaticeventsareperhaps
sometimes tobeunderstoodasacombinationofpent-up forces thatbuildupover long
spansoftime,notsimplytheprecipitatingfactorsthatreleasethem.Likewise,recentwork
has raised the possibility that democracy’s impact on policies and policy outcomes take
effect over a very long period of time (Gerring et al., 2005) and that there are indeed
sequencesintermsofnecessaryconditionsindemocratization(Wangetal.2015).Arguably,
short-termandlong-termeffectsarequitedifferent,whetherdemocracyisviewedasthe
cause or outcome of theoretical interest. For all these reasons, we believe that a full
understandingofdemocratizationdependsuponhistoricaldata.9
Theadvantageofourtopic–incontrastwithotherhistoricalmeasurementtaskssuch
asnational incomeaccounts– is thatmuchof theevidenceneeded to code featuresof
democracy is preserved in books, articles, newspapers archives, and living memory.
Democracyis,afterall,ahigh-profilephenomenon.Althoughasecretiveregimemayhide
thetruevalueofgoodsandservicesinthecountry,itcannotdisguisetheexistenceofan
election; those features of an election that might prejudice the outcome toward the
incumbent are difficult to obscure completely. Virtually everyone living in that country,
studyingthatcountry,orcoveringthatcountryforsomeforeignnewsorganizationoraid
organizationhasaninterestintrackingthisresult.
Thus,weregardthegoalofhistoricaldatagatheringasessentialandalsorealistic,
even if it cannot be implemented for every possible indicator of democracy. V-Dem
thereforeaimstogatherdata,wheneverpossible,backto1900forallterritoriesthatcan
municipalities, corporations), in contexts where the political community is vaguely defined (e.g.,transnationalmovements),oronagloballevel(e.g.,theUnitedNations).Thisisnottosaythattheconceptofdemocracyshouldberestrictedtoformalandwell-definedpolities.Itissimplytoclarifyourapproach,andtoacknowledgethatdifferentstrategiesofconceptualizationandmeasurementmayberequiredfordifferentsubjectareas.
9 This echoes a persistent theme presented in Capoccia and Ziblatt (2010), Knutsen, Møller & Skaaning(forthcoming), Teorell (2011), and in other historically grounded work (Nunn 2009; Mahoney &Rueschemeyer2003;Pierson2004;Steinmo,Thelen,&Longstreth1992).
16
claimasovereignorsemi-sovereignexistence(i.e.theyenjoyedadegreeofautonomyat
leastwithrespecttodomesticaffairs)andserveastheoperationalunitofgovernance.The
lattercriterionmeansthattheyaregoverneddifferentlyfromotherterritoriesandwemight
reasonablyexpectmanyofour indicators to varyacross theseunits. Thus, in identifying
political units we look for those that have the highest levels of autonomy and/or are
operationalunitsofgovernance.Thesesortsofunitsarereferredtoas“countries,”evenif
theyarenot fullysovereign.Thismeans, forexample, thatV-Demprovidesacontinuous
time-seriesforEritreacodedasanItaliancolony(1900-41),aprovinceofItalianEastAfrica
(1936-41),aBritishholdingadministeredunder the termsofaUNmandate (1941-51),a
federation with Ethiopia (1952-62), a territory within Ethiopia (1962-93), and an
independent state (1993-). For further details, see V-Dem Country Coding Units. In the
future,weplantoaddinformationinthedatasetanddocumentationtolinkpredecessor
andsuccessorstates,facilitatingpanelanalysiswithcontinuouscountry-levelunits.
V-Dem provides time-series ratings that reflect historical changes as precisely as
possible. Election-specific indicators are coded as events occurring on the date of the
election.Wecodeotherindicatorscontinuously,withanoption(thatsomecodersutilize)
tospecifyexactdates(day/month/year)correspondingtochangesinaninstitution.
Date-specificdatacanbeaggregatedat12-monthintervals,whichmaybeessential
for time-series where country-years form the relevant units of analysis. The V-Dem
“standard”dataset is inthecountry-year format,wheredate-specificchangeshavebeen
aggregatedtogetherattheyearlevel.However,wealsoprovideacountry-datedatasetfor
userswhowantgreaterprecision.Inthedataarchiveaccessibleviathedatadownloadpage
onourwebsite,wealsoprovidetherawcoder-leveldata.Doingsoallowsuserstoinspect
thedatadirectlyoruseitforalternateanalyses.Finally,inthesamearchivewealsoprovide
theposteriordistributionsfromtheBayesianordinalIRTmodelforeachvariabletofacilitate
theirdirectuseinanalyses.
Currently,weareworkingtoextendV-Demcodingbackfurtherinhistoricaltime,i.e.,
to1789,for85sovereigncountriesandforaselectionofindicators.Thiscodingwillenhance
ourknowledgeofdemocraticdevelopmentforcountrieswhoseprocessofdemocratization
began prior to the twentieth century. It will also enhance our knowledge of the pre-
democratic history of all countries, a history thatmay exert an enduring influence over
17
subsequentdevelopmentsinthe20thand21stcenturies.
CodingTypes
The350+V-DemspecificindicatorslistedinV-DemCodebookfallintofourmaintypes:(A*)
factualindicatorspre-codedbymembersoftheV-Demteamandprovidedinthesurveys
forCountryCoordinatorsand–Expertstoindicatetheirconfidenceregardingthepre-coded
data, (A) factual indicators codedbymembersof theV-Demteam, (B) factual indicators
coded by Country Coordinators and/or members of the V-Dem team, (C) evaluative
indicatorsbasedonmultipleratingsprovidedbyexperts,and(D)compositeindices.PartI
of V-Dem Codebook describes these indicators Parts II and III provide a fifth type of
indicators:(E)extantdata(bothfactualandsubjective).
We gather Type (A*) and (A) data from extant sources, e.g., other datasets or
secondarysources,aslistedintheCodebook.Thesedataarelargelyfactualinnature,though
somecoderjudgmentmayberequiredininterpretinghistoricaldata.PrincipalInvestigators
andProjectManagerssupervisethecollectionof thesedata,whichassistantresearchers
connectedtotheprojectcarryoutusingmultiplesources,withinputfromV-Dem’sCountry
Coordinators.
CountryCoordinators,underthesupervisionofRegionalManagers,gatherType(B)
datafromcountry-specificsourcesby.Foranumberofcountries,assistantresearchersat
theV-DemInstitutehavecodedtheseindicatorsduringtheupdateswhentheoriginalseries
goingfrom1900to2012havebeenextendedto2016.AswithType(A*)and(A)data,this
sortofcodingislargelyfactualinnature.
Type(C)datarequiresagreaterdegreeof judgmentaboutthestateofaffairs ina
particular country at a particular point in time. Country Experts code these data. These
experts are generally academics (about 80%) or professionals working media, or public
affairs(e.g.,senioranalysts,editors,judges);about2/3arealsonationalsofand/orresidents
in a country and have documented knowledge of both that country and a specific
substantive area. Generally, each Country Experts code only a selection of indicators
followingtheirparticularbackgroundandexpertise(e.g.thelegislature).
Type(D)dataconsistsofindicescomposedfrom(A),(B),or(C)variables.Theyinclude
18
cumulativeindicatorssuchas“numberofpresidentialelectionssince1900”aswellasmore
highlyaggregatedvariablessuchasthecomponentsanddemocracyindicesdescribedinthe
previoussectionanddetailedinAppendixA.
WedrawType(E)datadirectlyfromothersources.TheyarethereforenotaV-Dem
product.TherearetwogenresofE-data.Thefirstgenreconsistsofalternativeindicesand
indicators of democracy found in Part II of V-Dem Codebook, which may be useful to
compare and contrast with V-Dem indices and indicators. This genre also includes
alternativeversionsoftheV-Demindicesthatareordinalinsteadofinterval(Lindberg2015).
ThesecondtypeofE-indicatorsconsistoffrequentlyusedcorrelatesofdemocracysuchas
GDP.TheyarefoundinPartIII.
CountryExpertRecruitment
Type (C)coding–byCountryExperts– involvesevaluative judgmentson thepartof the
coder.Asaresult,wetakeanumberofprecautionstominimizeerrorinthedataandto
gaugethedegreeofimprecisionthatremains.10
An important aspect of these precautions is the fact thatwe endeavor to find a
minimumoffiveCountryExpertstocodeeachcountry-yearforeveryindicator.Thequality
and impartiality of C-data naturally depends on the quality of the Country Experts that
provide the coding. Consequently, we pay a great deal of care and attention to the
recruitmentofthesescholars,whichfollowsanexactingprotocol.
First,weidentifyalistofpotentialcodersforacountry(typically100-200namesper
country).RegionalManagers,inconsultationwithCountryCoordinators,usetheirintimate
knowledgeofacountrytocompilethebulkoftheexpertsonthislist.AssistantResearchers
locatedattheV-DemInstitute(UniversityofGothenburg)alsocontributetothislist,using
readilyavailableinformationdrawnfromtheInternet.11Othermembersoftheprojectteam
(PIs,PMs,andassociates)mayalsosuggestcandidates.Atpresent,ourdatabaseofpotential
CountryExpertscontainssome18,000names.
10ForaperceptivediscussionoftheroleofjudgmentincodingseeSchedler(2012).11ResearchAssistantsattheUniversityofNotreDamealsosuppliedmorethan3,000namesforallregionsin2011-2013,usinginformationfromtheInternet.
19
RegionalManagers and Country Coordinators thus play a critical role in the data
collectionprocess.V-Dem’sapproachistorecruitRegionalManagerswhoarenationalsor
residentsofoneofthecountriesineachregionwheneverpossible.TheRegionalManagers
aretypicallyprominentscholars inthefieldwhoareactiveasprofessors intheregion in
question.Insomecases,RegionalManagersarelocatedoutsideoftheregion,iftheyare
currentlyactiveinwell-respectedinternationalthinktanksorsimilarinstitutions.Country
Coordinatorsarealmostalwaysnationalsandresidentsofthecountrytobecoded.Theyare
alsoscholars,althoughtheyaretypicallymorejuniorthanRegionalManagers.
Using short biographical sketches, publications, website information, or similar
material we compile basic information for each Country Expert: their country of origin,
current location, highest educational degree, current position, and area of documented
expertise(relevantfortheselectionofsurveystheexpertmightbecompetenttocode)to
makesureweadheretothefiverecruitmentcriteria.
RegionalManagers,CountryCoordinators,andotherprojectteammembersreferto
five criteriawhen drawing up the list of potential Country Experts. Themost important
selectioncriterionisanindividual’sexpertiseinthecountry(ies)andsurveystheymaybe
assigned to code. This expertise is usually signified by an advanced degree in the social
sciences,law,orhistory;arecordofpublications;orpositionsinoutsidepoliticalsocietythat
establishtheirexpertiseinthechosenarea(e.g.awell-knownandrespectedjournalist;a
respectedformerhighcourtjudge).RegionalManagersandCountryCoordinatorsmayalso
indicatewhichsurveysapotential coderhasexpertise in.Naturally,potential codersare
drawntoareasofthesurveythattheyaremostfamiliarwith,andareunlikelytoagreeto
code topics they know little about. As a result, self-selection alsoworks to achieve our
primarygoalofmatchingquestionsinthesurveywithcoderexpertise.
Thesecondcriterionisconnectiontothecountrytobecoded.Bydesign,threeoutof
five(60%)oftheCountryExpertsrecruitedtocodeaparticularcountry-surveyshouldbe
nationalsorpermanentresidentsofthatcountry.Exceptionsaremadeforasmallnumber
of countries where it is difficult to find in-country coders who are both qualified and
independentofthegoverningregime,orwherein-countrycodersmightbeplacedatrisk.
ThiscriterionhelpsusavoidpotentialWesternorNorthernbiasesincoding.
The third criterion is the prospective coder’s seriousness of purpose, i.e. her
20
willingnesstodevotetimetotheprojectandtodeliberatecarefullyoverthequestionsasked
in the survey. Sometimes, personal acquaintanceship is enough to convince a Regional
ManagerandaCountryCoordinatorthatapersonisfit,orunfit,forthejob.Sometimes,this
featurebecomes apparent in communicationswithProgramManagers that precede the
offer to work on V-Dem. This communication is quite intensive, with an average of 13
interactionsbeforecodingisconcluded,andinvolvesrequiringthepotentialcodertoread
andworkwithseverallengthy,detaileddocuments.Thisprocessreadilyidentifiespotential
coderswhoarenotseriousenough.
The fourth criterion is impartiality. V-Demaims to recruit coderswhowill answer
surveyquestionsinanimpartialmanner.Wethereforeavoidthoseindividualswhomight
bebeholdentopowerfulactors–byreasonofcoercivethreatsormaterialincentives–or
whoserveasspokespersonsforapoliticalpartyorideologicaltendency.Closeassociation
(currentorpast)withpoliticalparties,seniorgovernmentofficials,politicallyaffiliatedthink-
tanksorinstitutesisgroundsfordisqualification.Incaseswherefindingimpartialcodersis
difficult,weaimtoincludeavarietyofcoderswho,collectively,representanarrayofviews
andpoliticalperspectivesonthecountryinquestion.
Thefinalcriterionisobtainingdiversityinprofessionalbackgroundamongthecoders
chosenforaparticularcountry.Forcertainareas(e.g.,themedia,judiciary,andcivilsociety
surveys)suchdiversityentailsamixtureofacademicsandprofessionalswhostudythese
topics.Italsomeansfindingexpertswhoarelocatedatavarietyofinstitutions,universities
andresearchinstitutes.
Afterweighingthesefivecriteria,wegivethe100-200potentialexpertsonourlistof
candidatesarankfrom“1”to“3,”indicatingtheorderofprioritywegivetorecruitingan
Expert.TheRegionalManagersandCountryCoordinatorsareprimarilyresponsibleforthe
ranking,butProgramManagersandoneof thePrincipal Investigatorsmay review these
choices.
Using this process,wehave recruitedover 2,800 scholars and experts fromevery
corneroftheworld.About26percentoftheCountryExpertsarewomen,12andover68
12ThenumberofwomenamongtheranksofourCountryExpertsislowerthanwewouldhaveliked,anditoccurreddespiteourstrenuousefforts.However,itreflectsgenderinequalitieswithregardtoeducationanduniversitycareersintheworld.
21
percenthavePhDsorMAsandareaffiliatedwithresearchinstitutions,thinktanks,orsimilar
organizations.
Inordertopreserveconfidentiality,V-Demhasadoptedapolicyofneitherconfirming
nordenyingtheidentitiesofCountryExperts.Wealsofollowaregulationofkeepingthe
Personal Identifiable Information (PII)protected, all this sensitive informationassociated
with Country Experts is handled and stored with care. During update 2017, only three
ProgramManagerswereactively involved in interactionswithCountryExpertsandwere
awareoftheidentitiesofthefinalchosenCountryExperts.Theseindividualsalsohandleall
correspondencewithCountryExperts,sothisconfidentialityisnotinadvertentlyrevealed
throughcommunication.
Thus,whiletheidentityofothermembersoftheV-Dementerpriseispublicizedon
ourwebsite,wepreservetheconfidentialityofCountryExperts.Severalreasonsliebehind
thisdecision.First,thereareanumberofcountriesintheworldwhereauthoritiesmight
sanctionCountryExperts,ortheirfamiliesorfriends,fortheirinvolvementintheproject.
Second,thereisnowaytopredictwhichcountrymayinthefuturebecomerepressiveand
thereforesanctiontheCountryExperts.Third,weanticipatethatV-Demdatamaybecome
used inevaluationsandassessments internationally inwaysthatcouldaffectacountry’s
status. Thus, one may foresee incentives for certain countries’ governments and other
actorstotrytoaffecttheirratings.Forallthesereasons,weconsideritessentialtopreserve
CountryExpertanonymity.
ExpertCodingProcess
TheProgramManagersattheV-DemInstitute(UniversityofGothenburg)issueinvitations
until thequotaof fiveCountryExpertspersurvey (country-year indicator).13 Weusually
recruit6-7expertstobeabletoreplacethosewhofailtobeginorcompletethesurveyin
time.Codersreceiveamodesthonorariumfortheirworkthatisproportionaltothenumber
ofsurveystheyhavecompleted.
C-indicatorsareorganizedintofourclustersandelevensurveys:
13BeforeJuly2014,therewasathirdProgramManagerattheKelloggInstituteoftheUniversityofNotreDamewhomanagedmostcountryexpertsinLatinAmericaandafewintheMiddleEastandNorthAfrica.
22
1. Elections Politicalparties/electoralsystems
2. Executive Legislature Deliberation
3. Judiciary Civilliberty Sovereignty
4. CivilsocietyorganizationsMedia
Politicalequality
Wesuggest(butdonotrequire)thateachCountryExpertcodeatleastonecluster.
In consultation with the Country Coordinators and Principal Investigators, Regional
ManagerssuggestwhichCountryExpertmightbemostcompetenttocodewhichsurveys.
WethenconsultwiththeCountryExpertaboutwhichcluster(s)theyfeelmostcomfortable
coding.Mostcodeonetotwoclustersofsurveys.Thismeansthat,inpractice,adozenor
more Country Experts provide ratings for each country (with a target of five for each
country/indicator/year,asstated).14
AllCountryExpertscarryouttheircodingusingaspeciallydesignedonlinesurvey.
Theweb-basedcodinginterfacesaredirectlyconnectedwithapostgresdatasetwherewe
storetheoriginalcoder-leveldata.Figure4providesanexampleofthecodinginterface.
ThecodinginterfaceisanessentialelementofV-Dem’sinfrastructure.Itconsistsof
aseriesofweb-basedfunctionsthatallowCountryExpertsandCountryCoordinatorsto(1)
logintothesystemusingtheirindividual,randomizedusernameandself-assigned,secret
password;(2)accesstheseriesofsurveysassignedtothemforaparticularcountry(orset
ofcountries);and(3)submitratingsforeachquestionoveraselectedseriesofyears.
Thecoding interfaceallows formanytypesofquestions (binary,ordinal,multiple
selection, numeric, range, text, date, and country-list selection), country and question-
specificyearmasks(e.g.,allowingthecodingofelectionsonlyinyearstheyoccurredforthat
14Insomerarecases---mainlysmallandunder-studiedcountries---weaskindividualexpertstocodethewholesetofsurveys,simplybecauseexpertsonthevariousspecificpartsofthesurveyarenotavailable.Similarly,itisalsonotalwayspossibletoreachthegoalofhavingfivecountryexpertscodeeachindicatorforthesecountries.
23
country),auto-filleddefaultdata(suchasnamesofheadsofstateforparticularcountry-
years),andquestion-specificinstructionsandclarifications.
Theinterfacealsorequiresthat,foreachrating,expertsassignalevelofconfidence,
indicatinghowconfidenttheyarethattheirratingiscorrect(onascaleof0-100,whereeach
5-percentintervalhasasubstantiveanchorpoint,inadditiondescriptivetextsareprovided
at0%,20%,40%,60%,80%,90%,and100% intervals),providinganother instrument for
measuringuncertaintyassociatedwiththeV-Demdata.Weincorporatethisconfidenceinto
themeasurementmodel.CountryExpertsalsohaveanopportunitytoregisteruncertainty
inthe“Remarks”fieldthatliesattheendofeachsectionofthesurvey.Here,expertscan
comment(inprose)onanyaspectoftheindicatorsorratingsthatshefoundproblematicor
difficulttointerpret.
Fig.4ExampleofCodingInterface
Finally, in order to ensure wide recruitment of potential experts, and minimize
confusiondue tounfamiliaritywith English,we translate all type-Cquestions, aswell as
24
coder-instructionsanddocumentationforthem,intofiveotherlanguages:Arabic,French,
Portuguese,Russian,andSpanish.Approximately82%ofcoderscodeintheEnglishversion
ofthequestionnairewhile18percentofthecoderscodeinanon-English(7%-French,6%
- Spanish, 3% - Russian, 1% - Arabic and 1% - Portuguese). Country Experts get a small
remunerationasatokenofappreciationfortheirtime.15
Wetakeanumberofstepstoassureinformedconsentandconfidentialityamong
participants.Theonlinesurveyprovides full informationabouttheproject (includingthis
document)andtheuseofthedata,sothatcodersarefullyinformed.Italsorequiresthat
prospectivecoderscertify that theyaccept the termsof theagreement.Theyaccess the
surveysonlywitharandomizedusernamethatweassignandasecretpasswordthatthey
createthemselves.Westorethedatatheysupplyonafirewall-protectedserver.Anydata
we release to thepublic excludes information thatmightbeused to identify coders.All
personally identifying information is kept in a separate database in order to ensure
confidentiality.
Aspeciallydesignedprogramminginterfaceisemployedtomanagethedatabaseof
potentialcountryexperts.Itincludesmanytoolsthatenableustohandleover2,800Country
Experts while guaranteeing their safety and confidentiality. These tools also ensure
consistency in instructions and information sent to Country Experts, quality control and
cleaningofdata,followupandevaluationofthecodingprocess.Itisdirectlylinkedtothe
postgresdatabasewhereratingsarestored.
Thecodermanagementtoolisjustoneofover50sophisticatedtoolsamongtheV-
Demmanagementinterfacesinthesoftware.Therearetoolsformanagementofcountries,
rounds of surveys, surveys and questions, country coordinators, regional managers, for
logging activities, analyses of progress on recruitment as well as coding, planning, and
generalmanagement.Aweb-interfaceportal isconnectedto themanagementsoftware,
allowing Regional Managers to securely upload Country Expert rosters to the database
15Fromwhatwecantell,thisisnotasignificantthreattocodingvalidity.Fewindividualsseemtohavebeenmotivatedtoconductthisarduouscodingassignmentforpurelymonetaryreasons:V-Dempaysverylittlerelativetowhathighlyqualifiedexpertscouldearnforthesameamountofworkfromotherpursuits.Furtherstrengtheningthispoint,thereseemstobenorelationshipbetweenthewealthofthecountryandourabilitytorecruitcoders:wehavefacedchallengesgettingexpertstoagreetoconductcodingforthepoorestaswellastherichestcountriesintheworld.
25
withouthavingtoshareconfidentialinformationviaemail.
Bridge-andlateralcoding
In addition to regular ratings by multiple Country Experts for C-type indicators, we
encourage Country Experts to conduct bridge coding (coding ofmore than one country
throughtime)andlateralcoding(codinglimitedtoasingleyear–2012).Thepurposeofthis
additionalcodingistoassurecross-countryequivalencebyforcingcoderstomakeexplicit
comparisonsacrosscountries.Thishelpsthemeasurementmodelestimate,andcorrectfor,
systematic biases across coders and across countries thatmay result if Country Experts
employvaryingstandards intheirunderstandingofaquestion,e.g.,aboutwhata“high”
levelofrepressionmightconsistof.
Throughoutimplementationoftheproject,wehaveencouragedCountryExpertsto
codemultiplecountriesover time -bridge coding.Anexpertwhoagrees tocodeoneor
moreadditionalcountriesreceivesthesamesetofsurveysforthesametimeperiodasthe
originalcountrytheycoded;bridgecodingthereforetypicallycoverstwotimeperiods:1900
topresent,or2005tothepresent.BridgecodinghelpsusbettermodelhowCountryExperts
makejudgmentsbetweendifferentresponsecategories,andallowsustoincorporatethis
informationintotheestimatedscoreforeachcountry-indicator-year/date.
Bridge coding is most useful when the chosen countries have different regime
histories.ThisgeneratesvarianceacrossaCountryExpert’sratings,whichinturnprovides
information about the coder’s judgments that can be used to inform themeasurement
model.Inordertomaximizevariance,andthereforegainasmuchinformationaspossible
abouteachexpert’s thresholdsand reliability,weencourageCountryExperts to select–
fromamongcountriestheyarefamiliarwith–thosethathavethemostdistinctivehistorical
trajectories.
AsofMarch2017,wehaveover530bridgecoders–about19percentofallCountry
Experts.Onaverage,theseexpertscode2.2countries.
ConstraintsoftimeorexpertisesometimespreventCountryExpertsfromconducting
bridgecoding. In thesesituations,weencourageCountryExperts toperformthesimpler
typeofcross-countrycomparisoncalledlateralcoding.Thatis,inadditiontotheiroriginal
26
codingofonecountryovertime(e.g.,from1900tothepresent),theycodeanumberof
countriesforasinglepointintime–January1,2012–focusingonthesamesetofquestions.
SomeCountryExpertshavecodedupto14countries.Moretypically,lateralcoding
extendstoafewcountries.Todate,350CountryExperts(about12%)haveperformedlateral
coding,coveringonaverageof5.5countriesand6.3surveys.Asaresult,lateralcodingby
regular Country Experts has provided linkages equivalent to over 1,100 “fully covered”
countries–inotherwords,countriesthathavebeen“cross-coded”bylateral/bridgecoding
acrossallindicatorsinthedataset.
Phasesofthedatacollection
ThefirstphaseofV-Dem,comprisingofdatacollectionfortheentireworldfrom1900to
2012,beganinMarch2012andwasconcludedinfall2013.167countries/territoriesexisting
todaywereincluded,thisrequiredtheinvolvementofsome2,000coders.
ByDecember2014,V-Dementeredintoanewphasebytheveryfirstupdateofthe
data.Goingforwardwehopetobeabletoupdatethedataonanannualorbiannualbasis
forall/asmanycountriesaspossible.
Firstdataupdate:Datawascollectedfor2013-2014for54countriesandcollected
dataforthefull-timeperiodfrom1900to2014forsixnewcountries.
Seconddataupdate:ByMarch2016thesecondupdatewasconcluded.Thistime
datafor76countriesfor2013-2015hadbeenupdated.
Third data update: Is launched in April 2017 and includes updated data for 174
countries,includingfournewcountries.Thisistheveryfirstcompletedataupdate
thattheV-Deminstituteisconducting.TheV-Demdatabasenowcontainsmorethan
17milliondatarecords.
Duetocoderattrition,codingfortheupdatesareconductedbyamixofreturningCountry
ExpertsandnewCountryExperts.Astoencourageconsistencyinratingsovertime,Country
Expertsareabletoseeandchangealltheirpreviousratings.WeasknewCountryExpertsto
codeform2005anonwardssoastoensurethattheirscoresoverlapbyanumberofyears
withreturningCountryExperts’ratings.
Forthesecondandthirdupdateofthedataweimplementedaseriesofvignettes
27
foreachsurvey togiveusadditional leverageonmeasurementerror.The fourthupdate
takesplaceOctober2017-March2018,withthereleaseofdatabyMarch2018.
3. Measurement
Havingdiscussedtheprocessofdatacollection,weproceedtothetaskofmeasurement.
Underthisrubric,weinclude(a)thequestionnaire,(b)ourmeasurementmodel,(c)methods
ofidentifyingerrorinmeasurement,(d)studiesofmeasurementerror,and(e)methodsof
correctingerror.Inprinciple,thediscussionsarerelevantfordifferenttypesofdata(A,B,
andCintheV-Demscheme)butmostifnotallofthemaremuchmoreacutewhenitcomes
toexpert-basedcodingofevaluative,non-factualyetcriticalindicators.Hence,mostofthe
followingfocusesontheC-typeindicators.
TheQuestionnaire
Themost important featureofasurvey is theconstructionofthequestionnaire itself. In
craftingindicatorstomeasuretheC-typedata,wehavesoughttoconstructquestionswith
bothspecificandclearmeanings,andwhichdonotsuffer fromtemporalorspatialnon-
equivalence.Todesignthesequestions,weenlistedleadingscholarsondifferentaspectsof
democracyanddemocratizationasProjectManagers.
We enrolled each Project Manager because of her record of scholarly
accomplishment in a particular area related to issues of democracy (e.g. legislatures,
executives, elections, and civil society), with the goal of creating a team that also had
substantiveexperiencesandexpertiseonallregionsoftheworld.ProjectManagersbegan
designingsurvey-questionsintheirareaofexpertisein2009,andwecollectivelyreviewed
andrefinedtheirquestionsoverthecourseoftwoyears.
WeimplementedapilotoftheV-Demsurveyin2011,whichservedasaninitialtest
ofourquestionnaire.Itwasimplementedfor12countries,two(one“easy”andone“hard”)
fromeachofthesixmajorregionsoftheworldenlistingover120pilot-CountryExpertsand
resultedinsome450,000ratingsonpreliminaryindicators.Theresultspromptedrevisions
inthenextroundofsurveys.Anotherroundofcollectivedeliberationfollowed, involving
consultationswithscholarsoutsideoftheprojectteam.TherevisedquestionsforC-coding
28
thusenduredseveralroundsofreviewwithProjectManagersandoutsideexpertsoverthe
courseoftwoyearsbeforeemergingintheirfinalform,asdescribedintheCodebook.
Identifying,Correcting,andQuantifyingMeasurementError
Evenwithcarefulquestiondesign,aprojectofthisnaturewillencountererror.Sucherror
maybetheproductoflinguisticmisunderstandings(mostofourcodersdonotspeakEnglish
astheirfirstlanguage,andsometakethesurveyinatranslatedform),misunderstandings
aboutthewayaquestionappliestoaparticularcontext,factualerrors,errorsduetothe
scarcityorambiguityofthehistoricalrecord,differinginterpretationsabouttherealityofa
situation,variationinstandards,coderinattention,errorsintroducedbythecoderinterface
orthehandlingofdataonceithasbeenenteredintothedatabase,orrandommistakes.
Someof theseerrorsare stochastic in the senseofaffecting theprecisionofour
estimatesbutnottheirvalidity.Othererrorsaresystematic,potentiallyintroducingbiasinto
theestimatesthatweproduce.Inthissection,wefirstdescribethemethodologicaltools
weusetomodelandcorrectforsystematicbiasincoders’answerstoourquestions,aswell
astoprovideestimatesofthereliabilityofthesecodings.Wethendescribetheprocedures
weusetoassessthevalidityofourestimates.Finally,weexplainhowweidentifythemost
serioussourcesofmeasurementerror,inordertocontinuouslyimprovehowwegatherand
synthesizedata.
MeasurementModels
ThemostdifficultmeasurementproblemsconcerntheC-typequestions,allofwhichrequire
substantialcaseknowledgeandgenerallysomedegreeofsubjectiveevaluation.Havingfive
codersforeachofthesequestionsisimmenselyuseful,asitallowsustoconductinter-coder
reliabilitytests.Thesesortsoftests–standardinmostsocialsciencestudies–areonlyrarely
ifeveremployedinextantdemocracyindices.
Whileweselectexpertscarefully,theyexhibitvaryinglevelsofreliabilityandbias,
and may not interpret questions consistently. In such circumstances, the literature
recommends that researchers usemeasurementmodels to aggregate diversemeasures
wherepossible,incorporatinginformationcharacterizedbyawidevarietyofperspectives,
29
biases,and levelsof reliability (Bollen&Paxton2000,Clinton&Lapinski2006,Clinton&
Lewis2008, Jackman2004,Treier& Jackman2008,Pemstein,Meserve&Melton2010).
Therefore,tocombineexpertratingsforaparticularcountry-indicator-yeartogeneratea
single“bestestimate”foreachquestion,weemploymethodsinspiredbythepsychometric
and educational testing literature (see, e.g., Lord&Novick 1968, Jonson&Albert 1999,
Junker1999,Patz& Junker1999).Theunderpinningsof thesemeasurementmodelsare
straightforward:theyusepatternsofcross-rater(dis)agreementtoestimatevariations in
reliabilityandsystematicbias.Inturn,thesetechniquesmakeuseofthebiasandreliability
estimatestoadjustestimatesofthelatent—thatis,onlyindirectlyobserved—concept(e.g.,
executiverespectfortheconstitution,judicialindependence,orpropertyrights)inquestion.
Thesestatisticaltoolsallowustoleverageourmulti-coderapproachtobothidentifyand
correctformeasurementerror,andtoquantifyconfidenceinthereliabilityofourestimates.
Variationintheseconfidenceestimatesreflectsituationswhereexpertsdisagree,orwhere
little information is available because few raters have coded a case. These confidence
estimatesare tremendouslyuseful. Indeed, to treat thequalityofmeasuresof complex,
unobservableconceptsasequalacrossspaceandtime,ignoringdramaticdifferencesinease
ofaccessandmeasurementacrosscases,isfundamentallymisguided,andconstitutesakey
threattoinference.
ThemajorityoftheC-typequestionsareordinal: theyrequireCountryExpertsto
rankcasesonadiscretescale. Take,forexample,thefollowingquestionaboutelectoral
violence:
Question:Inthisnationalelection,wasthecampaignperiod,electionday,andpostelectionprocessfreefromothertypes(notbythegovernment,therulingparty,ortheiragents)ofviolencerelatedtotheconductoftheelectionandthecampaigns(butnotconductedbythegovernmentanditsagents)?Responses:
0. No. There was widespread violence between civilians occurring throughout the electionperiod,orinanintenseperiodofmorethanaweekandinlargeswathsofthecountry.Itresultedinalargenumberofdeathsordisplacedrefugees.
1. Notreally.Thereweresignificantlevelsofviolencebutnotthroughouttheelectionperiodorbeyondlimitedpartsofthecountry.Afewpeoplemayhavediedasaresult,andsomepeoplemayhavebeenforcedtomovetemporarily.
2. Somewhat.Thereweresomeoutburstsoflimitedviolenceforadayortwo,andonlyinasmallpartofthecountry.Thenumberofinjuredandotherwiseaffectedwasrelativelysmall.
3. Almost.Therewereonlyafewinstancesofisolatedviolentacts,involvingonlyafewpeople;noonediedandveryfewwereinjured.
4. Peaceful.Noelection-relatedviolencebetweenciviliansoccurred.
30
Note, inparticular, thattheserankingsdonot followan interval-levelscale. Onecannot
subtractalmostfrompeacefulandgetnotreally.Furthermore,itneednotbethecasethat
thedifferencebetweennotreallyandsomewhatisthesameasthatbetweenalmostand
peaceful.Perhapsmostimportantly,althoughwestrivetowritequestionsandresponses
that are not overly open to interpretation, we cannot ensure that two coders look at
descriptionslikesomewhatinauniformway—evenwhensomewhatisaccompaniedbya
carefully formulated description—especially because coders have widely varying
backgroundsandreferences.Inotherwords,onecoder’ssomewhatmaybeanothercoder’s
not really; a problem known as scale inconsistency. Therefore, we use Bayesian item
response theory (IRT) modeling techniques (Fox 2010) to estimate latent polity
characteristicsfromourcollectionofexpertratingsforeachordinal(C)question.
Specifically, we fit ordinal IRTmodels to each of our ordinal (C) questions. (See
Johnson&Albert1999foratechnicaldescriptionofthesemodels.)Thesemodelsachieve
threegoals.First,theyworkbytreatingcoders’ordinalratingsasimperfectreflectionsof
interval-levellatentconcepts.Withrespecttotheexamplequestionabove,ourIRTmodels
assumethatelectionviolencerangesfromnon-existenttoendemicalongasmoothscale,
andcodersobservethislatentcharacteristicwitherror.Therefore,whileanIRTmodeltakes
ordinalvaluesasinput,itsoutputisaninterval-levelestimateofthegivenlatenttrait(e.g.
election violence). Interval-valued estimates are valuable for a variety of reasons; in
particular,theyareespeciallyamenabletostatisticalanalysis.Second,IRTmodelsallowfor
the possibility that coders have different thresholds for their ratings (e.g. one coder’s
somewhatmight fall above another coder’salmost on the latent scale), estimate those
thresholds from patterns in the data, and adjust latent trait estimates accordingly.
Therefore, they allow us to correct for this potentially serious source of bias, known as
differential itemfunctioning (DIF).16This isvery important inamulti-raterproject likeV-
Dem,wherecodersfromdifferentgeographic,cultural,andotherbackgroundsmayapply
differingstandardstotheirratings.Finally,IRTmodelsassumethatcoderreliabilityvaries,
16Givencurrentlyavailabledata,wemustbuild inassumptions—formally, theseareknownashierarchicalpriors—thatrestricttheextenttowhichcoders’thresholdestimatesmayvary.Informally,whileweallowcoderstolookatordinalrankingslikesomewhatandalmostdifferently,weassumethattheirconceptionsarenottoodifferent.Weareworkingtorelaxtheseassumptionsbycollectingmoredata.TechnicaldetailsareavailableinV-DemWorkingpaperno.21,andfullcodeisreleasedwiththedataset.
31
produceestimatesof raterprecision, anduse theseestimates—in combinationwith the
amountofavailabledataandtheextenttowhichcodersagree—toquantifyconfidencein
reportedscores.
Sinceourcodersgenerallyrateonecountrybasedontheirexpertise,itisnecessary
toutilizelateralcoders.Aspreviouslydescribed,thesecodersratemultiplecountriesfora
limitedtimeperiod(mostlyoneyear,butinsomecasesten).Wehaveatpresentsome350
lateralcoders.Inaddition,wehaveover390bridgecoders,asdiscussedabove.Theseare
coderswho code the full time series (generally 1900-2016) formore than one country,
coveringoneormoreareas (“surveys”).17 Essentially, this codingprocedureallowsus to
mitigate the incomparability of coders’ thresholds and the problem of cross-national
estimates’calibration(Pemsteinetal.2017).Whilehelpfulinthisregard,ourtestsindicate
that,giventhesparsityofourdata,eventhisextensivebridge-codingisnotsufficienttofully
solvecross-nationalcomparabilityissues.Wethereforeemployadata-collapsingprocedure.
At its core, this procedure relies on the assumption that as long as noneof the experts
changetheirratings(ortheirconfidenceabouttheirratings)foragiventimeperiod,wecan
treat the country-years in this period as one year. The results of our statistical models
indicate that this technique isextremelyhelpful in increasing theweightgiven tobridge
coders,andthusfurtheramelioratescross-nationalcomparabilityproblems.
Asafinalnote,ourmodeldivergesfrommorestandardIRTmodelsinthatitemploys
empiricalpriors.Specifically,wemodelacountry-year’slatentscoreforagivenvariableas
beingdistributedaccordingtoanormaldistributionwithanappropriatelywidestandard
deviationparameterandameanequaltotherawmeanofthecountry’sscores,weighted
bycoderconfidenceandnormalizedacrossallcountry-years.Moreformally,𝑍"~𝑁(𝜇", 1),
where𝑍isthelatentscoreforcountry-year𝑖,and𝜇isthenormalizedconfidence-weighted
average from the raw data.18 In contrast,most standardmodels employ a vaguemean
17Thuswehave lateral/bridgecodingcovering theequivalentofover1,100“full coverage”ofall country-questions.
18Therearetwosetsofexceptionstoouruseofthenormalizedconfidence-weightedaverageofcoderscoresasempiricalpriors. First,wedonot includedata from lateral coders in thecomputationof theempiricalpriors. We exclude these data from this procedure because the purpose of lateral codings is to betterestimatethresholdsofexperts,notprovidedataregardingthespecificcountryyeartheyarelateralcoding.Inprinciple,excludingthesedatawillassistintheestimationoflateralcoders'thresholds,sinceitanchorstheir thresholds tocountry-yearvalues forwhichwehaveagreatdealofdata (i.e. lateral-codedcountry
32
estimate,i.e.𝑍"~𝑁(0,1).Ourapproachofusingempiricalpriorsissimilartothestandard
approach: our wide standard deviation parameter still allows for the model output to
divergefrompriorasthedatawarrant.However,ourapproachincorporatesouractualprior
beliefsaboutacountry’sscoreandthusyieldsmoreaccuratemeasures.Especially inthe
caseofcountrieswithextremevalues,atraditionalapproachrisksbiasingoutputtoward
themean.
FutureversionsofourordinalIRTmodelswillimproveoncurrentestimatesintwo
primaryways.First,hierarchicalIRTmodelingtechniques(Patzetal.2002,Mariano&Junker
2007)wouldallowustoborrowstrengthfromdifferentvariableestimates,yieldingmore
precisemeasuresofeachvariable.Second,allraterscompleteapost-surveyquestionnaire
thatasksdemographicandattitudinalquestions.Codersalsoreportpersonalassessments
of confidence in their responses to each question. At present, of these data we only
incorporateconfidenceintothemodel,usingittoweightourpriormeanestimates;further
useoftheseformsofdatainourmodelswillallowustoteaseoutpatternsconcerningbiases
andreliabilityacrossdifferent typesofexperts,andgenerally improve thequalityofour
estimates.
Wealsouseconceptually-similarIRTtechniqueswhensufficientvariationexiststo
identifyraterthresholdsfornominalandsomedichotomousexpert-codedvariables.Forthe
remainingvariablesweprovidetheunweightedmean.
AnchoringVignettes
V-Dem’sthree-prongedapproachtodealingwithDIF—usingIRTmodels,recruitingbridge
years).Second,weoffsetthecontributionofhistoricalcoders(i.e.coderswhocodeyearsbefore1900)andnewcoders (i.e. coderswhoonlycodeyearsafter2005) to theempiricalpriorby theaveragedifferencebetweenthesecodersandthosecoderswhocodedtheyears1900-2012inoverlapyears(i.e.thoseyearsboth these sets of coders and the full time period coders coded). More specifically, we determine theconfidence-weightedaveragescoreofthefull-timeperiodcodersforaspecificcountryintheoverlapyears,andsubtracttheequivalentaveragefornewcodersofthesamecountryfromthisvalue.Wethenaddthisdifference to the new coders' scores for a given country for when computing the prior (restricting theresultingvaluessuchthattheycannotexceedtherangeoftheordinaldata).Weusethesameprocedureforhistoricalcoders (i.e.wecomputeoffsets fornewandhistoricalcodersseparately).Thepurposeof theseoffsetsisasfollows.Expertswhocodedifferenttimeperiodsmayhavedifferentcognitivereferencepointsforlevelsoftheordinalscale,andthusprovidedifferentvaluesforthesamelatentconstructduetoDIF.Theoffsetsamelioratethisproblembyfixingthepriorforagivencountry-yeartoaconsistentreferencepoint,i.e. thescoresof thosecoders forwhomwehavethemostdata (thoseexpertswhocodedthe full timeperiod).
33
andlateralcoders,andemployingempiricalpriors—hadhelpedtoproduceadatasetthat
standsupwelltotestsofvalidity(McMann2016,McMannetal2016,Sigman&Lindberg
2015,Teorell,Coppedge,Skaaning&Lindberg2016).Nonetheless,thereremainsroomfor
improvement.ThereforeV-Demhas includedanchoringvignettes (King&Wand2007) in
surveywavesstarting in2015/2016.Anchoringvignettesaredescriptionsofhypothetical
casesthatprovideinformationnecessarytoansweragivensurveyquestion.Weaskcoders
toratevignettesforV-Demquestionsbecausepatternsofvariationinhowratersevaluate
these synthetic cases provides information about difference how coders translate their
perceptions about cases into ordinal ratings, providing another tool formeasuring, and
adjustingforDIF.
Wefieldedourfirstwaveofvignettesduringthe2015/2016update,presenting116
vignettes for 31V-Demquestions to 599 coders from94 countries.We followedwith a
secondwaveofvignettesduringthe2016/2017update,presenting224vignettesfor66V-
Demquestions to1400coders from174countries.Codersarenot required tocomplete
vignettes,butwererequested.
Vignettesprovidebridgingdatathatrequiresnospecificcaseknowledge,enabling
ustoobtainbridginginformationacrossraterswhoarenotqualifiedtocodethesameset
of real-world cases. They also ensure that coders are considering the same information
when evaluating cases, helping us to isolate the effect of DIF on raters’ codes.We are
studying a variety of methods for incorporating information from anchoring vignette
responsesintoourmodelingstrategy.Currentlywetreatthemlikeanyotherobservation
whenfittingmeasurementmodels,therebyusingthebridginginformationthattheyprovide
toimprovetheDIFadjustmentsproducedbyourIRTmodels.
IdentifyingRemainingErrors
Toevaluatepossibleerrorsweemployanumberoftests,someofwhichareincorporated
into themeasurementmodels and others of which are applied ex post to examine the
validityofmodeloutput.
First,wehaveuseddatafromthepost-surveyquestionnairethateveryV-Demcoder
completestoidentifypotentialsourcesofbias.Thissurveydelvesintofactorsofpossible
relevance to coder judgments, such as personal characteristics like sex, age, country-of-
34
origin,educationandemployment.ItalsoinquiresintoopinionsthatCountryExpertshold
about thecountry theyarecoding,asking themtoassignapoint scoreona0-100scale
summarizing the overall level of democracy in the country on January 1, 2012, using
whateverunderstandingofdemocracy they choose to apply.Weask the samequestion
about several prominent countries from around the world that embody varying
characteristics of democracy/autocracy. Finally, the questionnaire contains several
questionsintendedtoelicitthecoder’sviewsabouttheconceptofdemocracy.Wehaverun
extensivetestsonhowwellsuchindividual-levelfactorspredictscountry-ratingsbuthave
foundthattheonlyfactorconsistentlyassociatedwithcountry-ratingsiscountryoforigin
(with“domestic”codersbeingharsherintheirjudgments).Thisisalsotheindividual-level
characteristicincludedinthemeasurementmodelestimates.
Inthefuture,weneverthelessplantouseeachpieceofinformationfromthispost-
surveyquestionnairetohelpinformthemeasurementmodel,i.e.,toenhanceprecisionand
limit possible undetected biases. The measurement model will also take into account
information we can glean from the performance of the coders that might serve as an
indicationoftheirlevelofattentiveness,effort,andknowledge.Thisinformationincludes
inter-coder reliability (assessed at the coder level across all codings), self-reported
confidence(ineachcoding),numberofcountry-yearscoded(alltogether),codingchanges
(thenumberoftimesthatacoderchangestheircodingfromT-1toTrelativetoothercoders
forthatcountry/indicator,aggregatedacrossallcodings),timeontask(thenumberofhours
a coder is logged into the on-line system, discounted by the number of
country/indicator/yearss/hehascoded),accesses(thenumberoftimestheon-linesurvey
isaccessed),contacts(writingcommentsoraskingquestionsoftheV-Demteamthatare
non-logistical in nature), and response rate (assessed at the country level). (With the
exceptionofinter-coderreliability,theseelementshavenotyetbeenincludedinthemodel.)
Eachoftheaforementionedfeatureswillalsobetestedindependently.Thus,wewill
beabletoreportonwhether,andtowhatextent,eachoftheobservedandself-reported
featuresofthecodersaffectstheirratings.Inparticular,byincludinghierarchicalpriorsthat
depend on observed rater characteristics and behavior in our latent variable model
specifications—anapproachoftenreferredtoas“empiricalBayes”—wecanevaluatethe
extenttowhichsuchfeatureshelptoexplainraterbiasandreliability,whilesimultaneously
35
incorporatingthatinformationintoindicatorestimates.
Inaddition,wewillapplyseveralexpost tests toevaluatethequalityof thedata
emanatingfromthemeasurementmodel.Onesortoftestreliesonthedistributionofthe
data.Ifthedistributionofresponsesforaparticularcountry/indicator/yearisbi-modalwe
haveanobviousproblem:codersdisagreewildly.Thisalsomeansthatthepointestimate
fromthemeasurementmodelisunstable:achangeofcodingforanysinglecoder,orthe
additionofanewcoder,islikelytohaveabigimpactonthepointestimate.Disagreement
as registeredbyabi-modaldistributioncould representa situation inwhich the truth is
recalcitrant – presumably because available information about a topic is scarce and/or
contradictory.Oritcouldrepresenterrorsthatarecorrigible.
A secondapproach tovalidationcomparesV-Dem indiceswithother indices that
purport to measure similar concepts, i.e., convergent validity. For example, a set of
regressionsusingallavailabledataoftheV-DemElectoralDemocracy/PolyarchyIndex–and
some of its constituent indicators – against Polity2 indicates relatively high correlations
(Pearson’s r= .85) and (separately) against FH Political rights (Pearson’s r= .90).
Unfortunately,techniquesofconvergentvalidityarelimitedintheirutility.First,wehave
some doubts about the validity of standard indices (see Comparisons and Contrasts).
Second, standard indices tend to hover at a higher level of aggregation, thus impairing
comparability betweenV-Dem indices andalternative indices. Indeed, only a fewextant
indicesarecloseenoughinconceptionandconstructiontoprovideanopportunityfordirect
corroborationwithV-Demindices.
A third approach to validation focuses on face validity. Once data collection is
completeforagroupofcountries,RegionalManagersandothermembersoftheV-Dem
teamlookcloselyatpointestimatesinanattempttodeterminewhethersystematicbias
mayexist.OnemajorsuchreviewwasconductedinOctober2013whenalmostallRegional
Managers,allProjectManagers,ResearchFellows,PIsandstaff,spentfourdayscollectively
reviewingalldatacollatedatthatpointtovalidatetheapproachandaggregationmethods.
Theprocessoffacevaliditycheckshassincethenbeenrecurrent.
CorrectingErrors
We correct problems with factual questions (B-type indicators) whenever the Principal
36
Investigators,inconsultationwiththerelevantProjectManagers,becomeconvincedthata
better (i.e., more correct) answer is available. Based on analysis of submitted data by
CountryCoordinators,certainvariablesweredesignatedasB+A.UsingtheoriginalB-data
as a point of departure and cross-checking with external resources, we designed and
implemented a coding scheme to re-code these indicators, as the Codebook describes.
Indicatorsaffectedincludeallindicatorsfromthedirectdemocracysurvey,fourindicators
ontheexecutive,fouronelectionsandnineonlegislature.Thedecisiontore-assignthese
indicators was also due to the interaction between question formulation and coder
interpretation, e.g. in some instances the meaning of “plebiscite” was interpreted in a
differentwaythanwhattheProjectManagerenvisaged,leadingtodiscrepanciesincoding.
Wehandleproblemswithevaluativequestions(C-typeindicators)withrestraint.We
fullyexpectthatanyquestionrequiringjudgmentwillelicitarangeofanswers,evenwhen
allcodersarehighlyknowledgeableaboutasubject.AkeyelementoftheV-Demproject–
settingitapartfrommostotherindicesthatrelyonexpertcoding–iscoderindependence:
eachcoderdoesherworkinisolationfromothercodersandmembersoftheV-Demteam
(apart fromclarifyingquestions about theprocess). Thedistributionof responses across
questions, countries, and years thus provides vital insight into the relative
certainty/uncertaintyofeachdatapoint.SinceaprincipalgoaloftheV-Demprojectisto
produceinformativeestimatesofuncertaintywedonotwishtotamperwithevidencethat
contributestothoseestimates.Arguably,thenoiseinthedataisasinformativeasthesignal.
Moreover,waywardcoders(i.e.,coderswhodivergefromothercoders)areunlikelytohave
a strong influence on the point estimates that result from the measurement model’s
aggregationacrossfiveormorecoders.Thisisespeciallythecaseifthewaywardcodersare
consistentlyoff-center (acrossall their codings); in this case, theirweight indetermining
measurementmodelscoresisreduced.
That said, there have been instances in which we have altered C-data. A few
questions were largely of factual nature (e.g. number of legislative chambers; if a local
governmentexists,whichofficeswereelectedinaparticularelection,etc.).Sincewelater
acquiredenoughfundingtohaveassistantsconductthefactualcodingbasedonsystematic
consultationofcrediblesources,wedischargedthedatasubmittedbyCountryExpertsfor
theseparticularquestionsandany“downstream”data.Forexample, ifaCountryExpert
37
indicatedthat thereweretwochambers in the legislature foraparticularyear,shethen
coded“downstream”inthequestionnaireaseriesofquestionsregardingboththelower
andupperchamber.Ifourresearchestablishedthatanupperchamberdidnotinfactexist
inthatparticularyear,wecleanedtherecordsofdataprovidedbytheexpertfortheupper
chamber.Thiscleaningaffected19%ofallexecutivedatasubmittedforthosedownstream
variables,7.7%ofthedataintheelectionsurveyand11%inthelegislativesurvey,forthe
initial data collection effort covering 1900 to 2012. These numbers reflect placeswhere
coders unnecessarily coded due either to a) problem with the skipping function in the
surveys,b)coders’ability tochangethepre-coded, factualdata,orc)an initialdecision,
subsequently reversed, to have Country Experts to answer some of the A-coded (more
factual) questions. After improving the coding interfaces and making it impossible for
country experts to change such factual pre-coded data during the coding during later
updates,theneedforsuch“downstream”cleaninghasbeenreducedtoclosetonil.
Inafinalcase,weremovedoriginalcodingbysomeCountryExpertsbecauseofa
factual misunderstanding (or misunderstanding about response-categories) about the
existenceoftheinternetineraspriortoitsinvention.
Inallthesesituations,wemaintaintheoriginalcoder-leveldatainarchivedfilesthat
mayberetrievedbyspecialrequestofthePIs.
VersionsofC-Variables
TheV-DemdatasetthencontainsA,B,C,andDindicatorsthatareallunique.Inaddition,to
facilitateeaseofuseforvariouspurposes, theC-variablesaresupplied inthreedifferent
versions(alsonotedintheV-DemCodebook):
1.“RelativeScale”-MeasurementModelOutput–hasnospecialsuffix(e.g.v2elmulpar).
This versionof thevariablesprovides country-year (country-date in thealternative
dataset)pointestimatesfromtheV-Demmeasurementmodeldescribedabove.The
pointestimatesarethemedianvaluesofthesedistributionsforeachcountry-year.
The scale of ameasurementmodel variable is similar to a normal (“Z”) score (i.e.
typically between -5 and 5, with 0 approximately representing the mean for all
country-years in the sample) though it does not necessarily follow a normal
distribution.Formostpurposes,thesearethepreferredversionsofthevariablesfor
38
time-seriesregressionandotherestimationstrategies.
“Measure of Uncertainty” –MeasurementModel Highest Posterior Density (HPD)
Intervals–havethesuffixes–"codelow"and"codehigh"(e.g.,v2elmulpar_codelow
and v2elmulpar_codehigh). These two variables demarcate one standard deviation
upperandlowerboundsoftheintervalinwhichthemeasurementmodelplaces68
percent of the probabilitymass for each country-year score. The spread between
"codelow" and "codehigh" is equivalent to a traditional one standard deviation
confidence interval; a larger range indicates greater uncertainty around the point
estimate.
2.“OriginalScale”–LinearizedOriginalScalePosteriorPrediction–hasthesuffix“_osp,”
(e.g.v2elmulpar_osp).Inthisversionofthevariables,wehavelinearlytranslatedthe
measurementmodelpointestimatesbacktotheoriginalordinalscaleofeachvariable
(e.g. 0-4 for v2elmulpar_osp) as an interval measure.19 The decimals in the _osp
version indicate the distance between the point estimate from the linearized
measurementmodelposteriorpredictionandthethresholdforreachingthenextlevel
on the original ordinal scale. Thus, a_osp value of 1.25 indicates that themedian
measurementmodelposteriorpredictedvaluewascloser to theordinalvalueof1
than2ontheoriginalscale.Sincethereisnoconventionaltheoreticaljustificationfor
linearlymappingordinalposteriorpredictionsontoan intervalscale,20 thesescores
shouldprimarilybeusedforheuristicpurposes.However,sincethe_ospversionmaps
ontothecodingcriteriafoundintheV-DemCodebook,andisstronglycorrelatedwith
theMeasurementModeloutput(typicallyat.98orhigher),someusersmayfindthe
_osp version useful in estimating quantities such as marginal effects with a clear
substantiveinterpretation.Usingthe“OrdinalScale”estimates---orincorporatingthe
19 More specifically, we use the measurement model to estimate the posterior distribution around thepredictedprobabilitythatatypicalcoderwouldplaceacountry-yearestimateateachleveloftheoriginalcodebook scale. We then linearly map these predicted probability distributions onto the original scale,producingadistributionofinterval-valuedscoresontheoriginalcodebookscaleforeachcountry-year.
20ThemaintheoreticalandpragmaticconcernwiththesedataisthatthetransformationdistortsthedistancebetweenpointestimatesintheMeasurementModeloutput.Forexample,thedistancebetween1.0and1.5inthe_ospdataisnotnecessarilythesameasthedistancebetweena1.5and2.0.
39
properties of ordinal probit models into the estimation procedure---is generally
preferabletousingthe_ospestimatesinstatisticalanalyses.Thatsaid,ifauseruses
_ospdatainstatisticalanalysesitisimperativethatshefirstconfirmthattheresults
arecompatiblewithestimationsusingMeasurementModeloutput.
“MeasureofUncertainty”–LinearizedOriginalScaleHPDIntervals–havethesuffixes
– "codelow" and "codehigh" (e.g., v2elmulpar_osp_codelow and
v2elmulpar_osp_codehigh).Weestimatethesequantitiesinasimilarmannerasthe
Measurement Model Highest Posterior Density Intervals. They demarcate one
standarddeviationupperandlowerboundsoftheintervalinwhichthemeasurement
model places 68 percent of the probabilitymass for each country-year score. The
spreadbetween"codelow"and"codehigh"isequivalenttoatraditionalonestandard
deviationconfidenceinterval;alargerrangeindicatesgreateruncertaintyaroundthe
pointestimate.
3.“OrdinalScale”-MeasurementModelEstimatesofOriginalScaleValue–hasthesuffix
"_ord" (e.g. v2elmulpar_ord). This method translates the measurement model
estimates back to the original ordinal scale of a variable (as represented in the
Codebook) after taking coder disagreement andmeasurement error into account.
Moreprecisely, itrepresentsthemost likelyordinalvalueontheoriginalcodebook
scale intowhichacountry-yearwould fall, given theaveragecoder’susageof that
scale. Specifically, we assign each country-year a value that corresponds to its
integerizedmedianordinalhighestposteriorprobabilitycategoryoverMeasurement
Modeloutput.
“MeasureofUncertainty”–OriginalScaleValueHPDIntervals–havethesuffixes–
"codelow" and "codehigh" (e.g., v2elmulpar_ord_codelow and
v2elmulpar_ord_codehigh). We estimate these values in a similar manner as the
Measurement Model Highest Posterior Density Intervals. They demarcate one
standarddeviationupperandlowerboundsoftheintervalinwhichthemeasurement
model places 68 percent of the probabilitymass for each country-year score. The
40
spreadbetween"codelow"and"codehigh"isequivalenttoatraditionalonestandard
deviationconfidenceinterval;alargerrangeindicatesgreateruncertaintyaroundthe
pointestimate.
AdditionalPossibilitiesforIdentifyingSourcesofMeasurementErrorintheFuture
A final approach to validationanalyzes various featuresof thedatagatheringprocess in
ordertogaugepossiblesourcesoferror.Thisanalysistakestheformofvariousstudiesin
whichaparticularissueisprobedinanintensivefashion.Thefollowingstudiesareunderway
oronthedrawingboard–thoughwecannotsayforsurehowlongitwilltakeustocomplete
them.
Onesuchstudywillfocusoncodertypes.Akeychallengetothevalidityisthatdata
maybesubjecttothesubjectiveperceptionsandopinionsofthechosencoders. Is itthe
casethatadifferentsetofcodersmightarriveataverydifferentsetofanswers?Features
ofthecoderscapturedinourpost-surveyquestionnairecanbetestedsystematicallyacross
theentiredataset,asnoted.However,wecannottestthepotential impactofadifferent
kindofcodernotincludedinourusualsample.Thisstudythereforefocusesoncomparisons
across different coder types, e.g., partisans, academics, civil society professionals,
businesspeople, cosmopolitans (those speaking foreign languages and with travel or
educationalexperienceabroad),educatedlaycitizens,andlesseducatedlaycitizens.Results
ofthisstudyshouldindicate(a)howfartheconsensusoncodingextends(i.e.,towhattypes
ofcoders),(b)howmuchdifferencethebackgroundofthecodermakes,(c)forwhattypes
of questions itmatters, and (d)which sorts of coders have themost positive view of a
country.Moregenerally,wehopetolearnmoreaboutthesensitivityofV-Demdatatoour
samplingofCountryExperts.
Asecondstudywouldbetofocusoncountrysequencing.Doesitmatterifcoders
have considered other countries prior to coding CountryA? Such a studywould involve
randomizing respondents into two groups. Group 1 is asked to code Country A. Several
weekslater,theyareaskedtocodeahandfulofcountriesincludingCountryA,whichthey
mustre-code.Thecomparisoncasesshouldincludethosethatareinthesameregionaswell
asacountry(preferablyinthesameregion,orwithahistoryofcolonialinvolvementinthe
region) generally regarded as highly democratic. Respondents arenot remindedof their
41
originalcodingsforCountryAandareencouragedtoadjusttheiroriginalcodingiftheyfeel
that a more accurate assessment is possible, in light of their consideration of other
countries.Group2repeatsthisprocedureinreverse.That is,theyfirstcodeahandfulof
relatedcountriesandthenareaskedtocodeCountryA.
Athirdstudywouldbetofocusonquestionordering.TheV-Demquestionnaireis
not randomized for several reasons. First, somequestionsmust be asked in a particular
order(laterquestionsareactivatedorskippeddependingupontheanswers).Second,we
wish tomaintain a logical flowacrossquestions and tomake the flowaspredictable as
possible,sothatinadvertenterrorsareminimized.Finally,wewishtomaintainequivalence
acrosssurveys.However,onemayalsowishtoknowwhethertheorderingofquestionson
thequestionnaireaffectsresponses,andifsohow.Toprobethisquestiononewouldhave
to randomize questions within a survey (but not across surveys), without upsetting
questionsthataredependentuponothers,andwhilemaintainingsomedegreeoflogical
flow.Forexample,wewillreversetheorderofquestionsthatareaskedfirstaboutmenand
nextaboutwomen.
A fourth study could explore the quality of model-based bias adjustment. In
particular,becausecodersfromdifferentcountriesmayunderstandbothquestionwordings
andconceptsindifferentways,twocodersoperatingindifferentcontextsmightratetwo
identical cases differently from one another. A common approach to addressing this
problemistoconstructanchoringvignettes—shorthypotheticaldepictionsofcases—and
thenaskcoders toevaluatevignettes inaddition to realcases,and tousedifferences in
vignette evaluations to correct for inter-personal differences in coder perceptions or
understandingsofconcepts (Kinget.al.2004;King&Wand2007;Hopkins&King2010).
Because the vignettes are fixed, these techniques assume that differences in rater
evaluationsmustrepresentdifferencesinpersonalinterpretation,andthensubtractthese
differences from responses for real cases, ostensibly correcting for respondent
incomparability. Similarly, given sufficient overlap in observed coding across raters, our
latentvariablemodelingtechniquescanusepatternsofinter-coderagreementtoidentify
andcorrectforsystematicdifferencesinraters'perceptionsandconceptualunderstandings.
Inotherwords,differencesinhowexpertsrateidenticalcaseshelptoidentifyinter-expert
variationininterpretationinmuchthesamewaythatvariationinratingsoffixedvignettes
42
does.Wecanvalidatethisfeatureofthemodelbycomparingitsperformancetoavignette-
basedapproachforcontrollingincomparabilityinsurveyresponses.Focusingonasubsetof
indicators, we would recruit country-experts to rate an anchoring vignette, their own
country,andsomecomparisoncountries.Thenwewouldapplybothvignette-basedand
measurement-model based corrections to responses to determine if they produce
comparableresults. Anexperimentalcomponentcanalsoseektodetermine ifvignettes
themselvesaltercoderbehavior.Inparticular,wecouldusepatternsofagreementbetween
raters to determine if treated experts (vignette condition) produce codings that are
systematicallydifferentfromacontrolpopulation(novignettecondition).
43
References
Almond,GabrielA.,SidneyVerba.1963/1989.TheCivicCulture:PoliticalAttitudesandDemocracyinFiveNations.NewburyPark,CA:Sage.
Bernhard,Michael,EitanTzelgov,Dong-JoonJung,MichaelCoppedge,&StaffanI.Lindberg.2015.TheVarietiesofDemocracyCoreCivilSocietyIndex.UniversityofGothenburg,VarietiesofDemocracyInstitute:V-DemWorkingPaperSeries,No.12.
Bernhard,Michael,ChristopherReenock,andTimothyNordstrom.2004.“TheLegacyofWesternOverseasColonialismonDemocraticSurvival.”InternationalStudiesQuarterly48(3),225-250.
Bollen,KennethA.,PamelaPaxton.2000.“SubjectiveMeasuresofLiberalDemocracy.”ComparativePoliticalStudies33(1):58–86.
Capoccia,Giovanni,DanielZiblatt.2010.“TheHistoricalTurninDemocratizationStudies:ANewResearchAgendaforEuropeandBeyond.”ComparativePoliticalStudies43(8-9):931-968.
Clinton,JoshuaD.,DavidLewis.2008.“ExpertOpinion,AgencyCharacteristics,andAgencyPreferences.”PoliticalAnalysis16(1):3–20.
Clinton,JoshuaD.,JohnS.Lapinski.2006.“MeasuringLegislativeAccomplishment,1877-1994.”AmericanJournalofPoliticalScience50(1):232–249.
Collier,DavidandJamesMahon(1993).“Conceptual‘Stretching’Revisited:AdaptingCategoriesinComparativeAnalysis.”AmericanPoliticalScienceReview87(4):845-855.
Coppedge,Michael,StaffanLindberg,Svend-ErikSkaaning,andJanTeorell.2015.MeasuringHighLevelDemocraticPrinciplesusingtheV-DemData.UniversityofGothenburg,TheVarietiesofDemocracyInstitute:V-DemWorkingPaperseriesNo.6
Coppedge,Michael,JohnGerring,StaffanI.Lindberg,Svend-ErikSkaaning,JanTeorell,DavidAltman,MichaelBernhard,M.StevenFish,AdamGlynn,AllenHicken,CarlHenrikKnutsen,KyleMarquardt,KellyMcMann,FarhadMiri,PamelaPaxton,DanielPemstein,JeffreyStaton,EitanTzelgov,Yi-tingWang,andBrigitteZimmerman.2015.V-Dem[Country-Year/Country-Date]Datasetv5.VarietiesofDemocracy(V-Dem)Project.
Coppedge,Michael,JohnGerring,StaffanI.Lindberg,Svend-ErikSkaaning,JanTeorell,withDavidAltman,MichaelBernhard,M.StevenFish,AdamGlynn,AllenHicken,CarlHenrikKnutsen,KellyMcMann,PamelaPaxton,DanielPemstein,JeffreyStaton,BrigitteZimmerman,FridaAndersson,ValeriyaMechkova,andFarhadMiri.2015.V-DemCodebookv5.VarietiesofDemocracy(V-Dem)Project.
Coppedge,Michael,JohnGerring,StaffanI.Lindberg,Svend-ErikSkaaning,JanTeorell,FridaAndersson,KyleMarquardt,ValeriyaMechkova,FarhadMiri,DanielPemstein,JosefinePernes,NataliaStepanova,EitanTzelgov,andYi-tingWang.2015.V-DemMethodologyv5.VarietiesofDemocracy(V-Dem)Project.
Coppedge,Michael,JohnGerring,StaffanI.Lindberg,Svend-ErikSkaaning,JanTeorell,andVladCiobanu.2015.V-DemCountryCodingUnitsv5.VarietiesofDemocracy(V-Dem)Project.
Coppedge,Michael,JohnGerring,StaffanI.Lindberg,Svend-ErikSkaaning,JanTeorell,FridaAndersson,ValeriyaMechkova,JosefinePernes,andNataliaStepanova.2015.V-DemOrganizationandManagementv5.VarietiesofDemocracy(V-Dem)Project.
44
Coppedge,Michael,JohnGerring,StaffanI.Lindberg,Svend-ErikSkaaning,andJanTeorell.2015.V-DemComparisonsandContrastswithOtherMeasurementProjects.VarietiesofDemocracy(V-Dem)Project.
Dahl,RobertA.1971.Polyarchy:ParticipationandOpposition.NewHaven:YaleUniversityPress.
Dahl,RobertA.1989.DemocracyanditsCritics.NewHaven:YaleUniversityPress.Epstein,DavidL.;RobertBates;JackGoldstone;IdaKristensen;SharynO’Halloran.2006.
“DemocraticTransitions.”AmericanJournalofPoliticalScience50(3):551-569.Fox,Jean-Paul.2010.BayesianItemResponseModeling:TheoryandApplications.New
York:Springer.Gallie,W.B.1956.“EssentiallyContestedConcepts.”ProceedingsoftheAristotelian
Society56:167–220.Gerring,John,PhilipBond,WilliamBarndt,andCarolaMoreno.2005.“Democracyand
Growth:AHistoricalPerspective.”WorldPolitics57(3):323–364.Goertz,Gary.2006.SocialScienceConcepts:AUser’sGuide.Princeton:Princeton
UniversityPress.Hadenius,AxelandJanTeorell.2005.“CulturalandEconomicPrerequisitesofDemocracy:
ReassessingRecentEvidence.”StudiesinComparativeInternationalDevelopment39(4):87–106.
Held,David.2006.ModelsofDemocracy,3ded.Cambridge:PolityPress.Hopkins,Daniel,andGaryKing.2010.“ImprovingAnchoringVignettes:DesigningSurveys
toCorrectInterpersonalIncomparability.”PublicOpinionQuarterly:1-22.Inglehart,RonaldandWelzel,Christian.2005.Modernization,CulturalChangeand
Democracy:TheHumanDevelopmentSequence.Cambridge:CambridgeUniversityPress.
Isaac,JeffreyC.n.d.“ThinkingAbouttheQualityofDemocracyanditsPromotion.”Unpublishedms.
Jackman,Simon.2004.“WhatDoWeLearnfromGraduateAdmissionsCommittees?AMultipleRater,LatentVariableModel,withIncompleteDiscreteandContinuousIndicators.”PoliticalAnalysis12(4):400–424.
Johnson,ValenE.,JamesH.Albert.1999.OrdinalDataModeling.NewYork:Springer.Junker,Brian1999.SomeStatisticalModelsandComputationalMethodsthatmaybe
UsefulforCognitively-RelevantAssessment.http://www.stat.cmu.edu/~brian/nrc/cfa/documents/final.pdf
King,Gary,ChristopherMurray,JoshuaA.Salomon,andAjayTandon.2004.“EnhancingtheValidityandCross-culturalComparabilityofMeasurementinSurveyResearch.”AmericanPoliticalScienceReview98(1):191–207.
King,Gary,andJonathanWand.2007.ComparingIncomparableSurveyResponses:NewToolsforAnchoringVignettes.PoliticalAnalysis15:46-66.
Knutsen,CarlHenrik.2010.“MeasuringEffectiveDemocracy.”InternationalPoliticalScienceReview31(2):109-128.
45
Knutsen,CarlHenrik,JørgenMøller,andSvend-ErikSkaaning.Forthcoming.GoingHistorical:MeasuringDemocraticnessbeforetheAgeofMassSuffrage.InternationalPoliticalScienceReview.
Lindberg,StaffanI.2015.OrdinalVersionsofV-Dem’sIndices:ForClassification,Description,SequencingAnalysisandOtherPurposes.UniversityofGothenburg,VarietiesofDemocracyInstitute:V-DemWorkingPapersSeriesNo.19.
Lord,FredericM.,andMelvinNovick.1968.StatisticalTheoriesofMentalTestScores.Reading,MA:Addison-Wesley.
Mahoney,James,andDietrichRueschemeyer,eds.2003.ComparativeHistoricalAnalysisintheSocialSciences.Cambridge:CambridgeUniversityPress.
Mariano,LouisT.andBrianW.Junker.2007.“CovariatesoftheRatingProcessinHierarchicalModelsforMultipleRatingsofTestItems.”JournaloftheEducationalandBehavioralStatistics32(2):287-314.
McMann,Kelly.2016.“MeasuringSubnationalDemocracy.”VarietiesofDemocracyInstituteWorkingPaper26(March).
McMann,Kelly,DanielPemstein,BrigitteSeim,JanTeorell,andStaffanI.Lindberg.2016.“StrategiesofValidation:AssessingtheVarietiesofDemocracyCorruptionData.”VarietiesofDemocracyInstituteWorkingPaper23(February).
Munck,GerardoL.2009.MeasuringDemocracy:ABridgebetweenScholarshipandPolitics.Baltimore:JohnHopkinsUniversityPress.
Munck,GerardoL.2016.“WhatisDemocracy?AReconceptualizationoftheQualityofDemocracy.”Democratization23(1):1-26.
Nunn,Nathan.2009.“TheImportanceofHistoryforEconomicDevelopment.”AnnualReviewofEconomics1(1):1–28.
Patz,RichardJ.,andBrianW.Junker.1999.“AStraightforwardApproachtoMarkovChainMonteCarloMethodsforItemResponseModels.”JournalofEducationalandBehavioralStatistics24:146-178.
Patz,RichardJ.,BrianW.Junker,MatthewS.Johnson,andLouisT.Mariano.2002.“TheHierarchicalRaterModelforRatedTestItemsanditsApplicationtoLarge-ScaleEducationalAssessmentData.”JournalofEducationalandBehavioralStatistics27(4):341-384.
Pemstein,Daniel,KyleL.Marquardt,EitanTzelgov,Yi-tingWang,JoshuaKrusellandFarhadMiri.2017.“TheV-DemMeasurementModel:LatentVariableAnalysisforCross-NationalandCross-TemporalExpert-CodedData”.UniversityofGothenburg,VarietiesofDemocracyInstitute:WorkingPaperNo.21,2ndedition.
Pemstein,Daniel,StephenMeserve,andJamesMelton.2010.“DemocraticCompromise:ALatentVariableAnalysisofTenMeasuresofRegimeType.”PoliticalAnalysis18(4):426–449.
Pierson,Paul.2004.PoliticsinTime:History,Institutions,andSocialAnalysis.Princeton:PrincetonUniversityPress.
Rose-Ackerman,Susan.1999.CorruptionandGovernment:Causes,Consequences,andReform.Cambridge:CambridgeUniversityPress.
Sartori,Giovanni.1970.“ConceptMisformationinComparativePolitics.”AmericanPoliticalScienceReview64(4):1033-1053.
46
Schedler,Andreas.2012.“JudgmentandMeasurementinPoliticalScience.”PerspectivesonPolitics10:1,21-36.
Shapiro,Ian.2003.TheStateofDemocraticTheory.Princeton:PrincetonUniversityPress.Sigman,RachelandStaffanI.Lindberg.2015.TheIndexofEgalitarianDemocracyandIts
Components:V-Dem'sConceptualizationandMeasurement.UniversityofGothenburg,VarietiesofDemocracyInstitute:V-DemWorkingPapersSeriesNo.21
Steinmo,Sven,KathleenThelen,andFrankLongstreth,eds.1992.StructuringPolitics:HistoricalInstitutionalisminComparativeAnalysis.Cambridge:CambridgeUniversityPress.
Teorell,Jan.2011.“OverTime,AcrossSpace:ReflectionsontheProductionandUsageofDemocracyandGovernanceData.”ComparativeDemocratization9:1(February)1,7.
Teorell,Jan,MichaelCoppedge,JohnGerring&StaffanLindberg.n.d.2016"MeasuringElectoralDemocracywithV-DemData:IntroducingaNewPolyarchyIndex.”UniversityofGothenburg,VarietiesofDemocracyInstitute:V-DemWorkingPapersSeriesNo.23
Teorell,Jan,RachelSigman,andStaffanI.Lindbergn.d.2016.V-DemIndices:RationaleandAggregations.UniversityofGothenburg,VarietiesofDemocracyInstitute:V-DemWorkingPapersSeriesNo.22
Teorell,JanandStaffanI.Lindberg.2015.TheStructureoftheExecutiveinAuthoritarianandDemocraticRegimes:RegimeDimensionsacrosstheGlobe,1900-2014.UniversityofGothenburg,VarietiesofDemocracyInstitute:V-DemWorkingPapersSeriesNo.5
Thomas,MelissaA.2010.“WhatDotheWorldwideGovernanceIndicatorsMeasure?”EuropeanJournalofDevelopmentResearch22(1):31–54.
Treier,Shawn,andSimonJackman.2008.“DemocracyasaLatentVariable.”AmericanJournalofPoliticalScience52(1):201–217.
Wang,Yi-ting,PatrikLindenfors,AkselSundström,FredrikJansson,andStaffanI.Lindberg.2015.NoDemocraticTransitionWithoutWomen’sRights:AGlobalSequenceAnalysis1900-2012.VarietiesofDemocracyInstitute:V-DemWorkingPapersSeriesNo.12.
Wang,Yi-ting,AkselSundström,PamelaPaxton,andStaffanI.Lindberg.2015.”Women’sPoliticalEmpowermentIndex:ANewMeasure.”UniversityofGothenburg,VarietiesofDemocracyInstitute:V-DemWorkingPapersSeriesNo.18.
Welzel,Christian.2007.“AreLevelsofDemocracyAffectedbyMassAttitudes?TestingAttainmentandSustainmentEffectsonDemocracy.”InternationalPoliticalScienceReview28(4):397–424.
47
APPENDIXA:V-DemIndices,Components,andIndicatorsDemocracyIndicesNames
Mid-LevelDemocracyandGovernanceIndicesNames
Lower-LevelDemocracyandGovernanceIndicesNames
NamesIndicators Tag Uniquenessscore*
Electoraldemocracyindex
v2x_polyarchy
Additivepolyarchyindex
v2x_api
Multiplicativepolyarchyindex
v2x_mpi
Expandedfreedomofexpressionindex
v2x_freexp_thick
Governmentcensorshipeffort-Media
v2mecenefm0,287399531
Harassmentofjournalists v2meharjrn 0,310029642 Mediaself-censorship v2meslfcen 0,299229721 Mediabias v2mebias 0,252931835 Print/broadcastmediacritical v2mecrit 0,235389702 Print/broadcastmediaperspectives v2merange 0,264891589 Freedomofdiscussionformen v2cldiscm 0,234735887 Freedomofdiscussionforwomen v2cldiscw 0,175262473 Freedomofacademicandcultural
expressionv2clacfree
0,328359319 Freedomof
associationindex(thick)
v2x_frassoc_thick
Partyban v2psparban 0,314515483 Barrierstoparties v2psbars 0,252064401 Oppositionpartiesautonomy v2psoppaut 0,257866799 Electionsmultiparty v2elmulpar 0,368900132 CSOentryandexit v2cseeorgs 0,34364043 CSOrepression v2csreprss 0,391819395 Shareof
populationwithsuffrage
v2x_suffr
Percentofpopulationwithsuffrage v2elsuffrage
Cleanelectionsindex
v2xel_frefair
EMBautonomy v2elembaut 0,414401744 EMBcapacity v2elembcap 0,476390647 Electionvoterregistry v2elrgstry 0,479468817 Electionvotebuying v2elvotbuy 0,619782147 Electionothervotingirregularities v2elirreg 0,400394507 Electiongovernmentintimidation v2elintim 0,464657467 Electionotherelectoralviolence v2elpeace 0,67570601
48
Electionfreeandfair v2elfrfair 0,394150747 Electedofficials
index v2x_elecoff
Legislaturebicameral v2lgbicam
Lowerchamberelected v2lgello
Upperchamberelected v2lgelecup
Legislaturedominantchamber v2lgdomchm
HOSselectionbylegislatureinpractice
v2exaphos
HOSappointmentinpractice v2expathhs
HOGselectionbylegislatureinpractice
v2exaphogp
HOGappointmentinpractice v2expathhg
HOSappointscabinetinpractice v2exdfcbhs
HOGappointscabinetinpractice v2exdjcbhg
HOSdismissesministersinpractice v2exdfdmhs
HOGdismissesministersinpractice v2exdfdshg
HOS=HOG? v2exhoshog
Chiefexecutiveappointmentbyupperchamber
v2exapup
Chiefexecutiveappointmentbyupperchamberexplicitapproval
v2exapupap
Liberaldemocracyindex
v2x_libdem
Electoraldemocracyindex
v2x_polyarchy
Liberalcomponentindex
v2x_liberal
Equalitybeforethelawandindividuallibertyindex
v2xcl_rol
Rigorousandimpartialpublicadministration
v2clrspct0,510897097
Transparentlawswithpredictableenforcement
v2cltrnslw0,372337936
Accesstojusticeformen v2clacjstm 0,106424425 Accesstojusticeforwomen v2clacjstw 0,307894313 Propertyrightsformen v2clprptym 0,165920394 Propertyrightsforwomen v2clprptyw 0,385230089 Freedomfromtorture v2cltort 0,380794137 Freedomfrompoliticalkillings v2clkill 0,390809329 Freedomfromforcedlaborformen v2clslavem 0,090370553 Freedomfromforcedlaborfor
womenv2clslavef
0,370180991 Freedomofreligion v2clrelig 0,591125139 Freedomofforeignmovement v2clfmove 0,437467753
49
Freedomofdomesticmovementformen
v2cldmovem0,379460353
Freedomofdomesticmovementforwomen
v2cldmovew0,276608073
Judicialconstraintsontheexecutiveindex
v2x_jucon
Executiverespectsconstitution v2exrescon 0,556533058 Compliancewithjudiciary v2jucomp 0,334738738 Compliancewithhighcourt v2juhccomp 0,354794616 Highcourtindependence v2juhcind 0,455176953 Lowercourtindependence v2juncind 0,434174586 Legislative
constraintsontheexecutiveindex
v2xlg_legcon
Legislaturequestionsofficialsinpractice
v2lgqstexp0,496293669
Executiveoversight v2lgotovst 0,386240561 Legislatureinvestigatesinpractice v2lginvstp 0,25393755 Legislatureoppositionparties v2lgoppart 0,402224961
Deliberativedemocracyindex
v2x_delibdem
Electoraldemocracyindex
v2x_polyarchy
Deliberativecomponentindex
v2xdl_delib
Reasonedjustification v2dlreason 0,013722755 Commongood v2dlcommon 0,018259475 Respectcounterarguments v2dlcountr 0,013655074 Rangeofconsultation v2dlconslt 0,012409352 Engagedsociety v2dlengage 0,012246084
EgalitariandemocracyIndex
v2x_egaldem
Electoraldemocracyindex
v2x_polyarchy
Egalitariancomponentindex
v2x_egal
Equalprotectionindex
v2xeg_eqprotec Socialclassequalityinrespectfor
civillibertiesv2clacjust
0,251630803 Socialgroupequalityinrespectfor
civilliberties v2clsocgrp 0,673137622 Weakercivillibertiespopulation v2clsnlpct 0,819008129
50
Equalaccessindex
v2eg_eqaccess
Powerdistributedbygender v2pepwrgen 0,491661585 Powerdistributedby
socioeconomicpositionv2pepwrses
0,491661585 Powerdistributedbysocialgroup v2pepwrsoc 0,4445505 Equal
distributionofresourcesindex
v2xeg_eqdr Means-testedvs.universalistic v2dlunivl 0,636117818
Encompassingness v2dlencmps 0,693676915 Educationalequality v2peedueq 0,243294135 Healthequality v2pehealth 0,171167797
Participatorydemocracyindex
v2x_partipdem
Electoraldemocracyindex
v2x_polyarchy
Participatorycomponentindex
v2x_partip
Civilsocietyparticipationindex
v2x_cspart
Candidateselection--National/local v2pscnslnl 0,758570546 CSOconsultation v2cscnsult 0,400943859 CSOparticipatoryenvironment v2csprtcpt 0,415201758 CSOwomensparticipation v2csgender 0,673605234 Directpopular
voteindex v2xdd_dd
Initiativespermitted v2ddlexci
Initiativessignatures% v2ddsigpci
Initiativessignature-gatheringtimelimit v2ddsiglci
Initiativessignature-gatheringperiod v2ddsigdci
Initiativeslevel v2ddlevci
Initiativesparticipationthreshold v2ddpartci
Initiativesapprovalthreshold v2ddapprci
Initiativesadministrativethreshold v2ddadmci
Initiativessupermajority v2ddspmci
Occurrenceofcitizen-initiativethisyear v2ddyrci
Referendumspermitted v2ddlexrf
Referendumssignatures% v2ddsigprf
Referendumssignature-gatheringperiod v2ddsigdrf
Referendumsparticipationthreshold v2ddpartrf
Referendumsapprovalthreshold v2ddapprrf
51
Referendumssupermajority v2ddspmrf
Referendumsadministrativethreshold v2ddadmrf
Occurrenceofreferendumthisyear v2ddyrrf
Plebiscitepermitted v2ddlexpl
Plebisciteparticipationthreshold v2ddpartpl
Plebisciteapprovalthreshold v2ddapprpl
Plebiscitesupermajority v2ddspmpl
Plebisciteadministrativethreshold v2ddadmpl
Occurrenceofplebiscitethisyear v2ddyrpl
Constitutionalchangespopularvote v2ddlexor
Obligatoryreferendumparticipationthreshold v2ddpartor
Obligatoryreferendumapprovalthreshold v2ddappor
Obligatoryreferendumsupermajority v2ddspmor
Obligatoryreferendumadministrativethreshold v2ddadmor
Occurrenceofobligatoryreferendumthisyear v2ddyror
Obligatoryreferendumcrediblethreat v2ddthreor
Popularreferencumcrediblethreat v2ddthrerf
Plebiscitecrediblethreat v2ddthrepl
Localgovernmentindex
v2xel_locelec
Localgovernmentelected v2ellocelc
Localofficesrelativepower v2ellocpwr
Localgovernmentexists v2ellocgov
Regionalgovernmentindex
v2xel_regelec
Regionalgovernmentelected v2elsrgel
Regionalofficesrelativepower v2elrgpwr
Regionalgovernmentexists v2elreggov
OtherDemocracyandGovernanceIndices
Names OtherLower-LevelDemocracyandGovernanceIndicesNames
NamesIndicators Tag
Womenpoliticalempowermentindex
v2x_gender
Womencivillibertiesindex
v2x_gencl
Freedomofdomesticmovementforwomen
v2cldmovew0,458950844
52
Freedomfromforcedlaborforwomen
v2clslavef0,49317791
Propertyrightsforwomen v2clprptyw 0,433149229 Accesstojusticeforwomen v2clacjstw 0,362024191 Womencivil
societyparticipationindex
v2x_gencs Freedomofdiscussionforwomen v2cldiscw 0,51935166
CSOwomensparticipation v2csgender 0,247076218 Percent(%)femalejournalists v2mefemjrn 0,512661832 Women
politicalparticipationindex
v2x_genpp Powerdistributedbygender v2pepwrgen
Lowerchamberfemalelegislators v2lgfemleg
Electoralregimeindex
v2x_elecreg
Legislativeorconstituentassemblyelection
v2xel_elecparl
v2eltype v2eltype_0
v2eltype v2eltype_1
v2eltype v2eltype_4
v2eltype v2eltype_5
Legislaturecloseddownoraborted
v2xlg_leginter
Legislaturebicameral v2lgbicam
Presidentialelection
v2xel_elecpres
v2eltype v2eltype_6
v2eltype v2eltype_7
Chiefexecutivenolongerelected
v2x_hosinter
HOS=HOG? v2exhoshog
HOGappointmentinpractice v2expathhg
HOSappointmentinpractice v2expathhs
Presidentialelectionaborted
v2x_hosabort
HOS=HOG? v2exhoshog
HOGappointmentinpractice v2expathhs
HOSappointmentinpractice v2expathhg
Presidentialelection v2xel_elecpres
53
Legislativeorconstituentassemblyelectionaborted
v2x_legabort
Legislaturebicameral v2lgbicam
Legislativeorconstituentassemblyelection
v2xel_elecparl
Executiveregimeindex
v2xex_elecreg
Presidentialelection
v2xel_elecpres
v2eltype v2eltype_6
v2eltype v2eltype_7
Chiefexecutivenolongerelected
v2x_hosinter
HOS=HOG? v2exhoshog
HOGappointmentinpractice v2expathhg
HOSappointmentinpractice v2expathhs
Presidentialelectionaborted
v2x_hosabort
HOS=HOG? v2exhoshog
HOGappointmentinpractice v2expathhs
HOSappointmentinpractice v2expathhg
Presidentialelection v2xel_elecpres
Legislativeelectoralregimeindex
v2xlg_elecreg
Legislativeorconstituentassemblyelection
v2xel_elecparl
v2eltype v2eltype_0
v2eltype v2eltype_1
v2eltype v2eltype_4
v2eltype v2eltype_5
Legislaturecloseddownoraborted
v2xlg_leginter
Legislaturebicameral v2lgbicam
Legislativeorconstituentassemblyelectionaborted
v2x_legabort
Legislaturebicameral v2lgbicam
Legislativeorconstituentassemblyelection
v2xel_elecparl
54
Corruptionindex v2x_corr
Legislaturecorruptactivities v2lgcrrpt
Judicialcorruptiondecision v2jucorrdc
Publicsectorcorruptionindex
v2x_pubcorr
Publicsectorcorruptexchanges v2excrptps 0,287505343 Publicsectortheft v2exthftps 0,186094553 Executive
corruptionindex
v2x_execorr
Executivebriberyandcorruptexchanges
v2exbribe0,339071586
Executiveembezzlementandtheft v2exembez 0,204393329
Electoralcomponentindex
v2x_EDcomp_thick
Freedomofassociationindex(thick)
v2x_frassoc_thick
Partyban v2psparban 0,314515483 Barrierstoparties v2psbars 0,252064401 Oppositionpartiesautonomy v2psoppaut 0,257866799 Electionsmultiparty v2elmulpar 0,368900132 CSOentryandexit v2cseeorgs 0,34364043 CSOrepression v2csreprss 0,391819395 Shareof
populationwithsuffrage
v2x_suffr
Percentofpopulationwithsuffrage v2elsuffrage
Cleanelectionsindex
v2xel_frefair
EMBautonomy v2elembaut 0,414401744 EMBcapacity v2elembcap 0,476390647 Electionvoterregistry v2elrgstry 0,479468817 Electionvotebuying v2elvotbuy 0,619782147 Electionothervotingirregularities v2elirreg 0,400394507 Electiongovernmentintimidation v2elintim 0,464657467 Electionotherelectoralviolence v2elpeace 0,67570601 Electionfreeandfair v2elfrfair 0,394150747 Electedofficials
index v2x_elecoff
Legislaturebicameral v2lgbicam
Lowerchamberelected v2lgello
Upperchamberelected v2lgelecup
Legislaturedominantchamber v2lgdomchm
55
HOSselectionbylegislatureinpractice
v2exaphos
HOSappointmentinpractice v2expathhs
HOGselectionbylegislatureinpractice
v2exaphogp
HOGappointmentinpractice v2expathhg
HOSappointscabinetinpractice v2exdfcbhs
HOGappointscabinetinpractice v2exdjcbhg
HOSdismissesministersinpractice v2exdfdmhs
HOGdismissesministersinpractice v2exdfdshg
HOS=HOG? v2exhoshog
Chiefexecutiveappointmentbyupperchamber
v2exapup
Chiefexecutiveappointmentbyupperchamberexplicitapproval
v2exapupap
Freedomofexpressionindex
v2x_freexp
Governmentcensorshipeffort-Media
v2mecenefm0,287568664
Harassmentofjournalists v2meharjrn 0,309101492 Mediaself-censorship v2meslfcen 0,324996159 Freedomofdiscussionformen v2cldiscm
Freedomofdiscussionforwomen v2cldiscw Freedomofacademicandcultural
expressionv2clacfree
0,299181418
Civillibertiesindex v2x_civlib
Privatecivillibertiesindex
v2x_clpriv
Freedomfromforcedlaborformen v2clslavem 0,090370553 Freedomfromforcedlaborfor
women v2clslavef 0,370180991 Propertyrightsformen v2clprptym 0,165920394 Propertyrightsforwomen v2clprptyw 0,385230089 Freedomofforeignmovement v2clfmove 0,381455761 Freedomofdomesticmovement
formen v2cldmovem 0,422904829 Freedomofdomesticmovement
forwomen v2cldmovew 0,422904829 Freedomofreligion v2clrelig 0,462254159 Religiousorganizationrepression v2csrlgrep 0,486998785 Physical
violenceindex v2x_clphy
Freedomfrompoliticalkillings v2clkill 0,239802867 Freedomfromtorture v2cltort 0,268822617
56
Politicalcivillibertiesindex
v2x_clpol
Governmentcensorshipeffort-Media v2mecenefm 0,32336033
Harassmentofjournalists v2meharjrn 0,336992175 Mediaself-censorship v2meslfcen 0,352861442 Freedomofdiscussionformen v2cldiscm 0,300186907 Freedomofdiscussionforwomen v2cldiscw 0,300186907 Freedomofacademicandcultural
expression v2clacfree 0,327849576 Partyban v2psparban 0,396406162 Barrierstoparties v2psbars 0,3333971 Oppositionpartiesautonomy v2psoppaut 0,341278346 CSOentryandexit v2cseeorgs 0,280345895 CSOrepression v2csreprss 0,291005396
Partyinstitutionalizationindex
v2xps_party
Partyorganizations v2psorgs 0,287962276 Partybranches v2psprbrch 0,286459634 Partylinkages v2psprlnks 0,838590265 Distinctpartyplatforms v2psplats 0,48433842 Legislativepartycohesion v2pscohesv 0,835063838
Corecivilsocietyindex
v2xcs_ccsi
CSOentryandexit v2cseeorgs 0,204224585 CSOrepression v2csreprss 0,327449254 CSOparticipatoryenvironment v2csprtcpt 0,478016631
Alternativesourcesofinformationindex
v2xme_altinf
Mediabias v2mebias 0,290619919 Print/broadcastmediacritical v2mecrit 0,265648013 Print/broadcastmediaperspectives v2merange 0,253035027 Dividedpartycontroloflegislatureindex
v2x_divparctrl
Nationalpartycontrol v2psnatpar Nationalpartycontrolordinal
version v2psnatpar_ord Divisionofpower
index v2x_feduni
57
Localgovernmentexists v2ellocgov
Regionalgovernmentexists v2elreggov
Localgovernmentelected v2ellocelc
Regionalgovernmentelected v2elsrgel
Localofficesrelativepower v2ellocpwr
Regionalofficesrelativepower v2elrgpwr
*Uniquenessisthevariancethatis‘unique’tothevariableandnotsharedwithothervariables.
**BecausetheaccountabilityindicesdonotusestandardBFAs,wedonotreportuniquenessscoreshere.
DetailsregardingmodelfitcanbefoundinthemethodologicalappendixofV-DemWorkingPaperNo.46