1

Measuring Party Competition from Legislative Speeches:

Analyses of Japanese Parliamentary Debates, 1953-2011

Airo Hino†, Luigi Curini∗, and Atsushi Osaki†

Paper prepared for the 2013 ECPR General Conference, Bordeaux

Work in progress – do not quote without Authors’ permission

Abstract

Among the existing measures of legislative behaviour, such as the use of roll call votes,

this paper attempts to add to the list by proposing a new measure of party competition

by using legislative speeches. Legislative speeches have become increasingly accessible

through digitalised databases in the recent years and we demonstrate how the use of

legislative speeches can contribute to our understanding of inter-party and intra-party

competition in the legislative arena. The use of legislative speeches to estimate positions

of political actors is indeed an important venue not only because of its recent

development in the scaling technique but also because of the limited availability of roll

call data in Japan. Our validations with external events and consequences seem to

suggest that the dimension extracted from Wordfish reflects a line of conflict between

government and opposition rather than an ideological left-right dimension.

† Waseda University. Email: airo@waseda.jp. ∗ University of Milan. Email: luigi.curini@unimi.it † Waseda University. Email: atsushi.osaki@gmail.com.

2

Introduction

In this paper, we propose a new measure of party competition which captures

party-dynamics based on the analysis of legislative speeches. Legislative speeches have

indeed become increasingly accessible through digitalised databases in the recent years

and the methods to analyse them, especially in an automated way, have increased

notably in the literature (Grimmer and Brandon 2013). The well-known method

advanced by Proksch and Slapin (2009), known as Wordfish, allows in particular to

extract the positions of actors along an underlying latent dimension. The substantial

content of such dimension, however, remains to be properly understood, contrary to

other methods, as we will discuss below. In this sense, by codifying speeches made by

prime ministers and party representatives in the parliamentary sessions of the Japanese

Diet from 1953 to 2011, we show how the recovered positions can be better understood

as reflecting the intensity of government and opposition confrontation and its

development over time, rather than a mere ideological scale. We then illustrate how

such government-opposition divide is conditioned by electoral cycles. Our analyses

show that parties oppose to each other more intensely prior to elections but to a lesser

extent in the middle of electoral cycles. The new measure also appears to account for

other dimensions of legislative politics such as the survival rate of Japanese

governments and the duration of time needed for governments to pass their proposed

bills. While this measure could also be applied to the studies of coalition politics and of

intra-party politics of factions, we hope to show in this paper some examples for an

illustration of the measure and illuminate a possible venue in the future research on

legislative politics. Below we will first review the literature that measures party

competition and discusses our strategies in coding parliamentary speeches.

3

Measuring party competition in political science

A variety of scaling methods are available to date in political science to measure

positions of political actors, yet most of them are confined to the positions of political

parties at the time of election. The rich list of these measures ranges from the coding of

party manifestos (Budge et al., 2001; Klingemann et al, 2006), expert surveys (Benoit

and Laver, 2006; Hooghe et al., 2010), and to voters’ evaluation in surveys (e.g.

Comparative Study of Electoral Systems series). Similarities and differences of each

approach can be summarised with a triangular form in Figure 1.

Figure 1. Estimating party positions

Party Manifestos

Election Data

Party Positions

Voters Evaluation Expert Surveys

The coding of party manifestos and the evaluation of party positions by voters in

surveys are similar in that political parties and voters both perform as actual actors in

the electoral processes. At the time of election, political parties propose their electoral

pledges and voters evaluate the stances of each party. Expert surveys do not fit in this

group as experts are not direct political actors in the electoral processes. Instead, expert

4

surveys and voters’ evaluation do share the feature of evaluation of parties’ positions as

both approaches evaluate party positions in a pre-determined survey. The coding of

party manifestos is different from this group of survey-based evaluation approaches

since it is based on the coding of actual political texts instead. Lastly, the coding of

manifestos and expert surveys have things in common with regard to the contexts of

data collection. As both data are collected in the uniform manner for every election, it is

relatively easier to compile a cross-national and time-series dataset. Recent efforts in

coordinating comparative and national election surveys could produce such

cross-sectional and time-series dataset but it is generally more costly and difficult to run

randomly sampled surveys for every election. In this regard, the coding of manifestos

and expert surveys could produce a dataset in a standardised form across different

countries over time with less difficulty than voters’ surveys.

All three approaches seem to be first limited with a range of cross-temporal dimension

and second confined to electoral settings. This is quite natural as all three approaches

are based on elections which take place in the 2-5 years cycles. But this does not mean

that positions of political actors do not change between the t-1 and t elections. We do

know from daily observations of particular parties that parties do change their positions

between cabinets and in relation to a particular legislation as well as in reaction to the

changes in society and the world affairs (Laver and Benoit 2003, Curini 2011).

Moreover, the same number of parties can be modified in the between-election periods

due to splits and mergers (Laver and Shepsle 1998). Similarly, the ways in which

coalition politics takes place in the legislature could also change in reaction to the above

conditions.

5

Secondly, the three approaches could also suffer from their limitations in applying to

other institutional settings such as a legislature and a government than electoral settings.

At the election time, parties mainly speak to voters. At the normal (i.e. non-election)

time, legislators mainly speak to themselves and to a lesser extent speak to voters. The

institutional incentives are clearly different when politicians are trying to garner votes at

an election and legislators are trying (not) to pass proposed bills or (not) to support the

government. Given that political parties shift their vote-seeking, policy-seeking, and

office-seeking incentives according to the institutional settings (Strom, 1990), one needs

to be careful in using the party positions derived only from the electoral settings. In the

normal time, parties’ behaviour could reasonably be principled by a government and

opposition divide. If this is the case, one would need to measure the degree of party

competition with a different scale that would reflect a divide in the parliamentary arena.

In this perspective, records available in a legislature are precious sources for extracting

a dimension that would capture the natural relationships among parties in the

parliamentary arena. The roll-call votes in a legislature have been analysed extensively

to estimate the stances of political parties, factions, and each individual legislator

(Clinton et al., 2004; Poole 2005; Hix et al. 2005; Curini and Zucchini 2010). There

have been continued discussions as to the nature of such extracted dimensions derived

from the roll-call scaling. At least when we focus on parliamentary democracies rather

than on presidential systems (on this point, see Laver 2006) what is measured from

roll-calls is indeed the structure of the “revealed behavioral space” (Hix and Jun 2009)

instead of the underlying ideological dimension. As a consequence, normally by

6

applying roll-call analysis to parliamentary democracies what a researcher gets is the

positions of MPs along a government-opposition divide (Curini and Zucchini 2012).

The measures produced by a roll-call analysis, however, exactly as the previous

methods, tend to remain relatively static: usually what we get is the positions of MPs

over one entire legislature. Albeit of course a roll-call analysis can be applied to a

specific sub-period during a legislature (Poole 2005), the need to have enough

observations (i.e., a relatively large number of roll-calls to which applying the analysis)

indirectly limits such possibility.

Moreover, a roll-call analysis cannot be applied in all those contexts in which roll-call

vote data are not available or are available only in a limited range, as it happens in the

Japanese case.1 The roll-call data are of limited use in Japan since most of the proposed

bills pass in both Houses of the Japanese Diet without an actual voting of each legislator

(but with a majority of legislators standing in the floor). One must thus bear in mind that

even with the roll-call vote data, which is available in the case of the Japanese

legislature only in a limited range, an extracted dimension needs to be carefully

validated if it were to be used in empirical analyses. Yet, for the purpose of measuring

the degree of party competition in Japan, one would need to look for other sources.

For the above reasons, this paper explores a scaling of party competition in the Japanese

Diet from legislative speeches. Legislative speeches are digitalised in recent years and

1 Only when the President of each House requests a formal voting or one-fifth of the legislators request a formal voting, a formal procedure of voting takes place according to Article 151 and 152 of the Rules of the House of Representatives and Article 138 and 139 of the Rules of the House of Councillors.

7

all the Diet record in Japan can be publicly accessed through the Internet2. If we can

somehow extract a dimension of party competition from parliamentary debates, changes

and variations of party positions can be measured in a richer cross-temporal dimension

at least in a yearly fashion as parliamentary sessions can be observed every year.

Scaling party positions from legislative speeches would be classified as the same

approach with the coding of manifestos seen in Figure 1 as positions are derived from

observing messages of actual political actors. We will thus briefly review some scaling

techniques below applied to the coding of party manifestos for our purposes.

Scaling legislative speeches in the Japanese Diet

The content analysis of political texts has developed through the stream of manual

approaches of human-coding and the stream of automated approaches of

computer-coding. The examples of human-based coding in measuring party competition

are exemplified in the coding projects of party manifestos. The Comparative Manifestos

Project (CMP) (Budge et al., 2001; Klingemann et al., 2006) requires trained coders to

classify quasi-sentences of manifesto texts into pre-determined policy domains. Another

manifesto project as demonstrated in Party Change Project (PCP) (Harmel et al., 1995)

asks coders to judge policy positions from the entire manifestos. In a similar vein,

projects in Japan classify electoral pledges publicly distributed during the election

campaign into administrative domains (Shinada, 2006; Kobayashi, 1997).

2 The National Diet Library makes the Diet record available through the following URL

(http://kokkai.ndl.go.jp/).

8

The stream of computer-assisted approaches ranges from a supervised method of

identifying words for human-coded quasi-sentences in the CMP data (Pennings and

Keman, 2002), of calculating Wordscores from training texts (Laver et al., 2003), and to

an unsupervised method of Wordfish which locates words on one dimension based on a

Poisson item response theory (Proksch and Slapin, 2008). The number of application of

these computer-assisted approaches has accelerated in the recent years. Among these

applications, an important contribution in the context of the Japanese politics has been

made by Proksch, Slapin, and Thies (2011) which applied Wordfish to the Japanese

party policies in news article (taken as surrogates for party manifestos) and extracted an

ideological left-right dimension in the Japanese political spectrum in the post-war years.

Although there are advantages and disadvantages of both human-coded scaling and

computer-assisted scaling, this paper pursues the latter approach of an automated text

analysis. The obvious issue would be a trade-off between validity and reliability in

adopting the human-based coding and computer-based coding (Krippendorff, 2004;

Riffe et al., 2005). The computer-assisted scaling has the virtue of holding high

reliability as the analyses can be replicated and same conclusions can be retained as far

as the procedures and data sources are made public. The computer-based coding might

thus have an edge against the manual human-based coding which would have to

repeatedly run inter-coder reliability test (e.g. Hayes and Krippendorff, 2007). Yet, the

downside of the automated scaling is a potential lack of validity in the scaled content

and one needs to check if the measurement really taps what is supposed to

operationalize. In this regard, the human-based coding naturally has the advantage.

9

To scale a dimension of party competition, we apply an unsupervised computer-

assisted scaling. The reason for applying an unsupervised method derives from our

approach to keep all options open and not to predetermine the nature of party

competition in the parliamentary arena. As an unsupervised computer-assisted scaling,

we apply the Wordfish technique of Proksch and Slapin (2008) to extract a dimension

of party competition from the Japanese legislative speeches. Like the Proksch, Slapin,

and Thies (2011) which applied the Wordfish to the Japanese party pledges, we repeat

the similar exercise with the legislative speeches in the Japanese Diet. While Proksch,

Slapin, and Thies (2011) found that the extracted dimension indeed captures the

ideological left-right dimension, we are not necessarily confined to the hypothesis of

ideological party competition in the parliamentary arena. Since the political text in our

analyses stem from a different institutional setting, the Japanese legislature, as opposed

to the electoral settings in most of the previous studies of political texts, we expect that

an extracted dimension could reasonably be different from an ideological dimension.

While keeping our reading of the scale unfixed, we tentatively hypothesize that the

dimension of party competition extracted from an unsupervised computer-assisted

scaling is concerned of a conflict between government parties and opposition parties. As

the language spoken on the floor is primarily directed to other delegates, or cabinets and

opposition parties rather than to voters, one could naturally expect that the dimension of

conflict may well be different from the ideological one found with different political

texts. On top of vote-seeking incentives, political parties have office-seeking incentives

as well as policy-seeking incentives (Strom, 1990). If that is the case, the line of conflict

may not be exclusively conditioned by an ideological dimension driven by

10

policy-seeking incentives but also be determined by a power-oriented dimension driven

by office-seeking incentives. This could be more likely to be the case in parliamentary

democracies where exchanges among parties tend to be driven by power-seeking games

in an arena type parliament rather than policy-seeking coordination in a transformative

legislature (Polsby, 1975). This provisional hypothesis is also in line with the existing

literature on the Japanese parliament which has suggested that it has high viscosity

attributed to opposition parties’ strategies (Mochizuki, 1982). Opposition parties try to

influence legislative processes by employing non-policy tactics, i.e. sabotaging sessions

and negotiating the parliamentary schedules, not necessarily by policy perspectives.

Given that the computer-assisted scaling requires some validations of the scaled content,

this paper attempts to do so by examining how it appears to be associated with

contextual variables in the Japanese parliamentary arena. By correlating the scale

obtained from the legislative speeches with external variables such as electoral cycles,

the percentage of succeeded bills, and the survival rate of government, we will check for

the “predictive validity” of the scale (Grimmer and Stewart, 2013: 21). This is one of

the recommended strategies for validating an unsupervised scale. Of course, one could

also compare different scaling approaches for validation for “semantic validity” and

“convergent validity” as outlined by Grimmer and Stewart (2013). Although this would

certainly be another possibility, this paper pursues cross-examinations with important

contextual variables often used in empirical analyses of legislative and election studies

to test for the “predictive validity” and external validity of the scale.

Extracting a dimension by Wordfish

11

We have selected all the speeches in the House of Representatives following the Prime

Minister’s general policy speeches and the questions from each party representative.

Where there are more than two speeches from the same party in the same session, we

have selected the speaker that is of higher rank in the party. Overall 429 speeches over

79 sessions, and almost 19,000 words/kanjis are collected from 1953 (Diet 18) to 2011

(Diet 179) from the database of the National Diet Library.

Coding procedures take the following two steps. First, we split the speeches into words

through tokenizer engines such as ChaSen and/or MeCab. The difficulty of

computer-assisted text analysis in Japanese is that a Japanese sentence is only

distinguished by commas, and words are put in sequence without spaces in between. We

thus applied the two main tokenizer engines developed in the past twenty years (and the

results remain largely identical). This procedure produces a data matrix in which the

frequencies of the words that appeared in the speech of each parties are recorded (i.e. all

the words that appeared in the speech listed in the rows and the parties in the columns).

Second, after having transformed from words to numbers, we estimate the positions of

each speaker (i.e. party position) through the automated scaling model of Wordfish

developed to run with the statistical software R (Proksch and Slapin 2008). The

Wordfish analyses textual documents with Poisson item response theory, comparing the

frequencies of words contained in each text, under the assumption that, for each

document, the words’ relative frequencies are informative of the general position of that

text. Through this, the Wordfish arrays documents along a single latent dimension,

providing estimates of their positions on the scale. The model is expressed as follows:

12

, where:

yij is the count of word j in actor i‘s document (in our case legislative speeches)

α (alpha) is a set of actor (i.e. speaker) fixed effects

ψ (gamma) is a set of word fixed effects

β (beta) is an estimate of a word specific weight capturing the importance of word j in

discriminating between positions

ω (omega) is an estimate of actor i's position.

The main advantage of Wordfish is its ability to track variations in actors’ preferences

across time by estimating the position of actor i at time t, regardless of its position at

time t-1. Indeed, any shift in the position of actor i is only due to a change in the set of

words used by that actor that expresses an actual modification of its

position/preferences, not an artefact of the model. Owing to this feature, Wordfish

makes it possible to produce valid time-series estimates. This statement holds, however,

only if the word usage and the political language remain relatively constant over time

(Proksch and Slapin, 2009). Given the wide cross-temporal range of our analyses, we

want to make sure that our results are not biased by changes in the actual meaning of

some political words. For this reason we split our data in two separate time periods,

choosing 1989 as the dividing line, assuming that political language changed after the

collapse of communism and the rise of new technology and globalised economy (see

Ceron 2012 for a similar choice).

13

Figure 1 shows a scatterplot of each word in the two dimensions of beta (on the

horizontal axis) and gamma (on the vertical axis) for the speeches before 1989 and

Figure 2 is a scatterplot of each word for the speeches after 1989. Both figures show

that some words are located on the two extremes on the horizontal axis, scoring high

positive or negative beta coefficients. This suggests that these words are powerful

discriminators of party positions. At the same time, these words tend to have low

gamma coefficient on the vertical axis, meaning that they appear rarely in the speeches.

Those words used often in the speeches tend to have high gamma coefficients but hardly

distinguish party positions (i.e. low beta coefficients). In general, most words are

located in the triangle of the high gamma / low beta corner, the low gamma / high

positive beta corner, and the low gamma / high negative beta corner.

14

Figure 2. Word estimates by Wordfish for the speeches before 1989

Figure 3. Word estimates by Wordfish for the speeches after 1989

15

Our choice to split the data before and after 1989 appears to be reasonable. Both figures

display some typical words related to the underlying dimension of beta coefficients.

Figure 2 and Figure 3 show some selected words used in the respective period.

Noticeably, recent words appear in Figure 3 such as “genome”, “interactive”, and

“working poor”. Likewise, relatively old-fashioned words such as “poor peoples” and

“mystery” appear in the speeches before 1989. Analysing the speeches all together

would have simply produced a dimension that captures the newness and oldness of the

words spoken in the speeches across the decades.

The semantic validity of the words appears to be mixed. On one hand, ideological

left-right terminologies seem to match with the dimension (“tradition” with a highly

positive beta coefficient and “poor people” with a highly negative beta coefficient in

Figure 1). On the other hand, government-opposition terminologies seem to be relevant

(“submitting bills” and “passed bills” with highly positive beta coefficients and “double

swords” with a highly negative beta coefficient). Further analyses are required to

cross-examine the semantic validity of the scales. Yet, the purpose of this paper is to

validate the scales with the contexts of the Japanese politics and to associate with other

contextual variables. We will turn below for the tests of “predictive validity” with

external events and political consequences at different levels of political processes.

Validating the Wordfish scale with political events

We will examine the validity of the extracted scale from Wordfish in two steps in the

following two sections. First, we will examine the changes of party positions in the

16

time-series from the 1950s to the present and cross-validate the scale with the political

events. Second, we will examine the Wordfish scale with other contextual variables. The

contextual variables consist of three levels of political processes: (i) electoral, (ii)

legislative, and (iii) executive. The details of these levels will be discussed in the next

section.

Before presenting the time-series changes of party positions, Figure 4 shows the overall

average of party positions throughout the decades. It appears that the parties are located

on the left-right ideological dimension in line with the conventional wisdom. Liberal

Democratic Party (LDP), the long-time ruling party of Japan, is located further to the

right; Japan Socialist Party (JSP), the major opposition party until the early 1990s, is

located on the left. The fact that Japanese Communist Party (JCP) is located further to

the left and that Japan Democratic Party (JDP), the major opposition party since 1998

and the governmental party between 2009 and 2012 is located on the moderate left seem

to further validate this axis as the standard left-right ideological dimension.

17

Figure 4. Average party positions on the wordfish scale, 1953-2011

However, the above reading of the dimension no longer holds once the estimated

positions are traced in the time-series. Figure 5 illustrates the shift of party positions

through the decades. As found in Figure 4, the overall trend is that LDP is located on the

upper side of the dimension (i.e. positive omega coefficient) and JSP and JDP are

located on the lower side (i.e. negative omega coefficients). The picture looks

remarkably different, though, when we look at the periods in which LDP went into

opposition. The first time when LDP went into opposition was in 1993 when eight

parties/groups including JSP formed the non-LDP coalition. JSP’s position jumps up to

the positive considerably and LDP’s position goes down to the negative for the first

time in the post-war period. The second time was in 2009 when LDP once again was

forced to be in opposition with the landslide victory of DPJ (JDP or DP in the figure).

18

DPJ goes up to the positive from the negative and LDP goes down to the negative from

the positive, virtually swapping their positions.

Figure 5. Changes of party positions on the Wordfish scale, 1953-2011

The time-series shifts of party positions seem to suggest that the extracted dimension is

the divide between government and opposition. It is quite unlikely to suppose that LDP

changed its ideological position from right to left in 1993 and 2009. Government parties

generally appear to be positive, while opposition parties mark negative coefficients. It

would be more reasonable to understand that the Wordfish score depicts a conflict

between government and opposition in the parliamentary debates.

19

It is important to note here that this divide of government and opposition is not an

artefact of Wordfish, nor the product of a uniqueness Japanese language. Proksch et al

(2011) has indeed demonstrated the ideological dimension applying the same scaling

method with manifesto documents. The only difference is the texts used for analyses.

While their analyses were based on electoral pledges of political parties, our analyses

were based on the words spoken in the parliamentary arena3. These differences seem to

suggest that institutional constraints clearly matter for the content of the scale.

Validating the Wordfish scale with political consequences

Based on the above finding, we will further validate the extracted scale with the

political trend variables. As described above, the trend variables consist of three levels

of political processes: (i) electoral, (ii) legislative, and (iii) executive. First, the electoral

level is concerned with the electoral cycles. We would expect that a conflict between

government and opposition intensifies when elections approach. Second, the legislative

level is concerned with the success rate of proposed bills. When a conflict between

government parties and opposition parties soars, we would expect that the success rate

would be diminished. Lastly, the executive level is concerned with the cabinet survival

rate. We would expect that when the conflict between government parties and

opposition parties is harsh, chances for cabinets to be terminated would be higher.

To put each of the three levels of analyses into the form of hypotheses, they should be

stated as follows.

3 The critique by Laver (2002) on using roll-call analysis in the context of a parliamentary (rather than in a presidential) democracy also applies to our case.

20

Hypothesis 1 (electoral cycles hypothesis)

When elections are closer in the electoral cycles, conflicts between government and

opposition intensify.

Hypothesis 2 (proposed bills hypothesis)

When conflicts between government and opposition intensify, chances are smaller for

cabinets to pass proposed bills.

Hypothesis 3 (cabinet survival hypothesis)

When conflicts between government and opposition intensify, chances are higher for

cabinets to be terminated.

To test these hypotheses, we need to operationalize the degree of Intensity of

Government and Opposition (IGO) divide. We could estimate such a measure over each

session k by adopting the Dalton’s index (2004) of party system polarization as follows:

, where VSjk is the seat share of party j during session k, Pjk is the position of party j

during session k over the latent government opposition scale, and is the average

position of parties along the same scale during session k.

Figure 6 shows the changes of the degree of IGO over the decades. The solid line

illustrates the weighted polarization (i.e. Dalton’s) index with interpolation and the

21

dotted line illustrates the index without interpolation. Clearly the IGO index is not

stable and it shows a relevant volatility in the short-term and across decades.

Figure 6. Changes of Intensity of Government and Opposition (IGO), 1953-2011

Hypothesis 1: Electoral Cycles Hypothesis

The IGO index is put as the dependent variable and the following possible determinants

of IGO are included in the model as independent variables. For the time variables, point

of the electoral cycle (both at the Lower and at the Upper house) and six dummies for

each decade starting since 1955 (omitted category: the decade between 1965-75) are

included. For the cabinet variables, majority margin of a cabinet, change in cabinet

composition between t and t-1 (a dummy), and length of the cabinet formula (i.e.,

number of sessions over which a given cabinet composition repeats itself) are included.

22

Table 1 shows the results. All the decades following the period of 1965-74 presents a

significantly lower value of IGO (but the decade 1995-2004). When a new government

formula is established, the confrontational attitude in the Parliament appears to decrease.

And most importantly, the variables concerning electoral cycles are found significant in

both the Lower House electoral cycles and the Upper House electoral cycles.

Table 1. Determinants of the IGO index

23

Figure 7. Electoral Cycles of the Lower House and the IGO index, 1953-2011

Figure 8. Electoral Cycles of the Upper House and the IGO index, 1953-2011

In both situations, the salience of the government and opposition divide decreases until

the half of the respective electoral cycles and the intensity index starts to increase as the

subsequent election approaches. These results suggest that parties become

confrontational as they face elections ahead, for the sake of appealing to voters that they

are fighting against their opponents and making themselves stand out in election.

24

Hypothesis 2: Proposed Bills Hypothesis

We analyze here a measure of the legislative outcome, which is operationalized as the

ratio of cabinet-bills successfully converted in law (period: 1953-2001). We expect a

negative relationship between IGO and this ratio (the larger the intensity of the

confrontation between the cabinet and the opposition, the more difficult for the cabinet

to convert into laws its proposed bills). Control variables are the set of dummies for

decades, majority cabinet margin, the existence of a different majority between the

Lower and the Upper House, and the total number of cabinet-bills proposed during each

session.

Table 2. The IGO Index and the Rate of Successful Cabinet Bills

25

The results are shown in Table 2. The IGO Index indeed show a negative sign and

significant at 95% level. The majority different at the Upper House (so-called ‘twisted

Parliament’ in the context of Japanese politics) also has a negative effect as well as the

decade dummy of 1995-2001. Even after controlling for these variables together with

the number of cabinet submitted bills, the IGO Index appears to be significant. The fact

that the IGO Index is related to the actual legislative outcome further validates the scale.

Hypothesis 3: Cabinet Survival Hypothesis

We now apply the IGO Index to explain the outcome at executive level. As the

hypothesis suggests, the higher the intensity between government and opposition is, the

more difficult it should be for a government to survive. We run Cox Proportional

Hazards Model to estimate the effects of the IGO Index. In this case we expect a

positive impact of IGO on the hazard rate of the survival of Japanese Cabinets, i.e., as

IGO increases, the predicted chances of cabinet termination also increases (period:

1953-2011). The effect is estimated together with the control variables: majority cabinet

margin, the number of component of the cabinets, the existence of a different majority

in the Upper House, and the electoral cycles.

The results are presented in Table 3. There are two scenarios. First, whenever a cabinet

lasts more than one session, for all of our independent variables, we consider just the

first value that each of independent variables displays in the first session of the cabinet.

Second, whenever a cabinet lasts more than one session, for all of our independent

variables we consider their average values across the sessions along which the cabinet

survives. The results are intriguing as the independent variables indeed show an

26

expected sign with significance. The margin of cabinet and the number of components

in cabinet both show positive signs, suggesting that the cabinet increases chances of

being terminated as the margin of cabinet is bigger and the number of parties in

coalition government increases. The electoral cycle variables are as expected negative

for the Lower House and positive for the Upper House elections. The ‘majority different

in the Upper House’ variable also has a positive sign as expected. These results are all

significant and are robust for both scenarios. And most importantly, the IGO Index has a

significantly positive effect on the chances of government termination. The fact that the

IGO Index is significantly related to the duration of cabinets even after controlling for

other relevant variables validates the extracted scale also at executive level.

Table 3. The IGO Index and the duration of Japanese cabinets, 1953-2011

27

Further validations with the manifesto scaling

Given that the Wordfish score was significantly related at all three levels of political

process, the electoral, the legislative, and the executive, one may argue that the

“predictive validity” is high enough to suggest that the extracted dimension is indeed

the index of government and opposition divide. Yet, we are still not sure if the extracted

dimension is really different from the findings in the previous studies which validated

the ideological left-right profile of the Wordfish score based on electoral pledges

(Proksch, Slapin, and Thies, 2011). To further validate the IGO Index, our final

analyses try to test the ideological scale and the IGO Index simultaneously.

First, we have tested the correlation between the left-right index from the CMP dataset

and the IGO Index. The result was a low correlation (.33) between the two indices. This

already testifies to the fact that these two indices measure to a large extent two different

phenomena. Second, we have estimated the same models of the proposed bills

hypothesis and of the cabinet survival hypothesis together with the CMP index.

Table 4 and Table 5 report that the IGO Index works even after controlling for the CMP

index, while the CMP index is never significant in both analyses. This once again

confirms that the IGO Index and the CMP Index are relatively independent of each

other and that the ideological left-right index derived from manifestos documents does

not have “predictive validity” of the political consequences at legislative and executive

levels examined in this paper.

28

Table 4. The IGO Index and the Rate of Successful Cabinet Bills (with CMP Index)

Table 5. The IGO Index and the duration of Japanese cabinets, 1953-2011 (with

CMP Index)

Conclusion

This paper attempted to scale party positions from legislative speeches in Japan. This

attempt had advantages: first to enrich cross-temporal variations of party positions, and

second to extract a dimension relevant to the parliamentary setting. We have tried to

validate the dimension extracted from the unsupervised scaling by Wordfish in relation

29

to the political events and consequences in Japan. The empirical analyses at electoral,

legislative, and executive level all appeared to support the idea that the Wordfish score

reflects a divide between government parties and opposition parties. We measured the

extent to which parties oppose to each other by the Index of Intensity of Government

and Opposition (the Index of IGO) for testing the respective hypotheses of electoral

cycles, proposed bills, and cabinet survival. The Index of IGO appeared to be

significantly related to all three levels of political process and this effect was robust

even after controlling for the ideological index derived from manifesto documents.

To scale a dimension from legislative speeches, we have taken an open-ended approach

and applied an unsupervised method. We have limited ourselves in this paper to test for

the “predictive validity” of the extracted Wordfish scale, but one should also validate

the scale through “semantic validity” and “convergent validity” as suggested by

Grimmer and Stewart (2013). Future research could take this venue to examine if the

Wordfish scale obtained from the Japanese legislative speeches really depicts the

government and opposition conflict. As Grimmer and Stewart (2013: 27) cautions, in

speaking of the Wordfish model, that “one should not assume that wordfish output

measures an ideological location without careful validation”, one should certainly

continue to validate the scale from various methods including the manual human-based

coding coupled with reliability test (Krippendorff, 2004) as well as the supervised

computer-assisted method of scaling.

If we take the words of Grimmer and Stewart (2013: 27) seriously when they state

“non-ideological locations that wordfish identifies are quite useful”, we could also

30

extend the discussions of the index of IGO to the context of intra-party politics where

factions fight each other within the same party. Given that many politicians are recorded

in the parliamentary records, we might be able to extract positions of each faction in

LDP or DPJ and such exercises might open up a new research venue in the field of

intra-party politics. The analyses are also not confined to the main session of the

parliament but the words spoken in various committees could also be collected. If one

builds a database for the scaled points for various politicians in various committees, it

would also be of use in future legislative studies and in the research of political parties.

One could also discuss if the dimension extracted in the Japanese Diet is applicable to

other democracies. The fact that such a government and opposition divide intensifies

might be also institutionally conditioned as parliamentary democracies or arena-type of

parliament. Future research could also take a comparative perspective into account and

put the Japanese legislative speeches in a relative perspective. It could also develop into

some measurements of arena type and/or transformative legislatures (Polsby, 1975). As

noted by Krehbiel and Peskowitz (2012), preferences of political actors are the

animating forces of politics. The method employed to derive such preferences can be

affected by the institutional context in which such preferences are captured. When this

is the case, those preferences become "institutionally endogenous". It would thus be

important to investigate such a relevance of institution on the legislative speeches.

31

References

Benoit, K. and Laver, M. 2006. Party Policy in Modern Democracies. Routledge.

Benoit, K., Laver, M., Mikhaylov, S., 2009. Treating words as data with error:

uncertainty in text statements of policy positions. American Journal of Political

Science, 53 (2): 495–513.

Budge, I., Klingemann, H., Volkens, A., Bara, J., and Tanenbaum, E., 2001. Mapping

Policy Preferences. Oxford University Press.

Ceron, A. 2012. “Bounded oligarchy: How and when factions constrain leaders in party

position-taking”. Electoral Studies, 31(4): 689-701

Clinton, J, Jackman, S, and Rivers, D. 2004. “The statistical analysis of roll call data”.

American Political Science Review, 98(2): 355-370.

Curini, L. 2011. “Government Survival the Italian Way: the Core and the Advantages of

Policy Immobilism during the First Republic”, European Journal of Political

Research, 50, 2011, 110-142

Curini L. and Francesco Zucchini. 2010. “Testing the law-making theories in a

parliamentary democracy. A roll call analysis of the Italian Chamber of Deputies

(1988-2008)” in König, Thomas; Tsebelis, George; Debus, Marc (Eds.) Reform

Processes and Policy Change. Veto Players and Decision-Making in Modern

Democracies,(189-211). Series Studies in Public Choice. Springer

Curini Luigi, and Francesco Zucchini. 2012. “Government Alternation and Legislative

Party Unity: The Case of Italy, 1988–2008”. West European Politics, 35(4), 2012,

826-846.

Dalton, Russell J. 2008. ‘‘The Quantity and the Quality of Party Systems’’.

Comparative Political Studies, 20(10): 1–22.

32

Grimmer, Justin, and Stewart, Brandon M. 2013. “Text as Data: The Promise and

Pitfalls of Automatic Content Analysis Methods for Political Texts”. Political

Analysis, 21(3): 267-297.

Hayes, A. F., & Krippendorff, K. 2007. “Answering the call for a standard reliability

measure for coding data”. Communication Methods and Measures, 1: 77-89.

Hix, Simon, Abdul Noury. 2008. Government-Opposition or Left-Right? The

Institutional Determinants of Voting in Fourteen Parliaments. Working Paper.

Hix, Simon, Abdul Noury, Gerard Roland, 2005. “Power to the Parties: Cohesion and

Competition in the European Parliament, 1979–2001”. British Journal of Political

Science, 35, 209-234.

Hix, Simon, Hae-Won Jun. 2009. “Party Behaviour in the Parliamentary Arena”. Party

Politics, 15(6), 667-694.

Klingemann, H., Volkens, A., Bara, J., and Budge, I., 2006. Mapping Policy Preferences

2. Oxford University Press.

Kobayashi, Yoshiaki. 1997. Gendai Nihon no Seijikatei. University of Tokyo Press.

Krehbiel, Keith, and Peskowitz, Zachary. 2012. “Legislative Organization and

Ideal-Point Bias”. Research Paper No.2124, Stanford Graduate School of Business.

Krippendorff, Kraus. 2004. Content Analysis. The Second Edition. Sage Publications.

Laver, M. 2006. “Legislatures and parliaments”. Chapter 7 in Barry Weingast and

Donald Wittman (eds.). Oxford Handbook of Political Economy. Oxford: Oxford

University Press.

Laver, M., Benoit, K., Garry, J., 2003. “Extracting policy positions from political texts

using words as data”. American Political Science Review, 97 (2): 311–332.

33

Laver, M. & Benoit, K. 2003. “The Evolution of Party Systems Between Elections”.

American Journal of Political Science, 47(2): 215-233.

Laver, M., Garry, J., 2000. “Estimating policy positions from political texts”. American

Journal of Political Science, 44 (3): 619–634.

Laver, M. & Shepsle, K. (1998). “Events, Equilibria, and Government Survival”.

American Journal of Political Science 42(1): 28-54.

Liesbet Hooghe, Ryan Bakker, Anna Brigevich, Catherine de Vries, Erica Edwards,

Gary Marks, Jan Rovny, Marco Steenbergen, Milada Vachudova. 2010. "Reliability

and Validity of Measuring Party Positions: The Chapel Hill Expert Surveys of 2002

and 2006". European Journal of Political Research, 42 (4): 684-703.

Massimiliano Landi & Riccardo Pelizzo (2013). “A Spatial Analysis of the Italian

Second Republic”. The Journal of Legislative Studies.

DOI:10.1080/13572334.2013.812342

Mochizuki, Mike. 1982. “Managing and influencing the Japanese legislative process:

The role of parties and the National Diet”. Ph.D Dissertation, Harvard University.

Pennings, Paul, and Keman, Hans. 2002. “Towards a New Methodology of Estimating

Party: Policy Positions”. Quality and Quantity, 36:55-79.

Polsby, Nelson W. 1975. "Legislatures" in Handbook of Political Science. Greenstein,

Fred I., and Polsby, Nelson W. Reading, MA: Addison-Wesley, pp. 257-319.

Poole, K. (2005). Spatial Models of Parliamentary Voting. New York: Cambridge

University Press.

Proksch, Sven-Oliber, and Slapin, Jonathan B. 2008. “A Scaling Model for Estimating

Time-Series Party Positions from Texts”. American Journal of Political Science,

52(3): 705-722.

34

Proksch, Sven-Oliber, and Slapin, Jonathan B. 2009. “How to Avoid Pitfalls in

Statistical Analysis of Political Texts: The Case of Germany”. German Politics,

18(3): 323-344.

Proksch, Sven-Oliber, Slapin, Jonathan B., and Thies, Micahel F. 2011. “Party system

dynamics in post-war Japan: A quantitative content analysis of electoral pledges”.

Electoral Studies, 30(1): 114-124.

Riffe, Daniel, Lacy, Stephen, and Fico, Frederick. 2005. Analyzing Media Messages:

Using Quantitative Content Analysis in Research. The Second Edition. Routledge.

Robert Harmel, Alexander C. Tan, Kenneth Janda, and Jason Matthew Smith. 1995.

“Substance vs. Packaging: An Empirical Analysis of Parties’ Issue Profiles”, Paper

delivered at the 1995 Annual Meeting of the American Political Science Association.

Shinada, Yutaka. 2006. “Senkyo Kouyaku Seisaku Data Nitsuite”. Nihon Seiji Kenkyu,

3(2).

Strom, Kaare. 1990. “A behavioral theory of competitive political parties”. American

Journal of Political Science, 34(2): 565-598.