Words as Data? Estimating the Fiscal Conservativeness of Provincial Premiers Using the Wordscore Procedure of Content Analysis

What is the usefulness of the new computer wordscoring
method developed by Laver, Benoit and Garry (2003)
for research in political science, and particularly for estimating
the policy position of key political actors? This is the central
question I address in this paper. Using the computer
wordscoring methodology I analyze the speeches given in the
legislative assemblies of Quebec and Ontario by provincial
premiers to assess their budgetary policy position. I present the
empirical results of the wordscoring content analysis and discuss
the validity and the effectiveness of this new technique.
Manual content analysis, but also traditional computer content
analysis (either using a dictionary approach or another one) has
involved large amounts of highly skilled labor required for
developing and testing the coding dictionaries that are central
to these approaches. In addition, important human involvement
brings the risk of potentially biased human coders. These two
shortcomings can be avoided by using a new probabilistic
word-scoring method developed recently by Laver, Benoit and
Garry (2002 & 2003) which extracts policy positions from
political texts by treating them as data in the form of words.
The technique using the software Wordscore estimates policy
positions by comparing a set of texts whose positions are known
or can be either estimated with confidence from independent
sources or assumed uncontroversially as representative for a
policy position (reference texts) and a set of texts whose policy
position one must search to uncover (virgin texts).
Using the computer wordscoring methodology I analyze the
policy position of provincial premiers in the budgeting process
in Quebec and in Ontario in the years 1968-2004 using the
'guardian-spender' budgetary framework (Wildavsky 1964 &
1984), which supposes the existence of two main roles attached
to the institutional positions held by the participants in the
budgetary negotiations: they are either 'guardians' of the treasury (participants from central agencies controlling the budget:
Minister of Finance, President of the Treasury board) or
advocates of program spending coming from program ministries
(minister of Health, minister of Education, etc).
Since we can not predict the fiscal behaviour of the government
on the base of a simple partisan left-right dichotomy (Imbeau),
the traditional models of budgetary theories employing
politicians' [here called 'actors'] preferences do not give us any
appropriate measure of the budgetary position of key political
'actors'. Thus, a way to uncover policy position of a premier is
to assess his or her preferences on an additional dimension,
differentiating between budgetary actors with a total vision of
budget (guardians) and those with a partial vision of it
(spenders). The vision is total if the budget balance is more
important than a party's taxing or spending preferences. On the
other hand, the vision of the budget is partial if the party
program (more or less tax and spending) is more important than
the budget balance.
Which texts are representative of the policy position of
budgetary 'actors' we are interested in? We certainly can assume
that the Speech from the Throne delivered by the
Lieutenant-Governor at the opening of each legislative session
expresses the Premier's policy preferences. Its delivery
constitutes one of the central moments in a session, thus it is
carefully crafted by the Premier's Office and its content is
reviewed many times before delivery. The Throne speeches
are compared to two reference texts: the budget speech
representing the guardians' expression of a total view of the
budget, and the preliminary remarks by the Ministers of Health
and of Education at budget hearings representing the spenders'
view of a partial vision of the budget. Although all three texts
are usually drafted so as to represent the position of the
government, they should be differentiated because of the policy
role that each assumes.
Based on these assumptions and using the wordscoring
technique of content analysis, I develop a fiscal conservatism
index of budgetary 'actors'. I then assess the internal validity
of my results using a variety of validity tests.
Bibliography
Imbeau, L.M. "Deficits and Surpluses in Federated States: A
Review of the Public Choice Empirical Literature." Paper
presented at the annual conference of the Canadian Political
Science Association, Winnipeg. 2004.
Laver, M., K. Benoit, and J. Garry. Placing Political Parties
in Policy Spaces. Dublin: Trinity College, 2002.
Laver, M., K. Benoit, and J. Garry. "Extracting Policy Positions
from Political Texts Using Words as Data." American Political
Science Review 97 (2003): 311-331.
Wildavsky, A.B. The Politics of the Budgetary Process.
Boston: Little, Brown and Company, 1964.
Wildavsky, A.B. The Politics of the Budgetary Process (4th
Edition). Boston: Little, Brown and Company, 1984.