This article was downloaded by: [University of Iowa Libraries]On: 13 November 2014, At: 01:05Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Educational Research and Evaluation:An International Journal on Theory andPracticePublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/nere20

The more, the better? Intensity ofinvolvement in private tuition andexamination performanceEmer Smyth aa Economic and Social Research Institute , Dublin, IrelandPublished online: 02 Oct 2008.

To cite this article: Emer Smyth (2008) The more, the better? Intensity of involvement in privatetuition and examination performance, Educational Research and Evaluation: An InternationalJournal on Theory and Practice, 14:5, 465-476, DOI: 10.1080/13803610802246395

To link to this article: http://dx.doi.org/10.1080/13803610802246395

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

The more, the better? Intensity of involvement in private tuition and

examination performance

Emer Smyth*

Economic and Social Research Institute, Dublin, Ireland

(Received 29 January 2008; final version received 29 April 2008)

Private tuition, or ‘‘shadow education’’, has become a prominent feature in manycountries internationally. A number of studies have found that such tuition is associatedwith improved examination performance and enhanced access to tertiary education.However, these studies have generally not taken into account important attitudinaldifferences between tuition participants and nonparticipants or explored variation in theintensity of young people’s involvement in paid tuition. This paper uses a large-scalesurvey of upper secondary students in the Republic of Ireland to explore the impact ofdifferent levels of involvement in private tuition on student performance. Multilevelregression and propensity score matching techniques indicate no significant perfor-mance advantage accruing to involvement in private tuition when adequate account istaken of key differences between participants and nonparticipants. All else being equal,even relatively high levels of involvement in private tuition do not enhance academicperformance.

Keywords: private tuition; academic performance

Introduction

In recent years, a number of countries have experienced a growth in ‘‘shadow education’’,that is, participation in paid private tuition as a supplement to regular schooling.Participation is found to be more prevalent around the key transition points in theeducational system, especially the transition to university education (Bray, 2005). Suchparticipation has been found to incur significant additional costs for students and theirparents in otherwise ‘‘free’’ educational systems (Bray & Kwok, 2003; Kang, 2006;Psacharopoulos & Papakonstantinou, 2005; Tansel & Bircan, 2005). The use of privatetuition has been found to be highly prevalent in a number of Asian countries, as well as inGreece and Turkey (Baker, Akiba, LeTendre, & Wiseman, 2001). However, there isevidence that private tuition has also become a growing feature in North America and anumber of Eastern European countries (Davies, 2004; Organisation for Economic Co-operation and Development [OECD], 2006a; Silova & Bray, 2005).

Research on private tuition has indicated that, as might be expected, participation ishigher among students from higher income families and/or those from professional

*Email: emer.smyth@esri.ie

Educational Research and Evaluation

Vol. 14, No. 5, October 2008, 465–476

ISSN 1380-3611 print/ISSN 1744-4187 online

� 2008 Taylor & Francis

DOI: 10.1080/13803610802246395

http://www.informaworld.com

Dow

nloa

ded

by [

Uni

vers

ity o

f Io

wa

Lib

rari

es]

at 0

1:05

13

Nov

embe

r 20

14

backgrounds with highly educated parents (see Psacharopoulos & Papakonstantinou,2005, on Greece; Tansel & Bircan, 2005, 2006, on Turkey; Stevenson & Baker, 1992, onJapan; Bray & Kwok, 2003, on Hong Kong). A number of studies have indicated thatparticipation in private tuition enhances academic performance. Many of these studieshave focused on Asian countries because of the historically high usage of private tuition.Stevenson and Baker (1992) found that some types of ‘‘shadow education’’ (includingpractice exams and correspondence courses) enhance the chances of entering Japaneseuniversities but that other types (such as private tutors) yielded no such advantage.These estimates control for gender, parental education, family income, and priorperformance. Kang (2006) indicates a significant boost in grades resulting fromexpenditure on private tuition among South Korean students, controlling for teacher-assessed initial ability, hours of self-study, and family characteristics. However, the effectis very modest in size, with 10% additional expenditure on tuition resulting in a 1.4%higher test score.

Fewer such studies have been carried out to date in European countries; however,research in the UK, Germany, and Turkey indicates some (but variable) benefitsassociated with private tuition. In Turkey, Tansel and Bircan (2005) found that those whohad received private tuition achieved higher grades in the competitive university entranceexamination. This difference held controlling for gender, family background character-istics, and having achieved honours at school. In the German context, Mischo and Haag(2002) found that tutoring was associated with improved performance across all subjects(Maths, English, Latin, and French), as well as improved self-confidence and reduced testanxiety. The study compared pairs of participants and nonparticipants matched in termsof prior school marks, but no other controls were used. Furthermore, the intensity ofinvolvement was quite high, with students receiving 90 min of tuition for 4 days a week,and so may not be generalisable to non-experimental situations where involvement levelsare lower. In the UK, Ireson and Rushforth (2005) found some effect of private tuition onlower secondary (GCSE) exam results, but this effect was variable across different groupsof students. Private tuition had a positive effect on mathematics grades among boys butnot girls. Furthermore, there was no significant effect of private tuition on English grades.This study controlled for gender, ethnic group, and first language but not for prioracademic performance.

Two key limitations are apparent in the existing research on the relationship betweenprivate tuition and academic performance. Firstly, studies do not always take into accountvariability in the nature of private tuition. Private tuition may be provided in a ‘‘cram’’school or by individual tutors, and students may differ in the number of subjects withwhich they receive help and in the amount of time they spend on private tuition (Ireson2004). Baker et al. (2001) explore the way in which the effect of private tuition varies acrossdifferent modes (e.g., taking correspondence courses as opposed to attending privatetuition schools). However, to date, no studies have explored the extent to which anyimpact of private tuition on educational performance reflects the intensity of involvementin such classes. Secondly, while a number of studies control for parental background inassessing the effect of private tuition on academic outcomes, studies rarely take account ofkey behavioural and attitudinal differences between those taking tuition and otherstudents.

In this paper, a large-scale study of upper secondary students is used to assess theimpact of the intensity of involvement in private tuition on academic performancein the Republic of Ireland, controlling for a number of important differences betweenparticipants and nonparticipants. The following section highlights some key

466 E. Smyth

Dow

nloa

ded

by [

Uni

vers

ity o

f Io

wa

Lib

rari

es]

at 0

1:05

13

Nov

embe

r 20

14

features of the Irish educational system, which are pertinent in a discussion of privatetuition.

Education in the Republic of Ireland

There is active school choice on the part of parents and students at secondary level in theRepublic of Ireland, with half of the student cohort not attending their nearest or mostaccessible school (Hannan, Smyth, McCullagh, O’Leary, & McMahon, 1996; Smyth,McCoy, & Darmody, 2004). There are three school sectors catering for young peoplebetween 12 and 18 years of age: voluntary secondary schools, vocational schools, andcommunity/comprehensive schools. These school sectors differ formally in their fundingand management structures and informally in the socioeconomic profile of their studentintake (Hannan et al., 1996). However, all school sectors operate within a nationallystandardised curriculum and assessment framework. The vast majority of schools acrossall sectors are non-fee paying, with fee-paying schools in the voluntary secondary sectormaking up 7% of all schools.

In the Irish educational system, students take two nationally standardised examina-tions: the Junior Certificate at the end of lower secondary education and the LeavingCertificate at the end of upper secondary education. Over four fifths of the student cohortreach the end of upper secondary education (Gorby, McCoy, & Watson, 2005). For themost part, upper secondary qualifications are assessed on the basis of student performancein a standardised examination, with students typically taking seven examination subjects.Students may take subjects at one of three levels: higher, ordinary, and foundation level.The level of education achieved is highly predictive of later life chances, with higher qualityemployment and pay levels found among those with tertiary qualifications (McCoy &Smyth, 2003). There is quite a marked difference in employment outcomes by educationallevel in Ireland in comparison with other OECD countries (OECD, 2006b). Furthermore,the actual grades achieved within national examinations are predictive of subsequentlabour market outcomes, with higher grades enhancing employment chances and paylevels (Breen, Hannan, & O’Leary, 1995; Gorby et al., 2005). Leaving Certificate gradesalso determine entry to tertiary education, due to the numerus clausus system in place, with‘‘points’’ allocated to the grades received in examination subjects and students rankedaccordingly for university entry purposes. Participation in tertiary education in Ireland hasexpanded rapidly in recent years, with over half of the cohort now making the transition totertiary education (O’Connell, Clancy, & McCoy, 2006).

Because of its importance for tertiary education entry and employment chances, theLeaving Certificate examination can be considered a ‘‘high stakes’’ test from the studentperspective, with an incentive for young people to seek additional assistance with study inorder to enhance their academic performance. As a result, there has been the emergence ofprivate tuition centres (termed ‘‘grind schools’’ in the Irish context), which providesupplementary tuition to students who attend mainstream secondary schools on either aregular part-time basis (evenings or weekends) or on a ‘‘block’’ basis (by offering intensiverevision courses during school holidays (Lynch & Moran, 2006). Tuition centres are notthe only mode of provision since a number of students take part in one-to-one tutoringgenerally given by teachers or university students. The costs involved can be quitesignificant. Part-time courses in some of the private tuition centres cost in the region of 895euro per year for one subject while block courses cost approximately 245 euro per subject.There are no systematic data available on the costs of one-to-one tuition, but Lynch andMoran (2006) suggest costs of 30 euro per hour for private tutors.

Educational Research and Evaluation 467

Dow

nloa

ded

by [

Uni

vers

ity o

f Io

wa

Lib

rari

es]

at 0

1:05

13

Nov

embe

r 20

14

Data and methodology

The paper draws on data on 4,709 students within 112 secondary schools in Ireland. Thesurvey was carried out in 1994, but more recent data indicate that the processes shapingacademic achievement and the profile of participants have remained stable over time(Smyth, in press). The sample of schools was drawn using stratified random samplingwith schools selected to be representative of the national distribution of schools in termsof school sector, gender composition, school size, and location. At the school level,detailed interviews with principals and guidance counsellors conducted in a representa-tive sample of schools (covering almost one sixth of all secondary schools) yieldedinformation on a number of aspects of school organisation and process. Within theseschools, questionnaires were administered to students in the middle of their final year ofupper secondary education.1 Classes were sampled within schools, taking roughly halfthe total number of classes from the relevant year in each selected school. In addition,the sampling frame allowed for differences between schools in the system of abilitygrouping (streaming as opposed to mixed ability) in order to provide a representativesample of the ability range of pupils within the school. In total, questionnaires werecompleted by 81% of Leaving Certificate students in the sampled classes. The group ofstudents was subsequently followed up to collect information from official records ontheir performance at upper secondary (Leaving Certificate) level. Performance wasmeasured in terms of grade point average, assigning ‘‘points’’ according to the gradereceived and subject level taken and averaging these points over all exam subjects (with arange of 0 to 20).

In the survey, students were asked whether they had taken part in paid private tuitionin the previous 3 months and the number of hours they had spent at such tuition, whichwas taken as a measure of intensity of involvement in private tuition. The questionnairesalso collected very detailed information on student characteristics prior to their taking theLeaving Certificate exam. The measures include: (1) family background, includingparental social class and parental education; (2) prior ability/achievement, as measured bystudent performance in the Junior Certificate exam 2 to 3 years earlier2; (3) studentmotivation and engagement, including educational aspirations and number of hours spenton homework and study on an average evening. The availability of such detailedinformation allows us to more accurately estimate the net effects of participation in privatetuition, comparing ‘‘like with like’’.

Because the survey involved sampling students within schools, traditional regressiontechniques are not appropriate since they assume no auto-correlation within the data(Goldstein, 1995). It cannot be assumed that students within the same school arecompletely ‘‘independent’’ of each other in this way, since they are not groups formedrandomly and they are bound to become more homogeneous as they share learning andsocial experiences. As a result, using traditional regression techniques on schools meansthat the estimates of the standard errors of conventional statistical tests are much toosmall, which may result in spurious ‘‘significant’’ results (see Guo & Zhao, 2000; Jones,1991). In contrast, multilevel (hierarchical linear) models have a number of advantages inthe estimation of school effects by explicitly taking the effects of school clustering intoaccount and providing precision-weighted estimates to reflect the number of observationson which they are based. For these reasons, multilevel modelling is used in this paper toprovide a more accurate estimate of the influence of participation in private tuition onexamination performance, controlling for background differences between students.Analyses here were carried out using the MLwiN computer package initially developed by

468 E. Smyth

Dow

nloa

ded

by [

Uni

vers

ity o

f Io

wa

Lib

rari

es]

at 0

1:05

13

Nov

embe

r 20

14

the Institute of Education, University of London (Rasbash, Steele, Browne, & Prosser,2005). The analyses are discussed in the following section.

Results

The characteristics of students taking private tuition

Among the upper secondary students sampled, around a third (32%) had taken paidprivate tuition in the 3 months prior to the survey. Among the group of students takingprivate tuition, the intensity of involvement tended to be relatively low. Over half (55%) ofthe students had spent 5 hr or fewer at private tuition, under a quarter (23%) had spentbetween 6 and 10 hr, 9% had spent between 11 and 20 hr, while only 4% had spent morethan 20 hr on such tuition in the previous 3 months (see Table 1).

The intensity of involvement in private tuition varies by a number of studentcharacteristics which have previously been found to be associated with examinationperformance (Smyth, 1999). These characteristics include gender, social class background,parental education, prior ability/achievement, educational aspirations, and investment oftime in homework and study. Young women make up a majority (63%) of those takingprivate tuition compared to under half of students not participating (see Table 1).Participation varies significantly by social background measured in terms of social classand parental education. A six-category social class classification developed for Irishpurposes by the Central Statistics Office (1995) was employed, ranging from ‘‘higherprofessional’’ to ‘‘unskilled manual worker’’. A dominance approach (Erikson, 1984) wasused to determine parental social class, whereby, if both parents were in employment,social class was based on the parent with the highest class. Students from a higherprofessional background make up a quarter of those taking private tuition and only atenth of nonparticipants. Furthermore, those from higher professional backgrounds arelikely to have more intensive levels of involvement, with this group making up almost athird of those taking more than 20 hr of tuition (Table 1). The measure of parentaleducation was based on the highest of mother’s and father’s educational level. As withsocial class, it was significantly associated with participation in private tuition and the

Table 1. Descriptive variables by tuition intensity.

NonparticipantsParticipants,of which:

No. of hours in previous3 months

1–5 6–10 11–20 420

Involvement in private tuition(% of whole sample)

68.1 (31.9) 17.5 7.6 2.7 4.1

% Female 48.9 63.3 62.7 61.7 68.7 64.9% Higher professional 11.0 25.4 25.6 22.8 24.0 31.1% Parents with tertiaryeducation

22.3 40.0 37.0 38.5 45.8 51.3

Lower secondary grade pointaverage

6.5 7.1 7.0 7.1 7.1 7.6

% Aspiring to degree-levelqualifications

34.4 59.1 57.4 54.8 59.5 73.6

Number of hours spent onhomework/study per night

3.7 4.1 4.0 4.1 4.2 4.2

Note: All between-group differences are significant at the p 5 .001 level.

Educational Research and Evaluation 469

Dow

nloa

ded

by [

Uni

vers

ity o

f Io

wa

Lib

rari

es]

at 0

1:05

13

Nov

embe

r 20

14

intensity of such involvement. Over half of the students who had spent more than 20 hr atprivate tuition had parents with tertiary qualifications compared with 22% of those whodid not attend private tuition.

In addition to background characteristics, students’ engagement with school is foundto be associated with participation in private tuition. Prior ability/performance wasmeasured in terms of the average grade achieved in the nationally standardised JuniorCertificate examination taken at the end of lower secondary education. Points wereassigned to each subject based on the level taken and grade achieved and the total ‘‘points’’averaged over all exam subjects taken. Scores ranged from 10 (for a higher level A grade)to 1 (for a foundation-level D grade) with zero allocated to ‘‘fail’’ grades (E, F, and NG).Students taking private tuition are disproportionately made up of those who have higherlevels of prior performance; there is a prior performance gap of 0.6 grade points betweenparticipants and nonparticipants (Table 1). Those who have intensive levels ofinvolvement in private tuition have particularly high levels of prior academic performance(7.6 grade points compared with 6.5 among nonparticipants).3

Previous international research has indicated that the desire to enter tertiary educationis often a motivating factor for participation in private tuition (see, e.g., Psacharopoulos &Papakonstantinou, 2005). This is also evident in the Irish context. Over half (59%) ofthose taking private tuition aspire to degree-level qualifications compared with a third(34%) of nonparticipants. Those with intensive levels of tuition involvement areparticularly oriented towards tertiary education, with almost three quarters of thosewho had taken more than 20 hr of tuition aspiring to degree-level qualifications. Inaddition, students who are highly motivated in terms of the amount of time they spend onhomework and study on the average weekday are more likely to take private tuition(Table 1).

In sum, students taking (high levels of) private tuition are disproportionately made upof those groups of students who tend to do well academically – being disproportionatelyfemale, from middle-class backgrounds, with high levels of prior ability/performance andhigher educational aspirations. In assessing the net impact of different levels ofinvolvement in private tuition, it is important to take account of these differencesbetween participants and nonparticipants. The following section provides estimates of thenet effect of tuition intensity on academic performance once the selective nature of theparticipant group is taken into account.

Impact of tuition intensity on performance

A multilevel regression model was used to estimate the effects of the number of hours ofprivate tuition on examination performance at upper secondary level:

yij ¼ b0 þ b1x1ij þ b2x2ij þ uj þ eij ð1Þ

Here, yij is the grade point average of student i in school j, x1 represents a set of dummyvariables indicating intensity of involvement in private tuition, and x2 represents a set ofstudent characteristics (including gender, social class, and so on). The school-level residual(uj) refers to the variation of a school’s performance from the overall mean, controlling forstudent characteristics and participation in tuition, while eij indicates the deviation foreach student from their school’s average.

The measure of Leaving Certificate performance was based on the average gradesreceived across all examination subjects taken. The points assigned were based on the

470 E. Smyth

Dow

nloa

ded

by [

Uni

vers

ity o

f Io

wa

Lib

rari

es]

at 0

1:05

13

Nov

embe

r 20

14

subject level taken and grades achieved, ranging from 20 (for an A1 grade on a higher levelpaper) to 1 (for a D3 grade on a foundation level paper) with zero assigned to ‘‘fail’’ grades(E, F, and NG). The null model (Table 2, Model 1) indicates the average grade across allstudents (7.9 out of a maximum of 20) with the random effects terms indicating significantvariation between students and schools around this average. A fifth of the total variationin performance is found to be at the school level, which is broadly consistent withestimates of the size of ‘‘school effects’’ from international studies (for an overview, seeTeddlie & Reynolds, 2000).

Intensity of involvement in private tuition is represented by a set of dummy variables(1–5 hr, 6–10 hr, 11–20 hr, and more than 20 hr), contrasted against those who have nottaken private tuition. Before including any controls, those who took part in private tuitionare found to achieve higher grades than nonparticipants, and the grades achieved arerelated to the intensity of involvement; those taking fewer than 20 hr of tuition achievearound one grade point per subject more than nonparticipants while those taking morethan 20 hr of tuition achieve 1.8 grade points more (Table 2, Model 2). However, asindicated in the previous section, participants differ from nonparticipants in terms of keycharacteristics associated with educational success.

In order to assess the net impact of involvement in private tuition on LeavingCertificate (upper secondary) performance, successive models take account of studentcharacteristics, including gender, social background, prior ability/performance, educa-tional aspirations, and amount of time spent on homework (Table 2, Models 3 to 7). Theimpact of these characteristics on performance is consistent with previous Irish research(see Smyth, 1999; Sofroniou, Cosgrove, & Shiel, 2002; Sofroniou, Shiel, & Cosgrove,2000). Higher grades are achieved by female students, those from middle-classbackgrounds (in terms of social class and parental education), those with higher levelsof prior academic ability/performance, those with higher educational aspirations, andthose who spend more time on homework and study. We can assess the net impact ofintensity of involvement in private tuition by comparing the coefficients for the dummyvariables across the models. For ease of comparison, the size of the private tuition ‘‘effect’’is depicted in Figure 1. The effect of private tuition is found to reduce in magnitude when

Figure 1. Estimated differences in Leaving Certificate exam grades by intensity of involvement inprivate tuition, compared to those with no involvement in private tuition.Note: Estimates are derived from the coefficients of the dummy variables in Table 2.

Educational Research and Evaluation 471

Dow

nloa

ded

by [

Uni

vers

ity o

f Io

wa

Lib

rari

es]

at 0

1:05

13

Nov

embe

r 20

14

Table

2.

Multilevel

regressionmodel

ofupper

secondary

perform

ance

(dependentvariable:gradepointaveragein

LeavingCertificate

exam).

Fixed

effects:

Model

1Model

2Model

3Model

4Model

5Model

6Model

7

Constant

7.874

7.543

7.220

6.076

7.096

5.414

5.035

Tookprivate

tuition:

1–5hr

0.991***

0.947***

0.627***

0.314**

70.031

70.048

6–10hr

1.065***

1.026***

0.679**

0.203

70.098

70.133

11–20hr

1.081**

1.019**

0.542

0.078

70.274

70.318

420hr

1.842***

1.819***

1.346***

0.831**

0.228

0.184

Fem

ale

0.646***

0.803***

0.261

0.280**

0.206

Socialclass:

Higher

professional

1.792***

0.824**

0.486*

0.490**

Lower

professional

1.376***

0.668**

0.417**

0.417**

Other

nonmanual

1.078***

0.530*

0.283*

0.284*

Skilledmanual

0.202

0.046

0.046

0.047

(Ref:semi/unskilledmanual)

Parentaleducation:

Upper

secondary

70.002

70.090

70.147

70.160

Tertiary

subdegree

0.562**

0.297

0.076

0.069

Tertiary

degree

1.977***

1.321**

0.825**

0.806***

(Ref:Lower

secondary/primary)

JuniorCertificate

GradePointAverage

1.248***

0.891***

0.887***

Educationalaspirations:

Tertiary

subdegree

0.975**

0.910***

Tertiary

degree

4.017***

3.894***

Homew

ork

hours

0.183***

Random

effects:

School-level

variation

3.969***

3.387***

3.276***

2.027***

2.278***

1.221***

1.217***

Student-level

variation

15.050***

14.840***

14.790***

13.860***

8.280***

6.548***

6.513***

Note:***p5

.001,**p5

.01,*p5

.05.

472 E. Smyth

Dow

nloa

ded

by [

Uni

vers

ity o

f Io

wa

Lib

rari

es]

at 0

1:05

13

Nov

embe

r 20

14

social background characteristics are taken into account, with a further reduction whenprior ability/performance is controlled for (see Figure 1). Controlling for gender, socialbackground, and prior grades, those spending the most time on private tuition score 0.8grade points more than nonparticipants compared with 1.8 grade points in the unadjustedmodel. Thus, a good deal of the apparent advantage accruing to those spending (largeamounts of) time at private tuition is due to the gender, social class, and ability profile ofparticipants. When aspirations and homework time are added to the model, no significantdifferences are found between participants in private tuition and nonparticipants (seeFigure 1); the coefficients for involvement in private tuition all become nonsignificant inthese models (Table 2, Models 6 and 7). Even more intensive levels of involvement inprivate tuition do not secure a performance advantage when we take account of themotivation and engagement levels of participants.4 In sum, the apparent positive effect of(high levels of) involvement in private tuition merely reflects the characteristics of thestudents taking such tuition.

Alternative estimates of between-group differences

There has been considerable debate about whether the kind of regression techniquesused so far provide accurate estimates of the effect of ‘‘treatment’’ on participants ifparticipants and nonparticipants differ markedly across key dimensions (Conniffe, Gash,& O’Connell, 2000; D’Agostino, 1998). Propensity score matching techniques have beenproposed as a way of more accurately comparing ‘‘like with like’’ in looking at theimpact of taking part in a particular programme or intervention. Exposure to privatetuition was treated as an ordinal variable, ranging from none (no involvement) to high(420 hr). An ordinal logistic regression model was used to predict the effects of gender,social class, parental education, prior ability/performance, and educational aspirationson the likelihood of having different levels of involvement in private tuition. Followingthe approach of Yanovitzky, Zanutto, and Hornik (2005), this model was used toconstruct a balancing score based on the values of the independent variables weighted bytheir estimated slope parameters. Cases were then divided into quintiles on the basis oftheir balancing scores. Estimates of upper secondary (Leaving Certificate) examinationperformance were then derived by treatment group within each quintile (see Table 3).There is no clear-cut pattern of increasing levels of examination performance withincreasing exposure to private tuition. Only in the second quintile did all of thetreatment groups outperform those who had not taken private tuition. Analyses ofvariance indicate that, when the balancing score is controlled for, no significantassociation is found between time spent at private tuition and academic performance. In

Table 3. Estimated examination performance (grade point average) by involvement in privatetuition within balancing score quintiles.

Quintile based onbalancing score: Nonparticipants

Participants, of which:

55 hr 6–10 hr 11–20 hr 420 hr

Quintile 1 4.85 5.11 4.79 4.48 4.76Quintile 2 6.31 6.66 6.94 6.76 7.67Quintile 3 7.79 7.34 8.48 7.31 8.53Quintile 4 9.65 10.35 10.24 9.26 10.33Quintile 5 11.91 11.43 11.39 11.82 11.97

Educational Research and Evaluation 473

Dow

nloa

ded

by [

Uni

vers

ity o

f Io

wa

Lib

rari

es]

at 0

1:05

13

Nov

embe

r 20

14

sum, the propensity score analyses confirm the findings of the multilevel regressionanalyses that increasing levels of involvement have no significant impact on examinationgrades when we compare ‘‘like with like’’.

Conclusions

There has been a growth in the use of private tuition internationally, with students mainlyusing such tuition at key transition points, such as the transition from school to college(Bray, 2003; Silova & Bray, 2006). Previous studies have indicated that such tuition securesadvantages to participants by boosting their academic performance and thus enhancingtheir access to tertiary education (Baker et al., 2001; Kang, 2006; Tansel & Bircan, 2005).However, such studies have not taken account of potential variability in the intensity ofyoung people’s involvement in private tuition. Nor have they always controlled for keyattitudinal and behaviour differences between participants and nonparticipants inestimating the net impact of paid tuition.

This article has sought to explore the impact of private tuition on upper secondaryexamination performance in the Republic of Ireland, taking account both of theintensity of involvement in tuition and of important differences between participants andnonparticipants. In the Irish context, the use of private tuition appears to be motivatedby the desire to secure a college place. As a result, participation is more prevalent amonggroups of students who tend to fare well academically: female students, students frommiddle-class backgrounds, those who have already secured higher grades, those withgreater levels of engagement in homework and study, and those with higher educationalaspirations. It is important, therefore, to take account of these differences whenestimating the impact of private tuition on academic performance. Comparing like withlike, no significant difference is found between those taking tuition and other students intheir final examination grades. Interestingly, there is no evidence that those with moreintensive levels of involvement in private tuition (i.e., spending more than 20 hr in theprevious 3 months) secure any grade advantages over nonparticipants or those withonly low levels of involvement. Given the considerable costs of private tuition, itdoes not therefore appear to be an effective strategy to maximise the chances of collegeentry.

Why does private tuition not appear to be successful in enhancing academicperformance? Two possible explanations are apparent. Firstly, such tuition is taken bystudents who tend to secure relatively high grades. There may therefore be a thresholdeffect whereby students with high grades find it difficult to boost these grades even further.Secondly, and perhaps more importantly, the amount of time students spend at privatetuition is much lower than that spent at their regular school. During term-time, youngpeople in secondary education spend a minimum of 28, and sometimes up to 35, hr perweek at school with additional investment of time outside school hours in school-basedextracurricular activities and homework/study. Students in exam-year classes spend anaverage of 3 to 4 hr per night on homework and study outside school time (Hannan et al.,1996; Smyth et al., 2004). It is perhaps not surprising, therefore, that school effects willdominate over any impact of outside-school tuition in the Irish context. However, anassessment of the impact of private tuition is still worthwhile since it involves a sizableminority (a third) of the student cohort and may involve considerable expenditure on thepart of families. Furthermore, analyses of the potential effectiveness of private tuitionmay contribute to the on-going policy debate on the marketisation of educationinternationally.

474 E. Smyth

Dow

nloa

ded

by [

Uni

vers

ity o

f Io

wa

Lib

rari

es]

at 0

1:05

13

Nov

embe

r 20

14

Notes

1. Students in their final year of lower secondary education were also surveyed but are not analysedfor the purposes of this paper.

2. Junior Certificate grade point average can be taken as a reliable proxy for underlying ‘‘ability’’.Firstly, there is a correlation of 0.8 between this measure and verbal and numerical ability testscores on entry to secondary education (Smyth, 1999). Secondly, the Junior and LeavingCertificate exams are separated by 2 to 3 years of full-time education. If taking private tuition forthe final exam is hypothesised to have an effect on performance, we would expect the effect ofsuch tuition to be positive and significant controlling for the influence of prior ability/performance.

3. Higher prior ability/performance among participants compared with nonparticipants is alsofound within social class groups. Thus, higher ability middle-class students are the most likely totake private tuition.

4. This paper focuses on the intensity of involvement in private tuition. Analyses comparing thosewho had spent no time at private tuition and those who had spent any time at private tuitionindicated no net difference between the two groups in performance controlling for the same setof student characteristics (see Smyth, in press). In addition, these analyses indicated no variationin the pattern across the ability distribution; in other words, private tuition did not yield anyadvantages for lower performing students.

Note on contributor

Dr Emer Smyth is a Senior Research Officer at the Economic and Social Research Institute.Her research interests centre on education, school to work transitions, and comparativemethodology. She is currently leading a mixed methods longitudinal study of secondary studentsin Ireland.

References

Baker, D.P., Akiba, M., LeTendre, G.K., & Wiseman, A.W. (2001). Worldwide shadow education:Outside-school learning, institutional quality of schooling, and cross-national mathematicsachievement. Educational Evaluation and Policy Analysis, 23(1), 1–17.

Bray, M. (2003). Adverse impacts of private supplementary tutoring. Paris: UNESCO.Bray, M. (2005, September). Private supplementary tutoring: Comparative perspectives on patterns

and implications. Paper presented at the Oxford International Conference on Education andDevelopment, Oxford, UK.

Bray, M., & Kwok, P. (2003). Demand for private supplementary tutoring: Conceptual con-siderations, and socio-economic patterns in Hong Kong. Economics of Education Review, 22,611–620.

Breen, R., Hannan, D.F., & O’Leary, R. (1995). Returns to education: Taking account ofemployers’ perceptions and use of educational credentials. European Sociological Review, 11(1),59–73.

Central Statistics Office (1995). Census of population of Ireland. Volume 6: Occupations. Dublin,Ireland: Stationery Office.

Conniffe, D., Gash, V., & O’Connell, P.J. (2000). Evaluating state programmes: ‘‘Naturalexperiments’’ and propensity scores. Economic and Social Review, 21(4), 283–308.

D’Agostino, R.B. (1998). Propensity score methods for bias reduction in the comparison of atreatment to a non-randomised control group. Statistics in Medicine, 17, 2265–2281.

Davies, S. (2004). School choice by default? Understanding the demand for private tutoring inCanada. American Journal of Education, 110, 233–255.

Erikson, R. (1984). Social class of men, women and families. Sociology, 18(4), 500–514.Goldstein, H. (1995). Multilevel statistical models (2nd ed.). London: Edward Arnold.Gorby, S., McCoy, S., & Watson, D. (2005). 2004 Annual School Leavers’ Survey of 2002/2003

Leavers. Dublin, Ireland: ESRI/Department of Education and Science.Guo, G., & Zhao, H. (2000). Multilevel modeling for binary data. Annual Review of Sociology, 26,

441–462.

Educational Research and Evaluation 475

Dow

nloa

ded

by [

Uni

vers

ity o

f Io

wa

Lib

rari

es]

at 0

1:05

13

Nov

embe

r 20

14

Hannan, D., Smyth, E., McCullagh, J., O’Leary, R., &McMahon, D. (1996). Coeducation and genderequality: Exam performance, stress and personal development. Dublin, Ireland: Oak Tree Press/ESRI.

Ireson, J. (2004). Private tutoring: How prevalent and effective is it? London Review of Education,2(2), 109–122.

Ireson, J., & Rushforth, K. (2005). Mapping and evaluating shadow education. London: Institute ofEducation.

Jones, K. (1991). Multi-level models for geographical research. Norwich, UK: University of EastAnglia.

Kang, C. (2006). Does money matter? The effect of private educational expenditures on academicperformance (Department of Economics Working Paper). Singapore: National University ofSingapore.

Lynch, K., & Moran, M. (2006). Markets, schools and the convertibility of economic capital: Thecomplex dynamics of class choice. British Journal of Sociology of Education, 27(2), 221–235.

McCoy, S., & Smyth, E. (2003). Educational expenditure: Implications for equality. In T. Callan, A.Doris, & D. McCoy (Eds.), Budget perspectives 2004 (pp. 65–97). Dublin, Ireland: ESRI/FFS.

Mischo, C., & Haag, L. (2002). Expansion and effectiveness of private tutoring. European Journal ofPsychology of Education, XVII(3), 263–273.

O’Connell, P.J., Clancy, D., & McCoy, S. (2006). Who went to college in 2004? A national survey ofentrants to higher education. Dublin, Ireland: Higher Education Authority.

Organisation for Economic Co-operation and Development. (2006a). Demand-sensitive schooling?Evidence and issues. Paris: Author.

Organisation for Economic Co-operation and Development. (2006b). Education at a glance. Paris:Author.

Psacharopoulos, G., & Papakonstantinou, G. (2005). The real university cost in a ‘‘free’’ highereducation country. Economics of Education Review, 24, 103–108.

Rasbash, J., Steele, F., Browne, W., & Prosser, B. (2005). A user’s guide to MlwiN. Bristol, UK:Centre for Multilevel Modelling, University of Bristol.

Silova, I., & Bray, M. (Eds.). (2006). Education in a hidden marketplace: Monitoring of privatetutoring. Budapest, Hungary: Open Society Institute.

Smyth, E. (1999). Do schools differ? Academic and personal development among pupils in the second-level sector. Dublin, Ireland: Oak Tree Press/ESRI.

Smyth, E. (in press). Buying your way into college? Private tuition and the transition to highereducation in Ireland. Oxford Review of Education. Available online prior to publication at http://www.informaworld.com/smpp/content*content¼a791325290*db¼all*order¼pubdate.

Smyth, E., McCoy, S., & Darmody, M. (2004). Moving up: The experiences of first year students inpost-primary education. Dublin, Ireland: Liffey Press/ESRI.

Sofroniou, N., Cosgrove, J., & Shiel, G. (2002). Using PISA variables to explain performance onJunior Certificate Examinations in mathematics and science. Irish Journal of Education, 33, 9–124.

Sofroniou, N., Shiel, G., & Cosgrove, J. (2000). Explaining performance on the Junior Certificateexamination. Irish Journal of Education, 31, 25–49.

Stevenson, D.L., & Baker, D.P. (1992). Shadow education and allocation to formal schooling:Transition to university in Japan. American Journal of Sociology, 97(6), 1639–1657.

Tansel, A., & Bircan, F. (2005). Effect of private tutoring on university entrance examinationperformance in Turkey (Discussion Paper). Bonn, Germany: IZA.

Tansel, A., & Bircan, F. (2006). Demand for education in Turkey: A tobit analysis of privatetutoring expenditures. Economics of Education Review, 25, 303–313.

Teddlie, C., & Reynolds, D. (2000). The international handbook of school effectiveness research.London: Falmer Press.

Yanovitsky, I., Zanutto, E., & Hornik, R. (2005). Estimating causal effects of public healtheducation campaigns using propensity score methodology. Evaluation and Program Planning, 28,209–220.

476 E. Smyth

Dow

nloa

ded

by [

Uni

vers

ity o

f Io

wa

Lib

rari

es]

at 0

1:05

13

Nov

embe

r 20

14