Luke Bishop Research Project

7/31/2019 Luke Bishop Research Project

1/40

Luke Bishop

Key Performance Indicators that discriminatewinning and losing in the knockout stages of

the 2011 Rugby World Cup!!E-mail: [email protected]


2/40

I have recently finished my BSc Sport and Exercise Science undergraduatedegree at Sheffield Hallam University. This is my final year dissertation,looking into the 2011 Rugby World Cup. I am looking to pursue a career inPerformance Analysis, and have been working as an analyst at Opta for the

past year.

If you choose to read my project, then thank you. Any comments or feedbackwould be much appreciated using the address provided on the first page.

Luke Bishop

!


3/40

Key Performance Indicators that discriminate winning and losing in the knockout stages of the 2011

Rugby World Cup. Luke Bishop

Page ii

Abstract

The objectives of the study were to investigate what performance indicators

discriminated winning and losing teams in the knockout stages of the 2011 Rugby

World Cup, looking to gain a better understanding of the influence of certain law

changes (Van Rooyen, Diedrick and Noakes 2010) on the tactical development of the

game. A review of preceding literature identified several performance indicators that

were found to be influential on match success in elite rugby; Lineout success,

turnovers conceded, frequency of kicks out of hand, tackle completion, line breaks,

penalties conceded and ruck frequency. An independent groups design was used,

comparing the means of winning and losing teams in the knockout matches (n=8). All

games were analysed by the primary investigator using a pre-set analysis template,

which allowed coding of the selected performance variables. Operational definitions

were assigned to each indicator prior to analysis. Intra- and inter-observer reliability

tests were completed, with correlation coefficients and percentage errors provided for

each variable. An effect size calculation was also applied to each variable. Following

a Mann Whitney U test, winners were found to concede a significantly higher

percentage of their penalties between halfway and the opposition 22m than losers (P

= 0.026) and losing teams were found to carry the ball significantly more than winning

teams (P= 0.035). Effect size calculations also identified meaningful, yet not

significant findings for several other variables. It was concluded that the ability to

concede penalties in more attacking positions and concede fewer turnovers, along

with a kicking, territory-based approach was more beneficial than a possession-

based one in the knockout stages of the 2011 Rugby World Cup.


4/40



Page 1

1.0 INTRODUCTION

Since its inception as a professional sport in the 1990s, Rugby Union has

advanced in terms of the scientific support that teams utilise. Nutritionists, strength

and conditioning coaches and psychologists have been introduced into modern day

rugby, all with the aim of enhancing the mental and physical performance of the

players and team. One of the most recent advances is the introduction of video

analysis support, aimed at identifying and improving match strategies by reviewing

video footage to gain a tactical advantage over the opposition.

Hughes (2004, p104) describes notational analysis as an objective way of

recording performance so that key elements of that performance can be quantified ina valid and consistent manner. The aim of notational analysis is to provide both

coaches and players with valuable information regarding sports performance that will

advance their decision-making (ODonoghue 2006) and understanding of tactical

issues in their sport, with an overall view to improving future performances (McGarry

2009). Prior to the introduction of performance analysis into sport, coaches would

have to make decisions based solely on their observation, which Franks and Miller

(1991) found to be poor. They reported that international level soccer coaches could

only recall 30% of what would be considered key events over the duration of a game.In addition to this, Bracewell (2002) suggests coaches struggle to remember rare

events and are unable to put events they do recall into context. It is suggested that

this is due to tension, emotion and individual bias a coach experiences during and

after a game. Jenkins, Morgan and ODonoghue (2007) successfully integrated

match analysis into the current coaching process of a netball team, highlighting that

using notational analysis as a way of overcoming the recall limitations of coaches, is

effective. Reilly and Gilbourne (2003) support these findings, suggesting that using

notational analysis as a coaching tool can provide detailed and accurate feedback forthe coach regarding positive and negative aspects of performance.

Lago-Ballesteros and Lago-Peas (2010) state that in order for identified

performance indictors to be useful, they have to be associated with success.

Therefore, studies have investigated how winning and losing teams differ with

regards to these performance indicators. There have been numerous studies across

several sports looking at the differences between winners and losers (Jones,

Mellalieu and James 2004, Csataljay et al. 2009, Lago-Peas, Lago-Ballesteros and

Rey 2011).


5/40



Page 2

To date limited research has been conducted into what performance indicators

differentiate winning and losing teams in Rugby Union. Indicators that distinguish

winning and losing found to be common across several studies are lineout success

(Ortega, Villarejo and Palao 2009, Vaz, Van Rooyen and Sampaio 2010, Jones,

Mellalieu and James 2004), turnovers conceded (Ortega, Villarejo and Palao 2009,

Vaz et al. 2011) and number of kicks out of hand (Stanhope and Hughes 1997,

Ortega, Villarejo and Palao 2009, Vaz et al. 2011). In addition, other secondary

indicators have been identified between winning and losing teams. These included

tackle completion percentage (Ortega, Villarejo and Palao 2009, Vaz et al. 2011), line

breaks (Ortega, Villarejo and Palao 2009), and penalties conceded (Jones, Mellalieu

and James 2004, Vaz et al. 2011). However, these studies have reported findings

from a mixture of international and domestic competitions, some of which are league

based and others are knockout based.

Van Rooyen, Diedrick and Noakes' (2010) study into ruck frequency is the only

article to look at knockout matches in the World Cup as a separate entity to the pool

stage matches. They found contrasting results for ruck frequency and match success

between the pool stages and knockout stages. This suggests that the format of the

competition may have an influence on a team's tactical approach and what

performance indicators are most important for success. Van Rooyen, Diedrick and

Noakes (2010) suggested that in the 2011 Rugby World Cup, the findings for ruck

frequency would be different to the 2007 tournament, due to the introduction of the

Experimental Law Variations (ELVs). Of the thirteen ELVs introduced in 2008, ten

were used at the 2011 World Cup.It is suggested that the ELVs may have an

influence on other performance indicators in modern rugby based on Van Rooyen,

Diedrick and Noakes (2010) findings.

This study sought to investigate the key performance indicators at the 2011

Rugby World Cup that distinguish winning and losing in the knockout stages. This

research will further our understanding of the tactical development of the modern

game, and how knockout rugby may or may not differ from other competition formats.

2.0 LITERATURE REVIEW

2.1. The use of Notational Analysis in Sport

2.1a. How Notational Analysis is used

Notational analysis aims to enhance performance by identifying the key

performance indicators (KPIs) in a particular sport. Performance Indicators are


6/40



Page 3

described as a combination of variables that identify a certain aspect of performance

and help to achieve success in a certain sport (Hughes and Bartlett 2002, Jones,

Mellalieu and James 2004). They are used to outline the differences between

winning and losing performances, thus allowing the coach to recognise which

elements are the most critical in determining the result (Csataljay et al. 2009).

Performance indicators for invasion games can be split into three groups: Match

Indicators (e.g. tries scored, number of lineouts), technical indicators (e.g. tackles

won/lost, number of completed passes) and tactical indicators (e.g. possession, pass

distribution) (Bartlett 2001, Hughes and Bartlett 2002). Match indicators help to

describe performance giving the analyst an idea of what the team or individual has

done successfully or unsuccessfully throughout a game. However, these indicators

can be misleading if presented on their own, offering no way of determining if

performance was good. They should be presented in comparison to previous

performances and/or the oppositions data, which puts the findings in context

(McGarry 2009). Technical indicators identify which particular skills a team or

individual are good or bad at. For example, if a rugby team consistently missed 50%

of their tackles in a game, it could be inferred that their tackling technique is poor,

meaning this could be addressed in future training sessions. Tactical indicators help

to identify how a team or individual has played the game, e.g. in soccer, a team may

focus their attack down the right wing, possibly to target the opposition left-back if

they have been identified as being poor. In rugby union, a team may kick the ball to

touch regularly if they feel that their opponents lineout is poor and can put the

opposition under pressure. Hughes and Bartlett (2002) note that all types of sports

(i.e. net and wall, striking and fielding) follow the same structure when identifying

performance indicators, although the indicators are adapted to the rules of each sport

(e.g. a match indicator in Tennis could be number of aces). However, the provision

and application of feedback has been suggested to be the most important factor in

improving performance (Hughes and Bartlett 2002, Liebermann et al. 2002), not

merely the identification of key performance indicators.

2.1b. Providing Feedback

Feedback is often given to the coach using video (ODonoghue 2006). Video

feedback improves clarity for coaches and players when trying to understand what is

being presented. This feedback is underpinned by statistics created from match

analysis. Statistics identify trends in performance indicators and give an unbiased

and objective record of the game, which can determine certain areas that require


7/40



Page 4

attention (ODonoghue 2007, Vaz et al. 2011). These trends govern what feedback is

given, providing an objective rationale for showing certain videos. It has been

suggested that any feedback given on the basis of statistical compilation must be put

into context (Bracewell 2002, McGarry 2009, Vaz et al. 2011) with regards to the

opposition, pitch position and previous performances. Vaz et al. (2011) and

Bracewell (2002) suggest using multivariate statistical techniques, by comparing

statistics with the opposition, in order to create more powerful results relative to that

particular match. This is supported by Tenga et al. (2009) who state that performance

analysis research must reflect the interaction between the two opponents, due to the

dynamic nature of invasion games, where the actions of each team or player is

influenced by that of their opposition (Grehaigne, Bouthier and Bernard 1997, Lames

and McGarry 2007). These statistics, if supported by video highlights, can provide an

important insight into relative match performance. If not put into context, statistics

produced are of little value, due to the lack of comparison with another amount. In

summary, the most effective form of feedback is using video, due to its facility to

improve clarity for both coaches and players. However, the underpinning of these

videos should be provided by trends identified in match statistics, which evidence

suggests should be reported in relation to the opposition and pitch position.

2.1c. Reliability and Validity of feedback

The most important methodological issue in performance analysis is ensuring

valid and reliable results. Valid and reliable results are imperative to making sure that

the aims and purposes of the study are effectively met (Tenga et al. 2009). Reliability

is described as the consistency of measurements made using an analysis system

(Wilson and Batterham 1999). In performance analysis, this would refer to the

consistency in which performance indicators are measured over time. James, Taylor

and Stanley (2007) report that reliability gives an indication of the validity of the

findings of a study. In performance analysis, validity is the extent to which the coded

events reflect what has happened during the analysed match.

James, Taylor and Stanley (2007) highlighted that there are three sources of error

in notational analysis; Operational Error, where the analyst presses the incorrect

button; Observational Error, where the analyst fails to code an event; Definitional

Error, where events are labelled incorrectly. It is suggested that to avoid definitional

errors in the analysis, that operational definitions are given to each variable. Hughes

(2004) states that these definitions need to be clear and precise, which will help to

enhance the reliability of the results. However, James, Taylor and Stanley (2007)


8/40



Page 5

suggest that poorly written definitions can lead to uncertainty for the analyst, meaning

events may be coded incorrectly; highlighting the influence they can have on the

findings of a study. An inter-rater reliability test can be used to address any errors

due to the misinterpretation of definitions. Atkinson and Nevill (1998) state that any

study that involves measurement has to have a measurement tool that is reliable. An

inter-rater reliability test will identify if the analyst is making regular mistakes.

However, if the second analyst is also coding these events incorrectly, it suggests

that there is possibly a problem with the operational definitions used (James, Taylor

and Stanley 2007). An intra-reliability test will not detect errors due to ambiguous

definitions, but will indicate how consistently the analyst codes events. Therefore, it is

suggested that if possible, both types of reliability tests should be completed to

provide a greater indication of the quality of the results presented.

Table 2.1. An analysis of different reliability tests carried out in recent performance

analysis research papers in Rugby Union.

(Eaves and Hughes 2003, Boddington and Lambert 2004, Jones, Mellalieu and James 2004,

Eaves, Hughes and Lamb 2005, James, Mellalieu and Jones 2005, Prim, Van Rooyen and

Lambert 2006, Van Rooyen, Lambert and Noakes 2006, Van Rooyen and Noakes 2006,

Sasaki et al. 2007, Williams et al. 2007, Ortega, Villarejo and Palao 2009, Van Rooyen,

Diedrick and Noakes 2010, Vaz, Van Rooyen and Sampaio 2010, Wheeler, Askew and

Sayers 2010, Williams, Hughes, ODonoghue 2010, Diedrick and Van Rooyen 2011, Vaz et

al. 2011).

The use of reliability procedures in previous performance analysis research has

been varied. Hughes, Cooper and Nevill (2002) considered the reliability of

procedures across 67 unspecified performance analysis studies. They found that

Reliability Test Reported Number %

Inter-analyst 3 17.5

Intra-analyst 7 41

Inter and Intra analyst 4 24

None 3 17.5

Total 17 100


9/40



Page 6

70% of studies did not mention any reliability procedures and 15% reported the use

of correlations. However, Bland and Altman (2010) suggest that using solely

correlations are often an inadequate method for confirming reliability. These studies

may have completed reliability tests, even though not reported. The minimal

discussion placed upon the reliability of their results suggests less importance was

placed upon this part of the research, with more discussion based on the overall

findings. It is suggested that more time should be spent discussing the reliability of

the study, as this helps to enhance the overall value of the presented findings

(Williams et al. 2007). A similar review (Table 2.1) is shown regarding the reported

reliability tests of recent Rugby Union literature.

2.2. Notational Analysis in Rugby Union

2.2a. Development of the Game at the World Cup

There has been a clear development in several areas of Rugby Union from

amateur rugby during the 1980's to the modern-day professional era (Eaves and

Hughes 2003, Eaves, Hughes and Lamb 2005, IRB 2003, 2005, 2007).

The first development has been the increase in total match time by approximately

five minutes from 1980's to 2000's. It has been suggested that this is due to an

advance in the laws in the modern game, with things such as blood replacements,yellow cards and the Television Match Official all adding on to the total match time

(IRB 2005). Half time is also now 3-4 times longer than during the 1980's (IRB 2005).

It is suggested that this is required in the modern game due to the increased ball-in-

play time (IRB 2005). Ball-in-play time has increased from averaging approximately

21-23 minutes in the 1980's (IRB 2005), compared to 35 minutes in the 2007 World

Cup (IRB 2007). An increased ball-in-play time leads to an increase in the number of

game actions, and therefore, the physical effort players exert is greater, meaning half

time is needed to be increased in order to allow players sufficient recovery time.In terms of game actions, there have been several changes identified over time.

Eaves and Hughes (2003) report the game has moved towards a quicker, more ruck

dominated game, with more phases of play in the professional era compared to when

the game was amateur. IRB (2003, 2007) support this, reporting that the average

number of rucks per game has increased from 69 per game (at the 1995 World Cup)

to 144 per game (at the 2007 World Cup). These findings coincide with how often

teams kick the ball, either in play or in to touch. Eaves, Hughes and Lamb (2005)

report a 22.5% decrease in the average number of kicks per game from the amateur


10/40



Page 7

to professional era. This is likely to be due to the introduction of laws that meant

teams gain no territory from kicking the ball directly in to touch from outside their 22m

area in the modern era. From an average of 80 passes per game in the 1980's (IRB

2005), numbers of more than 200 passes per game (IRB 2007) were reported at the

2007 World Cup. These findings relate back to the increased ball-in-play time in the

modern era, which result in increased game actions. Fewer handling errors per game

have been reported in the modern era (IRB 2005). It can be inferred that this relates

to an increase in skill level as the game has become professional. There has been a

decrease in the number of scrums over time, averaging 31 per game in the 1980's

compared to the 2000's where there is on average, 19 scrums per game (IRB 2005).

This could be linked to the reduction in handling errors, which are a common source

of scrums. In addition, a reduction in the number of lineouts is also apparent. Eaves,

Hughes and Lamb (2005) report a greater average frequency of lineouts during the

amateur era. An average of 52 per game was reported in the 1980's (IRB 2005)

compared to 31 per game at the 2007 World Cup (IRB 2007). This is possibly due to

the reduction of kicks in the modern era, which reduces the likelihood of a lineout

occurring. The most recent influence on the development of the game has been the

introduction of the ELVs in 2008. Of the thirteen laws introduced, three of them

(Table 2.2) were designed to influence how teams play, with their aim being to

reward attacking play.

In summary, it is suggested that due to law changes and an increase in

professionalism, teams have placed an emphasis on keeping possession of the ball,

through rucking and recycling the ball through several phase. There has become less

of a focus on kicking the ball either in play or in touch. These findings look at how the

game has changed in terms of the overall amount of game actions, but does not

identify what is the most preferential tactical approach to winning matches. The

constant development in teams tactical approaches due to increased fitness and skill

levels, and also due to changing laws, highlights the need for regular analysis into

what strategies and approaches are most suitable and relevant in elite Rugby Union.


11/40



Page 8

Table 2.2.Experimental Law Variations still in place during 2011 Rugby World Cup.

(IRB 2008)

2.3. Research in Rugby Union

Research into elite Rugby Union has varied. Some literature has looked at

patterns of play and its development over time (Eaves and Hughes 2003, Williams,

Hughes and ODonoghue 2005). Others have investigated the physiological

demands of sport, how this differs between positions and what the ideal physical

characteristics are at an elite level (Cunniffe et al. 2009, Austin, Gabbett and Jenkins

2011). More recently, the development and identification of performance indicators

has become more common (Hughes and Bartlett 2002, James, Mellalieu and Jones

2005). These indicators represent different aspects of the game, such as possession,

tackle completion and turnovers conceded. They are generally presented as either

times, frequency counts or as percentages. Developing from research into identifying

Law Effect on Game

If a team puts the ball back into its

own 22 and the ball is

subsequently kicked directly into

touch, there is no gain in ground.

This ensures that defending teams do not

have an unfair advantage over attacking

teams by encouraging tactical kicking and

counter-attacking skills.

A quick throw in may be thrown in

straight or towards the throwing

teams own goal line.

This increases the probability of a quick

throw-in, providing a positive opportunity for

the team taking the throw-in to run the ball

instead of choosing a lineout.

Introduction of an offside line 5

metres behind the hindmost feet

of the scrum.

This increases the space available to the

team who wins the ball at the scrum. By

having all the forwards committed at the

scrum itself and 10 metres between theback lines, more space is available to build

an attack in.


12/40



Page 9

general performance indicators, current research has begun to investigate what

performance indicators are related to success in elite Rugby Union (Ortega, Villarejo

and Palao 2009, Van Rooyen, Diedrick and Noakes 2010, Vaz, Van Rooyen and

Sampaio 2010). This is largely due to increased professionalism in the sport,

resulting in the need for improved scientific and analytic support aimed at improving

performance.

2.3a. Performance Indicators Associated With Success

Previous studies into which performance indicators determine winning and losing

in Rugby Union have found differing results, but some indicators have been found to

be common across several studies. Success at the lineout has been reported as

being a determining factor between winning and losing in Rugby Union. Ortega,

Villarejo and Palaos (2009) (p=


13/40



Page 10

lower percentage pass completion, which is likely to lead to situations where

turnovers will be conceded. However, both of these studies fail to suggest whether

pitch location of the turnovers have an influence on the match outcome, with neither

putting their findings into any context, other than comparing winners and losers. From

the limited evidence available, it appears that maintaining control of the ball when in

possession, and forcing errors by the opposition when in defence are related to

successful performance.

How teams use possession of the ball can help to identify their game tactics. A

team who produce a lot of passes and offloads may be typical of a team looking to

play fast, expansive rugby, whereas a team who kicks the ball a lot, but attempts

fewer passes, could be indicative of playing a more measured, territorial game.

Previous research (Ortega, Villarejo and Palao 2009, Vaz, Van Rooyen and Sampaio

2010 (p= 0.01)) has identified that winning teams produced more kicks in field and

kicks to touch than losing teams. Ortega, Villarejo and Palao (2009) also suggest

losing teams tend to pick and go from rucks, and pass the ball out wide more often

than winners. These findings suggest that kicking the ball as a form of attack appears

to be more beneficial than attacking with ball in hand. It could be suggested that if a

team kicks well, they will create a territorial advantage, thus increasing pressure on

the opposition. This will increase the likelihood of mistakes being made and point

scoring opportunities being presented to the team who has kicked the ball. Jones,

Mellalieu and James (2004) and Vaz et al. (2011) both report successful teams are

able to gain more penalties in attacking areas of the pitch. They suggest this is due to

a successful kicking and rucking game, which is used to build pressure. This

increases the likelihood of penalties being conceded by the defending team, again

suggesting a kicking oriented game is linked with match success. Van Rooyen and

Noakes (2006) in a comparison between South Africa, England, Australia and New

Zealand at the 2003 World Cup, reported that South Africa had more defensive kicks

than the other three teams, who had a higher number of kicks in attacking positions.

South Africa were the least successful out of these teams, so it could be inferred that

pitch position of kicks could also be an indicator of match success, not just the

frequency of kicks.

Diedrick and Van Rooyen (2011) suggest that try scoring is strongly linked with

overall success. This is supported by Jones, Mellalieu and James (2004) (p=


14/40



Page 11

Rooyen, Lambert and Noakes (2006) support the idea that game tactics alter during

knockout matches, as the most successful team at the 2003 World Cup, England

scored the most penalties and drop goals, with third placed New Zealand scoring the

most tries. However, Ortega, Villarejo and Palao (2009) (p= 0.01) report losing teams

to score more penalty goals than winning teams, perhaps devaluing their importance

for overall match success. This could possibly be due to the winning team having a

strong defence, meaning try scoring opportunities would be infrequent, leading to

goal kicks being their most favourable option to score points.

Ruck frequency was found to differ between winning and losing knockout matches

at the 2007 Rugby World Cup (Van Rooyen, Diedrick and Noakes 2010). They

reported that teams with a lower ruck frequency than their opponents won 100% of

the knockout matches. This finding was opposite to the pool stage matches, where

increased ruck frequency was associated with success. This suggests knockout

rugby requires a different tactical approach to league based matches. It appears that

the ability to take point scoring opportunities more often and the ability to defend well

as a team is more beneficial during knockout rugby, rather than having control of the

ball for longer than the opposition. Stanhope and Hughes (1997) found higher ruck

frequency led to more success throughout the 1991 World Cup. Eaves, Hughes and

Lamb (2005) suggest keeping possession and completing more rucks is more

important now than in previous years in an analysis of 5 Nations and 6 Nations

games from 1988 to 2002. They suggest a lineout almost guarantees possession to

the side throwing the ball in, meaning kicking to touch is effectively handing

possession to the opposition and is not an successful tactic. This suggests there has

possibly been a development in the importance of rucking and maintaining

possession of the ball, although there is not enough evidence to come to any definite

conclusions at present.

There have been other performance indicators reported to differ between winning

and losing teams, but have not been reported to be as influential as previously

mentioned indicators. Ortega, Villarejo and Palao (2009) report winning teams to

have a higher tackle completion percentage than losing teams, in addition to making

more tackles than their opponents during Six Nations matches. Vaz, Van Rooyen

and Sampaio (2010) support these findings, reporting successful teams to make

more tackles than the opposition during Super 12 fixtures. These findings suggest

winning teams spend long periods without possession, due to the greater number of

tackles they have to make. The fact winning teams are able to maintain a tackle

completion percentage greater than that of their opponents while making a greater


15/40



Page 12

total number of tackles, suggests the ability to defend is key to success in Rugby

Union. However, it could be that a combination of good attack from one team and

bad defence from the other is key to success in Rugby Union. Wheeler, Askew and

Sayers (2010) report top teams in the Super 12 competition are able to break tackles

with 19% of their carries, compared to 16% for the mid-table teams and 11% for the

bottom teams. These findings suggest top teams are more proficient in attack in

comparison to less successful sides. However, it remains unclear whether their

findings were as a result of good attacking by the successful sides or poor defending

from opponents.

Ortega, Villarejo and Palao (2009) report winning teams successfully break the

defensive line when attacking more often than losing teams do. This is supported by

Diedrick and Van Rooyen (2011) who reported that at the 2007 World Cup, an

increase in initial breaks was related to match success. However, Wheeler, Askew

and Sayers (2010) report that the number of line breaks achieved by a team is not

associated with success in Rugby Union, but the number of defenders beaten seems

to be a better predictor of success. However, these two studies looked at different

competitions (6 Nations and Super 12) where there are clear dissimilarities in how

teams play, offering an explanation as to why they reported contrasting results.

2.4. Aims and hypotheses

The purpose of this study was to examine which of the previously identified

performance indicators are significant in determining the result of Rugby World Cup

2011 knockout matches. These findings may provide a basis for possible training

interventions and suggestions in tactics for knockout rugby. A comparison of the

chosen performance indicators (Lineout success %, turnovers conceded, kicks out of

hand, total carries, total passes, ruck frequency, tackle completion %, line breaks and

penalties conceded) was made with findings from previous studies in order to allow

inferences to be made as made as to what the influence of the Experimental Law

Variations has been, and offer any suggestions as to how and why the game has or

has not changed tactically. Suggestions can then be made as to what was the most

suitable tactical approach during the 2011 Rugby World Cup knockout stages.

It is hypothesised that, based on findings and suggestions of previous research,

winning teams will have a more successful lineout, will concede fewer turnovers, kick

possession away more often and complete less attacking rucks than losing teams

during the knockout stages of the 2011 Rugby World Cup. In addition to these

primary performance indicators, secondary performance indicators will show that


16/40



Page 13

winning teams complete a higher percentage of their attempted tackles, break the

defensive line more often, and concede fewer penalties than losing teams.

3.0 METHOD

3.1. Design

The study was an independent groups design. Both winning and losing teams

from the knockout matches of the 2011 Rugby World Cup formed the basis for

comparison. The identified performance indicators for each group were compared in

an attempt to identify those factors that distinguish winning and losing teams.

3.2. Sample

Data were gathered from the knockout matches of the 2011 Rugby World Cup (n=

8), resulting in eight winning teams and eight losing teams. Only knockout matches

were chosen for analysis as teams at this stage are generally of a similar standard,

meaning tactical differences in a team's approach can be crucial in determining the

result. This differs from games in the pool stages, where a high proportion of games

are won due to clear differences in playing ability. Thus, meaning a teams tactics are

less important at this stage of the competition.

3.3. Procedure

The eight knockout matches were individually uploaded into the Rugby Union

DVD 11 (Opta Sports Data, Leeds, United Kingdom) analysis software. The primary

investigator undertook all of the match analysis. Each match was observed and

coded using a pre-set analysis template (Appendix 2), which allowed the coding of

the selected performance variables; Lineout success, Turnovers conceded, Kicks out

of hand, Passes, Carries (divided into types of carry; Pick and Go, One Out Drive,

Other Carry, Support Carry and Kick Return), Ruck frequency, Tackle completion,

Initial breaks, Penalties conceded. Each variable coded was recorded onto a

timeline; giving the match time it took place as well as the pitch co-ordinates, which

indicate where on the pitch the event took place.

After the games had been coded, the data recorded onto the timeline were

exported from the analysis software into Microsoft Excel, (Microsoft Corporation,

Washington, USA) where the final set of data for each match was displayed. Data

analysis was then completed.


17/40



Page 14

3.4. Identification of Performance Indicators

Performance Indicators were identified following a review of previous Rugby

Union literature in performance analysis. Primary indicators had been found to differ

between winning and losing teams common across several studies. Secondary

indicators were reported to discriminate between winners and losers in some studies,

but not to the extent of the primary indicators. The chosen indicators were adapted to

produce a list of the analysis variables for this study. Once these variables had been

identified, operational definitions (Table 3.1, 3.2) were given to each variable.

Hughes (2004) notes how it is important these definitions are clear and precise to

increase the reliability of the analysis.

3.5. Reliability Procedures

Both intra-observer (data compared between original analysis and re-analysis for

three matches) and inter-observer reliability (data compared with an analyst who is

equally familiar with the software for all matches) tests were undertaken. This was

done to ensure that the analyst reliably recorded each performance indicator, and

also to guarantee that the system used for the analysis had test-retest reliability

(James, Taylor and Stanley 2007). The inter-observer test was used to supplement

the information provided by the intra-observer test. Both reliability tests were

undertaken on each performance indicator individually, providing a percentage error

and correlation coefficient for each variable. Re-analyses for intra-observer reliability

were completed two weeks after the original analysis.

3.5a. Percentage Error

Each performance indicator was compared, and a percentage difference between

the two analyses calculated. This was measured using a simple test-retest

agreement method to find the total % error between the original test (A), and re-test

(B) scores, using the following equation (O'Donoghue 2010):

( A - B )

Total % Error = 100 x

( A + B ) / 2


18/40



Page 15

Using this method, the percentage error was calculated by expressing the

absolute error between two values as a percentage of the mean of the two values.

Cooper et al. (2007) suggest that there should be an agreement of!90% between

the compared analyses, i.e. "10% error.

3.5b. Correlation Coefficient

Pearson's coefficient of correlation was used for inter-observer reliability, whereas

an Intraclass Correlation was used for intra-observer reliability. These tests

determine the association between the sets of data. A correlation of 0.0 represents

no association between the values, whereas a correlation of 1.0 denotes perfect

agreement, and therefore good reliability. Values of#0.7 indicate a moderate

correlation, #0.8 is good and #0.9 denotes a very good association.

3.6. Data Analysis

Descriptive statistics (means, medians and standard deviations) were provided to

present the findings. The Shapiro-Wilk test showed that the majority of dependent

variables were normally distributed. However, several variables were not normally

distributed. This, accompanied by the small sample size, suggested a non-parametric

approach. A Mann Whitney U test was used to identify any statistical differences

between winning and losing teams at a statistical significance level of 95%.Due to the small sample size, effect sizes were reported for each variable, which

will help to identify trends in the data. From these trends, suggestions could be made

as to what variables may be found to be significant had the study been conducted

using more matches. Effect size (ES) was calculated using Cohens (1969) equation:

ES = (Mean1 Mean2) / Mean Standard Deviation

Cohen reports figures of 0.2 (small ES), 0.5 (medium ES) and!0.8 (large ES) as

guidelines for interpreting effect size.

All data analysis was completed using Excel 2011 (Microsoft Corporation,

Washington, USA) and SPSS 20.0.0 (IBM Corporation, New York, USA).


19/40



Page 16

Table 3.1. Operational definitions of Primary match indicators

Primary Match

Indicator

Definition

Lineout success (%) A successful lineout is determined when the attacking

team win the ball cleanly, win a free kick or penalty, or

win the ball in any other way. An unsuccessful lineout is

when the defending team wins the ball in any of the

methods identified above.

Turnovers conceded When the attacking team give possession to the

opposition through an error on their part (accidental

offside, bad pass, carried dead, forward pass, unforced

knock-on, carried in touch) or due to good work from

the defending side (jackal, forced in touch, forced

knock-on).

Kick in play When an attacker kicks the ball in open play, from

either inside or outside of their 22-metre area.

Passes When an attacker attempts a pass to a teammate. This

can be positive, i.e. complete, or negative (incomplete,

off target, forward, intercepted).

Carries When an attacker carries the ball towards the defensive

line. Types include; Pick and go, One out drive, Support

carry, Kick return and Other carry.

Pick and Go When an attacker carries the ball from the base of a

ruck or a maul.

One Out Drive A carry made following one (usually short) pass from a

ruck or maul, often to a forward, who takes the ball into

contact.

Support Carry A carry made following an offload from a team mate, or

after a receiving a pass from a team mate who has

broken the defensive line.

Kick Return A carry made in the first phase following the reception

of a kick by the opposition.

Other Carry Any carry made that does not fit into one of the other

four types of carry.


20/40



Page 17

Ruck Frequency The number of rucks the attacking team completes

successfully throughout the match. A ruck is where two

or more players from each team come together over the

ball (which is on the ground), often following a tackle.

Table 3.2 Operational definitions of Secondary match indicators

Secondary Match

Indicator

Definition

Tackle Completion (%). A completed tackle occurs when the ball carrier is held

by one or more defenders, causing the ball carrier to go

to ground, stay on their feet, or offload the ball from thetackle area. This will be recorded as a percentage of

the total number of attempted tackles.

Attempted Tackles Frequency of tackles attempted. This includes

successful tackles where the tackle is completed, and

unsuccessful ones, i.e. missed tackles.

Initial Breaks. When the ball carrier cleanly breaks the defensive line,

gaining a better territorial position for their team.

Penalties conceded This is when the referee awards a penalty against theteam or player who have broken the laws of the game.

4.0. RESULTS

4.1. Reliability

Reliability analyses undertaken on the data were completed in order to provide

information as to the value of the presented results. Pearsons Correlation Coefficient

(for inter-observer), Intraclass Correlation Coefficient (for intra-observer) and

percentage error were provided for the variables. Table 4.1 presents intra-observer

values, taken from the re-analysis of three matches. Table 4.2 shows inter-observer

values obtained from a comparison of all matches with another analysts data.


21/40



Page 18

Table 4.1. Correlation Coefficients and % Error for Intra-observer analysis.

aIntraclass Correlation Coefficient

4.1a. Intra-observer Reliability

Results from the Intra-observer analysis in Table 4.1 indicate that the majority of

variables for winners and losers showed at least a moderate to good relationship.

Four variables (Initial Breaks, Penalties Conceded, Turnovers Conceded and Lineout

Success %) showed a perfect correlation (1.000) between the two analyses for

winners and losers. The lone irregular correlation was found for tackle completion %

for losers (0.588), which, although is positively correlated, shows a less than

moderate relationship between the original analysis and re-analysis. Percentage

Error was found to be within the 10% limit suggested by Cooper et al. (2007) for

100% of the variables. The smallest percentage error was 0%, found for winners and

losers in initial breaks, penalties conceded, turnovers Conceded and lineout success

%. The largest error was reported for total passes (4.92%) for losing teams, which is

still well within the recommended 10% limit.

Winners Losers

Variable Correlation

Coefficient a% Error Correlation

Coefficient a%

Error

Tackle Completion (%) 0.758 2.56 0.588 2.98

Total Carries 0.990 2.91 0.879 3.72

Total Passes 0.998 0.42 0.993 4.92

Initial Breaks 1.000 0.00 1.000 0.00

Total Kicks 0.929 2.06 0.969 2.44

Penalties Conceded 1.000 0.00 1.000 0.00

Turnovers Conceded 1.000 0.00 1.000 0.00

Ruck Frequency 0.993 1.45 0.993 0.70

Lineout Success (%) 1.000 0.00 1.000 0.00


22/40



Page 19

Table 4.2. Correlation Coefficients and % Error for Inter observer analysis.

Winners Losers

Variable Correlation

Coefficient b% Error Correlation

Coefficient b%

Error

Tackle Completion (%) 0.905 1.09 0.789 2.36

Total Carries 0.994 2.98 0.982 2.00

Total Passes 1.000 0.00 1.000 0.00

Initial Breaks 1.000 0.00 1.000 0.00

Total Kicks 0.992 1.67 0.999 0.53

Penalties Conceded 1.000 0.00 1.000 0.00

Turnovers Conceded 1.000 0.00 1.000 0.00

Ruck Frequency 0.998 1.61 0.995 1.95

Lineout Success (%) 0.976 1.46 0.981 1.59

bPearsons Correlation Coefficient

4.1b. Inter-observer Reliability

Table 4.2 shows how good to perfect correlation coefficients were reported for all

variables for the inter-observer analysis. Perfect correlations (1.000) were reported

for four variables for winners and losers (total passes, initial breaks, penalties

conceded and turnovers conceded). Percentage error was found to be well within the

10% error limit, with the largest error being reported for Total Carries (2.98%).

Overall, there was very good agreement between the two analyses.

4.2. Key Performance Indicators

Table 4.3 presents descriptive statistics (Median, Mean and Standard Deviation

(SD)) for winners and losers. It also shows effect sizes from Cohens dtest and P

values from the Mann Whitney U test for each of the analysed performance

indicators. Variables are displayed as either frequency counts or percentages.

Overall, there were very few significant differences between winners and losers,

although there did appear to be some trends in the data. Two variables were found to

be significantly different between winners and losers. Winners conceded a

significantly higher percentage of their penalties between 50m and the opposition

22m than losers (P= 0.026) (Figure 4.2c) and losers carried the ball significantly


23/40



Page 20

more than winners (P= 0.035). While the total number of carries was significantly

higher for losing teams, there was no noticeable difference in the types of carries

between the groups (Figure 4.1).

Figure 4.1. Pie chart showing comparison between winners and losers in terms of

each type of carry as a percentage of total number of carries.

Figure 4.2. Bar chart showing comparison between winners and losers in distribution

of a) Turnovers conceded, b) Ruck frequency and c) Penalties conceded, as a

percentage of total number of each event. (* = Statistical significance, P= 0.026).

* *


24/40



Page 21

Effect sizes in Table 4.3 showed that some variables differed between winning

and losing teams, although not statistically significantly. Losers conceded more

turnovers than winners (ES = 0.89, P= 0.153); Winners conceded a higher

percentage of their turnovers between their own 22m and 50m than losers (ES =

0.87, P= 0.205) (Figure 4.2a); Winners kicked the ball out of hand more often than

losers (ES = 0.82, P= 0.223); Losers attempted more passes (ES = 0.91, P=0.078)

and had a higher pass completion (ES = 0.85, P= 0.157) than winners; Losers

completed more rucks than winners (ES = 0.81, P= 0.152), although no difference

was noted in terms of where on the pitch these rucks took place (Figure 4.2b); Losers

conceded a higher percentage of their penalties between their own 22m and 50m

than winners (ES = 0.92, P= 0.088) (Figure 4.2c)despite there being no difference inthe total number of penalties conceded.

.


25/40

KeyP

erform

a

nceIn

dic

ators

thatdis

crimin

atewinning

an

dlo

sin

gin

thekn

ock

outstagesof

the2

011R

ugbyW

orld

Cup.

Luk

eBish

op

P

age22

Table4.3.D

ifferen

cesink

eyperform

an

cein

dicators

betweenwinn

ers

an

d

losers

durin

gthekn

ock

outstag

esof

the2

011R

ugbyW

orld

Cu

p.

Winners

Losers

Variable

Median

Mean

SD

Median

Mean

SD

EffectSize

P1Value

LineoutSucce

ss(%)

89.0

0

85.6

3

11.3

9

87.0

0

91.0

0

12.

80

0.11

0.6

6

4

TurnoversCon

ceded

13.5

0

13.7

5

3.11

16.5

0

17.1

3

4.4

5

0.89*

0.1

5

3

Own22m(%)

3.0

0

5.2

5

7.0

9

8.5

0

8.2

5

5.04

0.4

9

0.24

5

22m-50m

(%)

34.5

0

35.3

8

9.5

9

30.5

0

25.8

8

12.14

0.87*

0.2

0

5

50m-Opposition22m(%)

33.0

0

37.3

8

18.21

46.0

0

45.7

5

10.

59

0.5

8

0.27

7

Opposition

22m(%)

23.0

0

21.8

8

13.62

20.0

0

20.12

8.92

0.1

6

0.6

9

7

TotalKicksou

tofhand

28.5

0

29.6

3

5.21

26.0

0

24.4

0

7.3

3

0.82*

0.22

3

Kicksin22

m(%)

30.0

0

30.8

8

12.2

9

35.0

0

35.7

5

10.48

0.4

3

0.3

6

8

KicksinPlay(%)

70.0

0

69.1

3

12.2

9

65.0

0

64.2

5

10.48

0.44

0.3

6

8

Passes

97.0

0

100.3

8

39.81

154.5

0

139.0

0

45.

01

0.91*

0.07

8

PassComp

letion(%)

96.0

0

96.0

0

2.62

98.0

0

97.5

0

0.9

3

0.85*

0.1

5

7

Carries

92.0

0

91.0

0

26.27

114.0

0

11

6.8

8

32.

30

0.88*

0.035

**

PickandG

o(%)

16.0

0

15.6

3

7.87

12.5

0

14.1

3

6.62

0.21

0.64

4

OneOutDrive(%)

16.5

0

19.0

0

7.4

3

19.0

0

21.0

0

9.32

0.24

0.52

1

SupportCa

rry(%)

4.0

0

2.5

9

4.1

3

6.5

0

5.2

5

2.87

0.41

0.3

9

3


26/40

KeyPerforma

nceIndicatorsthatdiscriminatewinningandlosingintheknockoutstagesofthe2011RugbyWorldCup.

LukeBishop

P

age23

KickReturn

(%)

11.50

13.00

6.97

9.00

10.00

4.14

0.54

0.51

4

OtherCarry

(%)

48.50

47.38

7.37

48.00

49.50

8.00

0.28

1.00

0

RuckFrequency

79.00

76.00

24.75

100.50

98.00

29.49

0.81*

0.15

2

Own22m(%

)

4.50

7.00

6.46

5.00

6.75

6.32

0.04

0.90

4

22m-50m(%)

22.50

27.38

13.76

22.00

25.13

6.24

0.23

0.89

8

50m-Oppo

sition22m(%)

47.00

43.63

13.55

45.00

47.00

14.59

0.24

0.62

7

Opposition

22m(%)

27.00

22.00

11.16

17.50

21.25

13.59

0.06

0.62

5

TackleComple

tion(%)

92.00

91.75

4.37

91.00

90.50

2.45

0.37

0.52

2

AttemptedTac

kles

124.5

122.75

32.87

103.00

98.13

39.75

0.68

0.16

9

InitialBreaks

2.00

2.50

1.77

2.50

2.63

1.06

0.09

0.62

1

PenaltiesConceded

7.00

7.75

2.19

8.50

8.38

2.67

0.26

0.55

5

Own22m(%

)

12.00

16.00

16.66

15.50

14.63

11.41

0.10

0.81

3

22m-50m(%)

27.00

29.38

20.87

43.50

45.44

13.99

0.92*

0.08

8

50m-Oppo

sition22m(%)

35.50

39.13

18.31

19.50

22.00

10.59

1.09*

0.026

**

Opposition

22m(%)

13.50

15.50

13.62

10.50

17.19

8.92

0.10

0.89

7

1MannWhitneyU.

*LargeEffectS

ize(>0.80).

**SignificantP

value(


27/40



Page 24

5.0 DISCUSSION

5.1. Summary

The purpose of the study was to identify performance indicators that discriminate

between winners and losers in the knockout matches at the 2011 Rugby World Cup.

This would then allow suggestions to be made as to the tactical development of the

game, and recommendations to be made about what was the most appropriate

tactical approach during the 2011 Rugby World Cup knockout stages.

The findings from the study indicate that there were very few clear differences

between winning and losing teams. The only significant differences were that winners

conceded a higher percentage of their penalties between the halfway line and the

opposition 22m than losers, and losing teams carried the ball more often than

winners. In addition to these significant differences, it was found that following an

effect size calculation, other variables may have been significantly different between

winners and losers had there been a larger sample size (winning teams kicked the

ball out of hand more often than losing teams; winners conceded a higher

percentage of their turnovers between their 22m and the halfway line than losing

teams; losing teams conceded more turnovers, attempted more passes, had a higher

pass completion, had a higher ruck frequency and conceded a higher percentage of

their penalties between their 22m and the halfway line than winning teams).

5.2. Differences between winners and losers

Winning teams conceded a significantly higher percentage of their penalties

between halfway and the opposition 22m (mean = 39.13%) than losing teams (mean

= 22%). When comparing this to the finding that losing teams conceded a higher

percentage of their penalties between their 22m and halfway (mean = 45.44%)

compared to winning teams (mean = 29.38%), it could be suggested that pitch

location of penalties conceded had an influence on the match outcome. Conceding

penalties in the defending half increases the likelihood of conceding points through

subsequent goal kicks, or from a possible territorial loss. This supports the view that

winning teams are able to gain more penalties in attacking positions than losing

teams (Jones, Mellalieu and James 2004, Vaz et al. 2011). Therefore, it is suggested

that a teams ability to concede penalties further up the field, outside the point scoring

range of the opposition, is indicative of success during the knockout stages of the

2011 Rugby World Cup. The pitch position seems to be of more importance than


28/40



Page 25

simply the frequency of penalties conceded, where there was a negligible difference

between winning and losing teams.

Losing teams (mean = 17.13) were found to concede more turnovers than winning

teams (mean = 13.75), which supports findings from previous research (Ortega,

Villarejo and Palao 2009, Jones, Mellalieu and James 2004). There was found to be

minimal difference in terms of pitch position of turnovers conceded between winners

and losers apart from between 22m and halfway in the defending half. Winning

teams (mean = 35.38) conceded more turnovers here than losing teams (mean =

25.88). This finding was not anticipated as tries most frequently originate from this

area of the pitch (Van Rooyen, Lambert and Noakes 2000, Boddington and Lambert

2004). This would suggest that a higher number of turnovers conceded here is likely

to be related to unsuccessful performance. However, it appeared not to be the case

in the 2011 Rugby World Cup.

Overall, findings from the current study suggest that the number of penalties

conceded did not discriminate between winning and losing teams at the 2011 Rugby

World Cup. However, pitch position of penalties conceded did appear to be influential

on the match outcome. The capability of teams to concede a high percentage of their

penalties in more attacking positions in comparison to their opponents was shown to

differentiate winning and losing teams. Contrary to this, the frequency of turnovers

conceded seems to help discriminate winners and losers more so than the pitch

position that these turnovers take place. A lower frequency of turnovers conceded

was apparent in winning teams.

5.2a. Style of Play

Losing teams (mean = 116.88) carried the ball significantly more than winning

teams (mean = 91). However, there was found to be no differences in the types of

carries by either group (Figure 4.2), which would give an indication of their tactical

approach, i.e. more Pick and Go carries would indicate a more narrow, less

adventurous approach, whereas more Other Carries would indicate a more

expansive approach. It is suggested that winning and losing teams adopted a similar

approach in terms of their distribution of carry types. These findings do not support

Ortega, Villarejo and Palao (2009), who suggested that losing teams pick and go

more often than winning teams. This difference is possibly due to the analysis being

completed on different competition formats (6 Nations and World Cup) where it has

appeared different tactical approaches are needed. Losing teams (mean = 139) were

also found to pass the ball more often than winning teams (mean = 100.38), which


29/40



Page 26

supports Ortega, Villarejo and Palao's (2009) findings that losing teams pass the ball

more frequently than winning teams. Losing teams percentage pass completion was

higher than winning teams, which contradicts the findings of Ortega, Villarejo and

Palao (2009), who reported a lower percentage pass completion for losing teams.

This was an unanticipated finding, as pass completion and turnovers conceded are

somewhat related (Ortega, Villarejo and Palao 2009), so it was expected that a

superior pass completion would result in an inferior number of turnovers conceded.

Winners (mean = 29.63) were found to kick the ball more often than losers (mean =

24.40), which supports the previous findings of Ortega, Villarejo and Palao (2009)

and Vaz, Van Rooyen and Sampaiao (2010) who all found winning teams to kick the

ball as a form of attack more frequently than losing teams. It does however disagree

with Eaves, Hughes and Lamb (2005) who suggested that keeping possession was

more related to match success than kicking the ball. Although it was found that

winning teams kicked more often in attacking positions than losing teams, this

difference was minor. While this agrees with Van Rooyen and Noakes' (2006)

findings that pitch position of kicks is linked to success, the findings from the current

study suggest that the frequency of kicks was more influential on match outcome

than the pitch position of these kicks. The findings suggest that kicking as a form of

attack is more associated with success than choosing to run with the ball. This

suggests that the impact that the Experimental Law Variations (Table 2.2) were

designed to have, has maybe not worked. The laws introduced were designed to

reward teams who look to counter-attack and run with the ball as opposed to kicking

it. Based on the results of this study, it is suggested that running and passing the ball

is not linked with success, signifying that the new laws have not favoured teams who

play more expansively. Contrary to this, perhaps winning teams have simply adapted

better to the law change with the use of better tactical kicking than losing teams and

as a result are being rewarded for this.

Losing teams (mean = 98) completed more rucks than winning teams (mean =

76). These findings are in agreement with what was found by Van Rooyen, Diedrick

and Noakes (2010) who reported that lower ruck frequency was linked to success in

the knockout stages of a World Cup. However, the findings in the current study do

not agree with the suggestion made by these authors that an increase in ruck

frequency will be related to success in the 2011 Rugby World Cup due to the

influence of the Experimental Law Variations. This again suggests that the

Experimental Law Variations have not had the influence the IRB had intended, with

more attacking play not appearing to relate to match success. It does seem that


30/40



Page 27

these laws are having an influence on the overall development of the game though.

There was an average of 174 rucks per game in the 2011 knockout stages compared

to 121 rucks per game in the 2007 knockout stages. Thus, suggesting that while the

laws have had an impact on the number of rucks completed during a game, perhaps

making the game a better spectacle for supporters, it appears that a rucking,

possession based approach is not linked with match success.

In terms of the style of play required for success in the knockout stages of the

2011 Rugby World Cup, it seems a game based around good tactical kicking and

territory was more effective than a possession dominated game plan, built around

rucking and carrying the ball.

5.3. Similarities between winners and losers

There were some variables that were found to be similar between winning and

losing teams, despite what previous research had suggested. These variables were;

Lineout success %, tackle completion % and initial breaks.

Firstly, a somewhat unexpected result was that the difference in lineout success %

between winners (mean = 85.63%) and losers (mean = 91%) was statistically

insignificant. In fact, the findings show that losing teams actually had a slightly better

average lineout success %. This suggests that success in the lineout was not

essential for success in the knockout stages of the 2011 Rugby World Cup. These

findings contrast to several studies, which all reported that success at the lineout was

indicative of match success (Jones, Mellalieu and James 2004, Ortega, Villaerjo and

Palao 2009, Vaz, Van Rooyen and Sampaio 2010, Vaz et al. 2011). A possible

reason for the findings of the current study opposing previous research is that the

previous studies looked at different competition formats, namely 6 Nations and Super

12, rather than specifically knockout matches. This further supports the idea that

different match indicators may be important in different competition formats.

There was also found to be no clear difference in tackle completion % between

winning (mean = 91.75%) and losing teams (mean = 90.50%), which differs from

what was reported by Ortega, Villarejo and Palao (2009). Despite, the tackle

completion % being similar, it was noted how winning teams (122.75) attempted a

greater average number of tackles per game than losing teams (98.13), although this

was not reported to be a significant difference. This agrees with Vaz, Van Rooyen

and Sampaio (2010) who report that winning teams make a greater number of

tackles than losing teams. From these results it could be suggested that during the

knockout stages of the 2011 Rugby World Cup, tackle completion % alone was not


31/40



Page 28

enough to give an indication of success. However, combining the percentage with the

number of attempted tackles gives a greater indication of a team's defensive

capabilities. This study suggests that maintaining a high tackle completion % whilst

attempting a greater number of tackles than the opposition is related to match

success. However, due to the insignificant difference between winners and losers

here, it is suggested that further research into this area will give a greater

understanding of the importance of this performance indicator.

There was also found to no clear difference in the number of initial breaks made

per game by winning (mean = 2.50) and losing (mean = 2.63) teams. This agrees

with Wheeler, Askew and Sayers (2010), who reported that the number of line breaks

by a team was not associated with success in Rugby Union. It does however,

disagree with Ortega, Villarejo and Palao (2009) and Diedrick and Van Rooyen

(2011) who both report winning teams to successfully break the defensive line more

frequently. It could be suggested that due to the mixed results found regarding this

performance indicator, that it perhaps is not vital to match success. It is suggested

that the frequency of initial breaks has no influence on the outcome of knockout

matches at the 2011 Rugby World Cup. Further research into this area is certainly

required to gain a clearer understanding as to the influence of initial breaks on match

outcome.

These findings have suggested that, despite previous research suggesting

otherwise, lineout success %, tackle completion % and initial breaks do not

discriminate between winning and losing teams at the 2011 Rugby World Cup

knockout stages. However, a high tackle completion % combined with a greater

number of attempted tackles than the opposition may contribute to discriminating

between winners and losers.

5.4. Limitations and Future Recommendations

5.4a. Sample Size

One particular limitation regarding the methodology of the study was the sample

size. As only the knockout matches were analysed, any firm conclusions can only be

made specifically to the 2011 Rugby World Cup. The issue with a small sample size

is that there is often not enough data to represent a population, meaning that results

that are presented as being significant are sometimes not the case. Effect size was

calculated in order to give an indication as to the meaningfulness of results, which

helps to overcome the issue of a small sample. Although this calculation suggested


32/40



Page 29

that some variables (turnovers conceded, % turnovers conceded between own 22m

and halfway, total kicks out of hand, total passes, pass completion, ruck frequency,

% penalties conceded between own 22m and halfway) would possibly be significant

had the sample size been larger, this calculation is not without its flaws. Cohen's d

calculation uses the means of the two groups to calculate effect size. This means

that a solitary unusual value may considerably affect the results (Thomas, Nelson

and Silverman 2011). This could give a false indication of the meaningfulness of the

results. Therefore, it is recommended that conclusions from this study should only be

applied to the 2011 World Cup and not to knockout rugby as a whole.

5.4b. Reliability

Overall, the reliability of the study was sound. However, what was unexpected was

that the inter-observer analysis showed more reliable results than the intra-observer

analysis (Table 4.1 and 4.2). A possible explanation for this was that the intra-

analysis was undertaken on only three matches in comparison to eight matches for

the inter-observer test, meaning that the slightest discrepancies were likely to be

magnified due to the very small sample size. It is recommended that reliability

analysis should be undertaken on the same size sample for both inter-observer and

intra-observer tests. Generally, the reliability of the study was strong, giving an

indication that the results presented possess good validity.

5.4c. Future Recommendations

As this was the first study to look at identifying what performance indicators

discriminate winners and losers in knockout rugby, specifically the 2011 World Cup, it

can act as a starting point for future research into this specific area. The review of

relevant literature, asserted by the findings from this study, have suggested that

performance indicators that relate to match success are varied across competition

formats. Therefore, future research should look to compare similar competition

formats to attempt to build profiles of what match indicators are key in each

respective format. It is suggested that to enhance the understanding of what

performance indicators are key in knockout rugby, follow-up studies should be

completed looking at recent knockout competitions, i.e. Heineken Cup, domestic

league play-off matches. This will then create a larger pool of information, meaning

better-founded conclusions can be made as to what is the best tactical approach in

knockout rugby. It is also recommended that a review of knockout rugby takes place


33/40



Page 30

annually in order to maintain awareness of the development of the game in this

format.

5.5. Conclusion

This study presents game statistics that help to identify what performance

indicators discriminate winning and losing teams during the knockout stages at the

2011 Rugby World Cup. These statistics can help to guide possible training

interventions for sides competing in future knockout tournaments. Key conclusions

are presented along with practical recommendations for each point.

Firstly, the results of the study showed that the Experimental Law Variations had

influenced the game in terms of the number of rucks that took place per game,

signalling that teams were more prepared to carry the ball at the 2011 World Cup in

comparison to previous years. However, despite this increase, it appeared that

carrying and rucking the ball was not the best approach to win matches. It was found

that kicking the ball, as a form of attack was more suitable, advocating a more

territory-based approach as opposed to a possession one. Therefore it is suggested

that kicking-based drills are included as an important part of training sessions prior to

knockout competitions.

In addition to this, a teams ability to concede fewer turnovers than the opposition

is related to match success. Therefore, offence sessions based around rucking and

ball retention in contact will help to reduce the incidence of conceding turnovers at

the breakdown. Defence sessions based around competing at rucks, slowing the ball

down and the jackal technique will also help to improve the chances of forcing

turnover ball from the opposition.

Conceding more penalties further away from the defensive try line also

discriminated between winning and losing teams at the 2011 Rugby World Cup. A

possible approach to help reduce the penalty frequency in the defensive half would

be to not commit many players to the breakdown when on defence. Fewer players

competing at the breakdown and more players set up in the defensive line will lead to

a reduction in the chance of conceding a penalty, whilst the defensive line will not be

outnumbered should the opposition move out wide in attack.

In summary, this study shows that there were few significant differences between

winning and losing teams at the 2011 Rugby World Cup. However, it was identified

that the most appropriate form of attack was kicking the ball more frequently,

whereas rucking, carrying and passing the ball more frequently was a less successful

approach. A good tactical kicking game, in combination with the ability to concede


34/40



Page 31

few turnovers and concede a higher proportion of penalties in the attacking half were

related to match success. A good defence was also found to have some importance,

with winning teams able to complete a similar percentage of tackles to losing teams,

whilst attempting a greater amount of tackles. It is important to note that the findings

of this study should be used carefully. They only represent the knockout stages at the

2011 World Cup, so to generalise them across different competition formats is ill-

advised. Further research should look at other knockout competitions in an attempt to

find the most effective tactical approach to knockout rugby.


35/40



Page 32

References

ATKINSON, G. and NEVILL, A. M. (1998). Statistical methods for assessing

measurement error (reliability) in variables relevant to sports medicine. Sports

Medicine (Auckland, N.Z.), 26 (4), 217-217.

AUSTIN, Damien, GABBETT, Tim and JENKINS, David (2011). The physical

demands of Super 14 Rugby Union. Journal of science and medicine in sport, 14 (3),

259-263.

BARTLETT, R. (2001). Performance analysis: Can bringing together biomechanics

and notational analysis benefit coaches? International Journal of Performance

Analysis in Sport, 1 (1), 122-122.

BLAND, J. Martin and ALTMAN, Douglas G. (2010). Statistical methods forassessing agreement between two methods of clinical measurement. International

Journal of Nursing Studies, 47 (8), 931-936.

BODDINGTON, Michele and LAMBERT, Mike (2004). Quantitative and qualitative

evaluation of scoring opportunities by South Africa in World Cup Rugby

2003. International Journal of Performance Analysis in Sport, 4 (2), 32-32.

BRACEWELL, P. (2002). Implementing statistics in a diagnostic coaching structure

for Rugby. Res. Lett. Inf. Math. Sci., 3 (1), 79-84.

COHEN, J. (1969). Statistical power analysis for the behavioral sciences. San Diego,

CA: Academic Press.

COOPER, Stephen-Mark et al. (2007). A simple statistical method for assessing the

reliability of data entered into sport performance analysis systems. International

Journal of Performance Analysis in Sport, 7(1), 87-87.

CSATALJAY, Gabor et al. (2009). Performance indicators that distinguish winning

and losing teams in basketball. International journal of performance analysis in

sport, 9 (1), 60-60.

CUNNIFFE, Brian et al. (2009). An evaluation of the physiological demands of elite

rugby union using global positioning system tracking software. Journal of Strength

and Conditioning Research / National Strength & Conditioning Association, 23 (4),

1195-1203.

DIEDRICK, Emile and VAN ROOYEN, Michele (2011). Line Break Situations in

International Rugby. International Journal of Performance Analysis in Sport, 11, (13)

522-534


36/40



Page 33

EAVES, Simon and HUGHES, Mike (2003). Patterns of play of international rugbyunion teams before and after the introduction of professional status. InternationalJournal of Performance Analysis in Sport, 3 (2), 103-103.

EAVES, Simon, HUGHES, Mike and LAMB, Kevin (2005). The consequences of theintroduction of professional playing status on game action variables in internationalnorthern hemisphere Rugby Union Football. International Journal of PerformanceAnalysis in Sport, 5 (2), 58-58.

FRANKS, I.M. and MILLER, G. (1991). Training coaches to observe andremember. Journal of Sports Sciences, 9 (3), 285-297.

GRHAIGNE, J. F., BOUTHIER, D. and DAVID, B. (1997). Dynamic-system analysisof opponent relationships in collective actions in soccer. Journal of Sports

Sciences, 15 (2), 137-149.

HUGHES, Mike (2004). Performance analysis - a 2004 perspective. InternationalJournal of Performance Analysis in Sport, 4 (1), 103-103.

HUGHES, Mike and BARTLETT, Roger (2002). The use of performance indicators inperformance analysis. Journal of Sports Sciences, 20 (10), 739-754.

HUGHES, M., COOPER, S-M. and NEVILL, A. (2002). Analysis procedures for non-parametric data from performance analysis. International Journal of PerformanceAnalysis in Sport, 2 (1), 6-6.

IRB (2003). RWC 2003: Statistical Review and Match Analysis. [online]. Lastaccessed 25th March 2012 at:http://www.irbplayerwelfare.com/pdfs/gameanalysis/RWC_Analysis_2003.pdf

IRB (2005). Changes in the Playing of International Rugby Over a 20 Year Period.[online]. Last accessed 25th March 2012 at:http://www.irb.com/mm/document/newsmedia/0/comparison1983to2003_541.pdf

IRB (2007). Statistical Review and Match Analysis: IRB game analysis. [online]. Lastaccessed 25th March 2012 at:http://www.irb.com/mm/document/home/0/071031ctrwc07statscomm_3811.pdf

IRB (2008). The IRB Guide to Experimental Law Variations. [online]. Last accessed19th March 2012 at:http://www.irb.com/mm/document/newsmedia/0/irbelvguideenfinal_5097.pdf

JAMES, Nic, MELLALIEU, Stephen D. and JONES, Nicholas M. P. (2005). Thedevelopment of position-specific performance indicators in professional rugbyunion. Journal of Sports Sciences, 23 (1), 63-72.


37/40



Page 34

JAMES, Nic, TAYLOR, Joe and STANLEY, Simon (2007). Reliability procedures for

categorical data in performance analysis. International Journal of Performance

Analysis in Sport, 7 (1), 1-1.

JENKINS, Rhian, MORGAN, Lynsey and O'DONOGHUE, Peter (2007). A case study

into the effectiveness of computerised match analysis and motivational videos within

the coaching of a league netball team. International Journal of Performance Analysis

in sport, 7 (2), 59-59.

JONES, Nicholas, MELLALIEU, Stephen and JAMES, Nic (2004). Team

performance indicators as a function of winning and losing in rugby

union. International Journal of Performance Analysis in Sport, 4 (1), 61-61.

LAGO-BALLESTEROS, J. and LAGO-PENAS, C. (2010). Performance in teamsports: Identifying the keys to success in soccer. Journal of Human Kinetics, 25 (3),

85-91

LAGO-PENAS, C., LAGO-BALLESTEROS, J. and REY, E. (2011). Differences in

performance indicators between winning and losing teams in the UEFA Champions

League. Journal of Human Kinetics, 27 (3), 137-148.

LAMES, Martin and MCGARRY, Tim (2007). On the search for reliable performance

indicators in game sports. International Journal of Performance Analysis in

Sport, 7 (1), 62-62.

LIEBERMANN, Dario et al. (2002). Advances in the application of information

technology to sport performance. Journal of Sports Sciences, 20 (10), 755-769.

MCGARRY, Tim (2009). Applied and theoretical perspectives of performance

analysis in sport: Scientific issues and challenges. International Journal of

Performance Analysis in Sport, 9 (1), 128-128.

O'DONOGHUE, Peter (2006). The use of feedback videos in sport. International

journal of Performance Analysis in Sport, 6 (2), 1-1.

O'DONOGHUE, Peter (2010). Research methods for sports performance

analysis. London, Routledge.

ORTEGA, E., VILLAREJO, D. and PALAO, J.M. (2009). Differences in game

statistics between winning and losing rugby teams in the Six Nations

tournament. Journal of Sports Science and Medicine, 8 (4), 523-527.

PRIM, Sebastian, VAN ROOYEN, Michele and LAMBERT, Michael (2006). A

comparison of performance indicators between the four South African teams and the


38/40



Page 35

winners of the 2005 super 12 rugby competition. What separates top from

bottom? International Journal of Performance Analysis in Sport, 6 (2), 126-126.

REILLY, Thomas and GILBOURNE, David (2003). Science and Football: A review of

applied research in the football codes. Journal of Sports Sciences, 21 (9), 693-705.

SASAKI, Koh et al. (2007). Scoring profiles and defence performance analysis in

Rugby Union. International Journal of Performance Analysis in Sport, 7 (3), 46-46.

STANHOPE, J. and HUGHES, M. (1997). An analysis of scoring in the 1991 rugby

union World Cup for men. In Hughes, M. (Ed.) (1997). Notational Analysis of Sport I

& II. University of Wales Institute Cardiff. Cardiff, Wales, pp. 167 176.

Luke Bishop Research Project

Documents

Transcript of Luke Bishop Research Project