jurnal novi 7.pdf

8/18/2019 jurnal novi 7.pdf

1/18

R E V I E W A R T I C L E

‘‘Not Just Little Adults’’: Qualitative Methods to Supportthe Development of Pediatric Patient-Reported Outcomes

Rob Arbuckle • Linda Abetz-Webb

Published online: 3 August 2013

Springer International Publishing Switzerland 2013

Abstract The US FDA and the European Medicines

Agency (EMA) have issued incentives and laws mandatingclinical research in pediatrics. While guidances for the devel-

opmentand validation of patient-reported outcomes (PROs) or

health-related quality of life (HRQL) measures have been

issued by these agencies, little attention has focused on pedi-

atric PRO development methods. With reference to the liter-

ature, this article provides an overview of specific

considerations that should be made with regard to the devel-

opmentof pediatricPRO measures,with a focus on performing

qualitative research to ensure content validity. Throughout the

questionnaire development process it is critical to use devel-

opmentally appropriate language and techniques to ensure

outcomes have content validity, and will be reliable and valid

within narrow age bands (0–2, 3–5, 6–8, 9–11, 12–14, 15–17

years). For qualitative research, sample sizes within those age

bands must be adequate to demonstrate saturation while taking

into account children’s rapid growth and development. Inter-

view methods, interview guides, and length of interview must

all take developmental stage into account. Drawings, play-doh,

or props can be used to engage the child. Care needsto be taken

during cognitive debriefing, where repeated questioning can

lead a child to change their answers, due to thinking their

answer is incorrect. For the PROs themselves, the greatest

challenge is in measuring outcomes in children aged 5–8 years.

In this age range, while self-report is generally more valid,

parent reports of observable behaviors are generally more

reliable. As such, ‘team completion’ or a parent-administered

child report is often the best option for children aged 5–8 years.

For infants and very young children (aged 0–4 years), patient

rating of observable behaviors is necessary, and, for adoles-

cents and children aged 9 years and older, self-reported out-comes are generally valid and reliable. In conclusion, the

development of PRO measures for use in children requires

careful tailoring of qualitative methods, and performing

research within narrow age bands. The best reporter should be

carefully considered dependent on the child’s age, develop-

mental ability, and the concept being measured, and team

completion should be considered alongside self-completion

and observer measures.

Key Points for Decision Makers

• Developing PROs for use in children is more complex thandeveloping in adults—the developmental stage of the child

must be considered throughout.

• During PRO development it is important that qualitative

methods are tailored to the age group—for younger children

(aged 6–9 years) the use of drawings, props and toys can

especially help to elicit information

• For the instruments themselves, when obtaining ratings for

infants and younger children (aged \5 years), an observer

report, usually completed by the parent, is necessary

• For children aged 6–9 years, either a parent observer report, a

parent-administered but child-completed PRO or an entirely

child-completed PRO can be appropriate, depending on the

concept

• Older children/adolescents can answer themselves but the

content and wording of questions might still be different fromthat used in adult measures

1 Background

1.1 Pediatric Legislation (‘Carrot and Stick’)

Historically, relatively few clinical studies were conducted

to evaluate the efficacy of pharmaceutical treatments in

R. Arbuckle (&) L. Abetz-Webb

Adelphi Values Ltd, Grimshaw Lane, Bollington,

Cheshire SK10 5JB, UK

e-mail: [email protected]

Patient (2013) 6:143–159

DOI 10.1007/s40271-013-0022-3


2/18

pediatric populations. The reasons included ethical con-

cerns about testing experimental treatments with children

and an expectation of limited return on investment. In

addition, the challenges posed by the continuous growth

and development of children makes dosing and staging of

disease difficult, and increases the sample sizes needed to

demonstrate safety and efficacy in all age groups. Devel-

opmental changes and their impact on children’s physicaland cognitive capabilities are additional challenges for self-

report endpoints in trials [1, 2].

The result is that many treatments are used ‘off-label’ in

pediatric populations—without marketing authorization or

licensing and without evidence of safety and efficacy. The

lack of pediatric clinical trials has resulted in misdosing of

children and withholding therapies from children because

there is insufficient evidence to make a valid risk/benefit

evaluation [3].

In response, between 2002 and 2007 both the US FDA

and European Medicines Agency (EMA) have issued leg-

islation to stimulate pediatric research [4–9].These legislations require by law that sponsors initiate a

pediatric development program for any drug which could

benefit children (characterized as ‘the stick’). If the sponsor

carries out that pediatric program to the satisfaction of the

respective regulators, the sponsor is then rewarded with a

6-month patent extension for marketing exclusivity in the

EU and US markets (‘the carrot ’). The extension is further

extended to 2 years for orphan drugs.

This legislation has generally been deemed a success,

with rapid increase in the number of treatments for which

pediatric studies are being conducted, or are at least plan-

ned [10–12].

1.2 Patient-Reported Outcomes (PRO)/Health-Related

Quality of Life (HRQL) Guidances

In adult indications, the use of patient-reported outcomes

(PROs) to support labelling claims has been the focus of a

great deal of interest from both regulators and the research

community in recent years [13–16]. In particular, the FDA

PRO guidance [14] and, to a lesser extent, the EMA HRQLreflection paper [13] have been influential in establishing

and driving standards for the development, validation and

use of PROs in clinical trials and other research studies. In

particular, the FDA has placed far greater importance on

input of patients to ensure the content validity of PROs

used to support label claims [17]. Figure 1 illustrates the

iterative process that the FDA recommends for the rigorous

development of a PRO instrument that will be used to

support product label claims.

However, the development, validation, and use of PROs

in pediatric studies has received less attention. One para-

graph in the FDA PRO guidance makes reference to thespecific considerations that must be taken into account for

pediatric PROs [14]. This paragraph highlights the need for

development within narrow age groupings and discourages

‘‘proxy-reported’’ outcome measures, stressing that obser-

ver reports that focus on events or behaviors that can be

observed should be used for patients who cannot respond

themselves. The EMA HRQL reflection paper makes no

specific mention of pediatric HRQL outcomes [13]. How-

ever, there is repeated acknowledgement in the literature

that there are many complicating issues and specific con-

siderations that must be taken account of in pediatric PRO

development [10, 18–20].

Fig. 1 Overview of the

pediatric patient-reported

outcome development process

144 R. Arbuckle, L. Abetz-Webb


3/18

Matza et al. [18] and, more recently, Solans et al. [21]

reviewed the state of the art in terms of pediatric HRQL

measurement, while Rothman and Kleinman [19] have

provided a summary of the expectations of regulators. In

addition, survey development theorists [22–24] provide

valuable guidance regarding cognitive interviewing in

pediatrics, and Bevans et al. [1] recently published a

valuable paper focussed on the conceptualization andunderstanding of child health, with guidance regarding

testing psychometric properties, and some suggestions

regarding the capabilities of different ages of children to

complete self-report measures. However, since the FDA

and EMA guidance documents have been released (in 2009

and 2005, respectively), there have been relatively few

publications in the literature that provide detailed guidance

regarding qualitative research methods for performing

concept elicitation and cognitive testing to support the

development of pediatric PRO measures.

Thus, in a rapidly evolving PRO development context,

there is a need for debate regarding specific considerationsfor pediatrics. How should qualitative PRO development

methods be tailored for pediatric populations? Are there

content validity considerations that are specific to the

development of pediatric PROs? For the instruments

themselves, at what age/developmental stage are children

too young to provide reliable reports? For these children,

how can we evaluate important concepts that cannot be

observed by caregivers or clinicians?

The objective of this article is to provide an overview of

specific methods and considerations for performing quali-

tative research to support the development of pediatric PRO

instruments. The methods described are particularly relevant

for pediatric PROs that will form clinical trial endpoints, but

can be relevant for PROs that will be used for other purposes.

With reference to examples from the literature and our own

research, the aim is to outline the ‘state of the art’ in terms of

qualitative methods for pediatric PRO development, and to

stimulate debate and research in this important area.

2 Pediatric Developmental Milestones and Age Ranges

for Pediatric PRO Development and PRO

Completion

The FDA PRO guidance for industry acknowledges that, for

children and adolescents, ‘‘instrument development and

validation testing within fairly narrow age groupings is

important to account for developmental differences and to

determine the lower age limit at which children can under-

stand the questions and provide reliable and valid responses

that can be compared across age categories’’ [14]. However,

to the knowledge of the present authors, the FDA has not

made any public statements regarding what the ‘‘fairly

narrow age groupings’’ should be, or what the lower age

limits should be.

It is generally acknowledged that the exact cut-offs for

age ranges will vary according to the disease population,

the context of use, and the concept targeted for measure-

ment. It is of relevance here to consider the literature

regarding pediatric developmental milestones, in order to

determine what children of different ages are likely to becapable of [1, 25]. However, it is important to bear in mind

that all boundaries are ‘fuzzy’ and in fact there can be wide

variation among different children of the same age in terms

of their motor, cognitive, and linguistic capabilities [26–

31]. Abilities will vary depending on genetic differences,

learning, experience, and socio-environmental factors [31].

A task that might be easy for one 6-year-old child to

complete may be very challenging for a less cognitively

advanced 8-year-old.

It should be noted that there are two separate issues to

consider here: the age at which a child can provide reliable

information in an interview (as part of PRO development)and the age at which a child can reliably respond to a

questionnaire which is an outcome measure in a clinical

trial or any other large, quantitative study.

2.1 Youngest Age at Which a Child can Report or be

Interviewed

For an understanding of child developmental milestones,

the Piagetian theory and information processing theory

provide competing theoretical perspectives. Piaget sug-

gested that a child’s cognitive development progresses

across the following four stages: sensori-motor develop-

ment, pre-operational thought (broadly age 2–6 years),

concrete operational thought (4–11 years), and formal

thought (12–17 years). In contrast, the information pro-

cessing theory posits that cognition evolves as children

develop improvements in attention, memory, and problem-

solving skills separately [32]. While no formal develop-

ment stages are outlined, the information processing theory

proposes that improvements in cognition develop during

middle childhood where the child is able to learn new

memory strategies (e.g. rehearsal), pay attention to relevant

information, and widen their knowledge basis through their

experiences [33]. Also of relevance for younger children is

the development of a ‘theory of mind’—the ‘‘ability to

attribute mental states to oneself and others’’—which is

considered to be central to the development of a child’s

reasoning, ability to relate to others, and development of

representation and language [34, 35]. Theorists suggest that

‘theory of mind’ and representational abilities initially

develop around the age of 3–5 years but that the ability to

introspect about one’s own thoughts does not seem to fully

develop until ages 6–8 years.

Not Just Little Adults 145


4/18

So, at what age can children self-report or be inter-

viewed? Matza et al. [18] suggest that children can begin

reporting on concrete domains between the ages of 4 and 6

years, whereas elsewhere it has been suggested that below

the age of 7 years children do not have sufficient cognitive

skills to be effectively and systematically questioned [24].

Borgers et al. [31] argue that the stages of cognitive

development are generally more overlapping than Piagetsuggested, but still argue that they provide a useful

framework, supporting the age of 6 years as a cut-off for

self-reporting [26, 27]. At the age of 5–6 years children can

talk about people and events, make drawings, and re-enact

events in play—all skills important for participation in

concept elicitation interviews. However, before the age of

6 years children do not necessarily take into account what

the listener can and cannot know, and may not understand

the difference between the past, present, and future. Thus,

the child may be able to understand and report on how they

feel right now, but have difficulty reporting on their

symptoms or quality of life over the previous week ormonth.

During middle childhood, children begin to think logi-

cally and are able to apply rules among objects and events

in the real world. Changes in perceptual and cognitive

abilities between the ages of 5 and 7 years include learning

to tell right from left and being able to draw the ‘biggest’

and ‘smallest’ squares [36, 37]. Before age 7 years most

children are not concerned if they have contradicted

themselves; after this age they avoid contradiction. From

an information processing perspective, children aged 5–7

years develop the ability to screen out irrelevant informa-

tion, learn new memory strategies, and have a better

understanding of their memory ability (meta-memory).

Such skills are relevant to concept elicitation and cognitive

interviews in which children are asked about their ability to

complete a PRO and report on their recall ability. Children

at this age are better able to recall specific concrete events

in time (e.g. the last time they threw up, were in hospital),

compared with recalling a more vague timeframe that

might require them to average over several days (e.g. the

last week).

Abstractions and hypothetical situations are still not

understood well. This can pose significant challenges for

cognitive debriefing of questionnaires, which often relies

on hypothetical questioning. It should also be noted that

children aged 4–7 years are highly suggestible [31] and

will often give interviewers responses that they want to

hear, and so care must be taken to ask questions in a neutral

manner. Children of this age can be very literal and may

have difficulty with questions that use logical operators

such as ‘or’ or negations [24]. Double negatives should

also be avoided. These findings provide support for the cut-

off for a child being able to be interviewed on their own

falling somewhere between ages 6 and 8 years. At 5 years

of age it is unlikely that linguistically and cognitively the

child will able to self-report on their own, but by 8 years of

age the majority of children will have moved to Piaget’s

concrete operational stage and will be able to self-report on

simple concepts asked in questions that are simply and

clearly worded.

Finally, adolescents are best able to think like adults andrespond to probes in an interview and self-report on

questionnaires. Adolescents are far better able to recall

over longer periods (e.g. 7 days) and to rate more abstract

concepts. Adolescents are able to understand questions in

an interview that ask about hypothetical events, e.g. ‘Tell

me about how you would answer this question if you had a

really bad asthma attack?’ In an interviewing context,

adolescents are much better at understanding the purpose

of the interview and what information is required. Com-

pared with children, adolescents are generally more

focused on the task and tend not to talk about irrelevant

topics. ‘Yes’ or ‘no’ answers are less frequent and, instead,adolescents are able to offer helpful suggestions and

insights.

Nevertheless, the issues that are most important to

adolescents may be quite different from adults. Adoles-

cents are developing their ego identities and peer percep-

tions become of paramount importance. Health problems

that impact on their body image or ability to socialize with

peers can be of far greater importance than other issues.

This egocentric approach will play a predominant role in

their answers to questions and the way in which a ques-

tionnaire should be worded. Likewise, adolescents may

become insulted if they perceive that they are given a

‘children’s questionnaire’ with faces as response continu-

ums or the like.

In summary, a broad age range of between 6 and 8 years

has strong support as the youngest age at which a child can

be meaningfully interviewed, and asked to self-report on a

PRO. Some researchers have shown that children as young

as 4 or 5 years of age may be capable of providing some

information on concrete aspects of their health status [10,

18, 38]. It has also been reported that children as young as

4 years can report pain [39]; however, children this young

will likely have difficulty with more complex questions.

As noted above, the developmental stages that each

child goes through must be taken into account when

designing both interview guides for qualitative research

and PRO questions. Table 1 provides a summary of

developmental milestones, appropriate qualitative methods

for questionnaire development, and who are appropriate

reporters for different age groups. Where health conditions

impact on the cognitive or physical development of the

child, (e.g. Down’s Syndrome, autism), these chronological

milestones may not apply. Details regarding questionnaire



5/18

development methods and the optimal reporters are pro-

vided in the remainder of the paper.

3 Methods for the Development of Pediatric PRO

Instruments

3.1 Reviewing the Literature and Pediatric ConceptElicitation Interviewing

In adults or children, the first steps in developing or

selecting a PRO must include identifying the con-

cept(s) targeted for measurement. The next step is to obtain

concept-relevant information to support the development

of a conceptual framework, item and response options to

measure each concept adequately [14]. This can be done

through review of the literature, and/or by performing

qualitative research with appropriate reporters (in pediat-

rics this can include children, parents/caregivers, and

occasionally other reporters such as teachers or clinicalstaff). Methods for reviewing the literature are much the

same in pediatrics as in adults. However, there is the added

challenge that the qualitative literature (which is preferable

for identifying concepts) in pediatrics is often sparse, and

rarely breaks findings down into the narrow age bands

required for thorough instrument development. One option

is to consider the ‘grey literature’ and online blogs and

chatrooms as additional sources of information. Otherwise,

the limited pediatric literature available places further

importance on conducting rigorous, open-ended qualitative

research. When conducting such research, the child’s age

and developmental stage must be taken into account and

methods tailored appropriately. Below we outline key

considerations.

3.1.1 Qualitative Research: Interviews vs. Focus Groups

In adults, open-ended qualitative research typically

involves conducting either interviews or focus groups (or

some combination of the two methods). Consideration of

appropriate methods for qualitative research with children/

adolescents should be based on their competencies and

interests [40], and not on the assumption that what is

appropriate for adults is also appropriate for children.

Focus groups with children have been successfully used to

support pediatric PRO development [41, 42]. It has been

suggested that focus groups have the advantage that they

remove the emphasis on the adult–child relationship, per-

haps reducing the likelihood that participants will respond

in the way they think the adult would desire [43]. There is

less pressure on a child to respond to every question,

reducing the chance that children will respond to questions

they do not fully understand [44]. It has also been

suggested that focus groups have the advantage of

acknowledging that the participants are the experts; how-

ever, the same could be equally said of one-to-one inter-

views if they are framed appropriately [45]. Focus groups

have also been suggested to generate greater depth due to

group members sparking ideas off one another, although

there has been found to be little empirical evidence in

support of this idea [46].However, there is reason for caution in conducting focus

groups in children and adolescents. Intimidation within the

group setting may inhibit interaction [43, 44], and this is

even more likely where children of mixed ages are inclu-

ded within a single group. Furthermore, children in focus

groups are more likely to adopt previously raised themes

than suggest their own [43, 44]. Even where children are of

the same age, conformity to peer pressure can exert con-

siderable influence on the comments individual children/

adolescents make [24]. Even adults are susceptible to social

desirability bias in group situations [47, 48], and it is likely

that this is further amplified in children and adolescents[24, 49]. This is of most concern among young adolescents

when peer pressure is perhaps most influential [43, 50].

However, Donaldson [51] has provided evidence that

younger children are also extremely suggestible, and may

give answers that they think adults are looking for if

questions are not posed in a very neutral way (a view

corroborated by survey development theorists) [24, 31].

While an interviewer can try to control such bias in a one-

on-one interview situation, it may be more difficult in a

focus group. In terms of practicalities, for children younger

than 12 years of age, it can be difficult to keep the attention

of all children in a focus group situation, even if focus

groups are kept relatively small in size (i.e. not more than

four participants). For all of these reasons, we suggest one-

on-one interviews in preference to focus groups in pediatric

populations.

If, because of practical and/or financial limitations,

focus groups must used, then limiting the size of the groups

is recommended. Heary and Hennessy [43] suggest the

optimum size is smaller than for adults, and propose

between four and six participants. Participants in each

group should be broadly similar in age, for the reasons

suggested above, and also due to developmental and cog-

nitive differences among children of different ages [43,

50]. Conducting groups with participants all of the same

gender should also be considered, particularly for sensitive

topics or health conditions, such as dermatological, uro-

logical, or bowel conditions [43, 50, 52].

3.1.2 Who to Interview?

There is debate in the pediatric PRO literature regarding

who is the most appropriate respondent, and who will



6/18

Table 1 Summary of developmental milestones, appropriate methods for questionnaire development, and who are appropriate reporters for

different age groups [22, 24, 31, 104]

Stage/

age

range

(years)

Developmental milestones Qualitative questionnaire development

methods

Best reporter in outcome measure

0–2 Sensori-motor stage:

Differentiates self from objects and recognizes

self as agent of action. Towards the end,

achieves object permanence, realising that

things continue to exist even when no longer

present

Parent/caregiver interviews or focus

groups only—cannot interview childdirectly

Child cannot self-report, must rely on a

parent/caregiver-completed outcomemeasure focussed on observable

behaviors

3–5 Preoperational stage:

Acquisition of language is primary cognitive

development task. Able to formulate mental

representations of objects that are not there

and to understand that symbols that stand for

something, such as a red light meaning ‘stop.’

Able to report how they feel right now, but

any recall likely to be challenging. Prone to

precausal reasoning: unable to distinguish

between psychological and physical causes.Likewise, thinking is based on concrete

mental images and memory is limited

Usually child interviews are not

recommended and only parent/

caregiver interviews or focus groups

are conducted

Can consider interviewing parent and

child together, but can only ask the

child about very simple concepts, and

about what they are experiencing at

that moment as accurate recall for

children of this age is limited

For most concepts, child is unable to

self-report, must rely on parent/

caregiver-completed outcome measure

focussed on observable behaviors

Parent can be asked to report on what

their child has told them (e.g. ‘Did

your child tell you he/she had a

stomach ache today?’)

For simple concepts (e.g. presence of

pain), can perhaps ask the child toreport in an interview-administered

format

6–11 ‘Concrete operational stage’ where logical

thinking becomes possible. These children

think more like adults, although they do tend

to think in terms of ‘black’ and ‘white’ rather

than grey areas in between. Capacity to store

and manipulate information still limited. At

this stage, the child can think back and recall

over short timeframes. However, he/she is

limited to thinking back to events that are

concrete. Abstractions and hypothetical

situations are not understood well

Recommend interviewing both the child

and a parent/caregiver (in separate

interviews)

For child interviews, must keep

questions simple and direct, focussed

on the here and now, and tangible,

salient concepts as much as possible.

Interviewer must be careful not to lead

the child. Parent should not be present

Use of toys and drawing activities

recommended to keep the child’s

attention

Recommend including both a child-

completed patient-reported outcome

(PRO) measure and a parent/caregiver-

completed measure. Which is

considered ‘primary’ will depend on

the concepts being measured and the

findings of the qualitative development

research (including both open-ended

interviews and cognitive interviews)

Can be valuable to ask the parent to

remind the child to complete the PRO

and regarding salient events relevant tothe child’s symptoms, but the child

should determine responses to each

question. Alternatively, child measure

can be interviewer administered

Child-completed measure must use very

simple, direct language, recall period

should not be more than 24 h for most

concepts and instantaneous/’right now’

recall periods are preferable

12–14 Formal operational:

Able to think logically and respond to

questionnaires. Can start to understand

‘hypothetical’ questions. Capacity to

temporarily store and process informationconsiderably superior to that of younger

children. Can think like adults but the issues

that are most important may be quite

different to adults. Developing ego identity

and peer perceptions are of paramount

importance. Thus, symptoms that inhibit

participation in social activities or are

embarrassing are likely to be perceived as the

most bothersome. This egocentric

perspective will play a predominant role in

answers to questions

Recommend interviewing both the

adolescent and a parent/caregiver (in

separate interviews). Parent interviews

less important and could be dropped

depending on the concept

Adolescent-completed PRO may be

sufficient, parent/caregiver-completed

measure focussed on observations

could add insight depending on the

condition and the concept of interest

15–17 Adolescent interviews or focus groups

essential. Interviews recommended in

preference to focus groups for most

concepts

Parent/caregiver interviews/focus groups

optional—their importance depends on

the concept

Adolescent-completed PRO likely to be

sufficient, parent/caregiver-completed

measure focussed on observations

could add insight depending on the

condition and the concept of interest,

but parents are less likely to be as

knowledgable about their child’s

condition/experiences



7/18

provide the most reliable and valid information to support

the generation of a PRO conceptual framework and items,

response scales and recall periods [18, 53]. As described

above, while there is some evidence children aged 4–5

years can report on specific, concrete aspects of their

health, they cannot answer more complicated questions,

and will often become very shy and uncommunicative. We

recommend therefore that only interviews with parent/ caregivers and clinicians are performed for this age group

and that age 6 years is the youngest age that should be

interviewed in a pediatric sample, to be confident of being

able to achieve saturation.

We recommend that both a parent/caregiver and the

child should be interviewed separately for children aged

6–11 years. In addition, for development of disease-specific

symptom measures, it is recommended that clinicians are

also interviewed. Similarly, when studying more severe

neurological conditions, a carer, psychologist, or neurolo-

gist who has worked closely with children with the targeted

condition can provide valuable information. Also for somesymptoms or domains of functioning or HRQL, teachers

can provide a valuable additional source of information

(e.g. in attention-deficit hyperactivity disorder (ADHD),

the child’s behavior and ability to concentrate in class

might be very different from their behavior at home, and

this is a key consideration).

Inclusion of many different categories of interview

participant increases the resources, costs, and time

required. However, for pediatric studies each different

respondent provides a different ‘piece of the puzzle’, giv-

ing information that is complementary and rarely entirely

redundant. For example, if interviewing about asthma

symptoms, the child him/herself can best report on how

irritating their cough feels and how much their asthma

symptoms limit their participation in sports. However, their

parent might be better able to recall and report on how

much the child’s asthma symptoms disturb their sleep.

It might be tempting to think that ‘mom knows best’ and

that parents provide more objective information than the

child; however, some information can only be known by

the children themselves, e.g. their experience of pain and

emotions. Parental views can be influenced by their own

health experiences and expectations, and the child can

learn to hide their true emotions from parents and carers

[54, 55]. Furthermore, for children of school age (and also

younger children where the child is in daycare) the parent

will often spend only a few hours a day with the child, and

so may not know the frequency or severity of symptoms, or

the impact on functioning or HRQL domains [54].

There is increasing consensus that parents should be

asked to only report on ‘observable signs, symptoms and

behavior’ [56] (defined by Donald Patrick in a breakout

session at the 2009 Drug Information Association

Outcomes Special Interest Group as ‘‘anything the care-

giver can see, hear, smell, feel or, strictly speaking, taste’’).

Arguably, this can include things the child tells the parent,

although they should be recorded separately from signs,

symptoms, and behaviors the parent observes directly [54].

Sometimes there may be differences in the information

provided by different interview participants that can be

difficult to resolve. Other evidence from the interviewsshould be used to judge who might be considered the more

reliable reporter, but in some cases it may be necessary to

flag both reports as questionable. One approach that can be

useful when parents and children are interviewed together

is to bring them together at the end of the interview. Then it

can be possible to highlight discrepancies for discussion,

taking care to avoid judgment or the suggestion that either

reporter is more trusted.

3.1.3 Presence of Parent/Caregiver when Interviewing

An issue for consideration when designing a pediatricinterview study is whether parent/caregivers should be

allowed to be present during the interview. Generally

parents should not be present as they may coach or influ-

ence the child. Even if asked to keep silent, their presence

might influence the child to give responses that he/she

knows the parent would prefer. Ethics review boards may

insist that the parent be given the option of being in the

room during the interview of children aged younger than

11 years, and some parents may insist upon it too. One

solution is to use a research facility that has a one-way

mirror viewing facility. This allows the parent to view the

interview from the next room (without being able to hear

the discussion) and be reassured that their child is content.

As the child is unable to hear or see their parent they are

typically less inhibited by their presence. However, for

ethical reasons, the child should be made aware that the

parent is behind the mirror watching.

When interviewing both the parent and child in the same

family it is recommended to have a team of two inter-

viewers so that both can be interviewed simultaneously.

This approach also allows for the possibility that the child

and parent can be brought together for a short section of

interviewing at the end.

3.1.4 Saturating and Analysing Concept Elicitation

and Cognitive Debriefing Interviews by Age

The idea of conceptual saturation is generally used to

determine sample sizes in qualitative interviews. Saturation

is commonly defined as ‘‘the point at which no new

information emerges’’ [57, 58]. In pediatric qualitative

research, it is important to demonstrate that saturation has

been achieved within narrow, pre-specified age ranges,



8/18

given the rapid developmental changes taking place. If

there is any doubt that saturation has been achieved, then

further interviews/groups should be conducted. The spe-

cific age ranges that are appropriate will depend on the

complexity of the concept(s) and condition being studied.

As an approximate guide we recommend demonstrating

saturation within each of the following age ranges, based

on the developmental milestones outlined in Table 1:

• 0–2 years (interview parent/caregivers only)

• 3–5 years (interview parent/caregivers only)

• 6–8 years (interview children and parent/caregivers)

• 9–11 years (interview children and parent/caregivers)

• 12–14 years (interview adolescents and parent/

caregivers)

• 15–17 years (interview children and possibly parent/

caregivers [parent/caregivers not essential in this age

band])

Researchers often combine the 6- to 8-year and 9- to

11-year age bands into a single group. However, given theconsiderable cognitive and linguistic changes that occur

during this period we suggest that it is critical that satu-

ration and psychometric validity are both evaluated within

each of these more narrow age ranges. It is during this

period that the majority of children develop causal rea-

soning and start to learn to read. The 12- to 14-year and 15-

to 17-year age bands are also frequently combined. The

differences between these age bands are less profound than

between the 6- to 8-year and 9- to 11-year age bands, and

combining the adolescent age bands can at times be

appropriate; this decision needs to be made based upon the

disease and concept(s) under study.

3.1.5 Interview Guides

When developing a pediatric concept elicitation discussion

guide, a key consideration is to ensure the guide is written

in child-friendly, age-appropriate language. This is partic-

ularly important for young children (those aged 6–8 years),

for whom questions should focus on the concrete and

abstract enquiries should be avoided. For example, in dis-

ease areas as diverse as asthma and restless legs syndrome

(RLS), the word ‘severe’ can be difficult for even young

adolescents aged 12–14 years to understand, thus making

the concept of symptom ‘severity’ more challenging to ask

about [20]. Even when using simple words, care must be

taken to ensure the child understands what is intended.

Words considered ‘naughty’ for children should be avoi-

ded. For example, ‘snot’ can be considered rude by some

children. Getting input from parents on appropriate termi-

nology that the child uses prior to the start of the interview

is essential.

In addition it should be considered that there are some

concepts relevant to adults, and perhaps adolescents, that

are inappropriate for children (e.g. work, sexual function-

ing) and vice versa.

Using role-play to test a guide is particularly useful for

pediatric interviewing where interviewers are likely to run

into difficulty or to find they are getting monosyllabic or

yes/no responses. Warm-up exercises can also be helpful toput the child at ease and to ensure they understand what is

required and the purpose of the questioning, although such

exercises should be kept short. Similarly, making it clear

during the consenting process what the reason for and

value of the interview is can help the child understand and

therefore engage with the process.

The interviewer should learn the guide well, allowing

them to improvise and adapt if they run into difficulty or

the child’s attention waivers. As much as possible the

children themselves should be encouraged to initiate dis-

cussions on topics so that the words used are ones that they

are familiar with, and the interviewer should then use thoseterms throughout. There are choices to be made regarding

the level of detail provided in the interview guide. It could

be argued that for a truly open-ended or ‘grounded theory’

approach an entirely blank canvas should be used. How-

ever, such an approach runs the risk that topics of interest

may not arise, or that the interviewer might forget to ask

issues of importance. In contrast, a guide that is too

detailed can be followed too prescriptively. Generally, a

balance can be struck in having a guide that reminds the

interviewer of the main topics to be covered without being

too inflexible.

Children easily become bored in interviews (particularly

children with ADHD or mental health conditions), so

having options that allow the interviewer to vary the focus

of the interview and utilize interactive techniques is

important. The interviewer should be careful to talk to the

child on their level, using appropriate language and

avoiding the use of any adult or clinical terminology. As in

all qualitative interviewing, open-ended questions and

taking time to engage with the respondent are of even more

importance with children and can help interviewers to

avoid some of the issues raised above.

3.1.6 Length of the Interview

The appropriate length of time for an interview with a child

is generally shorter than for adults [43]. The exact length of

time a child’s attention can be held will vary according to

the task but, typically, even adolescents struggle to con-

centrate for more than 1 h, and no more than 30 min

without a break is recommended for children aged 6–11

years. For focus groups, a slightly longer duration of



9/18

approximately 45 min is feasible. Taking breaks is rec-

ommended, as is allowing children to occasionally talk

about subjects that are ‘off-topic’ if it helps them to remain

engaged with the interviewer.

3.1.7 Use of Drawings and Props

One method that can be of value for helping children to talk about their health is the use of drawings [20]. Asking a child

to draw what their health condition feels like not only gives

the child a more fun task, but it can also help to initiate

discussion. This often results in powerful, unprompted

qualitative quotes—several examples of drawings with

accompanying quotes are provided in Fig. 2 [59]. The

drawings can also be subject to content analysis.

The use of props and toys can also be helpful, although

too many can be distracting. For example, play-doh has

proved useful when interviewing children about constipa-

tion, asking them to model their last bowel movement [60,

61]. Such aids can break the monotony of the interview and

engage the child. The downside of using drawings, props, or

toys is that they can be distracting, can waste time or lead

the interview off-topic, and comments can be more open to

interpretation. As such, these methods should be used

judiciously, and are of most value when children are having

difficulty focussing and are already distracted.

3.2 Generation or Modification of Items for PediatricPROs

When developing or modifying items that will be used in

pediatric populations the key is to ensure that the resulting

items are as simple and clear as possible, and to ensure they

will be understood by the target population [1]. This is

generally the aim for the development of adult PROs too

(in adult PRO development a common rule of thumb is to

aim for a 6th grade reading level, to ensure the PRO is

comprehensible to adults of all abilities), but with children

it is clearly of greater importance. Woolley et al. [22]

provides an example where an item worded ‘‘I am happywith myself’’ was misunderstood and interpreted to be

simply asking about mood, rather than self-esteem,

whereas simpler, more concrete items such as ‘‘I am smart’’

or ‘‘I am proud of myself’’ were better understood. Chil-

dren (especially those aged 7–10 years) can misinterpret or

have difficulty with vague terms because they tend to

interpret words literally [24].

Common wording pitfalls include asking about more

than one concept within a single item; items that ask an

individual to compare their current symptoms/functioning/

HRQL with their symptoms/functioning/HRQL at some

earlier time (the beginning of the study, or prior to having

the illness); use of double negatives; wording that is vague

and unclear; and response options that are vague or do not

fit with the question. In addition, using a large font and

laying out questions in a clear manner are additional

practical considerations that can aid comprehension con-

siderably for children but which are often neglected. In

addition, for electronic PROs (ePROs), having the device

present items to the child through audio as well as visually

is increasingly an option. While there is limited literature

on the use of this method in PROs, it is a method likely to

further aid comprehension, and so improve reliability.

If a symptom measure is being developed it is strongly

recommended that at least one expert physician is included

in the item generation process, to ensure that the items

generated are clinically relevant. Consulting the literature

is also recommended at this juncture, and input from par-

ents and teachers (in addition to incorporation of parent

interviews during concept elicitation) can also add insight.

Teacher involvement is particularly recommended for

pediatric measures that include behavioral or psychiatric

domains.

Fig. 2 Examples of drawings depicting restless legs sensations and

associated descriptions (reproduced from Picchietti et al. [59], with

permission). a A 7-year-old boy: ‘‘They (my legs) like feel weird, and

they want to kick.’’ b An 8-year-old boy: ‘‘It’s like my legs are wiggly

and like (inaudible).’’ c A 9-year-old boy: ‘‘It’s going to be hard to tell

that this is like a bruise. I’m trying to stretch my legs out, my thighs

out right here, so I’m going like this. I’m like trying to stretch my legs

out but at the same time it’s sort of hurting.’’ d An 11-year-old girl:

‘‘Well this picture shows like, see like it’s ant bites that’s kind of

showing you that it’s really hurting me like in those areas.’’



10/18

For a child self-report, modifying an adult questionnaire

is not recommended, even if guided by qualitative research.

The optimum item wording and ideal response options are

generally very different for children compared with adults.

Moreover, not all concepts relevant to adults are relevant or

appropriate for children and vice versa (e.g. questions

about work, sexual health, play). Instead, it is recom-

mended that items are generated based on qualitative datafindings, and then afterwards the concepts and items cre-

ated can be compared with the equivalent adult question-

naire to evaluate conceptual equivalence.

Parent/caregiver questionnaires (for infants and younger

children) should focus on asking about observable behav-

iors, as outlined in the FDA PRO guidance. For parent-

completed observer measures, there may be times when a

parent is only aware of a symptom or impact being expe-

rienced because of what their child tells them, not due to

direct observation. Questions should make it clear whether

a parent is to base their response on direct observation or

their child telling them about the event. (‘‘Did your childtell you his/her head hurt’’). Another option is to use a

single question and provide check boxes for the parent to

indicate if a rating is based on what he/she observed or

what the child told him/her. For item generation based on

both child and parent data, the child comments should be

given primacy over parent/caregiver quotes.

3.2.1 Response Options

Particular consideration should be given to developing

appropriate response options for pediatric PROs. As men-

tioned above, even children as old as 14 years can have

difficulty understanding what is meant by questions that

ask about ‘severity’ or use ‘severity’ response options (e.g.

‘mild’, ‘moderate’, ‘severe’) [20]. Wording that is better

understood by children and adolescents is to ask how ‘bad’

a symptom is.

Response options of ‘not like me’, ‘a little like me’ and

‘a lot like me’ have also been found to be misunderstood by

young children [22]. There is evidence that children prefer

adjectival response scales over a visual analog scale

(VAS), and find them easier to complete [62], perhaps

because the discrete number of choices is better suited to

the concrete and dichotomous thinking exhibited by young

children [63].

There is some evidence that children younger than 8

years of age cannot use Likert-type response scales [ 64,

65], and it has been suggested that fewer response options

are better for children younger than 8 years of age [18].

Using only dichotomous yes/no response options is cer-

tainly preferable in terms of the comprehension and

understanding, but limits the responsiveness of the measure

being developed. However, in a recent study, when the

response options were limited to ‘No/Yes, a little/Yes, a

lot’ it was found that several children, aged 7–17 years,

requested an ‘in between’ option [20]. It may be that with

the increasing use of computers and electronic games by

children, younger children are becoming increasingly lit-

erate at using such response scales.

A decision has to be made regarding whether the

response scale should ask about severity/intensity, fre-quency, bothersomeness, or use an agreement scale. Fre-

quency scales and severity scales that avoid the use of

high-level words are widely used and can be understood by

children, but the specific wording should be considered

carefully. Bothersomeness is a more complex concept and

so is best avoided, as are agreement scales. Generally, the

choice between severity and frequency should be made

based on which concept is more relevant to measure.

Graphics used to help make response scales more

meaningful to children include happy and sad faces [66,

67], circles of increasing sizes [18], and boxes that vary in

how full they are. See Fig. 3 for examples of each of these.Such graphics make response scales more concrete for

children [24]. Scales made up of faces that vary in terms of

happiness/unhappiness are commonly used, particularly

when rating pain, as in the Wong-Baker pain scale [68].

However, the extreme negative end of the scale typically

includes a very unhappy face with tears. This particular

response option has been critiqued on the basis that some

children, particularly boys, may not choose it if the

symptom/impact has not made them cry (or they do not

wish to admit they cried). In a study comparing three types

of graphic response scales (circles, faces, and thermome-

ter), the graded circles scale was found to have the highest

reliability [69]. In addition, ability items showed higher

reliability when the circles scale was used, but responses to

social items had higher reliability when the faces scale was

used, suggesting it is helpful to fit the graphic to the con-

cept [69]. Any visual aid may lead children to focus more

on the visuals than the words, and so should be considered

carefully. Moreover, some visual aids are harder to add into

ePROs than others, although the increasing availability of

larger screens and more sophisticated devices makes this

less and less of a problem.

3.2.2 Recall Period

There is evidence that children older than 8 years of age

can use a 4-week recall period [64, 70]. However, findings

from a recent study that included cognitive debriefing with

children have suggested that recalling accurately over a

period of more than 24 h can be challenging for children

younger than 11 years of age [20]. Survey-design theorists

recommend avoiding retrospective recall where possible,

and otherwise linking recall to concrete, salient,



11/18

memorable experiences [24]. For example, for a constipa-

tion measure it might be advised to ask children to recall

their last bowel movement (or ‘poop’ in child-friendly

language), rather than their bowel movements in the last

24 h. However, this is not always practical depending on

the symptom/concept being measured.

There is evidence that younger children develop

‘scripts’ about what usually happens for regularly occur-

ring events/activities [71, 72]. Such ‘scripts’ involve the

child generalizing their memories for events or activities

that occur repeatedly and routinely, and could limit the

accuracy of recall for specific events. The child might not

recall over the last 24 hours/7 days/4 weeks as they are

being asked to, but instead might choose an answer based

on their ‘script’ or general experience. During cognitive

debriefing, children, adolescents, and adults often indicate

that they are recalling over a longer time period than is

specified in a questionnaire. However, it is often unclear

whether this is due to the complicated hypothetical ques-

tioning that is necessary in a cognitive debriefing interview

(where the patient is usually only administered the PRO

once, and is asked to discuss it in detail).

To guard against the risk that recall periods might be

ignored, it is recommended to bold the recall period and

include it in every question, rather than just at the top of the

page or on an introductory screen on an ePRO. Moreover,

detailed training prior to PRO completion is essential to

increase the likelihood that the children understand and use

the recall period. Including graphical illustrations of the

recall period is also recommended. At a minimum,

including frequent reminders of the targeted recall period is

strongly recommended.

3.3 Conducting Cognitive Debriefing Interviews

with Children

Following item generation, the next step in development of

a PRO is cognitive debriefing in the target population. Suchcognitive debriefing involves administering the PRO to a

small number (typically between 12 and 20) of patients in

the target population and then interviewing them in detail

to ensure all instructions, questions, response options, and

the recall period (i.e. all aspects of the ‘instrument’) are

relevant and well understood [14, 17]. This testing of the

‘content validity’ of the PRO is considered an essential step

in the development of a PRO by the FDA [14], although it

has been pointed out that it is still not a method that is

consistently applied for all self-report instruments [22].

When performing pediatric cognitive debriefing, the pro-

cess is much the same as in adults; however, as with thesteps described in the previous sections, researchers must

adjust their expectations.

Cognitive debriefing can be repetitive and young chil-

dren in particular will likely lose focus after debriefing as

few as 10–15 items. Also of relevance here is the litera-

ture on processing speed. De Leeuw suggests that even a

child aged 12 years takes approximately 1.5 times longer

than an adult to process information; thus, children must

be given more time to consider their response during

debriefing [24]. Likewise, the level of detail that young

children can provide in relation to each item is more

limited. Children may rely on response sets or provide

inaccurate answers if their attention span is exceeded [18].

In adult cognitive debriefing, a single item can at times be

discussed for 5–10 minutes (with questions about the

relevance of the item content, the subject’s understanding

of the item, discussion of why the subject chose a par-

ticular response, and what would need to change for them

to choose a different response). With children aged 8–11

years and adolescents, the participant will generally only

be able to answer two or three questions about each item

before becoming bored. For children aged 6–8 years, even

less feedback is possible, and these children often only

give yes/no answers and are generally unable to explain

further, making it difficult to be completely confident that

they have understood correctly.

As in concept elicitation, taking regular breaks during

the cognitive debriefing is recommended, as is varying

what is being asked of the child to keep their interest. For

questionnaires longer than approximately 30 items it is

recommended that researchers only debrief half of the

questionnaire with any one child, or interview the same

child on two occasions.

Fig. 3 Examples of response scales



12/18

Another issue is whether the cognitive demands asso-

ciated with completing the cognitive debriefing interview

are too great for younger children. It has been suggested

that asking children to put questions in their own words is

too cognitively demanding for younger children [22].

Asking instead ‘‘what does the question mean?, what is it

asking?’’ has been suggested as being better understood

[22]. The researcher should be ready to try two or threeways of getting at the child’s ability to comprehend, as the

first approach used may not be successful.

A key issue that deserves greater attention with children

and adolescents compared with adults is their ability to

recall accurately. Unsurprisingly, there is evidence that

children aged 6–11 years recall symptoms and HRQL

impact more accurately if shorter recall periods are used

(e.g. 24 h) rather than longer recall periods (e.g. 7 days,

4 weeks) [73–75]. From our own work, we found that

while children as young as 9 years of age and their parents

had confidence in their ability to recall over the previous

24 h, they were not confident in their ability to recall over aweek, and certainly not over 4 weeks. Thus, it is very

important when conducting and analysing cognitive

debriefing interviews with children to pay a great deal of

attention to the time period children seem to be recalling

over and whether they are having difficulty recalling. Sleep

problems are one area where children generally report

having a great deal of difficulty recalling accurately [20].

3.3.1 Questionnaires Violating Conversation Norms Might

Confuse Young Children

There is evidence that young children first learn basic

conversation conventions, and only later learn how to deal

with exceptions where these conventions are violated [76].

If asked the same question more than once, young children

may change their answer because they assume that some-

thing was wrong with the first answer [77]. Therefore,

repeated questioning should be used sparingly with young

children, and interviewers should take care to ask questions

as neutrally as possible. In a similar vein, hypothetical

questioning can be particularly difficult for young children.

For daily diary instruments, one method that can help

researchers to avoid hypothetical questioning is to have the

child complete the diary for several days before the

debriefing interview. Then debriefing questions can focus

on their real experience with the diary and avoid hypo-

thetical ‘how would you answer if’ questions.

The work of Piaget [78] suggested that children aged

3–5 years have difficulty understanding cause and effect.

However, subsequent research demonstrated that if the task

is made very simple, young children’s reasoning far

exceeds the level Piaget thought possible [51, 79, 80]. In

fact, the children are able to reason logically on simple

problems, but they struggle with providing verbal justifi-

cation of their reasoning. The implications for interviewing

children about symptoms or functioning is that while

children as young as 3 or 4 years of age may be able to

report on their current health state, they may not be able to

explain their reasoning in an interview

There is evidence that when young children are asked

about events of personal significance to them (such aswhether or not they have been given an injection), they

usually provide correct answers [81]. However, young

children have also been shown to have trouble locating

events in time, or giving specific examples of recurrent

events—two issues of particular concern if asking young

children to report on frequent symptoms [82, 83]. In one

experiment, pre-schoolers were asked about what happens

when they eat lunch. The researcher found that if she asked

for information the children did not have, they would

provide answers from other similar scripted events [82].

All of these findings highlight that the researcher must

be very careful when conducting cognitive debriefinginterviews with children aged 6–11 years. For cognitive

debriefing, it is important that interviewers make it as clear

as possible that it is okay (and moreover helpful!) if the

child admits when he/she cannot understand. One useful

instruction in this context is to say at the beginning ‘we

need your help to make these questions better by making

them easier for younger kids’ [84].

4 PRO Completion

4.1 Parent vs. Child PRO Completion

So who should rate the symptoms, functioning, HRQL, and

other concepts typically measured using PROs (e.g. satis-

faction, adherence)? For children\3 years of age it is clear

that the children themselves cannot provide even an

interviewer-administered PRO rating, and either a parent/

caregiver- or clinician-completed measure is essential.

Ratings have been obtained on occasion from children as

young as 3–5 years. Obtaining a child rating for this age

group should only be considered for the most concrete and

the simplest of concepts, and even then, an interviewer- or

parent/caregiver-administered PRO is necessary as children

this age would not be able to complete the instrument

themselves

For the 6- to 11-year range is it recommended that either

complementary child and parent ratings are obtained or that

the child report is parent- or interview-administered, so that

the parent can help ensure the child can read and under-

stand the question as intended. This is particularly impor-

tant for the 6- to 8-year age range (given that this is the age

at which most children move from Piaget’s pre-causal



13/18

reasoning to causal reasoning), but is also recommended

for children aged 9–11 years. Where parent administration

of a child-completed PRO is employed (in any age group)

it is very important that the parent is given specific and

clear instructions on how he/she can and cannot help the

child. This should include clear guidance not to influence

the child’s response. Rather, the main function of the

parent should be to help the child read and understand thequestion. In the 6- to 8-year age range, generally neither the

child, the parent, nor the clinician can be considered the

‘gold standard’ but rather each reporter provides different

complementary information, which provides a different

piece of the puzzle.

As a result, including complementary parent and child

measures creates particular challenges when it comes to

designing clinical trials in which one (or a composite

including ratings from more than one reporter) endpoint

must be chosen as a ‘primary endpoint’ and considerations

must be made regarding multiplicity. Decisions regarding

the hierarchy of endpoints should be made taking intoaccount qualitative research findings from clinician, parent,

and child interviews and, if available, the evidence of the

psychometric validity of the different measures/versions.

Ideally these decisions should also be reached following

discussions with regulators.

Which rating (child, parent/caregiver, parent-adminis-

tered child measure, clinician, etc.) is considered primary

will vary depending on the concept being measured

(whether rating is of observable behaviors, observable

functional issues or non-observable symptoms, emotions,

or cognitions), the condition being studied, and, most

importantly, the age and ability of the child. As Matza et al.

[18] point out, for the 6- to 8-year age group, while the

child rating has more validity, the parent/caregiver rating

might have more reliability. As a result of these difficulties,

a parent-administered child report can prove the best

option.

In studies that include children across the range of ages

from 3–17 years, it is possible there will be the situation

that a parent report is used for children aged 3–5 years

(with no child rating at all for the 3- to 5-year age group),

and a child report is the primary endpoint for the children

aged 9–16 years, with a parent-administered version for

ages 6–8 years. However, inclusion of so many versions

has implications for pooling of data and sample sizes.

Data can only be pooled for scores created for which

there is the same number of items in each age group, and

the items can be considered conceptually equivalent. Often

with different age versions and separate child and parent

versions of PRO measures this is not the case, and so

pooling of data is not possible. Different age versions can

be considered analogous to different language versions of a

PRO [85]—the items need not be worded identically, but it

is important to show conceptual equivalence both from a

qualitative and quantitative perspective. If data cannot be

pooled it has the implication that sample sizes within each

age group must be sufficient to allow statistical differences

to be demonstrated within that age group.

4.2 Age Ranges Used in Existing Pediatric PROs

A summary of the different age-specific versions of some

of the most widely used pediatric HRQL instruments has

been provided by Matza et al. [18]. The majority of

instruments suggest a minimum age of around 6 years for

self-report, with a few allowing reporting for simple

questions to be used in children as young as 4 years of age

[10, 18, 38]. Moreover, most have separate versions for

adolescents and younger or middle-aged children (i.e. those

aged approximately 6–11 years). Among generic HRQL

measures, the Child Health Questionnaire has a parent

report version for children aged 4–17 years, and a child

report version for ages 10–17 years only [86]. The ChildHealth and Illness Profile (CHIP) [87], which later led to

the development of the Healthy Pathways measure, [1] has

child versions for ages 6–11 years and 12–17 years [88],

and a parent version for ages 6–11 years only [89]. There

are two widely used generic European HRQL measures—

the KINDL [90] and the KIDSCREEN [91]. The KINDL

[90] is unusual in having a simplified, interviewer-admin-

istered version for children as young as 4–7 years of age, as

well as versions for 8–11 years and 12–17 years. The

KIDSCREEN includes a version for self-report by children

and adolescents aged 8–18 years, as well as a parent ver-

sion. Lastly, the Pediatric Quality of Life Inventory

(PedsQL) [38]—the most widely used generic measure—

has child self-report versions for ages 5–7, 8–12, and 13–17

years, and a version for young adults aged 18–25 years.

4.3 Differences Between Child Self-Report and Parent

Report

Findings vary regarding the level of agreement between

child and parent ratings of health. Some researchers have

found high levels of agreement between parent and child

ratings [18, 92, 93], while others have much lower levels of

agreement [94–96]. One recent study found very low levels

of agreement between 5- and 8-year-olds and parents, with

intraclass correlation coefficients of just 0.02–0.23 [96]. In

a further recent study in asthma and epilepsy presented at

the International Society of Quality of Life Research [97] it

was reported that the low level of agreement between

parent and child decreased with the severity of the condi-

tion. Parents in this study rated their child’s HRQL as

worse than the child did (and also worse than the clinician

did) and the level of correlation was lower for children of



14/18

worse severity. This pattern has also been found in studies

of other pediatric chronic conditions such as chronic pain

[98, 99]. However, in studies comparing the responses of

healthy children and their parents, the opposite has been

found, with parents/caregivers rating their child’s health as

lower than the healthy child/adolescent [92, 96, 100, 101].

Finally, two studies provide evidence that the level of

agreement varies with age, with closer agreement for olderchildren/adolescents [102, 103].

5 Ethical Considerations in Relation to Pediatric PRO

Research

Broadly, pediatric PRO development research should be

conducted within the ethical guidelines that exist for

medical research. Clearly, ethical issues are of even greater

importance for a vulnerable population such as children.

All research should be conducted in line with the Decla-

ration of Helsinki, and should meet with national andregional guidelines for research in the country in which it is

conducted. Written informed consent should be obtained

from a parent/guardian for all child participants in inter-

view or questionnaire studies, and assent should be

obtained from the children themselves [43]. Of note, while

it is important to first obtain consent from the parent/

guardian to ask the child to participate, obtaining the

child’s assent to participate is also critical. To assent, the

child/adolescent needs to have the aims of the research and

what will be expected of them explained in simple,

developmentally appropriate language. This should be

through verbal explanation, but also through an assent form

that uses an appropriate level of language. It should also be

made very clear to children that they are able to withdraw

at any time, for any reason, with no adverse consequences

for them or their medical care.

6 Conclusions

The aim of this article was to provide an overview of the

state of the art for qualitative pediatric PRO development

research. It is most critical that PRO development methods

used in adults are carefully tailored to the age and devel-

opmental abilities of the children under study. Conceptual

saturation and comprehension should be evaluated within

narrow age ranges. A particular challenge is the issue of

how to evaluate children who are too young to report

themselves, but who experience symptoms that cannot be

adequately measured by any means other than asking the

child. Both parent reports that rely on observations and

parent-administered child self-reports are two options that

to some degree overcome these challenges. Moreover,

ePROs provide additional options to aid comprehension for

young children. Despite improvements in our understand-

ing, too often pediatric studies still rely on instruments that

are inappropriate for the population, poorly developed, or

inadequately measure the concept of interest. Improve-

ments in the art and the science of pediatric PRO research

are needed to help ensure that instruments used in pediatric

studies are truly ‘fit for purpose’, thus aiding in the accu-mulation of robust evidence regarding the health of

children.

Acknowledgments We would like to acknowledge the support

provided by Kate Bolton in preparing one draft of the manuscript, and

the support of Nicola Moss in helping with formatting and quality

checking. In addition, we are grateful for the helpful comments of two

anonymous reviewers.

Disclosure of interests Both Rob Arbuckle and Linda Abetz-Webb

are employees of Adelphi Values, a health outcomes consultancy that

specialises in working with healthcare companies on the develop-

ment, validation, and use of PRO instruments. As such, both authors

have been contracted to perform research for numerous pharmaceu-tical companies. Neither author owns stocks in any pharmaceutical

company, nor have they been a direct employee of a pharmaceutical

company.

Author contributions Both authors contributed to the conception

and writing of all parts of the manuscript and both authors read and

approved the final version.

References

1. Bevans KB, Riley AW, Moon J, Forrest CB. Conceptual andmethodological advances in child-reported outcomes measure-

ment. Expert Rev Pharmacoecon Outcomes Res. 2010;10(4):

385–96.

2. Forrest CB, Simpson L, Clancy C. Child health services

research: challenges and opportunities. JAMA. 1997;277(22):

1787–93.

3. Maldonado S. United States perspective. In: Mulberg A, Silber

S, van den Anker J, editors. Pediatric drug development: con-

cepts and applications. New Jersey: Wiley; 2009. p. 133–6.

4. The Best Pharmaceuticals for Children Act of 2002. Public Law

107-109, 115 Stat.1408-1424. 2002. http://www.fda.gov/

downloads/Drugs/DevelopmentApprovalProcess/Development

Resources/UCM049874.pdf .

5. The Pedatric Research Equity Act of 2003. Public Law 108-155,

117 Stat.1936-1943. 2003. http://www.fda.gov/downloads/Drugs/ DevelopmentApprovalProcess/DevelopmentResources/UCM077

853.pdf .

6. The Pediatric Research Equity Act of 2007. Public Law 110-85,

121 STAT.823 (Food and Drug Administration Amendments

Act (FDAAA) of 2007—Title IV). 2007. http://www.fda.gov/

downloads/Drugs/DevelopmentApprovalProcess/Development

Resources/UCM049870.pdf .

7. The Best Pharmaceuticals for Children Act of 2007. Public Law

110-85, 121 STAT.823 (Food and Drug Administration

Amendments Act (FDAA) of 2007—Title V). 22007. http://

www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/

DevelopmentResources/UCM049870.pdf .


http://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049874.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049874.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049874.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM077853.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM077853.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM077853.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049870.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049870.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049870.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049870.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049870.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049870.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049870.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049870.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049870.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049870.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049870.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049870.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM077853.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM077853.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM077853.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049874.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049874.pdfhttp://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/DevelopmentResources/UCM049874.pdf


15/18

8. Regulation (EC) no 1901/2006 of the european parliament and

of the council of 12 December 2006 on medicinal products for

paediatric use and amending Regulation (EEC) No 1768/92,

Directive2001/20/EC, Directive 2001/83/EC and Regulation

(EC) No 726/2004. Official Journal of the European Union.

2006. http://ec.europa.eu/health/files/eudralex/vol-1/reg_2006_

1901/reg_2006_1901_en.pdf .

9. Regulation (EC) no 1902/2006 of the european parliament and

of the council of 20 December 2006 amending Regulation

1901/2006 on medicinal products for paediatric use. Official

Journal of European Union. 2006. http://ec.europa.eu/health/

files/eudralex/vol-1/reg_2006_1902/reg_2006_1902_en.pdf .

10. Landgraf J, Abetz L. Measuring health outcomes in pediatric

populations: issue in psychometrics and application. In: Spilker

B, editor. Quality of life and pharmacoeconomics in clinical

trials. Philadelphia: Lippincott-Raven Publishers; 1996.

p. 793–802.

11. Pasquali SK, Burstein DS, Benjamin DK, Smith PB, Li JS.

Globalization of pediatric research: analysis of clinical trials

completed for pediatric exclusivity. Pediatrics. 2010;126(3):

e687–92.

12. Pharmaceutical Research and Manufacturers of America’s

(PhRMA). PhRMA Statement on Tremendous Success of BPCA

and PREA. 2011. http://www.phrma.org/media/releases/phrma-

statement-tremendous-success-bpca-prea .

13. European Medicines Agency. Reflection Paper on the Regula-

tory Guidance for the Use of Health-Related Quality of Life

(HRQL) Measures in the Evaluation of Medicinal Products.

2005. http://www.ema.europa.eu/docs/en_GB/document_library/

Scientific_guideline/2009/09/WC500003637.pdf .

14. US Department of Health and Human Services, Food and Drug

Administration, Center for Drug Evaluation and Research

(CDER), Center for Biologics Evaluation and Research (CBER),

Center for Devices and Radiological Health (CDRH). Guidance

for Industry—Patient-Reported Outcome Measures: Use in

Medical Product Development to support Labeling Claims. 2009.

http://www.fda.gov/downloads/Drugs/GuidanceCompliance

RegulatoryInformation/Guidances/UCM193282.pdf .

15. Leidy NK, Vernon M. Perspectives on patient-reported out-

comes: content validity and qualitative research in a changing

clinical trial environment. Pharmacoeconomics. 2008;26(5):

363–70.

16. Patrick DL, Burke LB, Powers JH, Scott JA, Rock EP, Dawisha

S, et al. Patient-reported outcomes to support medical product

labeling claims: FDA perspective. Value Health. 2007;10(Suppl

2):S125–37.

17. Lasch KE, Marquis P, Vigneux M, Abetz L, Arnould B, Bayliss

M, et al. PRO development: rigorous qualitative research as the

crucial foundation. Qual Life Res. 2010;19(8):1087–96.

18. Matza L, Swensen A, Flood E, Secnik K, Kline Leidy N.

Assessment of health-related quality of life in children: a review

of conceptual, methodological, and regulatory issues. Value

Health. 2004;7(Number 1):79–90.

19. Rothman M, Kleinman L. Patient-reported outcomes in pediatricdrug development. In: Mulberg A, Silber S, van den Anker J,

editors. Pediatric drug development: concepts and applications.

New Jersey: Wiley; 2009. p. 513–24.

20. Arbuckle R, Abetz L, Durmer J, Ivenenko A, Owens J, Croenlin

J, et al. Development of the Pediatric Restless Legs Syndrome

Severity Scale (R-RLS-SS): a patient reported outcome measure

of pediatric RLS symptoms and impact. Sleep Med. 2010;

11(9):897–906.

21. Solans M, Pane S, Estrada MD, Serra-Sutton V, Berra S,

Herdman M, et al. Health-related quality of life measurement in

children and adolescents: a systematic review of generic and

disease-specific instruments. Value Health. 2008;11(4):742–64.

22. Woolley ME, Bowen GL, Bowen NK. Cognitive pretesting and

the developmental validity of child self-report instruments:

theory and applications. Res Soc Work Pract. 2004;14(3):

191–200.

23. Fuchs M. The reliability of children’s survey responses: the

impact of cognitive functioning on respondent behavior. In:

Proceedings of Statistics Canada Symposium 2008: Data Col-

lection: Challenges, Achievements and New Directions; 2009.

24. de Leeuw ED. Improving data quality when surveying children

and adolescents: cognitive and social development and its role in

questionnaire construction and pretesting. Report prepared for the

Annual Meeting of the Academy of Finland: Research Programs

Public Health Challenges and Health and Welfare of Children and

Young People, May 10-12. Naantali, Finland; 2011.

25. Ravens-Sieberer U, Erhart M, Wille N, Wetzel R, Nickel J,

Bullinger M. Generic health-related quality-of-life assessment in

children and adolescents: methodological considerations. Phar-

macoeconomics. 2006;24(12):1199–220.

26. Inhelder B, Piaget J. The growth of logical thinking from

childhood to adolescence. London: Routledge; 1958.

27. Inhelder B, Piaget J. The early growth of logic in the child.

London: Routledge; 1964.

28. Vygotsky LS. Mind in society: the development of higher psy-

chological processes. Cambridge: Harvard University Press; 1978.

29. Bronfenbrenner U. The ecology of human development:

experiments by nature and design. Cambridge: Harvard Uni-

versity Press; 1981.

30. Kail RE. Children and their development. 5th ed. Englewood

Cliffs: Prentice Hall; 2006.

31. Borgers N, de Leeuw E, Hox J. Children as respondents in

survey research: cognitive development and response quality.

Bulletin de Methodologie Sociologique. 2000;66:60–75.

32. Siegler RS, Richards DD. The development of intelligence. In:

Sternberg RJ, editor. Handbook of human intelligence. Cam-

bridge: Cambridge University Press; 1982. p. 897–971.

33. Scarr S. Understanding development. New York: Harcourt

jurnal novi 7.pdf

Documents

Transcript of jurnal novi 7.pdf