Examination of the Validity of Nutrient Profiling Models for …€¦ · To Dr. Marie-Ève...
Transcript of Examination of the Validity of Nutrient Profiling Models for …€¦ · To Dr. Marie-Ève...
Examination of the Validity of Nutrient Profiling Models for Assessing the Nutritional Quality of Foods
by
Theresa H. K. Poon
A thesis submitted in conformity with the requirements for the degree of Master of Science
Department of Nutritional Sciences University of Toronto
© Copyright by Theresa H. K. Poon 2018
ii
Examination of the Validity of Nutrient Profiling Models for
Assessing the Nutritional Quality of Foods
Theresa H. K. Poon
Master of Science
Department of Nutritional Sciences University of Toronto
2018
Abstract
Nutrient profiling (NP) is a method for evaluating the healthfulness of foods. Although
many NP models exist, most have not been validated. This thesis aimed to examine
various aspects of validity of four models developed by authoritative bodies. All four
models demonstrated moderate content validity, which was assessed by examining the
nutrients/components considered in the models. However, different models exhibited
varying levels of construct or convergent validity with Ofcom, a previously validated
model which served as the reference, in classifying over 15,000 Canadian pre-
packaged foods from a branded, national database. Numerous incongruencies were
identified for certain models, which highlights the importance of examining
classifications across food categories, the level at which differences between models
become apparent. The challenges associated with validating models are discussed.
Overall, this thesis provides data that can inform regulators seeking to adapt existing NP
models and validate them for use in country-specific applications for nutrition policies.
iii
Acknowledgments I would like to express my deepest appreciation and sincerest gratitude to the following
individuals who made this experience more rewarding than I could have ever imagined.
To my supervisor, Dr. Mary L’Abbé, thank you for your generous mentorship and
unwavering support. Your energy, dedication, and work ethic is inspiring. Thank you
for giving me the opportunity to realize my long-time goal of undertaking graduate
studies and helping me grow as a researcher.
To Dr. Marie-Ève Labonté, thank you for your always helpful guidance, our excellent
collaboration, and your friendship.
To my committee members, Dr. JoAnne Arcand and Dr. Erin Hobin, thank you for your
insightful advice and encouragement on both academic and professional matters. Your
generosity in sharing your time is deeply appreciated.
To my appraiser, Dr. David Jenkins, thank you for your thoughtful questions and sharing
your valuable time.
To Dr. Kathy Musa-Veloso, thank you for your patience during this undertaking and for
my previous training, which has allowed me to truly appreciate and enjoy this learning
experience.
To the past and current members of the L’Abbé lab, thank you for our ever interesting
discussions that have enlightened me and made me laugh. Thank you for sharing in
this experience with me and making it that much more enjoyable.
Lastly, to my family, thank you for your unconditional support and encouragement.
Specifically to my partner, Chris Kwong, thank you for your selflessness, infinite
patience, and guidance. This milestone is as much yours as it is mine. I am forever
grateful.
iv
Table of Contents Acknowledgments ........................................................................................................... iii
Table of Contents ............................................................................................................ iv
List of Tables.................................................................................................................. vii
List of Figures ............................................................................................................... viii
List of Appendices ........................................................................................................... ix
List of Abbreviations ......................................................................................................... x
Chapter 1 ......................................................................................................................... 1
1 Introduction .................................................................................................................. 1
Chapter 2 ......................................................................................................................... 3
2 Background and Literature Review .............................................................................. 3
2.1 Overview of Nutrient Profiling (NP) ....................................................................... 3
2.1.1 Brief History of NP ...................................................................................... 3
2.1.2 Applications of NP ...................................................................................... 3
2.2 Key Characteristics of NP Models ......................................................................... 4
2.2.1 Intended Objectives and Scope of Models ................................................. 5
2.2.2 Exclusions and Exemptions ........................................................................ 6
2.2.3 Food Categories ......................................................................................... 7
2.2.4 Nutrients/components for Inclusion and Corresponding Nutrient Criteria ... 7
2.2.5 Reference Amounts .................................................................................. 10
2.2.6 Output of Models ...................................................................................... 11
2.2.7 Example NP Models ................................................................................. 12
2.3 Validation of NP Models ...................................................................................... 15
2.3.1 Types of Validity ....................................................................................... 15
2.3.2 Methods of Validity Testing ...................................................................... 18
v
2.4 Current Status of NP and Models ........................................................................ 21
2.4.1 Canadian Context ..................................................................................... 21
2.4.2 Global Context .......................................................................................... 22
2.4.3 Knowledge Gaps ...................................................................................... 26
Chapter 3 ....................................................................................................................... 31
3 Study on Validation of NP Models ............................................................................. 31
3.1 Manuscript ........................................................................................................... 31
3.1.1 Abstract .................................................................................................... 31
3.1.2 Introduction ............................................................................................... 32
3.1.3 Methods .................................................................................................... 34
3.1.4 Results ..................................................................................................... 41
3.1.5 Discussion ................................................................................................ 53
3.1.6 Supplementary Material ............................................................................ 59
3.2 Additional Analyses Not Included in Manuscript .................................................. 65
3.2.1 Summary of Results on Construct/Convergent Validity ............................ 65
3.2.2 Data Not Shown as Reported in Manuscript ............................................. 65
3.2.3 Modified-PAHO......................................................................................... 67
Chapter 4 ....................................................................................................................... 72
4 Overall Discussion ..................................................................................................... 72
4.1 Challenges Associated with Validating NP Models ............................................. 72
4.1.1 Ambiguity in the Definitions and Methods Used in Validating NP Models ...................................................................................................... 72
4.1.2 Lack of a Definition of a Validated Model ................................................. 75
4.2 Other Considerations .......................................................................................... 76
4.3 Significance and Implications of Research .......................................................... 77
Chapter 5 ....................................................................................................................... 80
vi
5 Conclusions ............................................................................................................... 80
References..................................................................................................................... 82
Appendix A .................................................................................................................... 90
Study on NP Models for Marketing Restrictions ............................................................. 90
Appendix B .................................................................................................................. 102
Ethics Review Application for Study on Proposed NP Model for Marketing Restrictions .............................................................................................................. 102
Appendix C .................................................................................................................. 133
Systematic Review of NP Models ................................................................................ 133
Copyright Acknowledgements...................................................................................... 134
vii
List of Tables Table 2-1 Food categories, nutrients/components, and nutrient criteria used in
the WHO EURO model .............................................................................. 9
Table 2-2 Summary of the key characteristics of several NP models used in this research ....................................................................................................13
Table 2-3 Definitions of different types of validity used by different researchers ......16
Table 2-4 Methods of testing different types of validity used by different researchers ...............................................................................................19
Table 2-5 Studies in which construct/convergent validity was assessed based on agreement between NP models (n=19) ...............................................28
Table 3-1 Authors’ contributions to study on validation of NP models ......................31
Table 3-2 Summary of NP models examined ...........................................................39
Table 3-3 Associations between the proportions (%, 95% CI) of foods classified as “less healthy” by the models and quartiles of Ofcom scores (n=15,227) ................................................................................................42
Table 3-4 Method based on the ingredient list used to estimate the FVN points of foods in the Ofcom model .....................................................................60
Table 3-5 Proportion (%) of “healthier” and “less healthy” foods classified by models compared to the Ofcom model for all foods (n=15,227) and by food category .......................................................................................62
Table 3-6 Results of the assessment of construct/convergent validity ......................65
Table 3-7 Proportion (%) of “healthier” and “less healthy” foods classified by the modified-PAHO model compared to the Ofcom model for all foods (n=15,227) and by food category ..............................................................70
Table 4-1 Descriptions for criterion, concurrent, convergent, and construct validity used by different researchers........................................................74
Table A-1 Authors’ contributions to study on NP models for marketing restrictions to children ...............................................................................90
Table B-1 Authors’ contributions to ethics review application for study on proposed NP model for marketing restrictions to children .......................102
Table C-1 Authors’ contributions to systematic review of NP models ......................133
viii
List of Figures Figure 2-1 Flow diagram of the publication and NP model selection. .........................24
Figure 3-1 Overall proportions (%, 95% CI) of “healthier” and “less healthy” foods and agreement (κ) between each model and the Ofcom model (n=15,227). .............................................................................................44
Figure 3-2 Agreement (κ, 95% CI) and discordance (%) between the Food Standards Australia New Zealand (FSANZ) and Ofcom model for all foods (n=15,183; data missing for n=44) and 22 food categories. ............45
Figure 3-3 Agreement (κ, 95% CI) and discordance (%) between the Health Canada Surveillance Tool (HCST) and Ofcom model for all foods (n=15,165; data missing for n=62) and 22 food categories. ......................47
Figure 3-4 Cross-classification analyses between four Health Canada Surveillance Tool (HCST) tiers versus quartiles of Ofcom scores for all foods (n=15,165; data missing for n=62) and 22 food categories. .......48
Figure 3-5 Agreement (κ, 95% CI) and discordance (%) between the World Health Organization Regional Office for Europe (EURO) and Ofcom model for all foods (n=15,182; data missing for n=45) and 22 food categories. ................................................................................................50
Figure 3-6 Agreement (κ, 95% CI) and discordance (%) between the World Health Organization Regional Office for the Americas/Pan American Health Organization (PAHO) and Ofcom model for all foods (n=15,182; data missing for n=45) and 22 food categories. ......................52
Figure 3-7 Cross-classification analyses between quartiles of Food Standards Australia New Zealand (FSANZ) and Ofcom scores for all foods (n=15,183; data missing for n=44) and 22 food categories. ......................66
Figure 3-8 Agreement (κ, 95% CI) and discordance (%) between the modified World Health Organization Regional Office for the Americas/Pan American Health Organization (modified-PAHO) and Ofcom model for all foods (n=15,177; data missing for n=50) and 22 food categories. ................................................................................................69
ix
List of Appendices Appendix A Study on NP Models for Marketing Restrictions to Children
Appendix B Ethics Review Application for Study on Proposed NP Model for Marketing Restrictions to Children
Appendix C Systematic Review of NP Models
x
List of Abbreviations CFG Canada’s Food Guide
CI confidence interval
EURO World Health Organization Regional Office for Europe
FDA Food and Drug Administration
FSANZ Food Standards Australia New Zealand
FVN/L fruits, vegetables, nuts, and/or legumes
HCST Health Canada Surveillance Tool
NP nutrient profiling
ONQI Overall Nutritional Quality Index
PAHO World Health Organization Regional Office for the Americas/Pan American Health Organization
PNIG Population Nutrient Intake Goals to Prevent Obesity and Related Non-communicable Diseases, World Health Organization
RR relative risk
U.S. United States
UK United Kingdom
WHO World Health Organization
1
Chapter 1
1 Introduction Nutrient profiling (NP) is defined as the science of classifying foods according to their
nutritional composition for the purpose of promoting health and preventing disease(1).
NP models have numerous applications in government and industry, for example, the
regulation of health claims, front-of-pack nutrition labelling, restrictions on marketing of
less healthy foods to children, and guidelines on product reformulation, amongst other
applications(1-3). Overall, NP models are important for ensuring that regulations and
health promotion strategies are aligned with dietary guidance and public health
objectives in the reduction of diet-related non-communicable diseases.
Based on a series of systematic reviews on the use of NP for various applications that
have been updated over the years(4-8), the number of potential NP models globally has
increased dramatically. Given the proliferation of NP models and the extensive
resources required to develop and validate a new model, the adaptation of an existing
NP model is preferred by the World Health Organization (WHO) and is becoming an
increasingly common practice for government and industry(1,2,9). However, although
many NP models exist, most models have not been fully evaluated for validity(2,7,8,10).
Even amongst NP models that have been evaluated for validity, most have been
validated only to a limited extent(8). Moreover, in the context of adaptation, NP models
need to be validated when used for different applications and in different countries, as it
is unknown whether the criteria that underpins a model developed to assess foods for
one application or from one population are appropriate for another(1,9). Furthermore, the
methods of validating NP models are in their infancy(2). As such, several researchers
have suggested that validity testing of NP models should be given the highest priority in
this field of research(3,10,11). Thus, the objective of this thesis was to examine the
content and construct/convergent validity of several NP models developed by
authoritative bodies for assessing the nutritional quality of foods in a Canadian context.
2
In Chapter 2, background on NP models and a review of the literature on the validation
of models is provided. In Chapter 3, the manuscript of the validation study is presented.
In Chapter 4, the challenges associated with validating NP models along with other
considerations are discussed. Overall, this thesis aims to explore whether four NP
models are valid tools to assess the nutritional quality of foods in Canada. This
research will contribute to the emerging science of NP and provide regulators with
pertinent data that will facilitate the selection and validation of NP models for public
health initiatives and nutrition policy.
3
Chapter 2
2 Background and Literature Review
2.1 Overview of NP
2.1.1 Brief History of NP
Although NP is a relatively new term in the field of nutrition research, the concept of NP
itself has been in use for some time(1,9). Several of the earliest forms of NP were
introduced by government bodies in the 1980s and 1990s(9), including the:
• minimum nutritional requirements for foods as part of the United States Department of Agriculture’s Special Supplemental Nutrition Program for Women, Infants, and Children in 1980(12);
• Swedish Keyhole by the Swedish National Food Administration in 1989(13); • disqualifying nutrient levels for health claims by the United States (U.S.) Food
and Drug Administration (FDA) in 1993(14); and • minimum nutrient criteria for the “healthy” implied nutrient content claim by the
U.S. FDA in 1993(15).
The term NP gained ground and began to be referred to as such following the
development of the Ofcom model by the United Kingdom (UK) Food Standards Agency
in 2004 and 2005(4,5) and the mention of nutrient profiles in Regulation (EC) No
1924/2006 on nutrition and health claims by the European Commission in 2006(16). In
2010, NP became even more widely known when the WHO provided to its Member
States a set of recommendations on the marketing of foods to children, one of which
advocated the use of NP models in defining the foods to be covered by the marketing
restrictions(17).
2.1.2 Applications of NP
As NP is increasingly recognized by authoritative bodies and industry as a transparent
and reproducible method for evaluating the healthfulness of foods, NP models are now
having numerous applications in government and industry, including for example(1-3,8):
• regulations of nutrition and health claims;
4
• guidelines for front-of-pack nutrition labelling; • restrictions on the marketing of foods to childrena; • food procurement policies for public institutions (e.g., schools, hospitals); • economic tools to orient food consumption (e.g., food subsidies or taxes); • standards for the composition of foods; • food fortification policies; and • product reformulation.
Overall, NP is important in ensuring that regulations and health promotion strategies are
aligned with dietary guidance and public health objectives in the reduction of diet-related
non-communicable diseases. NP should be thought of as a tool to be used in
conjunction with interventions aimed at improving diets within a jurisdiction(1).
It is important to differentiate between the NP model itself and the application of that
model(9). Ideally, a single model should be used across policies within a jurisdiction,
while the threshold criteria or cut-off scores of the model are adjusted for different
applications(9). For example, the Ofcom model may be used across multiple policies
within the UK; however, less stringent threshold criteria may be chosen to underpin the
regulations on front-of-pack labelling which is applicable to the general population,
whereas more stringent threshold criteria may be chosen to underpin the restrictions on
marketing to children in the context of protecting vulnerable populations.
2.2 Key Characteristics of NP Models This section provides an overview of NP models by summarizing the key characteristics
of these models, including:
• intended objectives and scope; • exclusions and exemptions; • food categories; • nutrients/components for consideration and corresponding nutrient criteria;
a We conducted a study on the application of NP models for the restriction of marketing unhealthy foods to children. This publication is presented in Appendix A (Labonté MÈ, Poon T, Mulligan C, Bernstein JT, Franco-Arellano B, L’Abbé MR. Comparison of global nutrient profiling systems for restricting the commercial marketing of foods and beverages of low nutritional quality to children in Canada. Am J Clin Nutr 106, 1471-1481.).
5
• reference amounts; and • output.
2.2.1 Intended Objectives and Scope of Models
The intended objectives of NP models may vary with regards to(1,9):
1) whether the model is used for a single application or multiple applications; 2) the end-user of the model; 3) whether the model aims to identify foods with a favourable profile (i.e., “healthier”
foods), an unfavourable profile (i.e., “less healthy” foods), or both; and 4) whether the model aims to shift consumption to “healthier” foods between or
within food categories.
The objectives of the model are crucial in determining the type and characteristics of the
model needed for a particular application. For example, a relatively simple model may
be preferable when the end-user of the model includes the general public viewing the
front-of-pack rating system on nutrition labels, whereas a relatively complex model may
be used when the end-user includes the regulatory officers who apply the model to
determine the eligibility of products for subsidies under food assistance programs.
Moreover, the inclusion of nutrients to limit (e.g., sodium) in a model may be sufficient
when it primarily aims to identify “less healthy” foods, whereas the inclusion of both
nutrients to limit and nutrients to encourage (e.g., fibre) may be preferable when the
model aims to identify foods across the full spectrum of healthfulness. Lastly, a model
with an “across the board” design, such that the same nutrient criteria are applied to all
foods, may be preferable when the model aims to encourage healthier choices between
categories (e.g., from ready-to-eat breakfast cereals high in sugar versus fruit)(1). In
contrast, a model with a “food category specific” design, such that the nutrient criteria
are applied at the food category level, may be preferable when the model aims to
encourage healthier choices within categories (e.g., from ready-to-eat breakfast cereals
high in sugar versus ready-to-eat breakfast cereals high in fibre)(1).
In addition, the scope of models may vary with regards to the form of food that is
specifically subjected or not subjected to the model(9). For example, models typically
cover either foods as sold or as consumed(9). Although the intended objectives and
scope of the models should be dictated by the aforementioned practical aspects
6
associated with the use of the models, often the objectives and scope are influenced by
the legislative frameworks and political context in which the models must thrive(9).
2.2.2 Exclusions and Exemptions
NP models may incorporate exclusions and exemptions, which refer to foods or food
categories that are explicitly allowed or not allowed as they relate to the application of
the model. Exclusions refer to foods that are never allowed by a model, while
exemptions refer to foods that are always allowed by a model. It is important to
differentiate between exclusions versus foods that are outside the scope of the model,
the latter which often includes foods for special medical purposes (e.g., termed “medical
foods” in the U.S.) and meal replacements because their nutritional composition is
usually defined in regulation. For example, in the case of school food models, soft
drinks are often exclusions because the models explicitly aim to prohibit soft drink
consumption in schools, whereas infant and follow-on formula are not explicitly
prohibited and would be considered outside the scope of these models with school-
aged children as the target population.
It should be noted that there is inconsistency with regards to this differentiation across
NP models, as illustrated in Table 2-2 in Section 2.2.7. According to the model
developed by the Food Standards Australia New Zealand (FSANZ) for the regulation of
claims(18), infant formula is explicitly prohibited from bearing nutrition content or health
claims; as such, infant formula should be considered as an exclusion. In contrast,
according to the model developed by the WHO Regional Office for Europe (EURO) for
the restrictions on the marketing of foods to children(19), infant formula is reported to be
“not covered by this model”; given that the decision on whether infant formula is
prohibited for marketing to children was not explicitly stated, infant formula should be
considered to be outside the scope of this model. Instead, seven food categories are
7
explicitly stated as not permitted for marketing to children and are considered as
exclusionsb.
2.2.3 Food Categories
The number and type of food categories included in a NP model vary widely across
different models(8). A model may have many, several, or even no food categories. The
most common method of defining food categories within a model is to align with existing
definitions in food regulations or categories in food composition databases(1). The food
categories may be organized simply as foods versus beverages, or by main categories
(e.g., bakery versus dairy products), sub-categories (e.g., milks versus yogurts), and
even sub-sub-categories (e.g., almond versus soy beverages). There are advantages
and disadvantages associated with various options(1). For example, while a model with
many food categories of different levels is more complex and may hinder the usability of
the model, it may be more effective in encouraging product reformulation(1);
alternatively, the application of a model with few food categories across different food
supplies is much simpler. Often the decisions related to food categories are made in
the context of balancing comprehensiveness of the model, feasibility of use of the
model, and consistency with other regulatory or policy tools in place(1).
2.2.4 Nutrients/components for Inclusion and Corresponding Nutrient Criteria
The number and type of nutrients/components included in a NP model vary widely
across different models(8). A model may have many or only several
nutrients/components with nutrient criteria. A model may include nutrients/components
to limit (e.g., energy, sodium, sugars), nutrients/components to encourage (e.g., protein,
fibre, fruits and vegetables), or both. There are advantages and disadvantages
associated with various models(1); for example, while a model with many
nutrients/components is more complex and may hinder usability of the model, it is more
b The exclusions included: 1) chocolate and sugar confectionery, energy bars, and sweet toppings and desserts; 2) cakes, sweet biscuits, and pastries; other sweet bakery wares, and dry mixes for making such; 3) beverages: juices; 4) beverages: energy drinks; 5) edible ices; 6) foods containing >1 g of industrially-produced trans fat per 100 g total fat; and 7) foods with alcohol content ≥0.5% of total energy.
8
comprehensive and may be more effective in differentiating the healthfulness of different
foods.
The consideration of which nutrients/components to include in a model is largely
dependent on the availability of nutrient composition data(1,9). For example, although
the WHO guidelines on sugars are for free sugarsc, added sugars were included in the
WHO EURO model due to the availability of added sugard data within the food
composition tables of European countries(19). In addition, the consideration of
nutrients/components to include is influenced by the co-linearity amongst nutrients such
that the inclusion of additional correlated nutrients lends itself to redundancy(9). For
example, according to the model adapted by the Queensland government in Australia
under their Food for Sport Guidelines, total fat and added sugars were intentionally
excluded from consideration; rather, the model opted to include a nutrient criterion for
total energy, which was said to have accounted for restricting the amount of fat or
sugars from a food(20).
As discussed in Section 2.2.1, the nutrient criteria are applied across all foods in “across
the board” models, whereas the nutrient criteria are applied at the food category level in
“food category specific” models(1). It should be noted that the way in which the nutrient
criteria are applied in “food category specific” models varies widely across different
models(8). The nutrient criteria may be applied at one or more food category levels
(e.g., for main categories only or for both main and sub-categories). In addition, the
nutrient criteria may differ for various food categories, as certain nutrients/components
may be more pertinent for specific food categories (e.g., the consideration of sodium
content, as opposed to sugar content, may be more pertinent for cheese products).
Thus, the number of food categories and the level at which the nutrient criteria are
applied may not always align and adds to the complexity to the model, as illustrated by
the WHO EURO model in Table 2-1.
c Free sugars are defined as “monosaccharides and disaccharides added to foods and beverages by the manufacturer, cook, or consumer, and sugars naturally present in honey, syrups, fruit juices, and fruit juice concentrates (WHO, 2015)(19). d Added sugars are defined as “all monosaccharides and disaccharides added to foods and beverages by the manufacturer, cook, or consumer during processing or preparation” (WHO, 2015)(20).
9
Table 2-1 Food categories, nutrients/components, and nutrient criteria used in the WHO EURO modela
Main and Sub-Categories
Nutrients/Components and Corresponding Criteriab Total Fat (g)
Saturated Fat (g)
Total Sugars (g)
Added Sugars (g)
Non-sugar Sweeteners (g)
Salt (g) Energy (kcal)
Savoury snacks – – – 0 – 0.1 – Beverages
Milk drinks 2.5 – – 0 0 – – Other beverages – – – 0 0 – –
Breakfast cereals 10 – 15 – – 1.6 – Yoghurts, sour milk, cream and other similar foods
2.5 2.0 10 – – 0.2 –
Cheese 20 – – – – 1.3 – Ready-made and convenience foods and composite dishes
10 4 10 – – 1 225
Butter and other fats and oils
– 20 – – – 1.3 –
Bread, bread products and crisp breads
10 – 10 – – 1.2 –
Fresh or dried pasta, rice and grains
10 – 10 – – 1.2 –
Processed meat, poultry, fish and similar
20 – – – – 1.7 –
Processed fruit, vegetables and legumes
5 – 10 0 – 1 –
Sauces, dips and dressings
10 – – 0 – 1 –
Abbreviation: WHO EURO, World Health Organization Regional Office for Europe. a Data are from WHO, 2015(19). The model consists of seven exclusions: 1) chocolate and sugar confectionery, energy bars, and sweet toppings and desserts; 2) cakes, sweet biscuits, and pastries; other sweet bakery wares, and dry mixes for making such; 3) beverages: juices; 4) beverages: energy drinks; 5) edible ices; 6) foods containing >1 g of industrially-produced trans fat per 100 g total fat; and 7) foods with alcohol content ≥0.5% of total energy. It also consists of 2 exemptions: 1) fresh and frozen meat, poultry, fish, and similar; and 2) fresh and frozen fruit, vegetables, and legumes. b If a food exceeds on a per 100 g basis any of relevant nutrient criteria for that food category, the food is classified as not permitted for marketing to children.
In addition to the exemptions for foods or food categories within models such that they
are not subject to any nutrient criteria (as discussed in Section 2.2.2), models may also
include exemptions to specific nutrient criteria for certain foods or food categories. For
10
example, this is commonly done for nuts. As certain types of nuts contain high levels of
mono- or polyunsaturated fat, their total fat content is correspondingly high. However,
given that salted nuts with high sodium content are widely available and consumed, it is
imperative that the food category for nuts not be treated as an exemption without
subjection to any nutrient criteria. Thus, in order to account for their high unsaturated
fat content, nuts are often exempted from only the total fat criterion, but not other
criteria.
Lastly, the basis of the nutrient criteria applied in a model varies widely across different
models(8). This is due to the fact it is advantageous to have coherence between the
nutrient criteria that underpin the model and nutrient criteria used in other regulatory or
policy tools within the same jurisdiction(1). For example, the nutrients and their criteria
included in the model developed by the WHO Regional Office for the Americas/Pan
American Health Organization (PAHO) were based on the WHO Population Nutrient
Intake Goals to Prevent Obesity and Related Non-communicable Diseases (PNIGs) and
their updates(21,22), including the WHO guidelines on sugars(23) and sodium(24), and the
expert consultation on fats(25). The PNIGs, defined as the average dietary intake
recommended for maintenance of good health in a population(22), were formulated after
comprehensive reviews of all the updated evidence related to specific nutrient intakes
and public health outcomes(21). According to the PAHO model, a food is classified as
“excessive” in one or more of the critical nutrients (i.e., total fat, saturated fat, trans fat,
sodium, and free sugars) if it contains more than the corresponding maximum level
recommended in the PNIGs(21,22). Although the PNIGs relate to the overall diet and not
to individual foods, the rationale for their use as the basis of the nutrient criteria of the
model was that the consumption of individual foods classified as “excessive” increases
the likelihood that the diet also will exceed the recommended nutrient goals(21).
2.2.5 Reference Amounts
The reference amount of a NP model refers to the standard amount of a food that
serves as the basis for the calculations of the nutrients/components present in the
food(1). The most commonly used reference amounts include per 100 g and/or 100 mL,
per 100 kcal, or per serving of the food(1,9). The reference amount used by a model
varies across different models, and there are advantages and disadvantages associated
11
with various options(1,8). For example, the reference amount per 100 g and/or 100 mL is
frequently used because it often corresponds to the data from food composition
databases or the basis for other nutrition-related tools in regulations (e.g., nutrition
labels and nutrition and health claims in the European Union are determined on a per
100 g or 100 mL basis(16)). However, this reference amount is problematic for foods
consumed in small amounts because the level of certain nutrients may appear to be
high when calculated on a per 100 g basis (e.g., sodium in yeast extracts)(1). While a
per 100 kcal basis may be advantageous because it can give an indication of the
nutrient density of a food, it is problematic for low or high energy foods(1). For example,
the WHO PAHO model utilizes a derivative of the per 100 kcal basis, such that the
sodium criterion (i.e., ≥1 mg/kcal per serving) resulted in beverages with very low
sodium and energy content (e.g., 5 mg sodium and 1 kcal per serving) to be classified
as “excessive” in sodium, when in fact, these beverages were not significant sources of
sodium(26)e. Furthermore, a combination of reference amounts may be used within a
model(1,9). For example, according to the WHO PAHO model, the reference amount
used for several of the nutrient thresholds is the contribution of those nutrients to the
total energy of the food provided in a single serving (e.g., ≥30% of total energy from
total fat in any given quantity of the food)(21).
2.2.6 Output of Models
Depending on the output generated by NP models, they can be categorized as scoring
models, threshold models, or both(1). Scoring models generate a score for each food
and allow for foods to be ranked (e.g., from “healthiest” to “least healthy”)(1). In contrast,
threshold models typically generate binary classifications of the foods (e.g., “healthier”
or “less healthy”) and do not allow foods to be ranked(1). However, threshold models
that generate more than two classifications (e.g., the red, amber, and green
classifications by the UK traffic light model(27)) allow foods to be ranked into categories
e In our study on the application of NP models for the restriction of marketing unhealthy foods to children, we accounted for this discrepancy by adjusting the sodium criterion in these beverages. This adjustment is described in our publication presented in Appendix A (Labonté MÈ, Poon T, Mulligan C, Bernstein JT, Franco-Arellano B, L’Abbé MR. Comparison of global nutrient profiling systems for restricting the commercial marketing of foods and beverages of low nutritional quality to children in Canada. Am J Clin Nutr 106, 1471-1481.).
12
of increasing or decreasing healthfulness. Models that generate both types of output
allow for ranking and classification by specifying pre-determined threshold criteria or
cut-off scores. For example, the Ofcom model generates a total score between -15 to
40 (from “healthiest” to “least healthy”), and if this total score is greater than or equal to
the cut-off score of 4 for foods or 1 for beverages, then the food is classified as “less
healthy”(28).
As discussed in Section 2.1.2, it is ideal to use a single model to underpin different
policies within a jurisdiction, with the threshold criteria or cut-off scores adjusted for
different applications of the model(9). For example in the case of the Ofcom model, the
cut-off scores (i.e., 4 for foods, 1 for beverages) underpinning the restrictions on
marketing to children, in the context of protecting vulnerable populations, could be
shifted higher along the scale from -15 to 40 (from “healthiest” to “least healthy”) to less
stringent cut-off scores for the regulations on front-of-pack labelling, which are
applicable to the general population.
2.2.7 Example NP Models
As discussed above, the characteristics of a NP model vary widely across different
models. This is illustrated in Table 2-2, which provides a summary of the key
characteristics of several NP models developed by authoritative bodies that are
evaluated in this thesis.
13
Table 2-2 Summary of the key characteristics of several NP models used in this research
Characteristic of NP Model Ofcom(28) FSANZ(18) HCST(29) EURO(19) PAHO(21) Intended objective/ application
Restrictions on marketing foods to children
Regulation of health claims
Surveillance of dietary intakes of Canadians relative to CFG
Restrictions on marketing foods to children
Restrictions on marketing foods to children
Scope of model • Within: as sold or as consumed
• Outside: not specified
• Within: as sold or as consumed
• Outside: 4 categoriesa
• Within: as consumed
• Outside: 3 categoriesb
• Within: as sold or as consumed
• Outside: follow-up formula and growing-up milks
• Within: not specified • Outside: foods for
special uses (breast milk substitutes), food supplements, alcoholic beverages
Exclusions (n) Not specified 3c Not specified 7d Not specified Exemptions (n) 0 Not specified 0 2e 3f Food categories (n) 2g 3h 4i 20j 5k Nutrients/components (n) 7 7 4 8 6 Nutrients to limit
Energy Total fat
Saturated fat
Trans fat (industrial sources) (total) Sodium
Total sugars Free/added sugars (added) (free) Sweeteners
Nutrients to encourage Protein Fibre Fruits/vegetables (FVN) (FVNL)
14
Table 2-2 Summary of the key characteristics of several NP models used in this research
Characteristic of NP Model Ofcom(28) FSANZ(18) HCST(29) EURO(19) PAHO(21) Level at which nutrient criteria applied
Main categories; across the board
Main categories; across the board
Main categories; food category specific
Main and subcategories; food category specific
Main categories; across the board
Reference amount 100 g 100 g or mL Serving 100 g % energy of foodl Output Score and
classification Score and classification
Classification Classification Classification
Abbreviations: CFG, Canada’s Food Guide; EURO, World Health Organization Regional Office for Europe; FSANZ, Food Standards Australia New Zealand; FVN/L, fruits, vegetables, nuts, and/or legumes; HCST, Health Canada’s Surveillance Tool; NP, nutrient profiling; PAHO, World Health Organization Regional Office for the Americas/Pan American Health Organization. a Foods that are 1) intended for further processing, packaging, or labelling prior to retail sale; 2) delivered to a vulnerable person by a delivered meal organization; 3) provided to a patient in a hospital or medical institution, other than food in a package; and 4) special purpose foods. b 1) "Other" foods and beverages recommended in CFG (e.g., unsaturated fats and oils, water); 2) "other" foods and beverages not in the food groups of CFG (e.g., saturated and/or trans fats and oils; beverages; uncategorized; alcoholic beverages; high fat and/or sugar foods); and 3) foods and beverages that are not classified (i.e., recipes; foods with missing nutrient data in the Canadian Nutrient File; infant formula). c 1) Kava; 2) foods with alcohol content >1.15% by volume; and 3) infant formula. d 1) Chocolate and sugar confectionery, energy bars, and sweet toppings and desserts; 2) cakes, sweet biscuits, pastries, other sweet bakery wares, and dry mixes for making such; 3) beverages: juices; 4) beverages: energy drinks; 5) edible ices; 6) foods containing >1 g of industrially-produced trans fat per 100 g total fat; and 7) foods with alcohol content ≥0.5% of total energy. e 1) Fresh and frozen meat, poultry, fish, and similar; and 2) fresh and frozen fruit, vegetables, and legumes. f 1) Unprocessed or minimally processed foods; 2) culinary ingredients; and 3) freshly prepared dishes. g 1) Beverages; and 2) foods. h 1) Beverages; 2) any food not in category 1 or 3; and 3) cheese high in calcium (i.e., >320 mg/100 g of food) and fats. i 1) Vegetables and fruits; 2) grain products; 3) milk and alternatives; and 4) meat and alternatives. j The model consists of 17 main categories, with the beverage category containing 4 sub-categories, for a total of 20 unique categories: 1) chocolate and sugar confectionery, energy bars, and sweet toppings and desserts; 2) cakes, sweet biscuits, pastries, other sweet bakery wares, and dry mixes for making such; 3) savoury snacks; 4) beverages: juices; 5) beverages: milk drinks; 6) beverages: energy drinks; 7) beverages: other beverages; 8) edible ices; 9) breakfast cereals; 10) yogurts, sour milk, cream, and other similar foods; 11) cheese; 12) ready-made and convenience foods and composite dishes; 13) butter and other fats and oils; 14) bread, bread products, and crisp breads; 15) fresh or dried pasta, rice, and grains; 16) fresh and frozen meat, poultry, fish, and similar; 17) processed meat, poultry, fish, and similar; 18) fresh and frozen fruit, vegetables, and legumes; 19) processed fruit, vegetables, and legumes; and 20) sauces, dips, and dressings. k 1) Ultra-processed foods; 2) processed foods; 3) unprocessed or minimally processed foods; 4) culinary ingredients; and 5) freshly prepared dishes. l While sweeteners were evaluated based on their absence or presence in the ingredient list and sodium was evaluated on a per kcal basis, the thresholds for the other nutrients were presented on as a % energy of the food (e.g., an excess of total fat is ≥30% of total energy of the food).
15
2.3 Validation of NP Models
2.3.1 Types of Validity
In general, validity is defined as the adequacy with which a measurement reflects what
is intended to be measured(30). Correspondingly, validation refers to the process of
establishing that a method is sound or that data are correctly measured(31). It should be
noted that validity is distinct from other related concepts, including accuracyf, precisiong,
and reliability or reproducibilityh. There are two fundamental categories for examining
validity: internal and external(30,31). Internal validity is the adequacy of the measurement
for the specific population being studied, whereas external validity (also referred to as
generalizability) is the adequacy of the measurement when applied to wider populations
not under study(30,31).
In alignment with the definition of NP, the validity of a NP model refers to the adequacy
with which the model classifies the healthfulness of foods for the purpose of promoting
health and preventing disease. There are different types of validity, including criterion
(further classified into predictive and concurrent), convergent, discriminant, construct,
content, and face validity(1,2,10,32-34). The different types of validity vary with respect to
their robustness in contributing to the validation of a model. The classic definitions of
the various types of validity according to Porta (2014)(31) and Cronbach and Meehl
(1955)(35) are summarized in Table 2-3. In addition to the classic definitions, the
definitions of validity used in the context of NP are provided to allow for comparisons of
the variations in definitions used by different experts on NP validation, namely:
Scarborough et al. (2007)(33), Arambepola et al. (2008)(32), Townsend (2010)(34), Rayner
(who prepared the WHO manual on NP models (2011, in press(1))), and Cooper et al.
(2016(10), 2017(36)).
f Accuracy is the degree to which a measurement represents the true value of the attribute that is being measured(27). g Precision is the quality of being sharply defined or stated through exact detail(27). h Reliability or reproducibility is the degree of stability or capacity to yield the same result when a measurement is repeated under identical conditions(2,27).
16
Table 2-3 Definitions of different types of validity used by different researchers (shaded cells indicate those upon which analyses in Chapter 3 were baseda)
Type Definitions Reference Criterion Extent to which the measurement correlates with an external criterion of the phenomenon under
study; ideally, a gold standard(31) (32), but it need not(33). • Porta, 2014(31) • Arambepola et al., 2008(32) • Scarborough et al., 2007(33)
Accuracy of the NP model scores based on an externally derived objective measure of healthfulness.
• Cooper et al., 2016(10)
Extent to which the method is accurately based on an externally derived gold standard; examines whether method correlates in a predicted manner with variables with which, theoretically, it should correlate.
• Townsend, 2010(34)
Predictive (type of criterion)
Extent to which the measurement is able to predict an external criterion of the phenomenon under study.b
• Porta, 2014(31) • WHO, 2011 (in press)(1)
Independent criterion measure is obtained after the test score. • Cronbach and Meehl, 1955(35) Extent to which the system reflects a population’s or an individual’s diet change over time; extent to which the system reflects a change in nutritional and health status.
• Townsend, 2010(34)
Ability of the tool to predict future outcomes, in which case the tests are performed at different times and then the correlation between the result of the tool and the secondary outcome is determined.
• Cooper et al., 2016(10)
Concurrent (type of criterion)
Measurement and criterion refer to the same point in time.c • Porta, 2014(31) • Cronbach and Meehl, 1955(35)
Requires that the tool be tested against some other method acknowledged as a gold standard for assessing the same variable, and the two tests are conducted at the same time.
• Cooper et al., 2016(10)
Convergent Extent to which the system is accurately based on self-report (internally derived) data with hypothesized relation; examines whether method correlates in a predicted manner with variables with which, theoretically, it should correlate.
• Townsend, 2010(34)
Extent to which the measurement correlates with an external criterion of the phenomenon under study at the same point in time.
• WHO, 2011 (in press)(1)
Comparison with other measures, not necessarily better measures, of the same variable or a closely related variable.
• Arambepola et al., 2008(32)
17
Table 2-3 Definitions of different types of validity used by different researchers (shaded cells indicate those upon which analyses in Chapter 3 were baseda)
Type Definitions Reference Discriminant Extent to which the system discriminates between groups expected to be different. • Townsend, 2010(34)
Comparison with other measures, not necessarily better measures, of variables that are not closely related.
• Arambepola et al., 2008(32)
Construct Extent to which the measurement corresponds to theoretical concepts (constructs) concerning the phenomenon under study.d
• Porta, 2014(31) • Arambepola et al., 2008(32) • WHO, 2011 (in press)(1)
Correlation between how the NP model ranks the healthfulness of foods in comparison to other measures that determine the healthfulness of food.
• Cooper et al., 2016(10)
Comparison with other measures of the same variable, or closely related ones. • Cooper et al., 2017(36) Content Extent to which the measurement incorporates the domain of the phenomenon under study.e • Porta, 2014(31)
• WHO, 2011 (in press)(1) Extent to which the system covers the full range of meaning for the concept being measured; assessment of the consistency between the science underlying the algorithms and the science published in the peer-reviewed literature.
• Townsend, 2010(34)
Consideration and analysis of the components that make up the NP model with reference to current scientific literature.
• Cooper et al., 2016(10)
Face Extent to which a measurement or a measurement instrument appears reasonable on superficial inspection.
• Porta, 2014(31)
Extent to which the system is a useful tool to the consumer making food-purchase decisions in the marketplace; determined by end users of the system (i.e., the consumer).
• Townsend, 2010(34) • Cooper et al., 2016(10)
Abbreviation: NP, nutrient profiling. a The shaded cells indicate the definitions upon which the analyses in the validation study (Chapter 3) were based. Refer to Section 4.1.1 for the full discussion. b An example is an academic aptitude test that is validated against subsequent academic performance(31). c An example is a visual inspection of a wound for evidence of infection validated against bacteriological examination of a specimen taken at the same time(31). d For example, if on theoretical grounds the phenomenon should change with age, a measurement with construct validity would reflect such a change(31). e For example, a measurement of functional health status should embrace activities of daily living (e.g., occupational, family, and social functioning)(31).
18
2.3.2 Methods of Validity Testing
The validity of NP models can be tested using a variety of methods(2,9,37). These
methods are summarized in detail in Table 2-4. In addition, these methods are cross-
referenced to the type of validity that can be inferred, which often differs between
various experts on validation(1,2,10,32-34).
Briefly, one can assess whether the classifications of foods determined using the model
are aligned with, for example(1,2,9,36):
1) food-based dietary guidelines; 2) views of nutrition professionals; 3) classifications determined by other NP models; 4) foods included as part of either self-reported or theoretically or modelled “healthy”
and “less healthy” diets; or 5) foods for which their consumption has been associated with various health
outcomes.
The methods listed above are associated with their respective strengths and
limitations(2,9,37). The first several methods (i.e., methods 1 to 4) can be relatively
simpler, but may be less robust methodologically. For example, the views of nutrition
professionals and dietary patterns self-reported by “healthy” eaters are subject to bias
by individual, socioeconomic, and cultural factors(38). While theoretically or modelled
“healthy” and “less healthy” dietary patterns are relatively more objective, these diets
simulate food choices based on a set of pre-defined and purposefully selected
constraints (e.g., for protein, >65 g/day for models of “healthy” diets and <65 g/day for
models of “less healthy” diets)(38), which limit their real-world relevance compared to
observed dietary patterns of free-living individuals. Furthermore, while biomarkers and
medical records are objective indicators of health status, the methods that incorporate
them are complex, costly, and time-consuming as they are typically collected from
prospective cohort studies or randomized clinical trials(2). As such, the WHO
recommends that simpler validation methods should be applied first, typically during
early development of the model, to ensure robust classifications of the foods(9). More
complex validation methods should be applied thereafter to increase the evidence-base
and confidence in support of the use of the model(9).
19
Table 2-4 Methods of testing different types of validity used by different researchers (shaded cells indicate those upon which analyses in Chapter 3 were baseda)
Methods of Validity Testing that have been used in NP
Validity Type and Reference Criterion Predictive
(type of criterion)
Concurrent (type of criterion)
Convergent Discriminant Construct Content Face
Outcome is an observed change in health status or disease risk using biomarkers/medical records:
(34)
1) Measured at different time points (i.e., based on longitudinal data)
(1,2,10)
2) Measured at the same time (i.e., based on cross-sectional data)
(10)
Outcome is an observed change in dietary intake of individuals:
1) Measured at different time points (i.e., based on longitudinal data)
(10,34)
2) Measured at the same time (i.e., based on cross-sectional data)
(34)
3) Used to predict change in health status
(10)
Outcome is an observed change in gross food sales receipts
(10,34)
Outcome is a predicted change in dietary intake of individuals using theoretically or modelled “healthy” or “less healthy” diets or dietary patterns
(34) (1,10,32,37)
Determined by convening a panel of a group of experts in relevant fields (e.g., comparison to views of nutrition professionals)b
(33) (1,34) (10,36) (1,34)
20
Table 2-4 Methods of testing different types of validity used by different researchers (shaded cells indicate those upon which analyses in Chapter 3 were baseda)
Methods of Validity Testing that have been used in NP
Validity Type and Reference Criterion Predictive
(type of criterion)
Concurrent (type of criterion)
Convergent Discriminant Construct Content Face
Comparison of the usage of the system between nutrition professionals and high school graduates
(34)
Comparison to other NP models (1) (10,36)
Comparison to food-based dietary guidelines
c (32) d (32) (10,36) (1)
Outcome is the agreement with nutritional endpoints (e.g., energy density of foods)
(10)
Outcome is the consistency between scientific underpinnings of algorithm of model and current scientific literature
(10,34)
Consideration of nutrients that are of major public health concern
(32)
Outcome is the consumer groups’ use and understanding (e.g., FOP systems)
(10,34)
Abbreviations: BGH, Balance of Good Health; FOP, front-of-pack; NP, nutrient profiling. a The shaded cells indicate the definitions and methods upon which the analyses in the validation study (Chapter 3) were based. Refer to Section 4.1.1 for the full discussion. b Although not explicitly part of my MSc thesis, we are currently conducting a study to examine the validity of the NP model proposed for use in Canada against the views of nutrition professionals (refer to Appendix B). c The UK’s national food guide, BGH, indicates that individuals should consume many of foods from ‘fruit and vegetables’, ‘bread, other cereals and potatoes’; intermediate amounts of foods from ‘milk and dairy foods’, ‘meat, fish and alternatives’; and ‘fatty and sugary foods’ sparingly. Therefore, testing for convergent validity should indicate that 1) foods classified as “healthier” by the NP model should be found more frequently within foods categorized by the BGH as ‘fruit and vegetables’ and ‘bread, other cereals and potatoes’, and 2) foods categorized by the model as “less healthy” should be found more frequently within foods categorized by the BGH as ‘fatty and sugary foods’(32). d Testing for discriminant validity should indicate no relationship between the way a NP model categorizes foods and the way ‘milk and dairy foods’ and ‘meat, fish and alternatives’ are categorized by the BGH(32).
21
2.4 Current Status of NP and Models
2.4.1 Canadian Context
In Canada, NP has been used primarily at the provincial level for school foods(8). Only
two models developed for use at the national level were identified, namely the Canada
Provincial and Territorial Nutrient Criteria for Foods and Beverages in Schools in
2013(39) and the Health Canada’s Surveillance Tool (HCST) tier system developed in
2014 for assessing the adherence of the dietary intakes of Canadians to Eating Well
with Canada’s Food Guide(8). More recently on October 24, 2016, the Minister of Health
announced the launch of Health Canada’s Health Eating Strategy, which aims to
improve healthy eating information, strengthen labelling and claims, improve the
nutritional quality of foods, protect vulnerable populations, and support increased
access to and availability of nutritious foods(40). As part of this initiative, the Canadian
government has proposed to use NP in their policies on front-of-pack labelling and
restrictions on the marketing of foods to children(40).
In order to inform the use of NP in the proposed policy related to the restrictions on
marketing to children, we conducted a study that compared, in the Canadian context,
the degree of strictness and agreement between several NP models developed by
authoritative bodies for application in the marketing restrictions of foods to children(26).
This publication, along with author contributions, is presented in Appendix Ai.
In June 2017, Health Canada published the discussion paper for the public consultation
on restricting unhealthy food and beverage marketing to children, detailing the NP
model proposed for use in this policy(26). We are currently conducting a study to
examine the construct and convergent/concurrent validity of this proposed NP model
against views of nutrition professionals. The Ethics Review Application that was
i Labonté MÈ, Poon T, Mulligan C, Bernstein JT, Franco-Arellano B, L’Abbé MR. Comparison of global nutrient profiling systems for restricting the commercial marketing of foods and beverages of low nutritional quality to children in Canada. Am J Clin Nutr 106, 1471-1481.
22
submitted to and subsequently approved by the Review Ethics Board at the University
of Toronto, along with author contributions, is presented in Appendix Bj.
2.4.2 Global Context
A series of systematic reviews on the use of NP for various applications have been
updated over the years, namely by Rayner and colleagues at the University of Oxford
and other global collaborators. As part of the work conducted in developing the Ofcom
model for the restrictions on marketing to children, a review was published in 2004k, and
very limited data on NP directly relevant to marketing to children were identified(4-6). The
2004 review was subsequently updated in 2008, and 39 NP models that met the
inclusion criteria were identified in the updated review(6)l. As part of the work conducted
in building an unpublished catalogue of NP models for the WHO, the 2008 review was
updated again in 2013, and a total of 119 models (both included and excluded) were
identified(7)m. This 2013 unpublished catalogue was subsequently updated by our
research group in 2017, and a total of 387 models (both included and excluded) were
identified from this latest systematic review(8,41)n. The manuscript of this systematic
review, along with author contributions, is presented in Appendix Co. Results pertaining
to the current status of the validity of NP models from this systematic review are
described below.
The objective of our systematic review was to develop an up-to-date, accessible, and
global resource which summarizes the key characteristics of NP models with
j L’Abbé MR, Poon T, Labonté MÈ, Bernstein JT (2017). Ethics Review Application Form: Validation of a nutrient profile model proposed for restricting the marketing of unhealthy foods to children by nutrition professionals (protocol ID 34962). k The systematic literature searches were executed in 2003. l The systematic literature searches identified studies published between 2004 and 2007. m The systematic literature searches identified studies published between 2008 and 2012. n The systematic literature searches identified studies published between 2012 and 2016. o Labonté MÈ, Poon T, Gladanac B, Ahmed M, Franco-Arellano B, Rayner M, L’Abbé MR (unpublished). Nutrient profile models with applications in government-led nutrition policies aimed at health promotion and noncommunicable disease prevention: a systematic review. (To be submitted to Adv Nutr in December 2017).
23
applications in government-led nutrition policies and programs in order to support the
adoption or adaptation of existing NP models(8). NP models were identified from three
sources: 1) the unpublished catalogue completed in 2013 by Rayner et al.(7); 2) search
of the peer-reviewed literature using 3 databases; and 3) search of the grey literature
using 15 databases(8). Models included in the systematic review must have been
developed or endorsed by government or intergovernmental organizations, allowed for
the evaluation of individual food items, had publicly available nutrient criteria, amongst
other criteria(8). As illustrated in Figure 2-1, 387p potentially relevant NP models were
initially identified, with 119 of the models identified from the 2013 unpublished
catalogue(7) and 268q additional models from the full text assessment of more than 600
publications(8). A total of 78 models were included(8).
p A total of 387 models include 368 potential models first assessed for eligibility and 19 additional models identified during the eligibility assessment stage and evaluated for inclusion/exclusion. q A total of 268 models include 249 potential models from full-text review and 19 additional models identified during the eligibility assessment stage and evaluated for inclusion/exclusion.
24
Figure 2-1 Flow diagram of the publication and NP model selection. Data are reproduced from the manuscript of the systematic review presented in Appendix C (Labonté MÈ, Poon T, Gladanac B, Ahmed M, Franco-Arellano B, Rayner M, L’Abbé MR (unpublished). Nutrient profile models with applications in government-led nutrition policies aimed at health promotion and noncommunicable disease prevention: a systematic review. (To be submitted to Adv Nutr in December 2017)). Data are current as of December 22, 2016. 1 Of these 19 models, 15 were specifically identified during the process of assessing the eligibility of the first 368 potential models (e.g., through additional documentation reviewed), and 4 were identified from other sources (e.g., personal communication or email newsletter received during the weeks that the eligibility assessment process occurred). Abbreviations: lit., literature; NP, nutrient profiling; WHO, World Health Organization.
25
Data on any form of validity testing were not identified specifically for 58% (n=45 of 78)
of these models developed or endorsed by government organizations(8). However, 6 of
these 45 models were considered to be indirectly validated to a minor extent because
either 1) data on validity testing were identified for a previous version of the specific
model which had been updatedr, or 2) the specific model was adapted from another
model for which some validity testing had been identifieds. In contrast, data on some
form of validity testing were identified for 42% (n=33 of 78) of models(8). Across these
33 models, data on:
face validity were identified for 5 models; content validity were identified for 8 models; construct/convergent validity were identified for 24 models; and criterion/predictive validity, the most robust assessment of validity, were identified
for 8 models.
Furthermore, adequate validity testing, with regards to both the variety in type and
robustness of validity testing (e.g., construct/convergent and criterion/predictive), was
identified for only 15% (n=12 of 78) of modelst.
The Ofcom model remained the most frequently validated model(8), a finding which was
also reported by Rayner et al. in 2013(7) and again by Cooper et al. in 2016(10). Given
that face validity has limited relevance for models with non-consumer end-users(34), the
Ofcom model has demonstrated all other types of validity, including content(4,5,42),
construct(10,32), convergent(32), discriminant(32), concurrent(33), and predictive(43-47) validity.
r This pertained to two models: 1) England Requirements for School Food Regulations; and 2) Minimum requirements and specifications for food items allowed in the Women, Infants and Children food packages (supplemental foods). s This pertained to four models: 1) Australian National Healthy School Canteens Project; 2) Government of Victoria (Australian state) Go For Your Life Healthy Canteen Kit; 3) Ireland broadcasting authority model for restricting the marketing of food and drink to children; and 4) Georgia (U.S. state) Women, Infants and Children (WIC) approved food list. t 1) International Choices model; 2) Keyhole model from Sweden, Denmark, Norway, and Iceland; 3) U.S. requirements for health claims for foods; 4) U.S. definition of a “healthy” food; 5) Health Canada’s Surveillance Tool tier system; 6) UK Ofcom model; 7) UK Traffic Light Labelling; 8) France Nutri-Score (or Five-Colour Nutrition Label); 9) France SAIN, LIM system; 10) Danish Code of responsible food marketing communication to children; 11) South Africa NP model; and 12) Food Standards Australia New Zealand’s Nutrient Profiling Scoring Criterion.
26
For the assessments of predictive validity, the Ofcom model was adapted into a dietary
index to represent the overall quality of an individual’s diet(43-47). This Ofcom-based
dietary index was validated based on data from a prospective cohort studyu conducted
in a UK population (n=10,308 adults)(43) and a longitudinal cohort nested in a
randomized controlled trialv conducted in a French population (n=13,017 adults)(44-47).
Diets that consisted of foods with lower Ofcom scores (i.e., “healthier” foods) were
significantly associated with decreases in the risk of all-cause mortality(43), cancer
mortality(43), long-term weight gain(44), metabolic syndrome(45), overall cancer(46), and
cardiovascular disease(47).
A detailed summary of the validity testing identified for all 78 included models is
presented in Table 6 in the manuscript of the systematic review (Appendix C).
2.4.3 Knowledge Gaps
Based on the series of systematic reviews discussed previously(4-8), the number of
potential NP models globally has increased dramatically, particularly within the last 5
years. Given this proliferation of models, there is less need for the development of new
models, which requires extensive resources and expertise(2). As such, the adaptation of
an existing model is preferred by the WHO and is becoming increasingly common
practice for government and industry(1,2,9). Specifically, the WHO provides guidance on
the process of adapting a model, and at the outset, there are two key considerations(9).
The first is the organization that developed the model(1,9), as data have shown that
industry-led models were less stringent with regards to the advertising of energy-dense
foods compared to government-led models(48). The second is the data on the validity of
the model(1,9). In fact, there is consensus amongst the scientific community that it is
prudent to adapt a model that has been developed by an authoritative body, and more
importantly, validated(1,2,9,10,34,37,48). Thus, in the context of adaptation, models need to
be validated when used for different applications and in different countries, as it is
u Whitehall II study. v The Supplémentation en Vitamines et Minéraux Antioxydants (SU.VI.MAX) cohort. The intervention study lasted up to eight years from 1994 to 2002, followed by the observational follow-up of health events until 2007.
27
unknown whether the criteria that underpins a model developed to assess foods for one
application or from one population are appropriate for another(1,9).
However, although many models exist, the majority of NP models have not been fully
evaluated, compared, or validated prior to their adaptation or implementation(1,2,8,10).
Even amongst models that have been evaluated for validity, most have been validated
only to a limited extent(8). In order to identify published studies with a design similar to
the study presented in Chapter 3 (i.e., assessment of validity based on agreement
between different NP models), references were reviewed from three data sources:
1) the systematic review on the construct and criterion-related validation of NP
models conducted by Cooper et al. (2016)(10), in which the literature searches
identified studies published up to June 30, 2015;
2) our aforementioned systematic review on the application of NP models(8), in
which the searches identified studies published between 2008 and 2016; and
3) additional PubMed literature searches conducted in 2017w to identify validation
studies published in 2016 and 2017 after the searches from the two other
systematic reviews were done.
A review of these data sources resulted in the identification of 19 studies in which
construct/convergent validity was assessed using the method of comparing agreement
between different NP models (Table 2-5). Studies in which different NP models were
compared but did not include discussions on the validity of the models, such as the
study by Eržen et al. (2015)(49)x, were not included in this table. None of the 19 studies
examined the construct/convergent validity of any NP model when applied to the
Canadian food supply.
w Three searches were conducted using PubMed in January, June, and December 2017 using the following search terms in any field of the records: (“nutrient profiling” or “nutrient profile” or “nutritional quality”) and (validity or validation). x Although the agreement between the classifications determined using the Ofcom and FSANZ models for 125 foods available in the Slovenian marketplace was assessed, the validation of the models was not an objective of the study, nor was it discussed within the study.
28
Table 2-5 Studies in which construct/convergent validity was assessed based on agreement between NP models (n=19)a
Reference Food Supply (n=Foods Examined)
NP Models Compared
Wicks et al., 2017(50)
South Africa (n=197) 1) South African health claims model 2) UK Ofcom model 3) WHO EURO model 4) WHO Eastern Mediterranean Regional Office model 5) South African Department of Health marketing to
children model Faulkner et al., 2014(51)
UK (n=32) 1) UK Traffic Light 2) UK Ofcom
Clerfeuille et al., 2013(37)
France (n=597) 1) Choices International 2) Green Keyhole 3) Agence Française de Sécurité Sanitaire des Aliments 4) European Commission draft model 5) Food Profiler
Rosentreter et al., 2013(52)
New Zealand (n=407) 1) UK Traffic Light 2) FSANZ
Scarborough et al., 2013(53)
UK (n=336) 1) Brazilian model 2) Danish Forum Code 3) Disney model 4) PepsiCo model 5) UK Ofcom 6) U.S. CSPI 7) U.S. Interagency 8) EU Pledge
Trichterborn et al., 2012(54)
Germany (n=307) 1) Swedish Keyhole 2) Choices International 3) UK Ofcom 4) SAIN/LIM score 5) U.S. FDA health claims criteria
Drewnowski and Fulgoni, 2011(55)
U.S. (n=1,045) 1) Nutrient Rich Food Index 2) U.S. National Heart, Lung, and Blood Institute “Go”,
“Slow”, “Whoa” Roberto et al., 2011(56)
U.S. (n=100) 1) U.S. Smart Choices 2) UK Ofcom
Trichterborn et al., 2011(57)
France, Germany, Spain, Sweden, UK (n=238)
1) Choices International 2) U.S. Smart Choices 3) UK Ofcom 4) LIM score 5) U.S. FDA health claims criteria
Trichterborn et al., 2011(58)
France, Germany, UK (n=242)
1) Swedish Keyhole 2) Choices International 3) U.S. Smart Choices 4) UK Ofcom 5) SAIN/LIM score 6) U.S. FDA health claims criteria
Eyles et al., 2010(59)
New Zealand (n=550) 1) Modified Heart Foundation tick 2) Modified FSANZ
Hebden et al., Australia (n not 1) Nestle nutritional profiling system
29
Table 2-5 Studies in which construct/convergent validity was assessed based on agreement between NP models (n=19)a
Reference Food Supply (n=Foods Examined)
NP Models Compared
2010(60) reported) 2) Kraft Sensible Solutions 3) Cereal Partners Worldwide Nutrition Foundation criteria 4) Kellogg’s global nutrient criteria 5) Fonterra Good Choice 6) FSANZ
Walker et al., 2010(61)
Australia (n=388) 1) UK Traffic Light 2) UK Ofcom
Drewnowski et al., 2009(62)
U.S. (n=378) 1) Nutrient Adequacy Score 2) Nutrient Density Score 3) Nutrient Rich Food models 4) LIM score 5) UK Ofcom
Walker et al., 2008(63)
Australia (n=1,933) 1) New South Wales School Canteens criteria 2) Uncle Toby's nutrition criteria 3) UK Traffic Light
Garsetti et al., 2007(64)
Netherlands, UK (n=62)
1) A Little A Lot (UK Ministry of Agriculture) 2) U.S. FDA health claims criteria 3) Netherlands Tripartite 4) UK Ofcom 5) U.S. CSPI
Labouze et al., 2007(65)
France, UK (n=98) 1) Nutrimap 2) UK Ofcom 3) Netherlands Tripartite
Quinio et al., 2007(66)
Belgium, Denmark, France, Ireland, Italy (n not reported)
1) UK Ofcom 2) Netherlands Tripartite 3) U.S. FDA health claims criteria
Azais-Braesco et al., 2006(67)
Europe/UK (n=125) 1) Calorie For Nutrient Index 2) Nutritious Food Index 3) Ratio of Recommended to Restricted Foods 4) UK Ofcom
Abbreviations: CSPI, Center for Science and the Public Interest; EU, European Union; FDA, Food and Drug Administration; FSANZ, Food Standards Australia New Zealand; UK, United Kingdom, U.S., United States. a The data for all studies, except the study by Wicks et al. (2017)(50), were adapted from Cooper et al. (2016)(10).
Moreover, as discussed in Section 2.2, different NP models can vary substantially in
their design and characteristics, and this variation ultimately gives rise to inherent
differences in the manner in which these models classify the healthfulness of foods.
Furthermore, as discussed in Section 2.3, the extent of the documentation and reporting
of validity testing varies widely across different NP models. This is due, in part, to the
infancy of the methods of validation(2). As such, several researchers have suggested
30
that validity testing of NP models should be given the highest priority in this field of
research(3,10,11).
Based on the data presented in this literature review, the need for validity testing of NP
models is a main knowledge gap in NP. Additionally, amongst models for which
construct/convergent validity testing has been conducted, the testing was typically done
in a small sample of indicator foods for countries other than Canada. Thus, the
objective of this thesis was to examine the content and construct/convergent validity of
several NP models developed by authoritative bodies in assessing the nutritional quality
of foods from a large, branded, Canadian database.
31
Chapter 3
3 Study on Validation of NP Models
3.1 Manuscript This manuscript has been submitted to the British Journal of Nutrition for publication:
Poon T, Labonté MÈ, L’Abbé MR. Comparison of nutrient profiling models for assessing
the nutritional quality of foods: a validation study (submitted October 12, 2017; revisions
to be submitted by January 29, 2018).
The contributions of the authors to the manuscript are summarized in Table 3-1.
Table 3-1 Authors’ contributions to study on validation of NP models
Task Author Designed research study Poon, Labonté, L’Abbé Generated data for nutrient profile models Poon, Labonté Analyzed data, ran statistical analyses Poon Wrote manuscript Poon Provided comments/revisions to draft and final manuscripts
Labonté, L’Abbé
Submitted manuscript to journal Poon
3.1.1 Abstract
Nutrient profiling (NP) is a method for evaluating the healthfulness of foods. Although
many NP models exist, most have not been validated. This study aimed to examine the
content and construct/convergent validity of four models from different regions:
Australia/New Zealand (FSANZ), Canada (HCST), Europe (EURO), and Americas
(PAHO). Content validity was assessed by examining the nutrients/components
considered by each model. Using data from the 2013 University of Toronto Food Label
Information Program (n=15,342 foods/beverages), construct/convergent validity was
assessed by comparing the classifications of foods determined by each model to a
previously validated model, which served as the reference (Ofcom). The parameters
assessed included agreement (kappa statistic (κ)), discordant classifications
(McNemar’s test), and associations (Cochran-Armitage trend test). Analyses were
32
conducted across all foods and by food category. All models exhibited moderate
content validity. Although positive associations were observed between each model
and Ofcom (all P<0.001 for trend), agreement with Ofcom was “fair” for HCST (κ=0.26)
and PAHO (κ=0.28), “moderate” for EURO (κ=0.54), and “near perfect” for FSANZ
(κ=0.89). There were discordant classifications with Ofcom for 5.3% (FSANZ), 22.0%
(EURO), 33.4% (PAHO), and 37.0% (HCST) of foods (all P<0.001).
Construct/convergent validity was confirmed between FSANZ and Ofcom, and to a
lesser extent between EURO and Ofcom. Numerous incongruencies with Ofcom were
identified for HCST and PAHO, which highlights the importance of examining
classifications across food categories, the level at which differences between models
become apparent. These results may be informative for regulators seeking to adapt
existing models for use in country-specific applications.
3.1.2 Introduction
Nutrient profiling (NP), defined as the science of classifying foods according to their
nutritional composition for the purpose of promoting health and preventing disease, is a
relatively new term in the field of nutrition research(1,9). Several of the earliest forms of
NP were introduced by government bodies in the 1980s and 1990s, including the United
States (U.S.) Special Supplemental Nutrition Program for Women, Infants, and Children
in 1980(12), Swedish Keyhole in 1989(13), and disqualifying nutrient levels for U.S. health
claims in 1993(14). The term NP gained ground following the development of the Ofcom
model by the United Kingdom (UK) Food Standards Agency in 2004 to 2005(4,5) and the
mention of nutrient profiles in Regulation (EC) No 1924/2006 on nutrition and health
claims by the European Commission in 2006(16). In 2010, NP became even more widely
known when the World Health Organization (WHO) provided to its Member States a set
of recommendations on the marketing of foods and beverages to children, one of which
advocated the use of NP models in defining the products to be covered by the
marketing restrictions(17). Globally, NP is recognised now as a transparent and
reproducible method of evaluating the healthfulness of foods, and for its use in
numerous applications in government and industry (e.g., front-of-pack food labelling,
food taxes, reformulation)(2,3).
33
Based on a series of systematic reviews that have been updated over the years, the
number of potential NP models identified globally was as follows: 39 (included models
only) in 2008(6), 119 in 2013(7), and 387 in 2016 (Labonté ME, Poon T, Gladanac B et
al., unpublished results). Given this recent proliferation of NP models and the extensive
resources required to develop and validate a new model, the adaptation of an existing
model is preferred by the WHO and is becoming increasingly common practice for
government agencies(1,2,9). It is prudent to adapt a model that has been developed by
an authoritative body, and more importantly, validated(1,2,9,48). However, although many
models exist, the majority of NP models have not been fully evaluated or validated prior
to their implementation(2,10). This is due, in part, to the infancy of the methods used to
validate NP models(2). Several researchers have suggested that validity testing of NP
models should be given the highest priority in this field of research(3,10,11).
There are several types of validity (e.g., content, construct, convergent), and there are
different methods of testing for validity(1,9,34). Content validity is defined as the extent to
which the model encompasses the full range of meaning for the concept being
measured(34). One method of testing for content validity is to assess the consistency
between the algorithmic underpinnings of a model and the current scientific literature(34),
such as whether a model considers nutrients of public health concern. Construct
validity is the extent to which the model corresponds to theoretical constructs under
study(1), and convergent validity is the extent to which the model correlates with
variables that theoretically should correlate at the same point in time(1,34). The
comparison of one NP model to another has been used as a method of testing either
construct(10) or convergent validity(1). In particular, the comparison to a model that has
been previously validated is considered by the WHO to be a reasonably strong form of
validation(1). Thus, the objective of this study was to examine the content and
construct/convergent validity of several NP models developed by authoritative bodies for
the assessment of the nutritional quality of pre-packaged foods from a large, national,
branded database.
34
3.1.3 Methods
NP models
The key characteristics of the models examined in this study are summarised in Table
3-2 and described below.
The Ofcom model was developed for the regulation of television advertising to
children(28). The model consists of two food categories: 1) beverages; and 2) foods. It
takes into consideration a total of seven nutrients to limit and nutrients/food components
to encourage, the latter including fruit, vegetable, and nut (FVN) content. In order to
estimate the FVN content of a food without quantitative declarations in the ingredient
list, which are not required in Canada, the presence and positions of the FVN
ingredients within the ingredient list were used (refer to Table 3.4 in Section 3.1.6)y.
Based on the level of nutrients/components present per 100 g of the food, the model
generates a summary score in which a lower score represents a food with a more
favourable nutritional profile. The model also classifies the food as “permitted” or “not
permitted” for advertising to children based on pre-determined cut-off scores for foods
and beverages.
The Food Standards Australia New Zealand (FSANZ) model was developed for the
regulation of health claims on foods(18). As a derivative of the Ofcom model, the FSANZ
model is similar, except for the following characteristics: it consists of an additional food
category of cheeses high in calcium (i.e., >320 mg/100 g) and fats; it considers nutrients
on a per 100 mL basis in addition to 100 g; and it considers legumes in addition to FVN
(i.e., FVNL). The method used to estimate the FVNL content for the FSANZ model was
similar to that used to estimate the FVN content for the Ofcom model, and is described
in detail elsewhere(68). The model classifies the food as “permitted” or “not permitted” to
carry health claims based on pre-determined cut-off scores for each food category.
The Health Canada Surveillance Tool (HCST) tier system was developed to assess the
adherence of the dietary intakes of Canadians to the dietary guidance provided by
y Table 3.4 is presented as a supplementary table in the manuscript.
35
Canada’s Food Guide (CFG)(29). The model consists of four food categories: 1)
vegetables and fruits; 2) grain products; 3) milk and alternatives; and 4) meat and
alternatives. It takes into consideration four nutrients to limit (i.e., total fat, saturated fat,
sodium, and total sugars). Based on the level of nutrients present per serving of the
food, the model classifies the food into one of four tiers as follows: tier 1 and 2 foods are
in line with CFG and should be consumed often; tier 3 foods are partially in line with
CFG and should be consumed less often; and tier 4 foods are not in line with CFG and
their consumption should be limited.
The WHO Regional Office for Europe (EURO) model was developed for the restriction
of marketing less healthy foods to children(19). The model consists of 20 food categories
(listed in a footnote to Table 3-2). It takes into consideration eight nutrients to limit,
including industrially-produced trans fat and added sugars. Given that quantitative
declarations of industrially-produced trans fat are not required in Canada, the amount
was estimated based on the presence of hydrogenated or partially hydrogenated oils in
the ingredient list and the total trans fat level declared in the Nutrition Facts table.
Instead of added sugars, free sugar levels were used in this study because the WHO
considers free sugars as part of their guidelines on sugars(23). In order to estimate the
free sugar content of a food without a quantitative declaration in the Nutrition Facts
table, which is not required in Canada, the University of Toronto’s free sugar algorithm
was used; this algorithm is described elsewhere(69). Based on the level of nutrients
present per 100 g of the food, the model classifies the food as “permitted” or “not
permitted” for marketing to children. It should be noted that seven food categories are
not subject to the nutrient criteria and are automatically classified as “permitted” (i.e.,
fresh and frozen meat/poultry/fish; fresh and frozen fruits/vegetables/legumes) or “not
permitted” for marketing (i.e., confectionery; sweet bakery products; juices; energy
drinks; edible ices).
The WHO Regional Office for the Americas/Pan American Health Organization (PAHO)
model was developed for a variety of regulatory purposes aimed at addressing the
obesity epidemic(21). The model consists of five food categories: 1) ultra-processed
foods; 2) processed foods; 3) unprocessed or minimally processed foods; 4) culinary
ingredients; and 5) freshly prepared dishes. It takes into consideration six nutrients to
36
limit. Based on the level of nutrients present per % energy of the food, the model
classifies the food as “not excessive” in any nutrient or “excessive” in one or more of the
nutrients. It should be noted that three food categories (i.e., unprocessed or minimally
processed foods; culinary ingredients; freshly prepared dishes) are not subject to the
nutrient criteria and are automatically classified as “not excessive” in any nutrient.
In this study, “healthier” foods were defined as those permitted for marketing to children
as per the Ofcom and EURO models, permitted to carry health claims as per FSANZ,
classified as tier 1 or 2 foods as per HCST, and not excessive in any nutrient as per
PAHO (Table 3-2). Correspondingly, “less healthy” foods were defined as those not
permitted for marketing to children as per the Ofcom and EURO models, not permitted
to carry health claims as per FSANZ, classified as tier 3 or 4 foods as per HCST, and
excessive in one or more nutrients as per PAHO.
Content validity
For the FSANZ, HCST, EURO, and PAHO models, content validity was assessed by
examining the consistency between the nutrients/food components included in the
models versus those considered by the WHO to be of immediate importance in
promoting health and preventing disease. These included the nutrients/food
components explicitly stated within objective three of the WHO’s Global action plan for
the prevention and control of non-communicable disease 2013-2020 as those to
encourage (i.e., unsaturated fat and fruits and vegetables) and those to limit (i.e.,
energy, total fat, saturated fat, trans fat, sodium, and sugars (type not specified))(70,71).
Construct/convergent validity
Construct/convergent validity was assessed by comparing the classifications of foods
determined by each of the models (FSANZ, HCST, EURO, and PAHO) versus those
determined by a reference model using several parameters. The Ofcom model was
chosen as the reference model because it has been extensively validated using various
methods for different applications in multiple countries(3). Specifically, the Ofcom model
has been demonstrated to have content(4,5,42), construct(10,32), convergent(32),
discriminant(32), and concurrent validity(33). In addition, the adaptation of the Ofcom
37
model into a dietary index to represent the overall quality of an individual’s diet has been
demonstrated to have predictive validity(43-47). Details of the food database and
parameters used in this assessment are provided below.
Food database
A total of 15,342 pre-packaged foods and beverages from the University of Toronto’s
Food Label Information Program (FLIP) 2013 database were examined. Data were
collected in 2013 across the four largest grocery chains in Canada, which represented
75.4% of the grocery retail market share(69). The product information collected included
the Nutrition Facts table and ingredient list, amongst other data. Details of FLIP 2013
are provided elsewhere(69). Foods were classified into the 22 food categories as per
Schedule M of the Food and Drug Regulations (version in force between March 2012
and December 2016)(72). A total of 115 products were excluded from the analyses: 55
products due to manufacturer errors in the nutrient declarations in the Nutrition Facts
table (i.e., >20% difference between the calories declared and calories calculated using
Atwater factors for macronutrients) and 60 products that did not align with any Schedule
M category (i.e., 55 meal replacements, 4 instant or dry yeast products, and 1 natural
health product). Thus, 15,227 unique products were available for analyses. In order to
generate the classifications of the foods for each NP model, the 15,227 foods in FLIP
2013 were first classified independently by two authors (T.P. and M.E.L.) into the food
categories specific to the NP models using information from the ingredient lists and/or
pre-classifications from Schedule M food categories and subcategories and sugar-
focused categories from Bernstein et al.(69). For all models, nutrient data for products in
their “as consumed” form were used. Subsequently, the nutrient criteria for each model
were applied to the foods.
Statistical analyses
Construct/convergent validity was assessed using four parameters. First, pairwise
agreement between each model and the Ofcom model in the proportions of foods
classified as “healthier” or “less healthy” was assessed across all foods (n=15,227) and
by the 22 Schedule M food categories using the kappa statistic (κ, 95% CI), as follows:
0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80
38
“substantial”; 0.81 to 0.99 “near perfect”(73). Second, discordant classifications
(hereafter referred to as “discordance”) between each model and Ofcom was defined as
the sum of the percentage of foods classified as “healthier” by a model but “less healthy”
by Ofcom and the percentage of foods classified as “less healthy” by a model but
“healthier” by Ofcom. Discordance was assessed across all foods and by food category
using McNemar’s test for paired data. Third, the association between the proportion
(and 95% CI) of foods classified as “less healthy” by each model and quartiles of Ofcom
scores was assessed across all foods using the Cochran-Armitage trend test.
Fourth, because the FSANZ and HCST models also generated ordinal outcomes (Table
3-2), cross-classification analyses between quartiles of FSANZ scores and the four
HCST tiers versus quartiles of Ofcom scores were conducted. Exact agreement was
defined as the classification of a food in the same quartiles/tiers using different models
(e.g., FSANZ Q1 and Ofcom Q1; HCST tier 4 and Ofcom Q4). In addition, agreement
within an adjacent (i.e., ±1) quartile/tier (e.g., FSANZ Q1 and Ofcom Q2; HCST tier 2
and Ofcom Q1) and disagreement (e.g., FSANZ Q1 and Ofcom Q3; HCST tier 2 and
Ofcom Q4) were assessed. Lastly, gross misclassification was defined as the
classification of a food in opposite quartiles/tiers (e.g., FSANZ Q1 and Ofcom Q4; HCST
tier 4 and Ofcom Q1). A P-value <0.05 was considered statistically significant.
Statistical analyses were conducted using SAS 9.4 (SAS Institute Inc., Cary, NC, USA).
39
Table 3-2 Summary of NP models examined
Ofcom (referencea) FSANZ HCST EURO PAHO Food categories (n) 2 3 4 20b 5 Reference amount 100 gc 100 g or mL Serving 100 g % energy of foodd Nutrients/food components (n)
7 7 4 8 6
Energye Total fate
Saturated fate
Unsaturated fatf Trans fate (industrially-
produced)g (total)
Sodiume
Total sugarse Free/added sugarse (added)h (free) Sweeteners
Protein Fibre Fruits/vegetablesf (FVN)i (FVNL)i
Continuous outcome (Ofcom score) (FSANZ score) Ordinal outcome (Ofcom quartile) (FSANZ quartile) (HCST tiers 1 to 4) Binary outcome
“Healthier” foods Permitted for marketing to children
Permitted to carry health claims
Tier 1 and 2 foods in line with CFG
Permitted for marketing to children
Not excessive in any nutrient
“Less healthy” foods Not permitted for marketing to children
Not permitted to carry health claims
Tier 3 foods partially in line with CFG and tier 4 foods not in line with CFG
Not permitted for marketing to children
Excessive in ≥1 nutrient(s)
Abbreviations: CFG, Canada’s Food Guide; EURO, World Health Organization Regional Office for Europe; FSANZ, Food Standards Australia New Zealand; FVN/L, fruits, vegetables, nuts, and/or legumes; HCST, Health Canada Surveillance Tool; NP, nutrient profiling; PAHO, World Health Organization Regional Office for the Americas/Pan American Health Organization.
40
a The Ofcom model was chosen as the reference model for the assessment of construct/convergent validity in this study. b The model consists of 17 main categories, with the beverage category containing 4 sub-categories, for a total of 20 unique categories: 1) chocolate and sugar confectionery, energy bars, and sweet toppings and desserts; 2) cakes, sweet biscuits, pastries, other sweet bakery wares, and dry mixes for making such; 3) savoury snacks; 4) beverages: juices; 5) beverages: milk drinks; 6) beverages: energy drinks; 7) beverages: other beverages; 8) edible ices; 9) breakfast cereals; 10) yogurts, sour milk, cream, and other similar foods; 11) cheese; 12) ready-made and convenience foods and composite dishes; 13) butter and other fats and oils; 14) bread, bread products, and crisp breads; 15) fresh or dried pasta, rice, and grains; 16) fresh and frozen meat, poultry, fish, and similar; 17) processed meat, poultry, fish, and similar; 18) fresh and frozen fruit, vegetables, and legumes; 19) processed fruit, vegetables, and legumes; and 20) sauces, dips, and dressings(19). c Although a reference amount of 100 g for foods and beverages was specified as part of the model, a reference amount of 100 mL for beverages and other products in liquid form was used for the assessment of construct/convergent validity in this study. d While sweeteners were evaluated based on their absence or presence in the ingredient list and sodium was evaluated on a per kcal basis, the thresholds for the other nutrients were presented on as a % energy of the food (e.g., an excess of total fat is ≥30% of total energy of the food)(21). e For the assessment of content validity in this study, the nutrients to limit according to the WHO include energy, total fat, saturated fat, trans fat, sodium, and sugars (type not specified)(70,71). f For the assessment of content validity in this study, the nutrients to encourage according to the WHO include unsaturated fat and fruits and vegetables(70,71). g For the assessment of construct/convergent validity in this study, the industrially-produced trans fat content of a food was estimated based on the presence of hydrogenated or partially hydrogenated oils in the ingredient list and the total trans fat level declared in the Nutrition Facts table. h Although added sugars were specified as part of the model, free sugar levels estimated using the University of Toronto’s free sugar algorithm(69) were used for the assessment of construct/convergent validity in this study. i For the assessment of construct/convergent validity in this study, the FVN/L content of a food was estimated based on the presence and positions of the FVN/L ingredients within the ingredient list.
41
3.1.4 Results
Content validity
Of the eight nutrients/food components considered by the WHO to be of immediate
importance in promoting health and preventing disease (i.e., energy, total fat, saturated
fat, unsaturated fat, trans fat, sodium, sugars (type not specified), and fruits and
vegetables)(70,71), each of the NP models considered at least four of these nutrients/food
components (Table 3-2). With respect to the nutrients to limit, saturated fat, sodium,
and some form of sugar (e.g., total, free, or added) all were considered in each of the
four models. Total fat was considered in the HCST, EURO, and PAHO models, energy
was considered in FSANZ and EURO, and trans fat was considered in EURO and
PAHO. With respect to the nutrients/food components to encourage, the fruit and
vegetable content of foods was considered in only the FSANZ model, and unsaturated
fat was not considered in any model.
Construct/convergent validity
Across all foods, a positive association was observed between each model and the
Ofcom model, such that the proportion of foods classified as “less healthy” by FSANZ,
HCST, EURO, or PAHO increased across quartiles of Ofcom scores, with the highest
quartile representing the “least healthy” foods (all P<0.001 for trend) (Table 3-3).
However, varying levels of agreement and discordance were observed between each
model and Ofcom, as described below.
42
Table 3-3 Associations between the proportions (%, 95% CI) of foods classified as “less healthy” by the models and quartiles of Ofcom scores (n=15,227)
NP model Foods (n) Quartiles of Ofcom (reference) scores P for trenda Q1 (“healthier”) Q2 Q3 Q4 (“less healthy”)
% 95% CI % 95% CI % 95% CI % 95% CI FSANZ 15,183b 0.7 0.5, 1.0 29.5 27.9, 31.1 92.3 91.5, 93.2 90.5 89.5, 91.4 <0.001 HCST 15,165c 22.4 21.2, 23.6 61.8 60.1, 63.5 49.1 47.5, 50.7 73.8 72.4, 75.3 <0.001 EURO 15,182d 36.2 34.9, 37.6 63.7 62, 65.4 94.4 93.6, 95.1 94.4 93.7, 95.1 <0.001 PAHO 15,182d 58.9 57.5, 60.4 93.8 92.9, 94.6 96.3 95.7, 96.9 95.7 95.1, 96.4 <0.001 Abbreviations: EURO, World Health Organization Regional Office for Europe; FSANZ, Food Standards Australia New Zealand; HCST, Health Canada Surveillance Tool; NP, nutrient profiling; PAHO, World Health Organization Regional Office for the Americas/Pan American Health Organization; Q, quartile. a Assessed using the Cochran-Armitage trend test. b Data were missing for 0.29% (n=44) of foods. c Data were missing for 0.41% (n=62) of foods. d Data were missing for 0.30% (n=45) of foods.
43
FSANZ model
Across all foods, there was “near perfect” agreement (κ=0.89; 95% CI 0.89, 0.90)
between the FSANZ and Ofcom model. Although the overall proportions of foods
classified as “healthier” by the two models were similar in magnitude (49.0% by FSANZ;
44.3% by Ofcom), significant discordance in the classifications between the models was
observed for 5.3% of the large sample of n=15,183 foods analyzed (P<0.001) (Figures
3-1 and 3-2). Specifically, significant discordance was observed for 11 of the 22 food
categories analyzed (all P<0.05); however, agreement was “near perfect” or
“substantial” for nine of these categories (κ=0.66 to 0.97; 1.2 to 17.6% discordance).
Notably, amongst the remaining two categories, “slight” agreement and higher
proportions of discordance were observed: potatoes/sweet potatoes/yams (κ=0.17;
27.9% discordance) and fats/oils (κ=0.07; 33.8% discordance). Across all foods, cross-
classification analyses using quartiles of FSANZ and Ofcom scores indicated that there
was exact agreement for 95.2% of foods and no gross misclassification of foods (data
across food categories not shown).
44
Figure 3-1 Overall proportions (%, 95% CI) of “healthier” and “less healthy” foods and agreement (κ) between each model and the Ofcom model (n=15,227). Data were missing for 0.01 to 0.34% (n=1 to 52) of foods within a model. Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). Abbreviations: EURO, World Health Organization Regional Office for Europe; FSANZ, Food Standards Australia New Zealand; HCST, Health Canada Surveillance Tool; PAHO, World Health Organization Regional Office for the Americas/Pan American Health Organization.
45
Figure 3-2 Agreement (κ, 95% CI) and discordance (%, indicated above each line) between the Food Standards Australia New Zealand (FSANZ) and Ofcom model for all foods (n=15,183; data missing for n=44) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). Significant discordance in classifications between models using McNemar’s test (*P<0.05, **P<0.001).
46
HCST tier system
Across all foods, there was “fair” agreement (κ=0.26; 95% CI 0.25, 0.28) between the
HCST and Ofcom model. Although the overall proportions of foods classified as
“healthier” by the two models were similar in magnitude (50.4% by HCST; 44.3% by
Ofcom), significant discordance in the classifications between the models was observed
for 37.0% of the foods (P<0.001) (Figures 3-1 and 3-3). With the exception of dairy
products/substitutes (P=0.38) and snacks (P=0.08), significant discordance was
observed for 20 of the 22 food categories analyzed (all P<0.05). However, agreement
was “substantial” or “moderate” for five of these categories (κ=0.53 to 0.77; 11.4 to
22.9% discordance). Amongst the remaining 15 categories, less agreement and higher
proportions of discordance were observed. Agreement was “fair” or “slight” (κ=0.02 for
fats/oils to κ=0.36 for marine products), and discordance ranged from 17.8% for
legumes to 81.5% for soups. Notably, there was disagreement more than expected by
chance, as indicated by a negative kappa statistic (κ=-0.07), and 56.4% discordance for
sauces/dips/gravies/condiments.
Across all foods, cross-classification analyses conducted using the HCST tiers and
quartiles of Ofcom scores indicated that the classifications were in exact agreement for
32.7%, within an adjacent tier/quartile for 49.0%, in disagreement for 15.9%, or grossly
misclassified for 2.4% of foods (Figure 3-4). Specifically, the three food categories with
the highest proportions of classifications that disagreed and/or were grossly
misclassified included eggs (75.0%), fats/oils (55.1%), and combination dishes (47.5%).
A total of 15 categories included grossly misclassified foods, with the highest
proportions amongst miscellaneous items (10.6%), combination dishes (9.2%), and
sauces/dips/gravies/condiments (6.2%) (the remaining categories had <5.0%).
47
Figure 3-3 Agreement (κ, 95% CI) and discordance (%, indicated above each line) between the Health Canada Surveillance Tool (HCST) and Ofcom model for all foods (n=15,165; data missing for n=62) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). Significant discordance in classifications between models using McNemar’s test (*P<0.05, **P<0.001).
48
Figure 3-4 Cross-classification analyses between four Health Canada Surveillance Tool (HCST) tiers versus quartiles of Ofcom scores for all foods (n=15,165; data missing for n=62) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Exact agreement occurs when a food is classified in the same tiers/quartiles (e.g., HCST tier 1 and Ofcom quartile 1). Agreement within an adjacent (±1) tier/quartile (e.g., HCST tier 1 and Ofcom quartile 2) and disagreement (e.g., HCST tier 1 and Ofcom quartile 3) also were assessed. Gross misclassification occurs when a food is classified in opposing tiers/quartiles (e.g., HCST tier 1 and Ofcom quartile 4).
49
EURO model
Across all foods, there was “moderate” agreement (κ=0.54; 95% CI 0.53, 0.55) between
the EURO and Ofcom model. While the overall proportions of foods classified as
“healthier” by the two models differed (29.8% by EURO; 44.3% by Ofcom), significant
discordance in the classifications between the models was observed for 22.0% of the
foods (P<0.001) (Figures 3-1 and 3-5). According to the EURO model, none of the
desserts or dessert toppings/fillings were classified as “healthier”, and none of the eggs
were classified as “less healthy” (refer to Table 3.5 in Section 3.1.6)z; thus, the kappa
statistic and McNemar’s test for significance in discordance, which required 2 by 2
tables to be generated, could not be conducted for these three food categories.
Significant discordance was observed for all 19 food categories analyzed using
McNemar’s test (all P<0.01); however, agreement was “substantial” or “moderate” for
nine of these categories (κ=0.45 to 0.80; 7.8 to 23.6% discordance). Amongst the
remaining 10 categories, less agreement and higher proportions of discordance were
observed. Agreement was “fair” or “slight” (κ=0.06 for legumes to κ=0.39 for dairy
products/substitutes), and discordance ranged from 3.0% for sugars/sweets to 62.7%
for fruit/fruit juices. Notably, there was disagreement more than expected by chance
(κ=-0.01) and 29.7% discordance for fats/oils.
z Table 3.5 is presented as a supplementary table in the manuscript.
50
Figure 3-5 Agreement (κ, 95% CI) and discordance (%, indicated above each line) between the World Health Organization Regional Office for Europe (EURO) and Ofcom model for all foods (n=15,182; data missing for n=45) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). Significant discordance in classifications between models using McNemar’s test (*P<0.01, **P<0.001). The “X” symbol represents a food category for which the kappa statistic and McNemar’s test could not be conducted because 2 by 2 tables could not be generated (i.e., none of the desserts or dessert toppings/fillings were classified as “healthier”, and none of the eggs were classified as “less healthy” by the EURO model).
51
PAHO model
Across all foods, there was “fair” agreement (κ=0.28; 95% CI 0.26, 0.29) between the
PAHO and Ofcom model. While the overall proportions of foods classified as “healthier”
by the two models differed (15.9% by PAHO; 44.3% by Ofcom), significant discordance
in the classifications between the models was observed for 33.4% of the foods
(P<0.001) (Figures 3-1 and 3-6). According to the PAHO model, none of the packaged
salads (e.g., pasta or potato salads) were classified as “healthier” (refer to Table 3.5 in
Section 3.1.6)aa; thus, the kappa statistic and McNemar’s test for significance in
discordance, which required 2 by 2 tables to be generated, could not be conducted for
this food category. Significant discordance was observed for all 21 food categories
analyzed using McNemar’s test (all P<0.01); however, agreement was “substantial” or
“moderate” for four of these categories (κ=0.41 to 0.77; 9.4 to 15.0% discordance).
Amongst the remaining 17 categories, less agreement and higher proportions of
discordance were observed. Agreement was “fair” or “slight” (κ=0.01 for legumes to
κ=0.32 for vegetables), and discordance ranged from 16.5% for snacks to 69.6% for
combination dishes. Notably, there was no agreement (κ=0) or there was disagreement
more than expected by chance (κ=-0.05 to -0.01) and a range of 12.3 to 87.9%
discordance for desserts, soups, fats/oils, dessert toppings/fillings, and sugars/sweets.
aa Table 3.5 is presented as a supplementary table in the manuscript.
52
Figure 3-6 Agreement (κ, 95% CI) and discordance (%, indicated above each line) between the World Health Organization Regional Office for the Americas/Pan American Health Organization (PAHO) and Ofcom model for all foods (n=15,182; data missing for n=45) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). Significant discordance in classifications between models using McNemar’s test (*P<0.01, **P<0.001). The “X” symbol represents a food category for which the kappa statistic and McNemar’s test could not be conducted because 2 by 2 tables could not be generated (i.e., none of the packaged salads were classified as “healthier” by the PAHO model).
53
3.1.5 Discussion
In this study, the four NP models were considered to have moderate content validity
because each model accounted for at least half of the nutrients/food components that
characterise “healthy” or “less healthy” diets according to the WHO(70,71). The WHO’s
Global action plan for the prevention and control of non-communicable disease 2013-
2020, which was updated recently in May 2017, was used because it represents
consensus in the current scientific literature with regards to the nutrients/food
components of immediate importance in promoting health and preventing disease(70,71).
Other studies that employed this method of testing for content validity (i.e., assessing
consistency between the underpinnings of a model and the current scientific
literature(34)) were not identified in the published literature, likely because content validity
is typically assessed during the early phases of NP model development(34) for which the
data are rarely published.
Several considerations should be noted. First, the WHO used the umbrella terms total
fat and total sugars as part of the recommendations for avoiding a “less healthy” diet.
Although recommendations specific to saturated, unsaturated, and trans fat also were
provided, sugars were not further differentiated, despite the WHO’s consideration of free
sugars as part of their sugar guidelines(23). Second, a model’s inclusion of
nutrients/food components for which their roles in health are still being debated (e.g.,
sweeteners, saturated fat) was not considered to detract from its content validity. Third,
a model’s lack of inclusion of nutrients/food components may not necessarily reflect a
lack of content validity. Although the fruit and vegetable content of foods was not
included as a food component in the EURO model, the model indirectly accounted for it
by treating the food category for fresh and frozen fruits/vegetables/legumes as an
exemption, such that these foods were automatically considered as “healthier”
irrespective of their nutrient content. Nevertheless, the high fruit or vegetable content of
foods belonging to other food categories (e.g., vegetable lasagne as combination
dishes) would remain unrecognised in this case. Furthermore, trans fat was not
considered in the FSANZ and HCST models, likely because several voluntary initiatives
introduced in Australia/New Zealand and Canada during the early 2000s were
successful in reducing trans fat levels in the food supply(74-76). Thus, the lack of
54
inclusion of nutrients/food components in a model should be considered in the context
of other policy frameworks.
With respect to construct/convergent validity, our findings related to the FSANZ and
Ofcom models were consistent with another study in that there was overall “near
perfect” agreement between the models, other than for fats/oils and dairy products
which were classified differently by the models(49). Although the overall discordance
between FSANZ and Ofcom observed for 5.3% of the foods was statistically significant,
this was likely due to the large sample of n=15,183 foods analyzed. Such a low
proportion of discordance would not be considered of practical significance; rather, this
result is meaningful in the context of establishing construct/convergent validity between
FSANZ and Ofcom when compared to the overall discordance observed for the other
models, which ranged from 22.0 to 37.0%. The “slight” agreement and high proportions
of discordance observed for fats/oils and potatoes/sweet potatoes/yams can be
explained by the differences in the characteristics of the two models, such as the
inclusion of fats in a third food category with a different pre-determined cut-off score and
the inclusion of potatoes and tubers as part of FVNL in the FSANZ model but not FVN in
the Ofcom model. Although our findings were expected given that FSANZ is a
derivative of the Ofcom model, the results confirmed construct/convergent validity
between these models when used to assess the nutritional quality of pre-packaged
foods available in the Canadian marketplace.
The case for construct/convergent validity with the Ofcom model was less convincing for
the EURO, PAHO, and HCST models. Limited published data on the validity of the
WHO EURO and PAHO models were identified. The data were primarily related to the
pilot testing conducted during the models’ development process. The EURO model
underwent expert consultation and field-testing using 10 country-specific food
composition databases, each of which contained approximately 200 foods that were
commonly consumed and/or advertised to children(3,19). To date, the EURO model has
not been tested for construct/convergent validity with healthful diets or predictive validity
with health outcomes(3). Nevertheless, the EURO model has been adapted for use by
other WHO Regional Offices, including the office for the Western Pacific in 2016(77),
South-East Asia in 2017(78), and the Eastern Mediterranean in 2017(79). The
55
development of these other WHO models underwent similar testing using country-
specific food databases in their respective regions(77-79). Although the development of
the PAHO model consisted of a comparison to three other NP models (i.e., Ofcom,
EURO, and a draft of the model for the Eastern Mediterranean region) for the
classification of 1,992 pre-packaged foods from five European countries, parameters
related to agreement or discordance between the models were not assessed, and
validity was not discussed(21).
Our findings related to the EURO and Ofcom models were consistent with other studies
in that there was overall “moderate” agreement(50), and that fats/oils and fruit juices were
often classified differently by the models(3). For example, there was only “slight”
agreement (κ=0.08) between the EURO and Ofcom model with a high level of
discordance for 62.7% (n=682 of 1,088) of the fruit/fruit juices, the majority of which
were fruit juices specifically (63.0%, n=430 of 682). This is likely because the EURO
model considers juices as exclusions, such that they are automatically considered as
“less healthy” irrespective of their nutrient content. The healthfulness of juices is viewed
differently in various jurisdictions; for example, the UK Eatwell Guide recommends that
consumption of fruit juices and/or smoothies should be limited to ≤150 mL/day as they
are a source of free sugars(80), while the 2015-2020 Dietary Guidelines for Americans
states that 100% fruit juices without added sugars and vegetable juices can be part of
healthy eating patterns(81). As the healthfulness of juices is currently under debate, it
remains to be seen whether the consideration of juices in NP models will change.
When PAHO was compared to the Ofcom model, there was only “fair” agreement
(κ=0.28) and discordance for 33.4% of the foods overall and across all 21 food
categories analyzed. For example, agreement was negligible (κ=0.01) with a high level
of discordance for 44.4% (n=80 of 180) of packaged legumes, all of which were
classified as “less healthy” by PAHO and “healthier” by Ofcom. For the majority of these
legume products (86.3%, n=69 of 80), their “less healthy” status was triggered because
the sodium threshold was exceeded. This discrepancy in classifying legumes is likely
because the sodium criterion is calculated per kcal of the food according to the PAHO
model, whereas it is calculated per 100 g of the food according to the Ofcom model.
56
Only a single study on the validity of the HCST model was identified, and only face
validity (i.e., extent to which the system is a useful tool for end users, such as
consumers making food-purchase decisions in the marketplace(34)) was discussed(82).
Based on the cross-classification analyses conducted to assess construct/convergent
validity between the HCST and Ofcom model in our study, adjustments to the nutrient
criteria may be prudent for the food categories with the highest proportions of food
classifications that disagreed and/or were grossly misclassified (i.e., eggs, fats/oils,
combination dishes, miscellaneous items, and sauces/dips/gravies/condiments). For
example, 75.0% (n=42 of 56) of the classifications for eggs disagreed because they
were classified as HCST tier 3 foods (i.e., partially in line with CFG such that fewer
choices should come from this tier), but had the lowest quartile of Ofcom scores (i.e.,
“healthiest” foods). For the majority of these egg products (73.8%, n=31 of 42), their tier
3 status was triggered only because the saturated fat threshold was exceeded, and not
because the thresholds for sodium or sugars were exceeded. In fact, although the
saturated fat content ranged from 2.8 to 3.3 g per CFG serving, these 31 egg products
were protein-rich sources that provided 10.9 to 13.2 g of protein per CFG serving. This
discrepancy is likely explained by the consideration of positive nutrients, such as
protein, by the Ofcom model, but not the HCST model. Although it should be noted that
NP was not an intended purpose of this model, adjustments to the HCST model may be
warranted before its continued use in public health policies and may in fact coincide well
with the revision of CFG, which is currently underway(83).
There are several strengths and limitations associated with this study. First, we
recognize that there are many other NP models developed by authoritative bodies that
could have been examined for content and construct/convergent validity in this study,
such as the Chilean “stop sign” warning labels and Nutri-Score, the latter which was
recently adopted by the French government as the voluntary front-of-pack nutrition
labelling scheme on 31 October 2017(84). In fact, based on the aforementioned
systematic review that was recently completed by our research group (Labonté ME,
Poon T, Gladanac B et al., unpublished results), we identified 387 potentially relevant
NP models, 78 of which met the inclusion criteria and were developed or endorsed by
government organizations. Given that it is neither feasible nor practical to examine the
57
validity of all models within a single research paper, our intent for the current study was
two-fold. We aimed to examine models developed by authoritative bodies, as data have
shown that industry-led models were less stringent with regards to the advertising of
energy-dense foods compared to government-led models(48). In addition, we aimed to
examine models with potentially wide applicability and/or relevance to the North
American context (e.g., WHO EURO and PAHO models), and so models from regions
with substantial differences in the food supply (e.g., Asia, Africa) were not examined.
Moreover, an important limitation of assessing validity by comparing NP models is the
lack of a gold standard measure for defining a “healthier” food, which was first
discussed by Arambepola et al.(32) nearly 10 years ago. Although validity testing ideally
should involve a gold standard comparator, some have reported that it is not a necessity
in the context of NP(32,33), given that all methods used in NP validity testing including the
most robust measurements are imperfect and subject to random and systematic
error(34). Indeed, depending on the research group, criterion validity has been described
as the extent to which the model is accurate based on “an externally derived gold
standard”(34), “an externally derived objective measure of healthfulness”(10), or “a
comparison of one measure with another(33). The method involving the comparison of
several models to the Ofcom model, a validated model which served as an objective
measure of healthfulness, is recognized by the WHO(1,9) and others(50,52,59) to be a valid
approach. It should be emphasised that we did not consider the Ofcom model to be
gold standard, as one does not exist in NP, and we did not consider models which
differed from the Ofcom model to be invalid. Rather, the Ofcom model was selected as
the reference because it is one of the most extensively studied NP models currently in
use, as discussed in the methods section. Our intent was to use the most extensively
validated NP model currently available as an objective measure of healthfulness and as
one method of identifying components of other models that may warrant adjustment or
further investigation. Nevertheless, a limitation of this method is that the scientific
underpinnings of the Ofcom model were established more than a decade ago, and the
science of NP has evolved since then. In fact, the Ofcom model is currently being
reviewed by Public Health England to ensure that the model reflects the latest dietary
guidance(85).
58
Furthermore, Townsend(34) aptly noted that a single validation study does not make a
NP model valid. Different methods of validity testing are associated with various
strengths and limitations(1,9,34). For example, the method of comparing the
classifications of foods determined using the NP model against food-based dietary
guidelines or views of nutrition experts typically involve a small sample of several
hundred indicator foods, are relatively simpler, and have been used more often(1,9,34). In
contrast, the method of assessing the association between the healthfulness of the
foods consumed by individuals, as determined using the NP model, and observed
changes in health outcomes is considered the most robust assessment of validity (i.e.,
predictive). However, this method is complex, costly in time and resources because it
requires data from prospective cohort studies, remains susceptible to recall bias from
self-reported diet recalls and food records, and has rarely been used(1,9,34). As such,
multiple validation studies using different comparison measures are required to validate
one model(10,34).
To our knowledge, this is the first study to assess the validity of several models
developed by authoritative bodies by using them to classify over 15,000 foods across
numerous food categories, in comparison to other validation studies which typically
used a small sample of several hundred indicator foods. Moreover, while data from
food composition databases are useful for NP purposes and have been used in most
validation studies, often the food categories are highly aggregated and the nutrient data
are presented as averages, and so variability across similar foods or different brands
cannot be ascertained(9). The use of foods from a large, national, branded database,
which is more reflective of the foods available in the marketplace, adds to the
robustness of the current analyses. However, the use of this database of pre-packaged
foods does not provide information with regards to how the NP models classify fresh
foods without Nutrition Facts tables on the labelling.
While the overall proportions of “healthier” and “less healthy” foods were similar in
magnitude between the FSANZ and HCST models versus the Ofcom model (all
approximately 50%), the kappa statistic indicated vast differences in agreement with
Ofcom (“near perfect” for FSANZ, but only “fair” for HCST); likewise, the discordance
parameter indicated vast differences in the classifications between each model and
59
Ofcom (only 5.3% for FSANZ, but 37.0% for HCST). This highlights the importance of
examining classifications across food categories, the level at which differences between
models become apparent. Because this study used a combination of parameters to
assess construct/convergent validity and consisted of detailed analyses across food
categories, it allowed for a comprehensive examination of the differences in
classifications between models.
In this study, all four NP models (FSANZ, HCST, EURO, and PAHO) demonstrated
moderate content validity. In contrast, the different models exhibited varying levels of
construct/convergent validity with the Ofcom model in classifying over 15,000 pre-
packaged foods. Construct/convergent validity was confirmed between the FSANZ and
Ofcom model, and to a lesser extent between the EURO and Ofcom model. Numerous
incongruencies with Ofcom were identified for the HCST and PAHO models, which
suggest that the classifications of several food categories may warrant further
investigation and adjustment. The results of this study may be informative for regulators
who wish to adapt existing models for use in country-specific applications. This
research is conducted as the WHO and many regulatory agencies are working to
establish transparent and reproducible methods to underpin their policies aimed at
curbing the obesity epidemic and preventing non-communicable diseases. In the span
of a year, the Canadian government has proposed to use NP in their policies on front-of-
pack labelling and restrictions on marketing to children(40). Research specifically on
validity testing is timely and globally relevant, as the number of NP models has
proliferated, but evidence on the adequacy of these models has lagged behind. As
several researchers have suggested that validity testing should be given the highest
priority in this field of research(3,10,11), this study contributes to addressing this need.
3.1.6 Supplementary Material
Tables 3-4 and 3-5, presented as supplementary material in the manuscript, are
provided here.
60
Table 3-4 Method based on the ingredient list used to estimate the FVN points of foods in the Ofcom modela
Ofcom criteria In-house criteria based on the ingredient list used to estimate FVN contentb Point(s) FVN content
of food (%) Non-concentrated FVN ingredients 0 ≤40 FVN is not one of the first two ingredients. 1 >40 FVN is the second ingredient. 2 >60 FVN is the first ingredient, but non-FVN ingredients appear to contribute
substantially to the product's weightc. 5 >80 FVN is the first ingredient, and only FVN ingredients contribute substantially
to the product's weight. Concentrated FVN ingredientsd 0 ≤40 FVN is not one of the first three ingredients. 1 >40 FVN is the second ingredient, but the amounts of the first and second
ingredients are not similar (see corresponding criteria for 2 pointse). FVN is the third ingredient and is estimated to represent ≥25% of the product’s weightf.
2 >60 FVN is the first ingredient, but non-FVN ingredients appear to contribute substantially to the product's weightc. FVN is the second ingredient, and the amounts of the first and second ingredients are similare.
5 >80 FVN is the first ingredient, and only FVN ingredients contribute substantially to the product's weight.
Abbreviation: FVN, fruits, vegetables, and nuts. a In the absence of quantitative declarations in the ingredient list, which are not required for food labelling in Canada, a method was developed by the research group at the University of Toronto in order to estimate the FVN content and corresponding FVN points of the foods for the Ofcom model. Given that the ingredients are listed in descending order by weight, the presence and positions of the FVN ingredients within the ingredient list were used to estimate the FVN content. b If sub-ingredients in brackets were presented for an ingredient, the ingredient was considered a FVN if one of the first two sub-ingredients was a FVN. If a food consisted of more than one component, and therefore more than one ingredient list (e.g., tuna kit with crackers), FVN points were calculated for the individual components, and the average of the FVN points was used to represent the food as a whole. If the ingredient list of a food was missing (<2% of foods), FVN points were not assigned unless it was evident from the product name that FVN contributed substantially to the product’s weight. c Non-FVN ingredients that were considered to contribute minimally to the product's weight included: salt, preservatives, colour, vitamins, minerals, oils, flavour extracts, antioxidants, and food additives. Other non-FVN ingredients were considered to contribute minimally if they appeared after salt or preservatives in the ingredient list. Non-FVN ingredients that were considered to contribute substantially to the product’s weight were other than those previously listed (e.g., sugar and water were considered to contribute substantially to the product’s weight). d Concentrated FVN ingredients are those in concentrated form (i.e., dried, evaporated, as pastes). According to the Ofcom model, concentrated FVN contribute less than non-concentrated FVN to the weight of the food; thus, the weight of concentrated FVN should be multiplied by two when calculating the FVN content of a food. Given that this criterion cannot be applied directly in the absence of quantitative declarations, the criteria were adjusted for concentrated FVN ingredients.
61
e A food may score two points if the total number of ingredients present in the ingredient list is low (e.g., ≤3), and if it is estimated that the amount of the first ingredient is only slightly higher than the amount of FVN in the food (e.g., estimated proportion of 55 versus 45%, respectively). f A food may score one point because at least 25% of the weight represented by a concentrated FVN equals to at least 40% FVN when multiplying the amount by two according to the Ofcom model.
62
Table 3-5 Proportion (%) of “healthier” and “less healthy” foods classified by models compared to the Ofcom model for all foods (n=15,227) and by food category
Schedule M category and description
Foods (n)
Ofcom FSANZ HCST EURO PAHO “Healthier”
“Less healthy”
“Healthier” “Less healthy”
“Healthier”
“Less healthy”
“Healthier” “Less healthy”
All 15,156 to 15,183a
“Healthier” 44.1 0.3 29.0 15.4 26.1 18.3 13.4 31.0 “Less healthy” 5.0 50.6 21.6 34.0 3.7 52.0 2.4 53.2
1. Bakery products
2082 to 2083
“Healthier” 24.2 0.0 23.9 0.2 13.9 10.3 3.1 21.0 “Less healthy” 3.5 72.4 39.5 36.3 0.9 75.0 0.2 75.7
2. Beverages 481 to 482
“Healthier” 49.2 1.0 47.9 2.3 11.4 38.9 9.5 40.7 “Less healthy” 5.6 44.2 9.1 40.7 0.0 49.7 0.0 49.8
3. Cereals, other grains
978 to 981
“Healthier” 75.5 0.0 75.0 0.5 69.5 6.0 67.2 8.4 “Less healthy” 6.1 18.4 22.4 2.2 1.8 22.6 1.0 23.5
4. Dairy products, substitutes
1,237 “Healthier” 37.1 0.1 27.8 9.4 16.3 20.9 6.1 31.1 “Less healthy” 17.5 45.3 8.3 54.5 5.2 57.6 2.4 60.4
5. Desserts 820 to 827
“Healthier” 32.3 0.7 24.0 9.3 0.0 33.0 0.1 32.9 “Less healthy” 0.6 66.4 6.0 60.7 0.0 67.0 0.0 67.0
6. Dessert toppings, fillings
115 “Healthier” 11.3 1.7 8.7 4.4 0.0 13.0 0.0 13.0 “Less healthy” 0.0 87.0 37.4 49.6 0.0 87.0 0.9 86.1
7. Eggs 56 “Healthier” 94.6 0.0 19.6 75.0 94.6 0.0 82.1 12.5 “Less healthy” 0.0 5.4 0.0 5.4 5.4 0.0 0.0 5.4
8. Fats, oils 535 “Healthier” 1.9 0.0 1.9 0.0 0.4 1.5 0.4 1.5 “Less healthy” 33.8 64.3 66.7 31.4 28.2 69.9 27.3 70.8
9. Marine products
440 “Healthier” 77.3 0.0 48.2 29.1 75.5 1.8 14.6 62.7 “Less healthy” 0.0 22.7 2.7 20.0 9.6 13.2 0.2 22.5
10. Fruit, fruit juices
1,088 “Healthier” 69.8 2.7 59.6 12.9 9.7 62.7 38.8 33.6 “Less healthy” 1.3 26.3 16.4 11.2 0.0 27.6 3.1 24.5
63
Table 3-5 Proportion (%) of “healthier” and “less healthy” foods classified by models compared to the Ofcom model for all foods (n=15,227) and by food category
Schedule M category and description
Foods (n)
Ofcom FSANZ HCST EURO PAHO “Healthier”
“Less healthy”
“Healthier” “Less healthy”
“Healthier”
“Less healthy”
“Healthier” “Less healthy”
11. Legumes 180 “Healthier” 99.4 0.0 81.7 17.8 85.0 14.4 55.0 44.4 “Less healthy” 0.0 0.6 0.0 0.6 0.0 0.6 0.0 0.6
12. Meat, poultry, their products, substitutes
895 “Healthier” 27.3 0.1 13.6 13.7 25.9 1.5 0.9 26.5 “Less healthy” 0.6 72.1 0.3 72.3 16.3 56.3 0.5 72.2
13. Miscellaneous
444 to 445
“Healthier” 20.9 0.0 21.0 0.0 7.6 13.3 10.1 10.8 “Less healthy” 1.6 77.5 63.3 15.8 0.2 78.9 3.4 75.7
14. Combination dishes
1,347 to 1,348
“Healthier” 71.6 0.0 8.0 63.6 50.5 21.1 2.0 69.6 “Less healthy” 1.2 27.2 2.1 26.4 1.6 26.8 0.0 28.4
15. Nuts, seeds 220 “Healthier” 65.5 0.0 47.7 17.7 47.7 17.7 62.7 2.7 “Less healthy” 10.5 24.1 4.1 30.5 5.9 28.6 12.3 22.3
16. Potatoes, sweet potatoes, yams
140 “Healthier” 67.9 0.0 49.3 18.6 32.1 35.7 12.9 55.0 “Less healthy” 27.9 4.3 4.3 27.9 7.1 25.0 0.0 32.1
17. Packaged salads
70 “Healthier” 75.7 0.0 27.1 48.6 38.6 37.1 0.0 75.7 “Less healthy” 5.7 18.6 0.0 24.3 4.3 20.0 0.0 24.3
18. Sauces, dips, gravies, condiments
1,219 to 1,224
“Healthier” 31.1 0.0 16.2 15.0 9.6 21.5 2.6 28.5 “Less healthy” 0.3 68.6 41.4 27.3 1.9 67.0 1.5 67.4
19. Snacks 794 “Healthier” 17.6 0.1 4.8 13.0 5.5 12.2 2.6 15.1 “Less healthy” 10.6 71.7 16.4 65.9 1.5 80.7 1.4 80.9
20. Soups 454 to 456
“Healthier” 89.3 0.0 7.9 81.5 86.8 2.4 1.3 87.9 “Less healthy” 0.2 10.5 0.0 10.6 6.4 4.4 0.0 10.8
21. Sugars, 739 “Healthier” 3.5 0.0 3.5 0.0 0.5 3.0 0.0 3.5
64
Table 3-5 Proportion (%) of “healthier” and “less healthy” foods classified by models compared to the Ofcom model for all foods (n=15,227) and by food category
Schedule M category and description
Foods (n)
Ofcom FSANZ HCST EURO PAHO “Healthier”
“Less healthy”
“Healthier” “Less healthy”
“Healthier”
“Less healthy”
“Healthier” “Less healthy”
sweets “Less healthy” 0.1 96.4 37.4 59.1 0.0 96.5 8.8 87.7 22. Vegetables 827 to
828 “Healthier” 70.7 0.1 65.7 5.1 48.8 22.0 31.9 38.9 “Less healthy” 0.1 29.1 24.8 4.5 0.0 29.2 0.0 29.2
Abbreviations: EURO, World Health Organization Regional Office for Europe; FSANZ, Food Standards Australia New Zealand; HCST, Health Canada Surveillance Tool; PAHO, World Health Organization Regional Office for the Americas/Pan American Health Organization. a Across the models, data were missing for 0.29 to 0.41% (n=44 to 62) of foods.
65
3.2 Additional Analyses Not Included in Manuscript
3.2.1 Summary of Results on Construct/Convergent Validity
The results of the four parameters used to assess construct/convergent validity are
summarized in Table 3-6.
Table 3-6 Results of the assessment of construct/convergent validity
Parameter Statistical Test or Analysis
NP Model FSANZ HCST EURO PAHO
Association between the proportion of “less healthy” foods classified by model and quartiles of Ofcom scores
Cochran-Armitage trend test
Positive association (P<0.001)
Positive association (P<0.001)
Positive association (P<0.001)
Positive association (P<0.001)
Agreement with Ofcom
Kappa statistica
Near perfect Fair Moderate Fair
Discordance in classifications with Ofcom (% of foods)
McNemar’s test
Significant (5.3) Significant (37.0) Significant (22.0)
Significant (33.4)
Agreement and misclassification between quartiles of FSANZ scores and 4 HCST tiers versus quartiles of Ofcom scores (% of foods)b
Cross-classification analyses
Exact agreement: 95.2
Exact agreement: 32.6
NA NA
Agreement ±1 Q: 4.2
Agreement ±1 Q: 48.8
Disagreement: 0.3
Disagreement: 15.9
Gross misclassification: 0
Gross misclassification: 2.3
Overall convergence with Ofcom Convergent Not convergent Moderately convergent
Not convergent
Abbreviation: NA, not applicable. a Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). b Data were missing for 0.3 and 0.4% of foods for the comparisons between FSANZ and HCST versus Ofcom, respectively.
3.2.2 Data Not Shown as Reported in Manuscript
As discussed in Section 3.1.4, “across all foods, cross-classification analyses using
quartiles of FSANZ and Ofcom scores indicated that there was exact agreement for
95.2% of foods and no gross misclassification of foods (data across food categories not
shown)”. These data are presented in Figure 3-7.
66
Figure 3-7 Cross-classification analyses between quartiles of Food Standards Australia New Zealand (FSANZ) and Ofcom scores for all foods (n=15,183; data missing for n=44) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Exact agreement occurs when a food is classified in the same quartiles (e.g., FSANZ quartile 1 and Ofcom quartile 1). Agreement within an adjacent (±1) quartile (e.g., FSANZ quartile 1 and Ofcom quartile 2) and disagreement (e.g., FSANZ quartile 1 and Ofcom quartile 3) also were assessed. There was no gross misclassification, which occurs when a food is classified in opposing quartiles (e.g., FSANZ quartile 1 and Ofcom quartile 4).
67
3.2.3 Modified-PAHO
As discussed in Section 3.1.3, the PAHO model was intended for the assessment of
processed or ultra-processed foods only; that is, unprocessed or minimally processed
foods, culinary ingredients, and freshly prepared dishes were not subject to the nutrient
criteria and were automatically classified as “not excessive” in any nutrient. Due to the
subjectivity associated with classifying foods according to their level of processing for
the PAHO model, our research group decided to evaluate the construct/convergent
validity of a modified version of the PAHO model (modified-PAHO), which consisted of
the following two adjustments: 1) the assessment of all foods irrespective of the level of
processing; and 2) a revised sodium criterion applied specifically to beverages with very
low sodium and energy content. These adjustments are described in detail in our
publication on the application of NP models for the restrictions on marketing to children
(Appendix A)(26).
Across all foods (n=15,177), a positive association was observed between the modified-
PAHO and Ofcom model, such that the proportion of foods classified as “less healthy”
by the modified-PAHO model increased across quartiles of Ofcom scores, with the
highest quartile representing the “least healthy” foods. Specifically, the proportions (%,
95% confidence intervals (CI)) of foods classified as “less healthy” were 71.3 (70.0,
72.6), 96.0 (95.3, 96.7), 99.2 (98.9, 99.5), and 100 (100, 100) in quartiles 1, 2, 3, and 4,
respectively (P<0.001 for trend).
Across all foods, there was “fair” agreement (κ=0.22; 95% CI 0.21, 0.23) between the
modified-PAHO and Ofcom model. While the overall proportions (%, 95% CI) of foods
classified as “healthier” by the two models differed (9.8 (9.4, 10.3) and 44.3 (43.5, 45.1),
respectively), significant discordance in the classifications between the models was
observed for 35.4% of the foods (P<0.001) (Figure 3-8). According to the modified-
PAHO model, none of the eggs, packaged salads (e.g., pasta or potato salads), or
sugars/sweets were classified as “healthier” (Table 3-7); thus, the kappa statistic and
McNemar’s test for significance in discordance, which required 2 by 2 tables to be
generated, could not be conducted for these three food categories. Significant
discordance was observed for all 19 food categories analyzed using McNemar’s test (all
68
P<0.01); however, agreement was “substantial” or “moderate” for two of these
categories (κ=0.56 or 0.76; 9.8 or 11.7% discordance). Amongst the remaining 17
categories, less agreement and higher proportions of discordance were observed.
Agreement was “fair” or “slight” (κ=0.01 for meat/poultry to 0.33 for fats/oils), and
discordance ranged from 1.5% for fats/oils to 73.4% for marine products. Notably, there
was no agreement (κ=0) or there was disagreement more than expected by chance (κ=-
0.02) and a range of 13.9 to 87.9% discordance for desserts, nuts/seeds, soups, and
dessert toppings/fillings.
Overall, the results pertaining to construct/convergent validity between the modified-
PAHO and Ofcom model were similar to those observed between the PAHO and Ofcom
model. Numerous incongruencies were identified between the modified-PAHO and
Ofcom model, which suggest that the classifications of several food categories may
warrant further investigation and adjustment.
69
Figure 3-8 Agreement (κ, 95% CI) and discordance (%, indicated above each line) between the modified World Health Organization Regional Office for the Americas/Pan American Health Organization (modified-PAHO) and Ofcom model for all foods (n=15,177; data missing for n=50) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). Significant discordance in classifications between models using McNemar’s test (*P<0.01, **P<0.001). The “X” symbol represents a food category for which the kappa statistic and McNemar’s test could not be conducted because 2 by 2 tables could not be generated (i.e., none of the eggs, packaged salads, or sugars/sweets were classified as “healthier” by the modified-PAHO model).
70
Table 3-7 Proportion (%) of “healthier” and “less healthy” foods classified by the modified-PAHO model compared to the Ofcom model for all foods (n=15,227) and by food category
Schedule M category and description
Foods (n) Ofcom Modified-PAHO “Healthier” “Less healthy”
All 15,177a “Healthier” 9.4 35.0 “Less healthy” 0.5 55.2
1. Bakery products 2,082 “Healthier” 3.1 21.0 “Less healthy” 0.2 75.7
2. Beverages 481 “Healthier” 12.3 38.1 “Less healthy” 0.0 49.7
3. Cereals, other grains 978 “Healthier” 66.7 8.8 “Less healthy” 1.0 23.5
4. Dairy products, substitutes
1,237 “Healthier” 1.5 35.7 “Less healthy” 0.4 62.4
5. Desserts 827 “Healthier” 0.1 32.9 “Less healthy” 0.0 67.0
6. Dessert toppings, fillings
115 “Healthier” 0.0 13.0 “Less healthy” 0.9 86.1
7. Eggs 56 “Healthier” 0.0 94.6 “Less healthy” 0.0 5.4
8. Fats, oils 535 “Healthier” 0.4 1.5 “Less healthy” 0.0 98.1
9. Marine products 440 “Healthier” 3.9 73.4 “Less healthy” 0.0 22.7
10. Fruit, fruit juices 1,088 “Healthier” 17.5 55.0 “Less healthy” 1.7 25.9
11. Legumes 180 “Healthier” 51.7 47.8 “Less healthy” 0.0 0.6
12. Meat, poultry, their products, substitutes
895 “Healthier” 0.1 27.3 “Less healthy” 0.0 72.6
13. Miscellaneous 445 “Healthier” 9.4 11.5 “Less healthy” 0.2 78.9
14. Combination dishes 1,348 “Healthier” 2.0 69.6 “Less healthy” 0.0 28.4
15. Nuts, seeds 220 “Healthier” 0.5 65.0 “Less healthy” 0.0 34.6
16. Potatoes, sweet potatoes, yams
140 “Healthier” 12.9 55.0 “Less healthy” 0.0 32.1
71
Table 3-7 Proportion (%) of “healthier” and “less healthy” foods classified by the modified-PAHO model compared to the Ofcom model for all foods (n=15,227) and by food category
Schedule M category and description
Foods (n) Ofcom Modified-PAHO “Healthier” “Less healthy”
17. Packaged salads 70 “Healthier” 0.0 75.7 “Less healthy” 0.0 24.3
18. Sauces, dips, gravies, condiments
1,224 “Healthier” 2.5 28.6 “Less healthy” 1.5 67.4
19. Snacks 794 “Healthier” 2.4 15.4 “Less healthy” 1.4 80.9
20. Soups 456 “Healthier” 1.3 87.9 “Less healthy” 0.0 10.8
21. Sugars, sweets 739 “Healthier” 0.0 3.5 “Less healthy” 0.0 96.5
22. Vegetables 827 “Healthier” 22.4 48.4 “Less healthy” 0.0 29.3
Abbreviation: PAHO, World Health Organization Regional Office for the Americas/Pan American Health Organization. a Data were missing for 0.33% (n=50) of foods.
72
Chapter 4
4 Overall Discussion
4.1 Challenges Associated with Validating NP Models As the methods of validation are in their infancy(2), there are several main challenges
associated with validating NP models, namely the ambiguity in definitions and methods
used in validation studies, lack of a definition of a validated model, and lack of a gold
standard for defining a “healthier” food. A comprehensive discussion on the lack of a
gold standard in NP and the rationale for using the extensively validated Ofcom model
as the reference model in the validation study is provided in Section 3.1.5; thus, this
discussion is not be repeated in this section.
4.1.1 Ambiguity in the Definitions and Methods Used in Validating NP Models
As illustrated in Tables 2-2 and 2-3, although there is some commonality across the
definitions of validity used by different researchers, it is evident that there is variability in
the definitions used even amongst experts of NP, including Scarborough et al.
(2007)(33), Arambepola et al. (2008)(32), Townsend (2010)(34), Rayner (who prepared the
WHO manual on NP models (2011, in press(1))), and Cooper et al. (2016(10), 2017(36))bb.
This is particularly true for the definitions of criterion, concurrent, convergent, and
construct validity, as illustrated in Table 4-1. For example, the description:
• “correlates with an external criterion” is used to describe criterion validity
according to Scarborough et al. (2007)(33), Arambepola et al. (2008)(32),
Townsend (2010)(34), and Cooper et al. (2016)(10), but convergent validity
according to Rayner (WHO, 2011, in press)(1);
bb The references to A Dictionary of Epidemiology by Porta (2014)(30) and the seminal publication on validity by Cronbach and Meehl (1955)(35) discuss validity in general terms and not in relation to NP specifically.
73
• “correlates... with variables with which, theoretically, it should correlate” is used
to describe criterion and convergent validity according to Townsend (2010)(34),
but construct validity according to Arambepola et al. (2008)(32) and Rayner
(WHO, 2011, in press)(1); and
• “comparison with other measures of the same variable or a closely related
variable” is used to describe convergent validity according to Arambepola et al.
(2008)(32), but construct validity according to Cooper et al. (2017)(36).
With respect to criterion validity, while the majority of researchers agree that the
external criterion to which the NP model is compared should ideally be a gold standard,
both definitions used by Scarborough et al. (2007)(33) and Cooper et al. (2016)(10) allow
for comparisons to non-gold standard objective measures of healthfulness. In addition,
there appears to be consensus on the definition of concurrent validity, such that it is a
type of criterion validity and requires the measurement and criterion to refer to the same
time point. However, concurrent validity is not widely used in the literature on NP, as it
was explicitly defined only by Cooper et al., 2016(10). In contrast, the term convergent
validity was used by Arambepola et al. (2008)(32), Townsend (2010)(34), and Rayner
(WHO, 2011, in press)(1). Thus, based on the similarities between the definitions for
convergent versus concurrent and criterion, it appears that convergent validity is the
equivalent term for concurrent validity, particularly when a non-gold standard
comparison is used. Lastly, it is curious that there is no mention of construct validity by
Townsend (2010)(34). In fact, Townsend (2010)(34) describes criterion and convergent
validity in terms of theoretical correlations, which is typically used to describe construct
validity.
74
Table 4-1 Descriptions for criterion, concurrent, convergent, and construct validity used by different researchersa
Description of Type of Validity
Scarborough et al., 2007(33)
Arambepola et al., 2008(32)
Townsend, 2010(34)
Rayner (WHO, 2011, in press)(1)
Cooper et al., 2016(10), 2017(36)
Extent to which the measurement correlates with an external criterion of the phenomenon under study
• Criterion (GS or non-GS)
• Criterion (GS)
• Criterion (GS)
• Convergent (non-GS; same time point)b
• Criterion (GS or non-GS)
• Concurrent (GS; same time point)
Examines whether the method correlates in a predicted manner with variables with which, theoretically, it should correlate
NA • Construct • Criterion (GS)
• Convergent (time point not specified)
• Construct NA
Comparison with other measures of the same variable, or closely related ones
NA • Convergent NA NA • Constructb
Abbreviations: GS, gold standard, NA, not applicable. a The shaded cells represent the definitions and corresponding method used to determine the type of validity assessed in the validation study presented in Chapter 3. b This type of validity was inferred based on the method involving the comparison of NP models to one another.
Due to this ambiguity in the definitions of the different types of validity, there is
inconsistency with regards the type of validity that can be inferred from using certain
methods. While the use of biomarkers or medical records as indicators of health status
is always presented as a method of assessing criterion-related validity (either predictive
or concurrent), it is less clear for other methods. For example, convening a panel of
experts in relevant fields has been reported as a method of establishing content validity
according to Townsend (2010)(34) and Rayner (WHO, 2011, in press(1)); specifically, the
comparison of a NP model to the views of nutrition professionals has been reported as a
method to assess criterion validity according to Scarborough et al. (2007)(33),
convergent validity according to Townsend (2010)(34) and Rayner (WHO, 2011, in
press(1)), or construct validity according to Cooper et al. (2016(10), 2017(36)).
As discussed in Chapter 3, the validation study conducted involved the comparison
between several NP models to the extensively validated Ofcom model, which served as
75
an external objective measure of healthfulness. This validation study can be considered
to have assessed criterion, convergent, and construct validity given that, as indicated by
the shaded cells in Table 4-1:
• Scarborough et al. (2007)(33) and Cooper et al. (2016)(10) do not consider a gold
standard comparator a necessity for the assessment of criterion validity; and
• the comparison of NP models to one another has been reported as a method to
assess convergent validity by Rayner (WHO, 2011, in press(1)) or construct
validity by Cooper et al. (2016(10), 2017(36)).
Overall, based on the data presented, there appears to be a need for improved
reporting on the types of validity in relation to the methods used in future validation
studies of NP models.
4.1.2 Lack of a Definition of a Validated Model
In light of the evidence presented for the Ofcom model in Sections 2.4.2 and 3.1.5,
several questions remain: at what point is a NP model considered valid? Does every
type of validity need to be demonstrated, or would demonstration of predictive validity
suffice? Is there a minimum number of validation studies that need to be conducted?
There is currently no known definition or consensus on a definition with regards to when
a model is considered valid.
While a validation study contributes evidence towards establishing validity of a model for
a specific application in a specific population, a single study does not make a model
valid, as all methods of validation remain susceptible to error(1,9,34). In discussing the
validation of models in the U.S., Townsend(34) has emphasized that different types of
validation studies and multiple studies with various sub-population groups should be
conducted, given the diversity of the population. Specifically, Townsend(34) called for a
gold standard that requires “many studies of convergent and/or criterion validity” to
validate a model that aims to help consumers with food selection in the U.S.
marketplace(34). Cooper et al. (2016)(10) echoed this sentiment and stated that “a large
number of carefully designed studies to establish both construct and criterion-related
validity” is required to validate one model. However, “many studies” and “a large
76
number of studies” was not further defined, and the scientific standard required to
establish that a NP model is sufficiently valid remains unclear.
In contrast, the WHO suggested that consensus regarding the validity of a NP should be
developed through active engagement with nutrition experts and stakeholders with an
interest in the application of the model(1). In the case of the Ofcom model, after its
adoption by the media and communication regulator Ofcom in 2007, an independent
review panel endorsed the model and the Food Standards Agency board confirmed that
the model was “generally scientifically robust and fit for purpose” in 2009(3). Given the
updated recommendations on sugars and fibres from the government’s Scientific
Advisory Committee on Nutrition, Public Health England initiated a review of the Ofcom
model in 2016(3,85)cc. As such, even if validity was established previously, it is prudent
for NP models to be re-assessed periodically for validity and to ensure consistency with
the latest dietary guidance.
4.2 Other Considerations While the nutrient content of a food is crucial to NP, bioavailability of the nutrients
provided by a food, which is modified by numerous factors, is not captured by NP(86). In
addition, while NP is primarily concerned with the energy and nutrient content of a food,
other characteristics of foods are typically not considered, for example, beneficial
components such as phytochemicals, or detrimental components such as food toxins,
pathogens, or contaminants(1,9). Moreover, NP does not encompass, nor is there an
avenue to incorporate, factors related to the environmental sustainability of foods or
ethical or religious concerns regarding certain foods(9).
Furthermore, the focus of this thesis has been on NP models either developed or
endorsed by authoritative bodies. While outside the scope of this thesis, it should be
recognized that there is extensive literature on non-government-led NP models, some of
which have been validated using robust methods. Specifically, of the 309dd models that
cc The outcome of this review is anticipated to be announced in June 2018. dd Out of a total of 378 models identified.
77
did not meet the inclusion criteria of the systematic review, 53% (n=164 of 309) of the
models were excluded because they were not developed or endorsed by a government
body(8). For example, the Overall Nutritional Quality Index (ONQI) was excluded
because it was developed by an academic institution (Yale University), was not
subsequently endorsed by a government organization, and the details of the model
were not publicly available due to the propriety nature of the algorithm(8). While the
ONQI is an algorithm that incorporates more than 30 dietary components and ranks
individual foods by relative healthfulness, it also generates an average ONQI score for
the diet consumed by individuals(87). This NP model was demonstrated to have
predictive validity based on longitudinal data from the U.S. Health Professionals Follow-
up Study and Nurses’ Health Study (n=42,382 and 62,284 healthy men and women,
respectively)(87). Men in the highest versus lowest quintile of ONQI scores had relative
risks (RR, 95% CI) of 0.77 (0.70, 0.85) for cardiovascular disease, 0.84 (0.73, 0.96) for
diabetes, and 0.89 (0.83, 0.97) for all-cause mortality(87). Similarly, women in the
highest versus lowest quintile of ONQI scores had RR (95% CI) of 0.79 (0.71, 0.88) for
cardiovascular disease, 0.86 (0.78, 0.96) for diabetes, and 0.90 (0.84, 0.97) for all-
cause mortality(87).
4.3 Significance and Implications of Research The research presented in this thesis has implications in both a global context and a
Canadian context.
There has been increasing emphasis that one should be more concerned with the
healthfulness of diets, as opposed to the healthfulness foods(3). This is primarily
because the healthfulness of diets is not solely determined by the nutrient composition
of individual foods, but numerous other factors, including the portion sizes of the
individual foods, frequency of consumption of foods, variety of foods that contribute to
the diet, and the combinations in which they are consumed(1,9,32). Although NP itself is
used to evaluate individual foods, as opposed to diets, and considers the food in
isolation such that the amount or frequency of consumption is not taken into
account(32,88), NP can be used as a tool to improve the nutritional quality of diets(88).
Certain techniques may be used, including the weighting of analyses according to the
78
amount of energy the food contributed to the diet(32). Alternatively, serving size or food
category (e.g., main dish or snack) may be incorporated as factors within the model(2,32).
Over the last decade, substantial strides have been made in the field of NP research,
and several NP models have been studied extensively, especially the Ofcom model. As
discussed in Section 2.4.2, the Ofcom model was successfully adapted into a dietary
index, which demonstrated predictive validity between overall diet quality and several
health outcomes(43-47). Thus, NP models have the potential to substantially contribute to
research on dietary patterns and disease risk.
It should be noted that the systematic review was not designed a priori to identify validity
testing of NP models specifically; therefore, the results pertaining to the validity of the
models from the systematic review should be interpreted with caution(8). For example, it
should be emphasised that the 58% (n=45 of 78) of models for which data on validity
testing were not identified are not necessarily invalid; rather, validity testing may not
have been identified because, as an example, content validity is typically assessed
during the early phases of NP model development(34) for which the data are rarely
disclosed, either in the published or grey literature. Nevertheless, data on some form of
validity testing were identified for 42% (n=33 of 78) of the models(8), an increase from
approximately 33% of models for which validity testing was identified by Rayner et al. in
the 2013 unpublished WHO catalogue(7). These results, albeit promising, indicate that
there is still substantial improvement to be made in the validation of NP models,
particularly with regards to the reporting and documentation of validity testing.
Nevertheless, as Townsend(34) noted, a single validation study contributes evidence
towards establishing the validity of that model for a specific application in a specific
population. Thus, the validation study presented in Chapter 3 contributes to the
scientific evidence in establishing the overall validity of the FSANZ, HCST, EURO, and
PAHO models for assessing the nutritional quality of foods.
Based on the literature review presented in Chapter 2, none of identified studies
examined the construct/convergent validity of any NP model when applied to the
Canadian food supply. To our knowledge, the validation study presented in Chapter 3 is
the first study to assess the validity of several models developed by authoritative bodies
by using them to classify over 15,000 foods from a branded, Canadian database.
79
Moreover, as Health Canada has proposed to use a newly developed NP model in their
policies on front-of-pack labelling and restrictions on marketing to children(40), this
research can inform the use of this proposed NP model for these applications. First, the
validation study highlighted the importance of examining classifications across food
categories and identified specific categories that are prone to misclassification when
assessed by different models, including combination dishes, soups, fats/oils, and eggs.
Regulators will be able to use these data to prioritize the classifications of these
“problem” food categories in order to ensure that the proposed NP model generates
accurate classifications.
Second, this thesis provided an understanding of the different characteristics of NP
models, and the validation study postulated reasons for the discrepancies in the
classifications determined using several models which differed in their characteristics.
For example, it was observed that the HCST model, which includes only four negative
nutrients, fared poorly in identifying “healthier” foods when compared to the Ofcom
model, which takes into account the positive attributes of foods. Thus, similar results to
the HCST will likely be observed for the proposed NP model, which includes only three
negative nutrients (i.e., saturated fat, sugars, and sodium). As such, regulators will be
able to use these data to inform the decisions on whether any exemptions or
adjustments to the nutrient criteria should be made, such as including the protein
content for eggs, mono- or polyunsaturated fat content for fats/oils, or
fruit/vegetable/legume content for combination dishes and soups.
Lastly, this thesis provided an understanding of the definitions and methods for NP
model validation. Regulators will be able to use these data to prioritize the different
validation studies, based on their strengths and limitations, that will need to be
conducted in order to validate their newly developed NP model. For example, studies
involving consumers will be important for establishing face validity of the proposed NP
model for use in front-of-pack labelling; however, studies involving nutrition
professionals, who may be more discerning towards the healthfulness of foods, will be
important for establishing criterion/convergent/construct validity of the model for use in
restrictions on marketing to children.
80
Chapter 5
5 Conclusions Based on the data presented in this thesis, it is evident that different NP models can
vary substantially in their design and characteristics, and this variation ultimately gives
rise to inherent differences in the manner in which these models classify the
healthfulness of foods. As such, validity testing is important to ensure that these NP
models are consistent with and complement other nutrition regulations or policies
already in place within a jurisdiction. In addition, NP is a rapidly evolving field with a
proliferation in the number of NP models globally. As the evidence on the adequacy of
these models has lagged behind, particularly in the Canadian context, validity testing of
NP models is of utmost importance in this field of research.
Furthermore, the data presented in this thesis illustrate the challenges associated with
validating NP models in a global context. Due to the ambiguity in the definitions of the
different types of validity, there is inconsistency with regards the type of validity that can
be inferred from using certain methods. Although there are promising results to indicate
that the identification of validity testing amongst NP models has increased in the last
several years, there is still substantial improvement to be made in the validation of NP
models, particularly with regards to the reporting and documentation of validity testing.
In addition, the scientific standard required to establish that a NP model is sufficiently
valid remains unclear.
In this thesis, a comprehensive examination of the validity of the FSANZ, HCST, EURO,
and PAHO models was conducted by classifying over 15,000 foods, using a
combination of parameters for assessment, and carrying out detailed analyses across
food categories. This research provides regulators with pertinent data that will facilitate
the selection and validation of NP models for public health initiatives and nutrition policy.
Specifically in a Canadian context, this research provides valuable information that will
help inform the application and validity testing of the new NP model proposed by Health
Canada to underpin the regulations on front-of-pack nutrition labelling and restrictions
81
on marketing to children. Overall, this research contributes to the scientific evidence in
establishing the validity of the four models for assessing the nutritional quality foods,
globally and in Canada, and to the emerging science of NP.
82
References 1. World Health Organization (in press) Guiding Principles and Framework Manual for the Development or Adaptation of Nutrient Profile Models (First Edition). Geneva: WHO.
2. Rayner M, Scarborough P, Kaur A (2013) Nutrient profiling and the regulation of marketing to children. Possibilities and pitfalls. Appetite 62, 232-235.
3. Rayner M (2017) Nutrient profiling for regulatory purposes. Proc Nutr Soc 76, 230-236.
4. Rayner M, Scarborough P, Stockley L (2004) Nutrient profiles: options for definitions for use in relation to food promotion and children’s diets (final report). London: Food Standards Agency. http://webarchive.nationalarchives.gov.uk/20100403014958/http://www.food.gov.uk/multimedia/pdfs/nutrientprofilingfullreport.pdf.
5. Rayner M, Scarborough P, Stockley L et al. (2005) Nutrient profiles: further refinement and testing of Model SSCg3d (final report). London: Food Standards Agency. http://www.food.gov.uk/multimedia/pdfs/npreportsept05.pdf.
6. Stockley L, Rayner M, Kaur A (2008) Nutrient profiles for use in relation to food promotion and children’s diet: update of 2004 literature review. Oxford: University of Oxford. http://webarchive.nationalarchives.gov.uk/20090808161247/http://text.food.gov.uk/cgi-bin/parser.pl/0005/www.food.gov.uk/multimedia/pdfs/publication/npliteraturereviewupdate.pdf.
7. World Health Organization (in press) Nutrient Profiling. Catalogue of Nutrient Profile Models: Summary Report. Geneva: WHO.
8. Labonte ME, Poon T, Gladanac B et al. (Unpublished) Nutrient profile models with applications in government-led nutrition policies aimed at health promotion and noncommunicable disease prevention: a systematic review. (To be submitted to Adv Nutr in December 2017).
9. World Health Organization (2010) Nutrient Profiling. Report of a WHO/IASO Technical Meeting, London, United Kingdom 4–6 October 2010. Geneva: WHO. http://www.who.int/nutrition/publications/profiling/WHO_IASO_report2010.pdf.
10. Cooper SL, Pelly FE, Lowe JB (2016) Construct and criterion-related validation of nutrient profiling models: a systematic review of the literature. Appetite 100, 26-40.
11. Drewnowski A, Fulgoni V (2008) Nutrient profiling of foods: creating a nutrient-rich food index. Nutr Rev 66, 23-39.
12. U.S. Department of Agriculture (2014) Federal Register Part II Vol. 79. No. 42. 7 CFR §246: Special Supplemental Nutrition Program for Women, Infants and Children
83
(WIC): Revisions in the WIC Food Packages – Final Rule. http://www.fns.usda.gov/sites/default/files/03-04-14_WIC-Food-Packages-Final-Rule.pdf (accessed June 2017).
13. National Food Agency of Sweden (2015) Regulations amending the National Food Agency's regulations (SLVFS 2005:9) on the use of a particular symbol. http://www.livsmedelsverket.se/globalassets/om-oss/lagstiftning/livsmedelsinfo-till-konsum---markning/livsfs-2015-1-particular-symbol-eng.pdf (accessed June 2017).
14. U.S. Food and Drug Administration (2017) 21 CFR §101.14. Health claims: general requirements. http://www.ecfr.gov/cgi-bin/text-idx?SID=c77c3ef247de8778a86e598ef4c2dcdc&mc=true&node=pt21.2.101&rgn=div5#se21.2.101_114 (accessed June 2017).
15. U.S. Food and Drug Administration (2017) 21 CFR §101.65. Implied nutrient content claims and related label statements. https://www.ecfr.gov/cgi-bin/text-idx?SID=c77c3ef247de8778a86e598ef4c2dcdc&mc=true&node=pt21.2.101&rgn=div5#se21.2.101_165 (accessed June 2017).
16. European Commission (2012) Regulation (EC) No 1924/2006 of the European parliament and of the council of 20 December 2006 on nutrition and health claims made on foods. http://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:02006R1924-20121129 (accessed June 2017).
17. World Health Organization (2010) Set of recommendations on the marketing of foods and non-alcoholic beverages to children. Geneva: WHO. http://apps.who.int/iris/bitstream/10665/44416/1/9789241500210_eng.pdf.
18. Australian Government (2015) Australia New Zealand Food Standards Code - Standard 1.2.7 - Nutrition, Health and Related Claims (F2015C00967). https://www.comlaw.gov.au/Details/F2015C00967 (accessed June 2017).
19. World Health Organization Regional Office for Europe (2015) WHO Regional Office for Europe nutrient profile model. Copenhagen: WHO Regional Office for Europe. http://www.euro.who.int/__data/assets/pdf_file/0005/270716/Nutrient-children_web-new.pdf?ua=1.
20. Queensland Government (2015) Food for Sport. Resource 4: red criteria and label reading. http://www.nprsr.qld.gov.au/industry-information/pdf/clubs/food-for-sport-resource-4.pdf (accessed August 2016).
21. Pan American Health Organization (2016) Pan American Health Organization Nutrient Profile Model. Washington, D.C.: PAHO. http://iris.paho.org/xmlui/bitstream/handle/123456789/18621/9789275118733_eng.pdf?sequence=8&isAllowed=y.
22. World Health Organization and Food and Agricultural Organization (2003) Diet, nutrition and the prevention of chronic diseases. Report of the joint WHO/FAO expert
84
consultation (WHO Technical Report Series No. 916). Geneva: WHO. http://www.who.int/dietphysicalactivity/publications/trs916/en/.
23. World Health Organization (2015) Guideline: sugars intake for adults and children. Geneva: WHO. http://apps.who.int/iris/bitstream/10665/149782/1/9789241549028_eng.pdf?ua=1.
24. World Health Organization (2012) Guideline: sodium intake for adults and children. Geneva: WHO. http://apps.who.int/iris/bitstream/10665/77985/1/9789241504836_eng.pdf.
25. Food and Agricultural Organization (2010) Fats and fatty acids in human nutrition. Report of an expert consultation (Food and Nutrition Paper No. 91). Rome: FAO. http://www.who.int/nutrition/publications/nutrientrequirements/fatsandfattyacids_humannutrition/en/.
26. Labonte ME, Poon T, Mulligan C et al. (2017) Comparison of global nutrient profiling systems for restricting the commercial marketing of foods and beverages of low nutritional quality to children in Canada. Am J Clin Nutr 106, 1471-1481.
27. UK Department of Health (2013) Guide to creating a front of pack (FoP) nutrition label for pre-packaged products sold through retail outlets. London: Department of Health. https://www.gov.uk/government/publications/front-of-pack-nutrition-labelling-guidance.
28. United Kingdom Department of Health (2011) Nutrient Profiling Technical Guidance. London: Department of Health. https://www.gov.uk/government/publications/the-nutrient-profiling-model.
29. Health Canada (2014) The Development and Use of a Surveillance Tool: the Classification of Foods in the Canadian Nutrient File According to Eating Well with Canada’s Food Guide. Ottawa: Ministry of Health. http://publications.gc.ca/collections/collection_2014/sc-hc/H164-158-2-2014-eng.pdf.
30. Gibson RS (2005) The design of nutritional assessment systems. In Principles of Nutritional Assessment, 2nd ed., pp. 11-15. New York: Oxford University Press.
31. Porta M (2014) A Dictionary of Epidemiology, 6th ed., [S Greenland, M Hernán, I dos Santos Silva and JM Last, editors]. New York: Oxford University Press.
32. Arambepola C, Scarborough P, Rayner M (2008) Validating a nutrient profile model. Public Health Nutr 11, 371-378.
33. Scarborough P, Boxer A, Rayner M et al. (2007) Testing nutrient profile models using data from a survey of nutrition professionals. Public Health Nutr 10, 337-345.
34. Townsend MS (2010) Where is the science? What will it take to show that nutrient profiling systems work? Am J Clin Nutr 91, 1109S-1115S.
85
35. Cronbach LJ, Meehl PE (1955) Construct validity in psychological tests. Psychol Bull 52, 281-302.
36. Cooper SL, Pelly FE, Lowe JB (2017) Assessment of the construct validity of the Australian Health Star Rating: a nutrient profiling diagnostic accuracy study. Eur J Clin Nutr, 1-7.
37. Clerfeuille E, Vieux F, Lluch A et al. (2013) Assessing the construct validity of five nutrient profiling systems using diet modeling with linear programming. Eur J Clin Nutr 67, 1003-1005.
38. Darmon N, Vieux F, Maillot M et al. (2009) Nutrient profiles discriminate between foods according to their contribution to nutritionally adequate diets: a validation study using linear programming and the SAIN,LIM system. Am J Clin Nutr 89, 1227-1236.
39. Federal Provincial Territorial Group on Nutrition Working Group on Improving the Consistency of School Food and Beverage Criteria (2013) Provincial and Territorial Guidance Document for the Development of Nutrient Criteria for Foods and Beverages in Schools 2013. http://foodsecurecanada.org/resources-news/resources-research/provincial-and-territorial-guidance-document-development-nutrient.
40. Health Canada (2016) Healthy Eating Strategy. Ottawa: Ministry of Health. https://www.canada.ca/content/dam/canada/health-canada/migration/publications/eating-nutrition/healthy-eating-strategy-canada-strategie-saine-alimentation/alt/pub-eng.pdf.
41. Labonte ME, Poon T, Gladanac B et al. (2015) Systematic review of nutrient profile models developed for nutrition-related policies and regulations aimed at noncommunicable disease prevention. http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42015024750 (accessed January 2017).
42. Rayner M, Scarborough P, Boxer A et al. (2005) Nutrient profiles: development of final model (final report). London: Food Standards Agency.
43. Masset G, Scarborough P, Rayner M et al. (2015) Can nutrient profiling help to identify foods which diet variety should be encouraged? Results from the Whitehall II cohort. Br J Nutr 113, 1800-1809.
44. Julia C, Ducrot P, Lassale C et al. (2015) Prospective associations between a dietary index based on the British Food Standard Agency nutrient profiling system and 13-year weight gain in the SU.VI.MAX cohort. Prev Med 81, 189-194.
45. Julia C, Fezeu LK, Ducrot P et al. (2015) The nutrient profile of foods consumed using the British Food Standards Agency nutrient profiling system is associated with metabolic syndrome in the SU.VI.MAX cohort. J Nutr 145, 2355-2361.
86
46. Donnenfeld M, Julia C, Kesse-Guyot E et al. (2015) Prospective association between cancer risk and an individual dietary index based on the British Food Standards Agency nutrient profiling system. Br J Nutr 114, 1702-1710.
47. Adriouch S, Julia C, Kesse-Guyot E et al. (2016) Prospective association between a dietary quality index based on a nutrient profiling system and cardiovascular disease risk. Eur J Prev Cardiol, 1-8.
48. Brinsden H, Lobstein T (2013) Comparison of nutrient profiling schemes for restricting the marketing of food and drink to children. Pediatr Obes 8, 325-337.
49. Eržen N, Rayner M, Pravst I (2015) A comparative evaluation of the use of a food composition database and nutrition declarations for nutrient profiling. J Food Nutr Res 54, 93-100.
50. Wicks M, Wright H, Wentzel-Viljoen E (2017) Restricting the marketing of foods and non-alcoholic beverages to children in South Africa: are all nutrient profiling models the same? Br J Nutr 116, 2150-2159.
51. Faulkner GP, Livingstone MB, McCaffrey TA et al. (2014) Supermarket own brand foods: lower in energy cost but similar in nutritional quality to their market brand alternatives. J Hum Nutr Diet 27, 617-625.
52. Rosentreter SC, Eyles H, Ni Mhurchu C (2013) Traffic lights and health claims: a comparative analysis of the nutrient profile of packaged foods available for sale in New Zealand supermarkets. Aust N Z J Public Health 37, 278-283.
53. Scarborough P, Payne C, Agu CG et al. (2013) How important is the choice of the nutrient profile model used to regulate broadcast advertising of foods to children? A comparison using a targeted data set. Eur J Clin Nutr 67, 815-820.
54. Trichterborn J, Drossard C, Kersting M et al. (2012) The potential impact of nutrient profiles on dairy-related energy and nutrient intake in German children and adolescents. Eur J Clin Nutr 66, 466-473.
55. Drewnowski A, Fulgoni V (2011) Comparing the nutrient rich foods index with "Go," "Slow," and "Whoa," foods. J Am Diet Assoc 111, 280-284.
56. Roberto CA, Bragg MA, Livingston KA et al. (2012) Choosing front-of-package food labelling nutritional criteria: how smart were 'Smart Choices'? Public Health Nutr 15, 262-267.
57. Trichterborn J, Harzer G, Kunz C (2011) Fine bakery wares with label claims in Europe and their categorisation by nutrient profiling models. Eur J Clin Nutr 65, 307-312.
58. Trichterborn J, Harzer G, Kunz C (2011) Nutrient profiling and food label claims: evaluation of dairy products in three major European countries. Eur J Clin Nutr 65, 1032-1038.
87
59. Eyles H, Gorton D, Ni Mhurchu C (2010) Classification of 'healthier' and 'less healthy' supermarket foods by two Australasian nutrient profiling models. N Z Med J 123, 8-20.
60. Hebden L, King L, Kelly B et al. (2010) Regulating the types of foods and beverages marketed to Australian children: how useful are food industry commitments? Nutr and Diet 67, 258-266.
61. Walker KZ, Woods J, Ross J et al. (2010) Yoghurt and dairy snacks presented for sale to an Australian consumer: are they becoming less healthy? Public Health Nutr 13, 1036-1041.
62. Drewnowski A, Maillot M, Darmon N (2009) Testing nutrient profile models in relation to energy density and energy cost. Eur J Clin Nutr 63, 674-683.
63. Walker KZ, Woods JL, Rickard CA et al. (2008) Product variety in Australian snacks and drinks: how can the consumer make a healthy choice? Public Health Nutr 11, 1046-1053.
64. Garsetti M, de Vries J, Smith M et al. (2007) Nutrient profiling schemes: overview and comparative analysis. Eur J Nutr 46 Suppl 2, 15-28.
65. Labouze E, Goffi C, Moulay L et al. (2007) A multipurpose tool to evaluate the nutritional quality of individual foods: Nutrimap. Public Health Nutr 10, 690-700.
66. Quinio C, Biltoft-Jensen A, De Henauw S et al. (2007) Comparison of different nutrient profiling schemes to a new reference method using dietary surveys. Eur J Nutr 46 Suppl 2, 37-46.
67. Azais-Braesco V, Goffi C, Labouze E (2006) Nutrient profiling: comparison and critical analysis of existing systems. Public Health Nutr 9, 613-622.
68. Bernstein JT, Franco-Arellano B, Schermel A et al. (2017) Healthfulness and nutritional composition of Canadian prepackaged foods and with and without sugar claims. Appl Phys Nutr Metab 42, 1217-1224.
69. Bernstein JT, Schermel A, Mills CM et al. (2016) Total and free sugar content of Canadian prepackaged foods and beverages. Nutrients 8, 582.
70. World Health Organization (2013) Global action plan for the prevention and control of noncommunicable diseases 2013-2020. Geneva: WHO. http://apps.who.int/iris/bitstream/10665/94384/1/9789241506236_eng.pdf?ua=1.
71. World Health Organization Regional Office for Europe (2017) Seventieth World Health Assembly (18 May 2017): Preparation for the third High-level Meeting of the General Assembly on the Prevention and Control of Non-communicable Diseases, to be held in 2018 – Report by the Director-General. Copenhagen: WHO Regional Office for Europe. http://www.euro.who.int/__data/assets/pdf_file/0010/340786/A70_27-en-cover.pdf?ua=1.
88
72. Government of Canada (2016) Food and Drug Regulations (C.R.C., c. 870), Sections B.01.001, B.01.002A and D.01.001. http://laws-lois.justice.gc.ca/eng/regulations/c.r.c.,_c._870/page-165.html#h-353 (accessed December 2016).
73. Viera AJ, Garrett JM (2005) Understanding interobserver agreement: the kappa statistic. Fam Med 37, 360-363.
74. Food Standards Australia New Zealand (2015) Monitoring of trans fatty acids. http://www.foodstandards.gov.au/science/monitoringnutrients/Pages/Monitoring-of-trans-fatty-acids.aspx (accessed September 2017).
75. Government of Canada (2016) Fats. http://healthycanadians.gc.ca/eating-nutrition/healthy-eating-saine-alimentation/nutrients-nutriments/fats-lipides-eng.php (accessed September 2017).
76. Arcand J, Scourboutakos MJ, Au JT et al. (2014) Trans fatty acids in the Canadian food supply: an updated analysis. Am J Clin Nutr 100, 1116-1123.
77. World Health Organization Regional Office for the Western Pacific (2016) WHO nutrient profile model for the Western Pacific Region: a tool to protect children from food marketing. Manila: WHO Regional Office for the Western Pacific. http://iris.wpro.who.int/bitstream/handle/10665.1/13525/9789290617853-eng.pdf.
78. World Health Organization Regional Office for South-East Asia (2017) WHO Nutrient Profile Model for South-East Asia Region. To implement the set of recommendations on the marketing of foods and non-alcoholic beverages to children. New Delhi: WHO Regional Office for South-East Asia. http://apps.who.int/iris/bitstream/10665/253459/1/9789290225447-eng.pdf.
79. World Health Organization Regional Office for the Eastern Mediterranean (2017) Nutrient profile model for the marketing of food and non-alcoholic beverages to children in the WHO Eastern Mediterranean Region. Cairo: WHO Regional Office for the Eastern Mediterranean. http://applications.emro.who.int/dsaf/EMROPUB_2017_en_19632.pdf?ua=1.
80. Public Health England (2016) The Eatwell Guide. Helping you eat a healthy, balanced diet. London: Public Health England. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/551502/Eatwell_Guide_booklet.pdf.
81. U.S. Department of Health and Human Services and U.S. Department of Agriculture (2015) 2015-2020 Dietary Guidelines for Americans (Eighth Edition). https://health.gov/dietaryguidelines/2015/resources/2015-2020_Dietary_Guidelines.pdf.
82. Jessri M, Nishi SK, L'Abbe MR (2015) Assessing the nutritional quality of diets of Canadian adults using the 2014 Health Canada Surveillance Tool tier system. Nutrients 7, 10447-10468.
89
83. Government of Canada (2017) Revision process for Canada's Food Guide. https://www.canada.ca/en/health-canada/services/canada-food-guides/revision-process.html (accessed September 2017).
84. Santé publique France (2017) Nutri-Score. https://www.santepubliquefrance.fr/Sante-publique-France/Nutri-Score (accessed November 2017).
85. Public Health England (2016) Review of the nutrient profiling model. https://www.gov.uk/government/collections/review-of-the-nutrient-profiling-model (accessed March 2017).
86. Ryckembusch D, Frega R, Silva MG et al. (2013) Enhancing nutrition: a new tool for ex-ante comparison of commodity-based vouchers and food transfers. World Development 49, 58-67.
87. Chiuve SE, Sampson L, Willett WC (2011) The association between a nutritional quality index and risk of chronic disease. Am J Prev Med 40, 505-513.
88. Breda J (2014) A WHO nutrient profile model: the European perspective. Platform for Action on Diet, Physical Activity and Health Brussels, 24 November 2014. http://ec.europa.eu/health/nutrition_physical_activity/docs/ev_20141124_co06_en.pdf.
90
Appendix A
Study on NP Models for Marketing Restrictions to Children This study has been published: Labonté MÈ, Poon T, Mulligan C, Bernstein JT, Franco-
Arellano B, L’Abbé MR. Comparison of global nutrient profiling systems for restricting
the commercial marketing of foods and beverages of low nutritional quality to children in
Canada. Am J Clin Nutr 106, 1471-1481.
Although this research was not explicitly part of my MSc thesis, I had a significant role in
this research, as indicated in Table A-1.
Table A-1 Authors’ contributions to study on NP models for marketing restrictions to children
Task Author Prior to January 2016 when TP started MSc Calculated FSANZ scores Labonté, Bernstein, Franco-Arellano After January 2016 when TP started MSc Designed research study Labonté, Poon, L’Abbé Assessed on-package marketing to children Mulligan Generated data for NP models Labonté, Poon Analyzed data, ran statistical analyses Poon Wrote manuscript Labonté Provided comments/revisions to draft manuscripts Poon, L’Abbé Provided comments/revisions to final manuscript Poon, Mulligan, Bernstein, Franco-Arellano, L’Abbé Submitted manuscript to journal Labonté Provided comments/revisions to post-submission responses
Poon, L’Abbé
Comparison of global nutrient profiling systems for restricting thecommercial marketing of foods and beverages of low nutritionalquality to children in Canada
Marie-Eve Labonte, Theresa Poon, Christine Mulligan, Jodi T Bernstein, Beatriz Franco-Arellano, and Mary R L’Abbe
Department of Nutritional Sciences, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
ABSTRACTBackground: The Canadian government recently committed to in-troduce legislation to restrict the commercial marketing of un-healthy foods and beverages to children.Objective: We compared the degree of strictness and agreementbetween nutrient profile (NP) models relevant to marketing restric-tions by applying them in the Canadian context.Design: With the use of data from the University of Toronto 2013Food Label Information Program (n = 15,342 prepackaged foods), 4 NPmodels were evaluated: the Food Standards Australia New Zealand-Nutrient Profiling Scoring Criterion (FSANZ-NPSC), theWHORegionalOffice for Europe (EURO) model, the Pan American Health Organization(PAHO) model, and a modified version of the PAHO model (Modified-PAHO), which did not consider the extent of food processing because theapplication of this characteristic was prone to ambiguity. The number andproportion of foods that would be eligible for marketing to children wascalculated with the use of each model, overall and by food category.Results: The Modified-PAHO and PAHO models would permit only9.8% (95% CI: 9.4%, 10.3%) and 15.8% (95% CI: 15.3%, 16.4%) offoods, respectively, followed by the EURO model [29.8% (95% CI:29.0%, 30.5%)]. In contrast, the FSANZ-NPSC would consider al-most half of prepackaged foods as eligible for marketing to children[49.0% (95% CI: 48.2%, 49.8%)]. Cross-classification analysesshowed that only 8.1% of foods would be eligible based on all models(e.g., most pastas without sauce). Subanalyses showed that eachmodel would be more stringent when evaluating food items thatspecifically target children on their package (n = 747; from 1.9% offoods eligible under Modified-PAHO to 24.2% under FSANZ-NPSC).Conclusions: The degree of strictness and agreement vary greatlybetween NP models applicable to marketing restrictions. The dis-crepancies between models highlight the importance for policymakers to carefully evaluate the characteristics underlying suchmodels when trying to identify a suitable model to underpin regu-lations restricting the marketing of unhealthy foods to children. Am JClin Nutr 2017;106:1471–81.
Keywords: food processing, healthfulness, marketing restrictions,children, nutrition policy
INTRODUCTION
Recent trends show that the prevalence of obesity in Canadianchildren and adolescents has reached a plateau ofw13% over the
last decade (1). Although the leveling off is promising, thoserates are still 3 times higher than those observed in the early1980s (2). The prevalence of both overweight and obesity inCanadian children and adolescents, estimated at 27.0%, alsoremains relatively high at the present time (1).
A strong evidence base now supports the role of unhealthyfood and beverage marketing as one of the key drivers of theglobal childhood obesity epidemic (3). Food promotion has in-deed been associated with increased food intakes in children (4–6). Marketing practices are also known to shape children’spreferences toward low–nutrient-dense food products high in fat,sugar, and salt (6, 7). Additionally, “less healthy” products, asdefined by various nutritional criteria, represent the ones that aremost heavily marketed to young individuals in Canada and inseveral other countries (8–10).
In 2016, the WHO Commission on Ending Childhood Obesityestablished a set of recommended actions for its Member States toeffectively combat childhood obesity (11). One of them is to
Supported by Canadian Institutes of Health Research (CIHR) Postdoctoral
Fellowship MFE-140953 (M-EL); Burroughs Wellcome Fund, Innovation in
Regulatory Science (TP, CM, MRL; 1014187); a CIHR Strategic Training
Grant in Population Intervention for Chronic Disease Prevention (JTB; TGF-
53893); the CIHR Collaborative Training Program in Public Health Policy
(JTB); a Graduate Student Fellowship of the Department of Nutritional
Sciences, University of Toronto (BF-A); a CIHR Strategic Operating Grant
(MRL; 201103SOK2118150); Canadian Stroke Network (MRL;
201103SOK201194-000); and an Earle W McHenry Research Chair un-
restricted research grant from the University of Toronto (MRL).
Supplemental Figure 1 and Supplemental Table 1 are available from the
“Online Supporting Material” link in the online posting of the article and
from the same link in the online table of contents at http://ajcn.nutrition.org.
Present address for M-EL: Institute of Nutrition and Functional Foods,
Laval University, Quebec City, Quebec, Canada.
Address correspondence to MRL (e-mail: [email protected]).
Abbreviations used: EURO, WHO Regional Office for Europe; FLIP,
Food Label Information Program; FSANZ-NPSC, Food Standards Australia
New Zealand-Nutrient Profiling Scoring Criterion; FVNL, fruit, vegetable,
nut, and legume; Modified-PAHO, modified version of the Pan American
Health Organization model; NFt, Nutrition Facts table; NP, nutrient profile;
PAHO, Pan American Health Organization.
Received May 25, 2017. Accepted for publication October 2, 2017.
First published online October 25, 2017; doi: https://doi.org/10.3945/ajcn.
117.161356.
Am J Clin Nutr 2017;106:1471–81. Printed in USA. � 2017 American Society for Nutrition 1471
at UN
IVE
RS
ITY
OF
TO
RO
NT
O G
ER
ST
EIN
SC
IEN
CE
INF
OR
MA
TIO
N C
TR
on Decem
ber 13, 2017ajcn.nutrition.org
Dow
nloaded from
61356.DCSupplemental.html http://ajcn.nutrition.org/content/suppl/2017/10/25/ajcn.117.1Supplemental Material can be found at:91
implement the WHO 2010 recommendations to reduce the ex-posure to, and the power of, the marketing of foods and beverageshigh in SFAs, trans fatty acids, free sugars, or salt to childrenand adolescents (11, 12). Another recommended action is todevelop nutrient profile (NP) models, which are based on ob-jective, transparent, and reproducible nutritional criteria, to de-termine whether a food product is eligible or not eligible to bemarketed to children (11). A number of countries, includingChile, Denmark, Ireland, Mexico, New Zealand, Norway,Singapore, South Korea, and the United Kingdom have alreadytaken steps in that direction and use NP models to regulatedifferent forms of marketing to children (13, 14). Other coun-tries, such as Finland and Sweden, instead have implemented atotal ban on the marketing of food products to children, there-fore not requiring the adoption of an NP model (13, 14).
In October 2016, Health Canada committed to introducingrestrictions on the marketing of foods and beverages to childrenas part of its Healthy Eating Strategy (15). In the event that thecomplete ban advocated for by various stakeholders in thecountry is not implemented (16–18), an NP model will need tounderpin the proposed restrictions. The main objective of thepresent study was therefore to compare, in the Canadian context,the degree of strictness and agreement between NP models de-veloped by authoritative organizations for application in re-stricting marketing to children. A secondary objective was toperform these comparisons on the subset of food products cur-rently carrying on-package marketing to children. Based onprevious studies that compared classifications made by NPmodels meant for a variety of purposes (9, 10, 19–23), we hy-pothesized that the degree of strictness and agreement wouldvary substantially between the studied models.
METHODS
Study design
This was a cross-sectional analysis of the Canadian pre-packaged food supply with the use of the University of TorontoFood Label Information Program (FLIP) 2013 database, de-scribed in detail elsewhere (24). Briefly, FLIP 2013 containsnutritional information on 15,342 unique food products with aNutrition Facts table (NFt) from the 4 largest grocery chains inCanada (Loblaws, Metro, Sobeys, and Safeway), representing75.4% of the grocery retail market share (25). Data were collectedbetween May and September 2013 by systematically scanninggrocery store shelves with the use of a smartphone application.The information collected includes product information(e.g., company, brand, Universal Product Code), container size,price, NFt information, ingredient list, package marketing(e.g., nutrient content claims, front-of-package labeling, andmarketing to children), and photos of all sides of the packages.Only 1 package size/food was captured, but all flavors and va-rieties were collected. Nutritional information was recordedfor products in their “as sold” form and, if necessary(e.g., condensed soup), values for the “as consumed” form werecalculated according to package instructions with the use ofESHA Food Processor software (version 10.13.1; ESHA Re-search) and food composition data from the Canadian NutrientFile, version 2010b (26).
Foods in FLIP 2013 were classified into 22 distinct foodcategories and 153 subcategories as defined in Schedule M of theFood and Drug Regulations (version in force between 15 March2012 and 13 December 2016) (27). Specific types of products notclassified in any Schedule M category were excluded from thepresent analyses as follows: meal replacements (n = 55), instantor dry yeast (n = 4), and a natural health product (n = 1). A totalof 55 products were further excluded because of errors in nu-trient declarations in the NFt, as determined by Atwater calcu-lations that were .20% from the declared caloric values. Thus,15,227 products were included in the analyses.
Selected NP systems
Table 1 summarizes the key characteristics of each NP modelselected for the present study, consisting of 2 internationalsystems recently developed by regional offices of the WHO[Europe (28) and the Americas (29)] and an NP system from theAustralia and New Zealand governments (30). These NP modelsbuilt by authoritative sources were specifically retained for theirpotential wide applicability, meaning that they have been de-veloped or tested for use in several countries. Further details oneach model and their application to the FLIP 2013 databasefollow.
Food Standards Australia New Zealand-Nutrient ProfilingScoring Criterion
Although primarily designed to assess the eligibility of a foodproduct to carry a health claim, the Food Standards Australia NewZealand-Nutrient Profiling Scoring Criterion (FSANZ-NPSC) (30)was retained because it represents a modified version of thewell-established Ofcom model used for marketing restrictionsin the United Kingdom (31). The potential effectiveness ofthe FSANZ-NPSC as an NP model to restrict marketing tochildren has also previously been investigated (10, 32). Ad-ditionally, the original Ofcom model is currently undergoingrevision (33).
Briefly, foods were first classified into 1 of 3 possible cate-gories as follows: 1) beverages, 2) any food item not in category1 or 3, and 3) cheese with a high calcium content (.320 mgCa/100 g) and fats (e.g., oil and butter). The third categoryrepresents the main difference between the FSANZ-NPSCand the Ofcom model, which includes only 2 categories.That third category takes into account the higher fat and so-dium content that can be found in fats and cheese products ascompared with other food items that fall into category 2. Asummary score was then calculated for each food productbased on points for both nutrients to limit (energy, saturatedfat, total sugars, and sodium) and nutrients or food compo-nents to encourage [protein, fiber, and percent composition offruits, vegetables, nuts, and legumes (FVNLs) in a product].Points for each nutrient to limit and each nutrient to en-courage were assigned based on nutrient values per 100 g/mL.Because of the absence of quantitative ingredient declarationsin Canada, a method was established by our group, based onthe presence and position of FVNL ingredients within theingredient list, to estimate FVNL points for each food prod-uct. The method is detailed elsewhere (34). Predefined cutoffscores, which vary depending on the FSANZ-NPSC category,were used to classify food products either as eligible or not
1472 LABONTE ET AL.
at UN
IVE
RS
ITY
OF
TO
RO
NT
O G
ER
ST
EIN
SC
IEN
CE
INF
OR
MA
TIO
N C
TR
on Decem
ber 13, 2017ajcn.nutrition.org
Dow
nloaded from
92
eligible to be marketed to children. The version of theFSANZ-NPSC in force in Australia and New Zealand before1 March 2016 was used in the present analyses (30).
Pan American Health Organization NP model
The Pan American Health Organization (PAHO), representingthe WHO Regional Office for the Americas, published an NPmodel in 2016 designed for applications in a wide range ofnutrition policies, including, among others, restrictions on themarketing of unhealthy food and beverages to children, theregulation of school food environments, the establishment offront-of-package warning labels, and the establishment of tax-ation policies (29).
The application of the PAHO NP model first required fooditems to be classified into 1 of 5 possible categories, based ontheir extent of processing: 1) ultra-processed products, 2) pro-cessed products, 3) unprocessed or minimally processed prod-ucts, 4) culinary ingredients, or 5) freshly prepared products. AsFLIP is a database of packaged foods, no product in FLIP 2013was a freshly prepared product. Processed or ultra-processedproducts were thereafter classified as containing “excessive” or“not excessive” amounts of critical nutrients (sodium, freesugars, other sweeteners, total fat, saturated fat, and trans fat),based on predetermined thresholds in the model. Other sweet-eners were specifically evaluated based on their presence (yesor no) in the ingredient list and sodium was evaluated on a
TABLE 1
Summary of the 4 governmental or intergovernmental nutrient profile models evaluated1
Model and
application(s)
Food
categories, n
Nutrients
n Energy Total fat
Saturated
fat trans Fat
Total
sugars
Free/added
sugars Sweeteners Sodium/salt Protein Fiber FVNL
FSANZ-NPSC
Nutrition content
and health claims233 7 U U U U U U U
PAHO
Marketing of foods
to children
54 7 U5 U U U U (free) U U
School food environments
FOP labeling
Taxation policies
Agricultural subsidies
Food provision guidelines
for social programs
Modified-PAHO
Same as PAHO 16 7 U5 U U U U (free) U U
EURO
Marketing of foods
to children
177 88 U U U U U U (added)9 U U
1 EURO, WHO Regional Office for Europe; FOP, front-of-package; FSANZ-NPSC, Food Standards Australia New Zealand-Nutrient Profiling Scoring
Criterion; FVNL, fruits, vegetables, nuts, and legumes; Modified-PAHO, modified version of Pan American Health Organization model; PAHO, Pan American
Health Organization.2 The FSANZ-NPSC was retained because it consists of an updated version of the Ofcom model used for marketing restrictions in the United Kingdom.3 The 3 food categories are as follows: 1) beverages, 2) any food item not in category 1 or 3, and 3) cheese with a calcium content.320 mg/100 g, edible
oil spreads, margarine, and butter.4 The 5 food categories are as follows: 1) processed products, 2) ultra-processed products, 3) unprocessed or minimally processed products, 4) culinary
ingredients, and 5) freshly prepared dishes. The same nutrient criteria are applied only to processed and ultra-processed products. Nutrient criteria are not
applied to unprocessed or minimally processed products, culinary ingredients, and freshly prepared dishes; that is, the PAHO model considers that these
categories are always eligible to be marketed to children.5 Total energy provided by the food is not a criterion; however, the criteria for the other nutrients are presented on a per-total-energy basis (e.g., the
threshold to indicate an excess of total fat is $30% of total energy).6 All foods are evaluated with the use of the nutrient profile criteria, irrespective of the extent of food processing.7 There are 17 food categories, but the beverages food category contains 4 subcategories. These categories are as follows: 1) chocolate and sugar
confectionery, energy bars, and sweet toppings and desserts; 2) cakes, sweet biscuits, pastries, other sweet bakery wares, and dry mixes for making such; 3)
savory snacks; 4) beverages: a) juices, b) milk drinks, c) energy drinks, and d) other beverages; 5) edible ices; 6) breakfast cereals; 7) yogurts, sour milk,
cream, and other similar foods; 8) cheese; 9) ready-made and convenience foods and composite dishes; 10) butter and other fats and oils; 11) bread, bread
products, and crisp breads; 12) fresh or dried pasta, rice, and grains; 13) fresh and frozen meat, poultry, fish, and similar; 14) processed meat, poultry, fish, and
similar; 15) fresh and frozen fruit, vegetables, and legumes; 16) processed fruit, vegetables, and legumes; and 17) sauces, dips, and dressings.8 The number and types of nutrients or food components considered varies depending on the category, except for trans fat. Indeed, according to the WHO,
“marketing is prohibited if the product contains .1 g per 100 g total fat in the form of industrially-produced trans fatty acids,” irrespective of the food
category. Because the exact contribution of ruminant compared with industrially produced trans fat is not known for products from the Food Label Information
Program, this criterion was evaluated based on the presence of hydrogenated oils or partially hydrogenated oils in the ingredient list of a product and the
amount of total trans fat per 100 g total fat; refer to the Methods section for further details.9 According to the WHO, added sugars are used as a criterion in the EURO model because available data in food composition tables refer to added sugars.
In the current analyses, data on free sugars were used instead.
NUTRIENT PROFILING FOR MARKETING RESTRICTIONS 1473
at UN
IVE
RS
ITY
OF
TO
RO
NT
O G
ER
ST
EIN
SC
IEN
CE
INF
OR
MA
TIO
N C
TR
on Decem
ber 13, 2017ajcn.nutrition.org
Dow
nloaded from
93
per-kilocalorie basis, whereas all other nutrients were evaluatedas a percentage of energy. The free sugar content of products inFLIP 2013 was calculated with the use of the University ofToronto’s free sugar algorithm, which is based on the WHOdefinitions, as described by Bernstein et al. (24). Food productsthat exceeded $1 of the predetermined thresholds for criticalnutrients were considered not eligible to be marketed to chil-dren, whereas food products that did not exceed any thresholdwere considered eligible. Unprocessed or minimally processedproducts and culinary ingredients were not subject to the ap-plication of the thresholds and therefore were all consideredeligible in the present study. The PAHO model specifies thatthese types of products usually form part of a healthy diet or, inthe case of culinary ingredients (e.g., olive oil), are used inconjunction with unprocessed or minimally processed foods forproducing freshly prepared dishes.
Modified-PAHO NP model
It was decided to test a modified version of the PAHO NPmodel (Modified-PAHO), which took into account 2 modifica-tions. First, an across-the-board approach was used, in which allfoods in FLIP 2013 (i.e., not only processed or ultra-processedproducts, but also the minimally processed and unprocessedfoods) were evaluated against the thresholds and classified asexcessive or not in critical nutrients. This decision was made tosimplify the application of the PAHONPmodel by not taking intoaccount the extent of food processing, because the delimitationbetween processed or ultra-processed products as compared withunprocessed products, minimally processed products, or culinaryingredients was ambiguous for certain food items. An across-the-board approach therefore eliminated any possible subjectivity inthe initial categorization of foods.
Second, an adjustment to the PAHO sodium criterion wasapplied to beverages with a zero- or low-calorie content. Somebeverages with a very low sodium content (e.g., 5 mg Na/serving),but also a very low energy content (e.g., 0 calories), were initiallyclassified as being in excess for sodium (i.e., sodium was wellabove the threshold of$1 mg Na/kcal), however, those beverageswere clearly not a significant source of that nutrient. Based on theFood and Drug Regulations (35), the “low in sodium” nutrientcontent claim cutoff of #140 mg Na per reference amount andper serving of stated size was used as the threshold in these cases[beverages with both a sodium content #140 mg/serving andan energy content #140 kcal/serving were considered to have,1 mg Na/kcal (i.e., “not in excess”)]. A total of 134 beverageswere affected by this change. The “low in sodium” cutoff waschosen considering that most of the non–calorie-reduced ver-sions of the beverages had an energy content close to or ,140kcal/serving, therefore allowing for a 1:1 ratio. Also, beverageswith an energy content #140 kcal, but a sodium content.140 mg/serving were considered as having a ratio .1 (i.e., inexcess).
WHO Regional Office for Europe NP model
The WHO Regional Office for Europe (EURO) model, in-troduced in 2015, was specifically designed for restricting themarketing of unhealthy foods and beverages to children (28). Itwas built based primarily on the government-developed andgovernment-endorsed models from Norway and Denmark.
Foods from FLIP 2013 were first classified into 1 of 17 possiblecategories, among which the beverages category was furtherdivided into 4 subcategories (juices, milk drinks, energy drinks,and other beverages). As specified by the model, foods classifiedin 7 of the food categories had no nutritional criteria and wereautomatically considered either not eligible to be marketed tochildren [categories 1, 2, 4a, 4c, and 5 (e.g., juices)] or alwayseligible [categories 13 and 15 (e.g., frozen fruits)]. Foods in othercategories were evaluated against predetermined thresholds per100 g/mL for the following nutrients or foods components, whichvaried depending on the category: total fat, saturated fat, totalsugar, added sugar, nonsugar sweetener, salt, and energy(e.g., category 6, breakfast cereals, included limits for total fat,total sugar, and salt). The source document for the model pro-vides additional details on the nutrients considered in eachcategory and on how their thresholds were determined (28).Where applicable, free sugars were taken into account instead ofadded sugars because their amounts could be calculated for foodsin FLIP 2013 (see the PAHO NP model), and they are the type ofsugars specifically considered as part of the current WHOguidelines on sugars (36). Additionally, the model specifies that“marketing is prohibited if the product contains .1 g per 100 gtotal fat in the form of industrially-produced trans fatty acids”(28), irrespective of the food category. A total of 55 productsfrom different categories, which were initially classified as eli-gible to be marketed to children based on the criteria for theabove nutrients or food components, were reclassified as noteligible because hydrogenated or partially hydrogenated oilswere present in their ingredient list, and their total trans fattyacids content was .1 g/100 g of total fat. The total trans fattyacid content was used because the exact proportion of in-dustrially produced trans fatty acids in a product could not bedetermined. Thus, the use of total trans fatty acids represented amore conservative approach.
Considerations for all selected models
Under each NP model, food products from FLIP 2013 werefirst classified into their appropriate category independently by 2authors (M-EL and TP), and any discrepancy was resolved byconsensus. The classification of FLIP products into eachmodel’s categories was completed with the use of a combinationof information from Schedule M categories and subcate-gories (described above), sugar-focused categories detailed inBernstein et al. (24), and the ingredient list. For consistency, andbecause the FSANZ-NPSC and EURO models specify that foodproducts should be evaluated with the use of nutritional com-position data in the “as consumed” form (if necessary), this typeof data was used for all models. Nevertheless, the “as sold” datacorrespond to the “as consumed” data for $92.0% of products(n = 14,115/15,342) in the database.
On-package marketing to children
For the purpose of the subanalyses, the presence of on-packagemarketing to children was determined with the use of the ele-ments previously established and tested in our group (37), basedon information from Colby et al. (38) and Elliot (39) as follows: 1)children’s product lines, 2) child-focused lettering or graphics, 3)allusions to fun or play, 4) unconventional flavors, colors, orshapes, 5) toys, coupons, prizes, or contests, 6) games, and 7)
1474 LABONTE ET AL.
at UN
IVE
RS
ITY
OF
TO
RO
NT
O G
ER
ST
EIN
SC
IEN
CE
INF
OR
MA
TIO
N C
TR
on Decem
ber 13, 2017ajcn.nutrition.org
Dow
nloaded from
94
characters appealing to children. A product had to meet $1 ofthe previous criteria to be considered as marketed to children.The classification of all products in FLIP 2013 as having on-package marketing to children or not was performed by one ofthe authors (CM) and verified by a team member not involved inthe present project. Any uncertainty was resolved by consensuswith the other authors.
Statistical analyses
The degree of strictness was determined by the number andproportion (percentage with accompanying 95% CIs) of foodproducts considered eligible to be marketed to children in FLIP2013 and was reported overall and by Schedule M category foreach of the selected NP models. Agreement between the variousmodels was determined by cross-classification analysis(i.e., number and proportion of food products classified similarlyor differently between any 2 models). Cohen’s k statistic wasalso used, and agreement was interpreted as follows: slight,
0.01–0.20; fair, 0.21–0.40; moderate, 0.41–0.60; substantial,0.61–0.80; and almost perfect, 0.81–0.99 (40). The number andproportion of foods classified as eligible by all models or noteligible by any model was also determined. Subanalyses con-sisted of repeating the above analyses with the use of only thesubset of foods with on-package marketing to children. Allstatistical analyses were carried out in SAS (version 9.4; SASInstitute Inc.).
RESULTS
The degree of strictness, as shown by the proportion of Ca-nadian prepackaged foods classified as eligible to be marketed tochildren, varied considerably between the different NP models(Table 2). The Modified-PAHO and PAHO models were thestrictest overall by allowing only 9.8% (95% CI: 9.4%, 10.3%)and 15.8% (95% CI: 15.3%, 16.4%) of foods, respectively,followed by the EURO model [29.8% (95% CI: 29.0%, 30.5%)].In contrast, almost half of the Canadian prepackaged foodproducts [49.0% (95% CI: 48.2%, 49.8%)] were considered as
TABLE 2
Number and proportion (%) of Canadian prepackaged foods that would be eligible to be marketed to children according to 4 nutrient profile models, overall
and by food category (n = 15,227)1
Schedule M category
number and description2Foods
analyzed, n
Foods eligible to be
marketed to children
FSANZ-NPSC PAHO Modified-PAHO EURO
n % (95% CI) n % (95% CI) n % (95% CI) n % (95% CI)
All 15,227 74553 49.0 (48.2, 49.8) 24133 15.8 (15.3, 16.4) 14973 9.8 (9.4, 10.3) 45343 29.8 (29.0, 30.5)
1. Bakery products 2084 5753 27.6 (25.7, 29.5) 693 3.3 (2.5, 4.1) 693 3.3 (2.5, 4.1) 307 14.7 (13.2, 16.3)
2. Beverages 482 264 54.8 (50.3, 59.2) 46 9.5 (6.9, 12.2) 593 12.2 (9.3, 15.2) 553 11.4 (8.6, 14.3)
3. Cereals, other grain products 988 8014 81.1 (78.6, 83.5) 6753 68.3 (65.4, 71.2) 6625 67.0 (64.1, 69.9) 707 71.6 (68.7, 74.4)
4. Dairy products and substitutes 1240 6763 54.5 (51.7, 57.3) 105 8.5 (6.9, 10.0) 24 1.9 (1.2, 2.7) 265 21.4 (19.1, 23.7)
5. Desserts 827 272 32.9 (29.7, 36.1) 1 0.1 (0, 0.4) 1 0.1 (0, 0.4) 0 0 (0, 0)
6. Dessert toppings, fillings 116 134 11.2 (5.4, 17.0) 2 1.7 (0, 4.1) 2 1.7 (0, 4.1) 0 0 (0, 0)
7. Eggs and substitutes 56 53 94.6 (88.6, 100) 46 82.1 (71.8, 92.5) 0 0 (0, 0) 56 100 (100, 100)
8. Fats, oils 535 191 35.7 (31.6, 39.8) 148 27.7 (23.9, 31.5) 2 0.4 (0, 0.9) 153 28.6 (24.8, 32.4)
9. Marine, fresh water animals 440 340 77.3 (73.3, 81.2) 65 14.8 (11.4, 18.1) 17 3.9 (2.1, 5.7) 374 85.0 (81.7, 88.3)
10. Fruit, fruit juices 1089 7733 71.0 (68.3, 73.7) 4563 41.9 (38.9, 44.8) 2083 19.1 (16.8, 21.4) 106 9.7 (8.0, 11.5)
11. Legumes 180 179 99.4 (98.3, 100) 99 55.0 (47.7, 62.3) 93 51.7 (44.3, 59.0) 153 85.0 (79.7, 90.3)
12. Meat, poultry, their
products, substitutes
895 249 27.8 (24.9, 30.8) 12 1.3 (0.6, 2.1) 1 0.1 (0, 0.3) 378 42.2 (39.0, 45.5)
13. Miscellaneous 446 1003 22.4 (18.5, 26.3) 60 13.5 (10.3, 16.6) 43 9.6 (6.9, 12.4) 35 7.8 (5.3, 10.4)
14. Combination dishes 1357 9814 72.3 (69.9, 74.7) 27 2.0 (1.2, 2.7) 27 2.0 (1.2, 2.7) 708 52.2 (49.5, 54.8)
15. Nuts, seeds 220 167 75.9 (70.2, 81.6) 165 75.0 (69.2, 80.8) 1 0.5 (0, 1.4) 118 53.6 (47.0, 60.3)
16. Potatoes, sweet potatoes,
yams
140 134 95.7 (92.3, 99.1) 18 12.9 (7.2, 18.5) 18 12.9 (7.2, 18.5) 55 39.3 (31.1, 47.5)
17. Salads 70 57 81.4 (72.1, 90.8) 0 0 (0, 0) 0 0 (0, 0) 30 42.9 (31.0, 54.7)
18. Sauces, dips, gravies,
condiments
1229 3853 31.3 (28.7, 33.9) 52 4.2 (3.1, 5.4) 493 4.0 (2.9, 5.1) 144 11.7 (9.9, 13.5)
19. Snacks 794 224 28.2 (25.1, 31.3) 32 4.0 (2.7, 5.4) 30 3.8 (2.4, 5.1) 56 7.1 (5.3, 8.8)
20. Soups 456 408 89.5 (86.6, 92.3) 6 1.3 (0.3, 2.4) 6 1.3 (0.3, 2.4) 425 93.2 (90.9, 95.5)
21. Sugars, sweets 749 275 3.6 (2.3, 4.9) 655 8.7 (6.7, 10.7) 05 0 (0, 0) 4 0.5 (0, 1.1)
22. Vegetables 834 5864 70.3 (67.2, 73.4) 2643 31.7 (28.5, 34.8) 1853 22.2 (19.4, 25.0) 405 48.6 (45.2, 52.0)
1 EURO, WHO Regional Office for Europe; FSANZ-NPSC, Food Standards Australia New Zealand-Nutrient Profiling Scoring Criterion;
Modified-PAHO, modified version of Pan American Health Organization model; PAHO, Pan American Health Organization.2 Schedule M categories are defined in the Canadian Food and Drug Regulations (27). Detailed descriptions for each category and subcategory can also be
found in Supplemental Table 1.3Missing data for ,0.5% of food items.4Missing data for 0.5% to ,1.0% of food items.5Missing data for 1.0% to ,1.5% of food items.
NUTRIENT PROFILING FOR MARKETING RESTRICTIONS 1475
at UN
IVE
RS
ITY
OF
TO
RO
NT
O G
ER
ST
EIN
SC
IEN
CE
INF
OR
MA
TIO
N C
TR
on Decem
ber 13, 2017ajcn.nutrition.org
Dow
nloaded from
95
eligible to be marketed to children according to the FSANZ-NPSCmodel. Differences in the overall strictness of the various NPmodels were also observed quite consistently across different foodcategories, with the Modified-PAHO model being the strictestmodel and the FSANZ-NPSC being the most permissive model inthe majority of categories (Table 2).
The Modified-PAHO model eliminated large proportions ofsome core foods that contained high amounts of a negativenutrient. For example, only #0.5% of nuts and seeds and eggsand egg substitutes were allowed by the Modified-PAHO model,whereas .53% of foods in those categories were allowed underthe other models (Table 2). Products in both food categorieswere essentially eliminated because of their high content in totalfat and saturated fat (Supplemental Figure 1).
In contrast, substantial proportions of food products notconsistent with dietary guidelines were classified as eligible formarketing to children under the FSANZ-NPSC model, whereasthose same products were permitted in very low proportions(#8%) or not permitted at all under the other models(i.e., carbonated and noncarbonated beverages; most types offrozen desserts; custard, gelatin, and pudding; hors d’oeuvres;sauces for dipping; legume- or dairy-based dips; minor mainentree sauces, such as gravy; and snacks, such as chips andpretzels; Supplemental Table 1).
Cross-classification analyses showed that about half of thefoods identified as not eligible under the FSANZ-NPSC modelwere also not eligible under the other models, whereas thisproportion was .65% for the comparison between the EUROmodel and each of the PAHO models and at 84% between theoriginal PAHO model and its modified version (Table 3). Theseresults are consistent with the observed degrees of agreement
between the NP models as assessed by the k statistic. Indeed,agreement was considered slight to fair between each of thePAHO models and the FSANZ-NPSC model (Modified-PAHO:k = 0.19; PAHO: k = 0.27), whereas it was substantial (k = 0.73)between the PAHO and Modified-PAHO models.
Another way to evaluate agreement between models was bydetermining the number and proportion of foods that would beclassified as eligible by all 4 models or not eligible by any model(Table 4). Overall, 8.1% of foods would be eligible based on allmodels. The top 5 types of such foods, based on Schedule Msubcategories, were as follows: pastas without sauce (n = 414);vegetables without sauce (n = 165); grains, such as rice andbarley (n = 107); fresh, canned, or frozen fruit (n = 90); andbeans, peas, and lentils (n = 89) (data not shown). Additionally,47.4% of foods would not be allowed by any model. The pro-portion of noneligible foods varied considerably between foodcategories, with none of the food items in eggs and egg sub-stitutes not allowed by any model, and $86.5% of products insugars and sweets and dessert toppings and fillings not allowedby any model. Supplemental Table 1 also shows subcategories inwhich 100% of products would not be allowed by any model(e.g., brownies; toaster pastries; and candies, including choco-late bars).
Tables 5 and 6 present the results of subanalyses based onlyon food products that had on-package marketing tochildren, representing 4.9% (n = 747/15,227) of all productsanalyzed. The Modified-PAHO model was still the strictest byallowing only 1.9% (0.9%, 2.8%) of products that were targetingchildren, whereas the most permissive model remained theFSANZ-NPSC, with 24.2% (95% CI: 21.2%, 27.3%) of suchproducts targeting children classified as being eligible to be
TABLE 3
Agreement between classifications made by 4 nutrient profile models applied to Canadian prepackaged foods (n = 15,227)1
Foods eligible or not eligible to be marketed to children, %
PAHO Modified-PAHO EURO
Eligible
(n = 2413)
Not eligible
(n = 12,802)
Eligible
(n = 1497)
Not eligible
(n = 13,705)
Eligible
(n = 4534)
Not eligible
(n = 10,692)
FSANZ-NPSC2
Eligible (n = 7455) 14.5 34.5 9.5 39.5 27.4 21.6
Not eligible (n = 7728) 1.3 49.5 0.4 50.4 2.3 48.5
k (95% CI)3 0.27 (0.26, 0.29) 0.19 (0.18, 0.20) 0.52 (0.51, 0.53)
PAHO4
Eligible 9.7 6.1 11.5 4.4
Not eligible 0.1 83.9 18.3 65.8
k (95% CI)3 0.73 (0.71, 0.74) 0.37 (0.36, 0.39)
Modified-PAHO5
Eligible 8.4 1.4
Not eligible 21.3 68.7
k (95% CI)3 0.33 (0.31, 0.34)
1 EURO, WHO Regional Office for Europe; FSANZ-NPSC, Food Standards Australia New Zealand-Nutrient Profiling Scoring Criterion;
Modified-PAHO, modified version of Pan American Health Organization model; PAHO, Pan American Health Organization.2 There were missing data for 0.3% of the food items for each of the comparisons between the FSANZ-NPSC model and the PAHO, Modified-PAHO, and
EURO models.3Missing data were excluded from the calculation of the simple k statistic. Agreement between the models was assessed as follows: slight, 0.01–0.20;
fair, 0.21–0.40; moderate, 0.41–0.60; substantial, 0.61–0.80; and almost perfect, 0.81–0.99 (40).4 There were missing data for 0.2% of the food items for the comparison between the PAHO model and the Modified-PAHO model and missing data for
0.1% of the food items for the comparison between the PAHO model and the EURO model.5 There were missing data for 0.2% of the food items for the comparison between the Modified-PAHO model and the EURO model.
1476 LABONTE ET AL.
at UN
IVE
RS
ITY
OF
TO
RO
NT
O G
ER
ST
EIN
SC
IEN
CE
INF
OR
MA
TIO
N C
TR
on Decem
ber 13, 2017ajcn.nutrition.org
Dow
nloaded from
96
marketed to them (Table 5). The overall proportion of foods thatwould be eligible for marketing to children under each NPmodel in the present subanalyses represented, at most, half ofthe one observed in the main analyses when all foods from FLIP2013 were analyzed [e.g., 24.2% (Table 5) compared with49.0% (Table 2) of foods considered eligible for marketing,respectively, under the FSANZ-NPSC]. Agreement between thevarious models as assessed by the k statistic was also lower thanthat observed in the main analyses (Table 6). Cross-classificationanalyses showed that .73% of foods with on-package market-ing to children not permitted by the FSANZ-NPSC model werealso not permitted by the PAHO, Modified-PAHO, and EUROmodels, whereas this proportion was .90% for all other pos-sible comparisons. In the categories of dessert toppings andfillings, miscellaneous, and nuts and seeds, none of the productstargeting children were allowed by any model examined (Table5).
DISCUSSION
This study showed that the proportion and types of foods andbeverages in the Canadian prepackaged food supply that wouldbe permitted to be marketed to children based on governmental orintergovernmental NP models that have been developed formarketing restrictions vary greatly (i.e., from less than one-sixth
to almost half of Canadian prepackaged foods) depending on theselected model. Subanalyses also showed that each of the modelswould be more stringent when evaluating only food items car-rying attributes that are appealing to children. This suggests thatfood items specifically targeting children essentially representproducts for which consumption should be limited to an occa-sional basis, and that the adoption of an NP model for marketingrestrictions could largely reduce children’s exposure to thosefoods.
Consistent with our results, studies in other countries haveshown that the selection of an NP model has a large impact on theproportion and types of either packaged food products or food-and drink-related media advertisements considered eligible formarketing to children (9, 10, 21–23). Combined with these otherstudies, our study highlights the importance of carefully exam-ining the underlying characteristics of NP models that could beadapted or developed for use as part of a specific public healthpolicy. Those characteristics include the types and number ofnutrients or food components considered, the definition of foodcategories, the selected reference amounts, and the establishedthresholds, among others (41, 42).
Similar to observations by Scarborough et al. (21) and NiMhurchu et al. (22), the various models evaluated were moreconsistent in identifying foods that would not be eligible formarketing to children than in identifying eligible foods. This
TABLE 4
Canadian prepackaged foods that would be eligible to be marketed to children by all 4 nutrient profile models and not
eligible by any model, overall and by food category1
Schedule M category
number and description2Foods
analyzed, n
Foods eligible by all
models, n (%)
Foods not eligible by
any model, n (%)
All 15,2273 1234 (8.1) 7220 (47.4)
1. Bakery products 20843 43 (2.1) 1490 (71.5)
2. Beverages 4823 35 (7.3) 217 (45.0)
3. Cereals, other grain products 9884 648 (65.6) 169 (17.1)
4. Dairy products and substitutes 12403 21 (1.7) 494 (39.8)
5. Desserts 827 0 (0.0) 555 (67.1)
6. Dessert toppings, fillings 1163 0 (0.0) 101 (87.1)
7. Eggs and substitutes 56 0 (0.0) 0 (0.0)
8. Fats, oils 535 2 (0.4) 327 (61.1)
9. Marine, fresh water animals 440 17 (3.9) 58 (13.2)
10. Fruit, fruit juices 10893 93 (8.5) 284 (26.1)
11. Legumes 180 91 (50.6) 1 (0.6)
12. Meat, poultry, their products, substitutes 895 1 (0.1) 504 (56.3)
13. Miscellaneous 4463 30 (6.7) 329 (73.8)
14. Combination dishes 13575 17 (1.3) 346 (25.5)
15. Nuts, seeds 220 1 (0.5) 49 (22.3)
16. Potatoes, sweet potatoes, yams 140 8 (5.7) 4 (2.9)
17. Salads 70 0 (0.0) 12 (17.1)
18. Sauces, dips, gravies, condiments 12293 31 (2.5) 817 (66.5)
19. Snacks 794 6 (0.8) 554 (69.8)
20. Soups 456 6 (1.3) 19 (4.2)
21. Sugars, sweets 7494 0 (0.0) 648 (86.5)
22. Vegetables 8345 184 (22.1) 242 (29.0)
1 The 4 nutrient profile models evaluated were the Food Standards Australia New Zealand-Nutrient Profiling Scoring
Criterion, the Pan American Health Organization nutrient profile model, a modified version of the Pan American Health
Organization model, and the WHO Regional Office for Europe nutrient profile model.2 Schedule M categories are defined in the Canadian Food and Drug Regulations (27).3Missing data for ,0.5% of food items.4Missing data for 1.0% to ,1.5% of food items.5Missing data for 0.5% to ,1.0% of food items.
NUTRIENT PROFILING FOR MARKETING RESTRICTIONS 1477
at UN
IVE
RS
ITY
OF
TO
RO
NT
O G
ER
ST
EIN
SC
IEN
CE
INF
OR
MA
TIO
N C
TR
on Decem
ber 13, 2017ajcn.nutrition.org
Dow
nloaded from
97
could be expected considering that all the models primarily tookinto account nutrients to limit, such as sodium and sugars. Onlyone of the models, the FSANZ-NPSC, considered the contri-bution of nutrients or food components to encourage, and thismay, at least in part, explain the higher percentages of foodsallowed by the FSANZ-NPSC overall and in the majority of foodcategories as compared with the other models. A limited numberof studies that assessed New Zealand or European packagedfoods have similarly shown that the UK Ofcom model or itsderivatives (e.g., the FSANZ-NPSC and Health Star Ratingsystem) are 7–15% and 25% more permissive than the EUROand PAHO models, respectively (22, 29).
A permissive model, such as the FSANZ-NPSC model, alsoclassified a substantial number of items that were not consistentwith dietary guidelines as eligible for marketing to children,whereas those same products were permitted in very low pro-portions or not permitted at all under more restrictive models,such as the EURO and PAHO models. From the perspective ofprotecting vulnerable populations (3), opting for a mandatory andfairly stringent model would be the approach most consistentwith the intent of restricting the exposure to, and the power of, thepromotion of unhealthy foods to children in various mediachannels. Questionable or undesirable food items would be lesslikely to pass the criteria of the system. This is particularlyimportant considering that a complete ban of food and beveragemarketing to children would likely represent the most effective
policy to implement. For example, a total ban in place since 1980in the province of Quebec, Canada (43) has been linked to lowerobesity rates in children aged 6–11 y than in children in otherprovinces (17). However, such bans are not always feasible,depending on the legislation system in place.
To limit subjectivity in the application of an NP model to afood supply, clear, nonambiguous definitions for food categoriesneed to be established. Although we acknowledge that consid-ering the extent of food processing (NOVA system) is a novel andinteresting avenue given that it has been shown to predict dietquality among Canadians (44), the categorization of foodsaccording to this method in the original PAHO model resulted inmuch higher discrepancies between team members than thecategorization of foods in the other NP models. For example,PAHO’s definition of minimally processed foods includes“combinations of 2 or more unprocessed or minimally processedfoods” (29). Because fruit juices without added sugars fall intothis category, a food product, such as sliced pineapples with aningredient list that indicates “pineapple, pineapple juice,” wouldalso, by definition, fall in that same category. In contrast, itappears more accurate to consider this product as processedgiven the substantial amount of sugars added by the inclusion ofpineapple juice. The idea of exempting fruit juice without addedsugars from being assessed against the criteria is also concerningbecause juice itself is a source of free sugars (36) and thereforenot considered in line with many dietary guidelines. As
TABLE 5
Number and proportion (%) of Canadian prepackaged foods specifically targeting children that would be eligible for marketing according to 4 nutrient
profile models, overall and by food category (n = 747)1
Schedule M category
number and description2
Foods with on-package
marketing to children
analyzed, n
Foods eligible to be
marketed to children
FSANZ-NPSC PAHO Modified-PAHO EURO
n % (95% CI) n % (95% CI) n % (95% CI) n % (95% CI)
All 747 1813 24.2 (21.2, 27.3) 26 3.5 (2.2, 4.8) 14 1.9 (0.9, 2.8) 46 6.2 (4.4, 7.9)
1. Bakery products 173 4 2.3 (0.1, 4.6) 1 0.6 (0, 1.7) 1 0.6 (0, 1.7) 3 1.7 (0, 3.7)
2. Beverages 11 2 18.2 (0, 45.4) 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0)
3. Cereals and other grain products 51 11 21.6 (9.9, 33.3) 1 2.0 (0, 5.9) 1 2.0 (0, 5.9) 2 3.9 (0, 9.4)
4. Dairy products and substitutes 74 38 51.4 (39.7, 63.0) 0 0 (0, 0) 0 0 (0, 0) 5 6.8 (0.9, 12.6)
5. Desserts 144 39 27.1 (19.7, 34.4) 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0)
6. Dessert toppings and fillings 7 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0)
9. Marine and fresh-water animals 2 2 100 (100, 100) 0 0 (0, 0) 0 0 (0, 0) 2 100 (100, 100)
10. Fruit and fruit juices 58 33 56.9 (43.8, 70.0) 8 13.8 (4.6, 22.9) 5 8.6 (1.2, 16.1) 4 6.9 (0.2, 13.6)
12. Meat, poultry, their products,
and substitutes
4 4 100 (100, 100) 0 0 (0, 0) 0 0 (0, 0) 4 100 (100, 100)
13. Miscellaneous 14 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0)
14. Combination dishes 69 39 56.5 (44.5, 68.5) 0 0 (0, 0) 0 0 (0, 0) 24 34.8 (23.3, 46.3)
15. Nuts and seeds 9 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0)
16. Potatoes, sweet potatoes, and yams 4 4 100 (100, 100) 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0)
19. Snacks 54 3 5.6 (0, 11.9) 7 13.0 (3.7, 22.2) 7 13.0 (3.7, 22.2) 0 0 (0, 0)
20. Soups 1 1 100 (100, 100) 0 0 (0, 0) 0 0 (0, 0) 1 100 (100, 100)
21. Sugars and sweets 72 14 1.4 (0, 4.2) 9 12.5 (4.7, 20.3) 0 0 (0, 0) 1 1.4 (0, 4.2)
1 EURO, WHO Regional Office for Europe; FSANZ-NPSC, Food Standards Australia New Zealand-Nutrient Profiling Scoring Criterion;
Modified-PAHO, modified version of Pan American Health Organization model; PAHO, Pan American Health Organization.2 None of the foods in Schedule M categories 7 (egg and egg substitutes), 8 (fats and oils), 11 (legumes), 17 (salads), 18 (sauces, dips, gravies, and
condiments), and 22 (vegetables) were specifically marketed to children on their package. Schedule M categories are defined in the Canadian Food and Drug
Regulations (27).3Missing data for 0.1% of food items.4Missing data for 1% of food items.
1478 LABONTE ET AL.
at UN
IVE
RS
ITY
OF
TO
RO
NT
O G
ER
ST
EIN
SC
IEN
CE
INF
OR
MA
TIO
N C
TR
on Decem
ber 13, 2017ajcn.nutrition.org
Dow
nloaded from
98
described previously, such an ambiguous situation explains whywe also tested a modified, across-the-board version of the PAHOmodel that did not exempt any product from being evaluatedagainst the nutritional criteria. It was also observed that 86.5%of foods in FLIP 2013 (n = 13,166/15,227; data not shown) wereclassified as processed or ultra-processed, supporting the ideathat the use of a model applicable to the entire packaged foodsupply might be the best approach in the Canadian context. Still,the use of the Modified-PAHO model was not without concerns.Our results highlight that applying a model to the entire foodsupply may require clear exemptions to be established for foodsin line with dietary guidelines that naturally contain highamounts of a negative nutrient included in the model’s algorithm(e.g., nuts, which have a high total fat content even in the ab-sence of added fat). With prespecified exemptions, core foods,such as nuts, without any added ingredients would not be in-appropriately ruled out.
A number of limitations and strengths need to be pointed out.First, the present analyses did not weight products by marketshare and did not specifically consider the foods most commonlyconsumed by children, nor the foods most heavily marketed tochildren in various media channels, such as television or theInternet. However, our analyses did provide a comprehensiveevaluation of a large sample (.15,000 items) of foods that areavailable to children and their parents in Canadian grocerystores. Our analyses also provided specific data on foods thatcurrently carry on-package marketing to children, representing amarketing channel that has continued to remain outside currentindustry- or government-based regulatory frameworks, such asthe Canadian Children’s Food and Beverage Advertising Ini-tiative (45) or the Consumer Protection Act in Quebec (43).Second, we recognize that several other NP models meant formarketing restrictions exist worldwide (13, 14), but we decidednot to consider models from regions (e.g., Asia) that were
unlikely to be relevant in a North American context due todifferences in food supplies. Additionally, we decided not toconsider proposed models that are not currently in use [e.g., thenutrition standards of the Interagency Working Group on FoodMarketing to Children, United States (46)]. We also only optedfor models developed by authoritative bodies, which are morelikely to be used by other government bodies and to be trustedand supported by consumers than industry-based models. This isimportant given that industry-based models have been shown tobe less stringent than government-based models (23).
In conclusion, the present study showed wide variations in thedegree of strictness and agreement between NP models havingapplications in restricting the commercial marketing of foods andbeverages of low nutritional quality to children. This highlightsthe importance of carefully evaluating the characteristicsunderlying a model that is being developed or adapted for use in aspecific public health policy. Where total bans of the commercialmarketing of foods and beverages to children are notintroduced, a relatively stringent and mandatory NP model thatuses clearly defined categories and exemptions, and that isconsistent with other nutrition-related policies in the jurisdiction(e.g., front-of-package labeling system), should be considered.Such a model would more closely align with the public healthobjective of protecting vulnerable populations and would ensureconsistency between the country’s policies and national dietaryguidelines.
We thank Mavra Ahmed and Sheida Norsen, who assisted in preliminary
calculations of FVNL points that supported the determination of FSANZ-
NPSC scores used in the analyses of the present study. We also thank Laura
Vergeer, who assisted in evaluating the presence of on-package marketing
to children.
The authors’ responsibilities were as follows—M-EL, TP, and MRL:
designed the research; M-EL, JTB, and BF-A: calculated the FSANZ-NPSC
scores; CM: assessed the presence of marketing to children on food
TABLE 6
Agreement between classifications made by 4 nutrient profile models applied to Canadian prepackaged foods with on-package marketing to children
(n = 747)1
Foods eligible or not eligible to be marketed to children, %
PAHO Modified-PAHO EURO
Eligible (n = 26) Not eligible (n = 721) Eligible (n = 14) Not eligible (n = 733) Eligible (n = 46) Not eligible (n = 701)
FSANZ-NPSC2
Eligible (n = 181) 1.6 22.7 1.2 23.1 5.9 18.4
Not eligible (n = 565) 1.9 73.9 0.7 75.1 0.3 75.5
k (95% CI)3 0.06 (0.00, 0.11) 0.06 (0.01, 0.11) 0.32 (0.25, 0.40)
PAHO
Eligible 1.9 1.6 0.8 2.7
Not eligible 0.0 96.5 5.4 91.2
k (95% CI)3 0.69 (0.53, 0.86) 0.13 (0.01, 0.25)
Modified-PAHO
Eligible 0.8 1.1
Not eligible 5.4 92.8
k (95% CI)3 0.18 (0.04, 0.31)
1 EURO, WHO Regional Office for Europe; FSANZ-NPSC, Food Standards Australia New Zealand-Nutrient Profiling Scoring Criterion;
Modified-PAHO, modified version of Pan American Health Organization model; PAHO, Pan American Health Organization.2 There were missing data for 0.1% of the food items for each of the comparisons between the FSANZ-NPSC model and the PAHO, Modified-PAHO, and
EURO models.3Missing data were excluded from the calculation of the simple k statistic. Agreement between the models was assessed as follows: slight, 0.01–0.20;
fair, 0.21–0.40; moderate, 0.41–0.60; substantial, 0.61–0.80; and almost perfect, 0.81–0.99 (40).
NUTRIENT PROFILING FOR MARKETING RESTRICTIONS 1479
at UN
IVE
RS
ITY
OF
TO
RO
NT
O G
ER
ST
EIN
SC
IEN
CE
INF
OR
MA
TIO
N C
TR
on Decem
ber 13, 2017ajcn.nutrition.org
Dow
nloaded from
99
packages; M-EL and TP: conducted the research; TP: analyzed the data; M-EL:
assisted in data analysis and wrote the manuscript; MRL: had primary respon-
sibility for the final content; and all authors: critically reviewed and read and
approved the final manuscript. TP is a graduate student and is employed part
time by Intertek Scientific & Regulatory Consultancy. BF-A was a PepsiCo
employee (2009–2015). These companies were not involved in any way in the
present research. None of the remaining authors reported a conflict of interest
related to the study.
REFERENCES1. Rodd C, Sharma AK. Recent trends in the prevalence of overweight
and obesity among Canadian children. CMAJ 2016;188:E313–20.2. Janssen I. The public health burden of obesity in Canada. Can J Di-
abetes 2013;37:90–6.3. WHO. Consideration of the evidence on childhood obesity for the
Commission on Ending Childhood Obesity. Report of the Ad hocWorking Group on Science and Evidence for Ending ChildhoodObesity [Internet]. Geneva (Switzerland); WHO Press. 2016 [cited2017 Jan 30]. Available from: http://apps.who.int/iris/bitstream/10665/206549/1/9789241565332_eng.pdf?ua=1).
4. Boyland EJ, Nolan S, Kelly B, Tudur-Smith C, Jones A, Halford JC,Robinson E. Advertising as a cue to consume: a systematic review andmeta-analysis of the effects of acute exposure to unhealthy food andnonalcoholic beverage advertising on intake in children and adults. AmJ Clin Nutr 2016;103:519–33.
5. Boyland EJ, Whalen R. Food advertising to children and its effects ondiet: review of recent prevalence and impact data. Pediatr Diabetes2015;16:331–7.
6. Sadeghirad B, Duhaney T, Motaghipisheh S, Campbell NR,Johnston BC. Influence of unhealthy food and beverage marketing onchildren’s dietary intake and preference: a systematic review and meta-analysis of randomized trials. Obes Rev 2016;17:945–59.
7. Cairns G, Angus K, Hastings G, Caraher M. Systematic reviews of theevidence on the nature, extent and effects of food marketing to chil-dren. A retrospective summary. Appetite 2013;62:209–15.
8. Potvin Kent M, Dubois L, Wanless A. A nutritional comparison offoods and beverages marketed to children in two advertising policyenvironments. Obesity (Silver Spring) 2012;20:1829–37.
9. Rincon-Gallardo Pati~no S, Tolentino-Mayo L, Flores Monterrubio EA,Harris JL, Vandevijvere S, Rivera JA, Barquera S. Nutritional quality offoods and non-alcoholic beverages advertised on Mexican television ac-cording to three nutrient profile models. BMC Public Health 2016;16:733.
10. Watson WL, Johnston A, Hughes C, Chapman K. Determining the‘healthiness’ of foods marketed to children on television using theFood Standards Australia New Zealand nutrient profiling criteria. NutrDiet 2014;71:178–83.
11. WHO. Report of the Commission on Ending Childhood Obesity [In-ternet]. Geneva (Switzerland); WHO Press. 2016 [cited 2017 Feb 1].Available from: http://apps.who.int/iris/bitstream/10665/204176/1/9789241510066_eng.pdf.
12. WHO. Set of recommendations on the marketing of foods and non-alcoholic beverages to children. Resolution of the Sixty-third WorldHealth Assembly Adopted 21 May 2010. WHA63.14 Marketing of foodand non-alcoholic beverages to children [Internet]. Geneva (Switzerland);WHO Press. 2010 [cited 2017 Jan 30]. Available from: http://apps.who.int/iris/bitstream/10665/44416/1/9789241500210_eng.pdf.
13. Labonte ME, Poon T, Gladanac B, Ahmed M, Franco Arellano B,Rayner M, L’Abbe MR. Systematic review of nutrient profile modelsdeveloped for nutrition-related policies and regulations aimed at non-communicable disease prevention [Internet]. PROSPERO #CRD42015024750.2015 [cited 2017 Jan 24]. Available from: http://www.crd.york.ac.uk/PROS-PERO/display_record.asp?ID=CRD42015024750.
14. Labonte ME, Poon T, Gladanac B, Ahmed M, Franco Arellano B,Rayner M, L’Abbe MR. Systematic review of nutrient profile modelsdeveloped for nutrition-related policies and regulations aimed at non-communicable disease prevention. Appl Physiol Nutr Metab 2017;42:S23 (abstr).
15. Health Canada. Healthy eating strategy [Internet]. 2016 [cited 2017 Jan30]. Available from: https://www.canada.ca/content/dam/canada/health-canada/migration/publications/eating-nutrition/healthy-eating-strategy-canada-strategie-saine-alimentation/alt/pub-eng.pdf.
16. Senate of Canada. Child Health Protection Act. An act to amend theFood and Drugs Act (prohibiting food and beverage marketing directedat children). Bill S-228. 1st Sess., 42nd Parliament, 64-65 ElizabethII, 2015–2016 [Internet]. [cited 2017 Jan 30]. Available from: http://www.parl.gc.ca/content/hoc/Bills/421/Private/S-228/S-228_1/S-228_1.PDF.
17. Heart and Stroke Foundation of Canada. The kids are not alright. Howthe food and beverage industry is marketing our children and youth todeath. 2017 Report on the health of Canadians [Internet]. 2017 [cited2017 Feb 1]. Available from: http://www.heartandstroke.ca/what-we-do/media-centre/report-on-health.
18. Stop Marketing to Kids Coalition. The Ottawa principles [Internet].2016 [cited 2017 Mar 13]. Available from: http://stopmarketingtokids.ca/the-ottawa-principles/.
19. Azaıs-Braesco V, Goffi C, Labouze E. Nutrient profiling: comparisonand critical analysis of existing systems. Public Health Nutr 2006;9:613–22.
20. Clerfeuille E, Vieux F, Lluch A, Darmon N, Rolf-Pedersen N. Assessingthe construct validity of five nutrient profiling systems using diet modelingwith linear programming. Eur J Clin Nutr 2013;67:1003–5.
21. Scarborough P, Payne C, Agu CG, Kaur A, Mizdrak A, Rayner M,Halford JC, Boyland E. How important is the choice of the nutrientprofile model used to regulate broadcast advertising of foods to children?A comparison using a targeted data set. Eur J Clin Nutr 2013;67:815–20.
22. Ni Mhurchu C, Mackenzie T, Vandevijvere S. Protecting New Zealandchildren from exposure to the marketing of unhealthy foods anddrinks: a comparison of three nutrient profiling systems to classifyfoods. N Z Med J 2016;129:41–53.
23. Brinsden H, Lobstein T. Comparison of nutrient profiling schemes forrestricting the marketing of food and drink to children. Pediatr Obes2013;8:325–37.
24. Bernstein JT, Schermel A, Mills CM, L’Abbe MR. Total and free sugarcontent of Canadian prepackaged foods and beverages. Nutrients 2016;8.
25. Canadian Grocer. Executive report, Canadian grocery industry 2012–2013. Toronto (Canada): Rogers Publishing Limited; 2012.
26. Health Canada. Food and nutrition. Nutrient data. Canadian nutrient file,version 2015 [Internet]. 2015. [updated 2016 Jun 3; cited 2017 Feb 3].Available from: http://www.hc-sc.gc.ca/fn-an/nutrition/fiche-nutri-data/index-eng.php.
27. Government of Canada. Food and drug regulations (C.R.C., c. 870),sections B.01.001, B.01.002A and D.01.001 [Internet]. Ottawa (Canada):Minister of Justice. 2016 [cited 2016 Mar 16]. Available from: http://laws-lois.justice.gc.ca/eng/regulations/c.r.c.,_c._870/page-165.html#h-353.
28. WHO Regional Office for Europe. WHO Regional Office for Europenutrient profile model [Internet]. 2015 [cited 2016 Nov 1]. Availablefrom: http://www.euro.who.int/en/health-topics/disease-prevention/nutrition/publications/2015/who-regional-office-for-europe-nutrient-profile-model-2015.
29. Pan American Health Organization. Pan American Health Organizationnutrient profile model [Internet]. 2016 [cited 2016 Oct 24]. Avail-able from: http://www.paho.org/hq/index.php?option=com_content&-view=article&id=11662%3Apaho-nutrient-profile-model&catid=1370%3Amicronutrients&Itemid=41739&lang=fr.
30. Australian Government. ComLaw. Legislative instrument compilation.Australia New Zealand Food Standards Code - Standard 1.2.7 - nu-trition, health and related claims - F2015C00967 [Internet]. 2015 [cited2016 Jan 17]. Available from: https://www.comlaw.gov.au/Details/F2015C00967.
31. Department of Health (UK). Nutrient profiling technical guidance[Internet]. 2011 [cited 2016 Nov 15]. Available from: https://www.gov.uk/government/publications/the-nutrient-profiling-model.
32. Hebden L, King L, Kelly B, Chapman K, Innes-Hughes C,Gunatillaka N. Regulating the types of foods and beverages marketedto Australian children: how useful are food industry commitments?Nutr Diet 2010;67:258–66.
33. Government of UK, Public Health England. Review of the nutrientprofiling model [Internet]. 2016 [cited 2017 Mar 13]. Available from:https://www.gov.uk/government/collections/review-of-the-nutrient-profiling-model.
34. Bernstein JT, Franco Arellano B, Schermel A, Labonte ME,L’Abbe MR. Healthfulness and nutritional composition of Canadianprepackaged foods with and without sugar claims. Appl Physiol NutrMetab 2017. In press.
1480 LABONTE ET AL.
at UN
IVE
RS
ITY
OF
TO
RO
NT
O G
ER
ST
EIN
SC
IEN
CE
INF
OR
MA
TIO
N C
TR
on Decem
ber 13, 2017ajcn.nutrition.org
Dow
nloaded from
100
35. Government of Canada, Canadian Food Inspection Agency. Specificnutrient content claim requirements. Sodium (salt) claims [Internet].2016 [cited 2017 Mar 13]. Available from: http://www.inspection.gc.ca/food/labelling/food-labelling-for-industry/nutrient-content/specific-claim-requirements/eng/1389907770176/1389907817577?chap=9.
36. WHO. Guideline: sugars intake for adults and children [Internet].Geneva (Switzerland): WHO. 2015 [cited 2017 Mar 13]. Available from:http://apps.who.int/iris/bitstream/10665/149782/1/9789241549028_eng.pdf?ua=1.
37. Murray C. Examining the nutritional content of prepackaged foods andbeverages marketed to children in Canada. Canada: University ofToronto; 2014.
38. Colby SE, Johnson L, Scheett A, Hoverson B. Nutrition marketing onfood labels. J Nutr Educ Behav 2010;42:92–8.
39. Elliott C. Marketing fun foods: a profile and analysis of supermarketfood messages targeted at children. Can Public Policy 2008;34:259–73.
40. Viera AJ, Garrett JM. Understanding interobserver agreement: thekappa statistic. Fam Med 2005;37:360–3.
41. Sacks G, Rayner M, Stockley L, Scarborough P, Snowdon W,Swinburn B. Applications of nutrient profiling: potential role in diet-related chronic disease prevention and the feasibility of a core nutrient-profiling system. Eur J Clin Nutr 2011;65:298–306.
42. Scarborough P, Rayner M, Stockley L. Developing nutrient profilemodels: a systematic approach. Public Health Nutr 2007;10:330–6.
43. Gouvernement du Quebec. P-40.1 - Consumer Protection Act [Inter-net]. 2016 [cited 2016 Oct 31]. Available from: http://legisquebec.gouv.qc.ca/en/ShowDoc/cs/P-40.1.
44. Moubarac JC, Batal M, Louzada ML, Martinez Steele E, Monteiro CA.Consumption of ultra-processed foods predicts diet quality in Canada.Appetite 2017;108:512–20.
45. Advertising Standards Canada. Canadian Children’s Food & BeverageAdvertising Initiative [Internet]. 2016 [cited 2017 May 16]. Availablefrom: http://www.adstandards.com/en/childrensinitiative/default.htm.
46. Federal Trade Commission, CDC, FDA, USDA. Interagency WorkingGroup on Food Marketed to Children tentative proposed nutritionstandards [Internet]. 2009 [cited 2016 Dec 2]. Available from: https://cspinet.org/sites/default/files/attachment/ftcnewstandards.pdf.
NUTRIENT PROFILING FOR MARKETING RESTRICTIONS 1481
at UN
IVE
RS
ITY
OF
TO
RO
NT
O G
ER
ST
EIN
SC
IEN
CE
INF
OR
MA
TIO
N C
TR
on Decem
ber 13, 2017ajcn.nutrition.org
Dow
nloaded from
101
102
Appendix B
Ethics Review Application for Study on Proposed NP Model for Marketing Restrictions to Children We are currently conducting a study to examine the construct and
convergent/concurrent validity of the proposed NP model for use in Canada against
views of nutrition professionals. The Ethics Review Application was approved by the
Review Ethics Board (REB) at the University of Toronto. The pilot study was conducted
between November 3 to 10, 2017, and an amendment to the approval was submitted to
the REB on December 11, 2017: L’Abbé MR, Poon T, Labonté MÈ, Bernstein JT
(2017). Ethics Review Application Form: Validation of a nutrient profile model proposed
for restricting the marketing of unhealthy foods to children by nutrition professionals.
Although this research was not explicitly part of my MSc thesis, I had a significant role in
this research to date, as indicated in Table B-1. In addition, I plan to submit the results
of this study for publication in 2018.
Table B-1 Authors’ contributions to ethics review application for study on proposed NP model for marketing restrictions to children
Task Author
Designed research study Poon, L’Abbé Wrote research proposal Poon
Provided comments/revisions to draft proposals L’Abbé Provided comments/revisions to final proposal Labonté, Bernstein, L’Abbé Wrote REB application Poon
Provided comments/revisions to draft REB application
Labonté, Bernstein, L’Abbé
Generated master list of foods Bernstein Generated survey Poon
Conducted pilot survey testing and analyzed feedback
Poon
Wrote amendment to REB application Poon, Bernstein Provided comments/revisions to amendment to REB application
L’Abbé
Office of the Vice-President, Research and Innovation Human Research Ethics Program
UT HREP – Application Form for Faculty Researchers 1 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
Office Use Only Application Number:
ETHICS REVIEW APPLICATION FORM – FACULTY
(For use by University of Toronto Faculty Researchers only)
SECTION A – GENERAL INFORMATION 1. TITLE OF RESEARCH PROJECT
Validation of a nutrient profile model proposed for restricting the marketing of unhealthy foods to children by nutrition professionals 2. INVESTIGATOR INFORMATION Principal Investigator (must be a UofT faculty member with research privileges): Title (e.g., Dr., Ms., etc.): Dr.
Name: Mary R. L’Abbé
Department: Nutritional Sciences Mailing address: Phone: Institutional e-mail: Alternate Contact (e.g., Research Coordinator): Title: Ms. Name: Alyssa Schermel Phone: Institutional e-mail: Co-Investigators: Are co-investigators involved? Yes No Title: Ms. Name: Theresa Poon Department (or organization if not affiliated with U of T): Nutritional Sciences Mailing address: Phone: Institutional e-mail: Title: Dr. Name: Marie-Eve Labonté Department (or organization if not affiliated with U of T): Institute of Nutrition and Functional Foods (INAF), Laval University Mailing address:
Before you start, familiarize yourself with:
TCPS2 Application instructions
Office FAQs
103
UT - Office of Research Ethics – Application Form for Faculty Researchers 2 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
Phone:
Institutional e-mail:
Title: Ms. Name: Jodi Bernstein Department (or organization if not affiliated with U of T): Nutritional Sciences Mailing address: Phone: Institutional e-mail: Please append additional pages with co-investigators’ names if necessary. 3. UNIVERSITY OF TORONTO RESEARCH ETHICS BOARD:
Social Sciences, Humanities and Education Health Sciences HIV/AIDS To determine which Research Ethics Board (REB) your application should be submitted, please consult: http://www.research.utoronto.ca/about/boards-and-committees/research-ethics-boards-reb/ 4. LOCATION(S) WHERE THE RESEARCH WILL BE CONDUCTED:
(a) If the research is to be conducted at a site requiring administrative approval/consent (e.g., in a school), please include all administrative consent letters. It is the responsibility of the researcher to determine what other means of approval are required, and to obtain approval prior to starting the project.
University of Toronto Hospital specify site(s) School board or community agency specify site(s) Community within the GTA specify site(s) International specify site(s) Other specify site(s) (b) For all off-campus research, whether in the local community or internationally, the researcher should consult with the Framework on Off-Campus Safety, Guidelines on Off-Campus Safety, and Guidelines on Safety in Field for institutional requirements. (c) The University of Toronto has an agreement with the Toronto Academic Health Sciences Network (TAHSN) hospitals regarding ethics review of hospital-based research where the University plays a peripheral role. Based on this agreement, certain hospital-based research may not require ethics review at the University of Toronto. If your research is based at a TAHSN hospital, please consult the following document to determine whether or not your research requires review at the University of Toronto. http://www.research.utoronto.ca/faculty-and-staff/research-ethics-and-protections/humans-in-research/ - “Administrative review” heading toward the bottom of the page. 5. OTHER RESEARCH ETHICS BOARD APPROVAL(S) (a) Does the research involve another institution or site? Yes No (b) Has any other REB approved this project? Yes No
If Yes, please provide a copy of the approval letter upon submission of this application. If No, will any other REB be asked for approval?
104
UT - Office of Research Ethics – Application Form for Faculty Researchers 3 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
Yes (please specify which REB) No 6. FUNDING OF THIS PROJECT
(a) Funding Status Source and Type Details Funded Agency: Canadian Institutes of Health
Research Fund #: (6 digits)
Agency: Burroughs Wellcome Fund #: (6 digits) Applied for funding
Agency: Submission date: Agency: Submission date:
Unfunded If unfunded, please explain why no funding is needed: 7. CONTRACTS AND AGREEMENTS (a) Is this research to be carried out as a contract or under a research agreement? Yes No If yes, is there a University of Toronto funding or non-funded agreement associated with the research? Yes No
If Yes, please append a copy of the agreement with of this application.
Is there any aspect of the contract that could put any member of the research team in a potential conflict of interest? Yes No If yes, please elaborate under #10. (b) Is this a Division 5, Health Canada regulated clinical trial that involves drugs, devices or natural health products? Yes No (if so, the application must be reviewed by the full board) 8. PROJECT START AND END DATES Estimated start date for the component of this project that involves human participants or data: September 2017 Estimated completion date of involvement of human participants or data for this project: September 2018 9. SCHOLARLY REVIEW:
(a) Please check one:
I. The research has undergone scholarly review by thesis committee, departmental review committee, peer review committee or some other equivalent (Specify review type – e.g., departmental research committee, supervisor, CIHR, SSHRC, OHTN, etc.): CIHR, Health Canada
II. The research will undergo scholarly review prior to funding (Specify review committee – e.g., departmental research committee, SSHRC, CIHR peer-review committee, etc.):
III. The research will not undergo scholarly review (Please note that all research greater than minimal risk requires scholarly review)
(b) If box I or II above was checked, please specify if:
The review was/will be specific to this application
105
UT - Office of Research Ethics – Application Form for Faculty Researchers 4 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
The review was/will be part of a larger grant 10. CONFLICTS OF INTEREST (a) Will the researcher(s), members of the research team, and/or their partners or immediate family members: (i) Receive any personal benefits (e.g., financial benefit such as remuneration, intellectual property rights, rights of employment, consultancies, board membership, share ownership, stock options, etc.) as a result of or in connection with this study? Yes No (ii) If Yes, please provide further details and discuss how any real, potential or perceived conflicts of interest will be managed during the project. (Do not include conference and travel expense coverage, or other benefits which are considered standard for the conduct of research.) Not applicable (b) Describe any restrictions regarding access to or disclosure of information (during or at the end of the study) that have been placed on the investigator(s). These restrictions include controls placed by the sponsor, funding body, advisory or steering committee. Not applicable (c) Where relevant, please explain any pre-existing relationship between the researcher(s) and the researched (e.g., instructor-student; manager-employee; clinician-patient; minister-congregant). Please pay special attention to relationships in which there may be a power differential – actual or perceived. There is no expected pre-existing relationship between the participants and the research team members in contact with them. SECTION B – SUMMARY OF THE PROPOSED RESEARCH 11. RATIONALE Describe the purpose and scholarly rationale for the proposed project. State the hypotheses/research questions to be examined. The rationale for doing the study must be clear. Please include references in this section. Background: In Canada, the Child Health Protection Act (Bill S-228) was introduced in September 2016 by Senator Greene Raine to amend the current Food and Drugs Act to prohibit food and beverage marketing directed at children under 13 years of age1. Shortly after, Health Canada also committed to introducing marketing restrictions as part of its Healthy Eating Strategy2. In response to this commitment, Health Canada has proposed the use of a new nutrient profile (NP) model specific to Canada for the identification of foods that would be subject to these marketing restrictions3. According to the World Health Organization’s (WHO’s) Guiding Principles and Framework Manual for the Development or Adaptation of NP Models, validation of a model is recommended for all newly developed or adapted models because various approaches of validation aid in the development, consistency, and credibility of the NP model4. One method of ensuring the credibility of the classifications of the foods generated a NP model is to assess convergent validity with food-based dietary guidelines4. Convergent validity is defined as “the extent to which the measurement correlates with an external criterion of the phenomenon under study at the same point in time” 4. The comparison to food-based dietary guidelines can be conducted using two approaches, involving either 1) a set of “indicator” foods that are the focus of food-based dietary guidelines (e.g., fruits, vegetables, confectionery, savoury snacks) and a small number of nutrition professionals as part
106
UT - Office of Research Ethics – Application Form for Faculty Researchers 5 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
of an expert panel involved in the development of the model, or 2) a representative sample of foods and a larger number of nutrition experts4. The latter approach is relatively more robust than the former because the representative sample of foods is intended to cover the entire spectrum of foods, and the classifications by the nutrition experts, albeit subjective, are determined independently without influence of the NP model under investigation4. Objective: The objective of this study is to assess convergent validity by comparing the marketing classifications of a representative sample of foods determined using Health Canada’s proposed NP model against the classifications determined by a sample of Canadian nutrition professionals. The marketing classifications refer to whether each food will be classified as “allowed” or “not allowed” to be marketed to children. Hypothesis: It is hypothesized that the marketing classifications of foods determined using the proposed NP model will align with the classifications determined by Canadian nutrition professionals. Significance: This research will be conducted as part of the Canadian Institutes of Health Research (CIHR) Project Grant, entitled “the IMPACT of food environment policies on the Canadian food supply, dietary intakes and health: evidence to inform policy action”. The research team will share the results of this study with Health Canada to inform the use of the proposed NP model in the policy on restricting the marketing of unhealthy foods and beverages to children. References: 1. Senate of Canada. Bill S-228. An Act to amend the Food and Drugs Act (prohibiting food and beverage
marketing directed at children). First reading, September 27, 2016. First Session, Forty-second Parliament, 64-65 Elizabeth II, 2015-2016. Internet: http://www.parl.gc.ca/content/hoc/Bills/421/Private/S-228/S-228_1/S-228_1.PDF (Accessed: 30 January 2017).
2. Health Canada. Healthy Eating Strategy. October 2016. Internet: https://www.canada.ca/content/dam/canada/health-canada/migration/publications/eating-nutrition/healthy-eating-strategy-canada-strategie-saine-alimentation/alt/pub-eng.pdf (Accessed: 30 January 2017).
3. Canada H. Toward Restricting Unhealthy Food and Beverage Marketing to Children: Discussion paper for public consultation. June 2017. Internet: https://s3.ca-central-1.amazonaws.com/ehq-production-canada/documents/attachments/9bced5c3821050c708407be04b299ac6ad286e47/000/006/633/original/Restricting_Marketing_to_Children.pdf (Accessed: 17 June 2017).
4. Rayner M. WHO Guiding Principles and Framework Manual for the development or adaptation of nutrient profile models (first edition). UNPUBLISHED 2011.
12. METHODS (a) Please describe all formal and informal procedures to be used. Describe the data to be collected, where and how they will be obtained and how they will be analyzed. Minor revisions to the study are bolded and are based on the results of the pilot study (section
12.2.4.6) 12.1 OVERVIEW The validation study will compare the marketing classifications of a representative sample of foods determined using the proposed NP model against the classifications determined by Canadian nutrition professionals. This work will be done through an online survey. The marketing classifications refer to whether each food will be classified as “allowed” or “not allowed” to be marketed to children.
107
UT - Office of Research Ethics – Application Form for Faculty Researchers 6 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
12.2 DATA COLLECTION 12.2.1 Representative Food Sample The representative food sample will consist of a master list of 200 branded foods from two databases (described in Section 12.2.2) across a variety of food categories pertinent to children’s marketing and Canadian diets. The pertinent food categories will be determined using four sources (refer to Appendix A): 1) 21 food categories across the 220 indicator foods, as per Health Canada’s discussion paper for public
consultation1; 2) 16 food categories across foods specifically marketed to children on food and beverage packaging, as
per Labonté et al. (2017 [submitted])2; 3) eight food categories according to the Canadian Children’s Food and Beverage Advertising Initiative
(CAI) Uniform Criteria3; and 4) 19 food categories across the foods specifically marketed to children on television programs, as per
Potvin Kent et al. (2017 [submitted])4. Specific food items will be selected from these categories for the master list, including a variety of foods that range in healthfulness. Specifically, a widely used NP model (e.g., Foods Standards Australia New Zealand5) will be applied to the foods in the master list to ensure that a range in healthfulness is achieved and that foods contain a range of both nutrients to limit and nutrients to encourage. In addition, the proposed NP model will be applied to the foods in the master list to ensure that both foods that will be permitted or not permitted by the proposed model will be represented. Participants in this study will be randomized into two groups and assigned to either be presented with foods without a Principal Display Panel (PDP) (“without PDP” group) or with a PDP (“with PDP” group). For participants in the “without PDP” group, 40 unique foods will be randomly selected from various food categories from the 200 foods in the master list. For participants in the “with PDP” group, 40 unique foods will be randomly selected from various food categories from a subset of 100 of the 200 foods in the master list. The subset of 100 foods will enable more classifications for each product, but from relatively fewer participants. Food items in the subset will be selected based on the availability of high-quality images of the PDP for each food. One hundred foods with high-quality images will be randomly selected if high-quality images for more than 100 foods are available. For participants in both groups, the random selection of foods will be done at the food category level to ensure that participants are exposed to a variety of food categories. The nutrition professionals will classify these 40 foods for which the Nutrition Facts table (NFt), ingredient list and either a PDP (“with PDP” group) or the generic product name (e.g., chocolate covered granola bar) (“without PDP” group), will be presented. The provision of a generic product name versus a PDP will enable the researchers to differentiate between classifications for 100 foods made solely on nutrition and ingredient information versus classifications that may have been influenced by a brand or symbols/logos. Refer to Section 12.2.4 for details. 12.2.2 Source of Nutrient Data of Representative Food Sample The nutrient data for each of the foods in the master list will be obtained from the following databases: Food Label Information Program (FLIP) and Menu-FLIP 2016. FLIP is a branded food composition database that has been developed and maintained since 2010 by the Principal Investigator (L’Abbé) with peer‐reviewed funding from the Canadian Institutes of Health Research (CIHR), the Canadian Stroke Network, Heart and Stroke Foundation, UofT McHenry Chair, and IT support from Dietitians of Canada (DC)6. This database of packaged foods is updated every 3-4 years for testing hypotheses and monitoring the Canadian food supply. FLIP 2013, contains nutritional information on n=15,342 unique food products with a NFt from the four largest national grocery retailers (Loblaw’s, Metro, Sobeys, and Safeway), representing approximately 75% of the Canadian retail food market share. In-store data collection occurred between May and September 2013 by systematically scanning grocery store shelves using a smartphone application, and web and mobile database software have been developed for efficient database management, processing and analyses. Information collected include product information (e.g.,
108
UT - Office of Research Ethics – Application Form for Faculty Researchers 7 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
company, brand, Universal Product Code), container size, price, NFt information, ingredient list, package marketing [e.g., nutrient content claims, front-of-pack (FOP) labelling, marketing to children], and photos of all sides of the packages. Only one package size per food was captured, but all flavours and varieties were collected. Nutritional information was recorded for products in their form “as sold” and “as consumed”, if necessary (e.g., condensed soup). Foods in FLIP 2013 were classified into 22 distinct food categories and 153 subcategories as defined in Schedule M of the Food and Drug Regulations (version current between March 15, 2012 and December 13, 2016)7. A 2017 collection is currently underway with a grant from the Centre for Child Nutrition & Health (now renamed the Lawson Centre for Child Nutrition), UofT, and from CIHR, and the complete data set is anticipated to be ready for analyses in late 2017 or early 2018. Currently, FLIP 2017 contains nutrition information for approximately 11,000 foods collected from the Loblaw’s website, which will be used for this study. The Menu-FLIP database is a unique UofT database containing nutrient composition information (serving size plus up to 13 nutrients commonly found on food labels) from over 80 sit-down and takeout restaurant chains in Canada, and includes data on ~9,000 foods and beverages per cycle. Data collections occurred in 2010, 2013, and 2016. One additional Ontario specific Menu-FLIP data collection supported by Public Health Ontario occurred in January-March 2017. For each collection, inclusion criteria included all food service establishments with ≥ 20 retail locations in Canada and with ≥1 outlet in at least three provinces. Nutrition information has been collected from publicly available resources such as company Canadian websites, and companies were contacted to clarify data if necessary. Establishments were classified according to type (fast-food, full service), and foods by type (meal, entrée, side dish etc). For this study, Menu-FLIP 2016 will be used to identify a small proportion of restaurant foods for inclusion under one or two categories within the master list of foods: 1) foods from fast food restaurants, and 2) foods from dine-in restaurants that are specifically marketed to children, as per Potvin Kent et al. (2017 [submitted])4. 12.2.3 Proposed NP Model Health Canada’s proposed NP model will allow marketing of foods that Health Canada encourages for healthy eating and child development, but will restrict marketing foods that contain sugars, sodium, or saturated fat above specific thresholds1. Two levels of nutrient threshold options are proposed: 5% of the DV of saturated fat, sugars, or sodium (option 1), or 15% of the DV of saturated fat, sugars, or sodium (option 2)1. Marketing would be restricted for foods exceeding one or more of these thresholds1. Ultimately, the proposed NP model will classify each food as “allowed” or “not allowed” to be marketed to children. Both these threshold options will be assessed. Each option will be applied to the master list and will be compared to the results from the survey of nutrition professionals (refer to Section 12.2.4). 12.2.4 Survey of Nutrition Professionals 12.2.4.1 Nutrition Professionals This study is modelled after a validation study conducted by Scarborough et al. (2007), who tested a NP model using data from a survey of nutrition professionals8,9. In the study by Scarborough et al. (2007), the survey was sent to a total of 3,517 nutrition professionals, of which 850 were from the British Dietetic Association and 2,667 were from the British Nutrition Society, with approximately 15% of individuals belonging to both organizations8,9. A total of 850 responses to the survey were received, resulting in a 24% initial response rate8,9. Of the 850 responses, 148 (17%) were considered non-suitable according to pre-defined criteria (described below) and were excluded; thus, 702 responses were eligible for analysis, which equates to a final response rate of 20%8,9. For this study, a non-nationally representative sample of nutrition professionals will be recruited through outreach from the three organizations that represent nutrition professionals in Canada to their constituents: DC with approximately 6,000 members; l’Ordre professionnel des diététistes du Québec (OPDQ) with approximately 3,175 members; and the Canadian Nutrition Society (CNS) with approximately 400 members. Based on an estimate whereby 15% (n=60) of the CNS members also belong to their respective dietetic organizations, the survey link will be sent 9,515 nutrition professionals across the three organizations (6,000 from DC; 3,175 from OPDQ; 340 from CNS).
109
UT - Office of Research Ethics – Application Form for Faculty Researchers 8 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
Recruitment will be conducted via dissemination of the introductory email (refer to Appendix B) to all members of the three organizations (DC, OPDQ, and CNS). The recruitment email will provide a brief introduction to the study and a weblink to the online survey, which will be hosted by SurveyGizmo. As discussed in Section 12.2.4.5, the information letter and informed consent form (refer to Appendix C) will be provided as preamble to the survey questions (refer to Appendix D). Based on the response rates observed in the Scarborough et al. (2007) study8,9, initial and final response rates of 24% and 20% would correspond to 2,284 and 1,903 responses, respectively, in this study. If a more conservative estimate is used involving initial and final response rates of 10% and 8%, respectively, it is estimated that 952 responses will be received and 761 responses will be eligible for analysis in this study. The same response quality exclusion criteria used by Scarborough et al. (2007) 8,9 to ensure that the survey questions were not completed carelessly or maliciously will be applied in this study; specifically, responses will be excluded if: 1) The respondent categorized less than 30 of the 40 (<75%) foods presented; 2) The respondent placed more than 80% of foods within a 15% range of the continuous scale; or 3) The respondent categorized more than 50% of foods using discordant classifications and scores
(e.g., a food that was selected as “not allowed” for marketing was given a score of 100, which indicates that the food is “healthiest”).
With 2/3 of participants randomized into the “without PDP” group and the remaining randomized into the “with PDP” group, it is anticipated that of the estimated 761 eligible responses, 510 will be from the “without PDP” group and 251 will be from the “with PDP” group. With 40 foods viewed per participant each of the foods from the master list of 200 products (presented without PDP) and from the subset of 100 products (presented with PDP), each product will be viewed about 100 times within the randomized groups. 12.2.4.2 Provision of Nutrition Information The following information for each food will be provided to the nutrition professionals on a single screen: 1) either the generic product name (e.g., chocolate covered granola bar) for participants in the “without PDP” group or the PDP for participants in the “with PDP” group; 2) NFt; and 3) ingredient list (refer to Appendix D). Given the inclusion of OPDQ members, the languages in which the nutrition information is presented must be considered. For certain food products in FLIP 2017, the French text may not be available for the PDP, NFt, and/or ingredient list. As such, foods with PDPs containing both English and French text will be presented whenever possible; otherwise, foods with PDPs containing English only will be presented. NFts containing both English and French will be presented. For all foods, the ingredient lists will be presented in English only. For foods in Menu-FLIP 2016, French text may not be presented in the image of the restaurant food nor on the NFt. As such, the images of the restaurant foods containing English only will be presented; however, NFts containing both English and French will be presented. Ingredient lists are not available for restaurant foods. Due to the inclusion of both English and French nutrition professionals, a disclaimer that some of the information will be presented in English only will be included in the survey (refer to Appendix D). 12.2.4.3 Classification of Foods Nutrition professionals will be asked two questions related to the healthfulness of each food: 1) “Should the food be allowed or not allowed to be marketed to children?”, and 2) “How would you rate the healthfulness of the food, with two anchor descriptors ‘healthiest’ and ‘least healthy’?” The first question will permit analysis of the results as a categorical variable, whereas the second question will permit analysis of the results as a continuous variable. Nutrition professionals will be asked to score the foods compared to all foods, rather than foods from a similar category.
110
UT - Office of Research Ethics – Application Form for Faculty Researchers 9 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
12.2.4.4 Informed Consent The information letter and informed consent form (refer to Appendix C) will be provided as preamble to the online survey questions. The information letter and informed consent will detail the study procedures, conditions for participation, risks and benefits, confidentiality, compensation, and contact information for the study. Nutrition professionals will be able to check one of two boxes corresponding to “I agree” or “I do not agree” to study participation. Those who selected “do not agree” will be prompted to exit the webpage. Those who selected “agree” will be directed to start the survey questions (refer to Appendix D). Participants will be able to decline participation in the study at any time by simply closing the webpage. 12.2.4.5 Survey Questions A single survey will be administered to all nutrition professionals. The survey platform, SurveyGizmo, will allow the participant to select the language (English or French) of the survey. The time for completion of the survey will be approximately 20 minutes. The survey questions are presented in Appendix D. The survey will consist of two parts for a total of 50 numbered questions. With respect to Part A, seven questions will be asked to ascertain demographics of the nutrition professionals (e.g., degrees acquired, years of nutrition experience, type of nutrition experience, experience working with children, etc.). Part B will be related to the classifications of the foods. Each participant will be randomized to classify either i) 40 foods for which the generic product name, NFt, and ingredient list will be presented or ii) 40 foods for which the PDP, NFt, and ingredient list will be presented. A disclaimer that some of the nutrition information is presented in English only will be included at the start of the food classifications (Part B). After classifying all 40 foods, two questions will be asked to ascertain the factors considered in the rating of foods (e.g., name of food, Nutrition Facts information, ingredient list, brand, claims, etc.). A final question will be asked to ascertain whether the respondent is familiar with Health Canada’s consultation document on marketing to children that was published in June 2017. The survey will be available for two weeks until a specified end-date, at which time the weblink to the survey will expire. 12.2.4.6 Pilot Study A pilot study with a convenience sample of approximately 20 English and French respondents (e.g., from the University of Toronto and Health Canada) will be conducted to ensure feasibility of the completion of the survey within the targeted time limit (i.e., 20 minutes) and clarity of the questions. Adjustments will be made to the survey if there are any concerns regarding clarity or time. 12.3 DATA ANALYSIS As discussed in Section 12.2.2.4.1, the same response quality exclusion criteria used by Scarborough et al. (2007) 8,9 to ensure that the survey questions were not completed carelessly or maliciously will be applied in this study; specifically, responses will be excluded if: 1) The respondent categorized less than 30 of the 40 (<75%) foods presented; 2) The respondent placed more than 80% of foods within a 15% range of the continuous scale; or 3) The respondent categorized more than 50% of foods using discordant classifications and scores
(e.g., a food that was selected as “not allowed” for marketing was given a score of 100, which indicates that the food is “healthiest”).
12.3.1 Primary Outcome The primary outcome will be the agreement in the proportions of “allowed” or “not allowed” foods determined using option 1 (5% DV) of Health Canada’s proposed NP model and by the nutrition professionals, assessed using the Kappa statistic and McNemar’s test for discordant pairs. Responses from participants randomized to the “without PDP” group will be used to determine this outcome. Each participant in the “without PDP” group will assess 40 foods using the generic name, NFt, and ingredient list. These 40 foods will be selected from the master list of 200 foods. It is anticipated that approximately 510 participants randomized to the “without PDP” group will have responses eligible for analyses; thus, each of the 200 foods will be assessed by approximately 100 participants.
111
UT - Office of Research Ethics – Application Form for Faculty Researchers 10 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
12.3.2 Secondary Outcomes A secondary outcome will be the agreement in the proportions of “allowed” or “not allowed” foods determined using option 2 (15% DV) of Health Canada’s proposed NP model and by the nutrition professionals. Additional secondary outcomes that may be explored include: • An analysis of the factors (e.g., claims, FOP logos or symbols) that were used in making the determination of whether a food should be allowed to be marketed to children. • Determine whether or how the PDP influenced the nutrition professionals’ classifications. Responses from participants randomized to the “with PDP” group will be used to determine this outcome. Each participant in the “with PDP” group will assess 40 foods using the PDP, NFt, and ingredient list. These 40 foods will be selected from a subset of 100 foods from the master list. It is anticipated that approximately 251 participants randomized to the “with PDP” group will have responses eligible for analyses; thus, each of the 100 foods will be assessed by approximately 100 participants. The difference in the 100 ratings of each food from the “with PDP” and the “without PDP” groups will be determined. • Although the intent of the proposed NP model is not to generate a score, the derivation of a NP score may allow for comparisons to the score generated by the nutrition professionals. For example, the derivation of a score may be based on the three nutrients of public health concern (i.e., saturated fat, sugars, and sodium) considered in the proposed NP model, such that a score of 1 as “healthiest” when none of the limits are exceeded; score of 2 when the limit for one of the three nutrients is exceeded; score of 3 when the limits for two of the three nutrients are exceeded; or score of 4 when the limits for all three nutrients are exceeded. The secondary outcomes that may be examined include, for example:
- Difference in the average scores of a food determined using option 1 (5% DV) of the proposed NP model and by the nutrition professionals.
- Difference in the average scores of a food determined using option 2 (15% DV) of the proposed NP model and by the nutrition professionals.
- Based on the classification and scores generated by the nutrition professionals, a comparison of the mean score across all “allowed” foods versus mean score across all “not allowed” foods. 12.4 REFERENCES 1. Canada H. Toward Restricting Unhealthy Food and Beverage Marketing to Children: Discussion paper
for public consultation. June 2017. Internet: https://s3.ca-central-1.amazonaws.com/ehq-production-canada/documents/attachments/9bced5c3821050c708407be04b299ac6ad286e47/000/006/633/original/Restricting_Marketing_to_Children.pdf (Accessed: 17 June 2017).
2. Labonté ME, Poon T, Mulligan C, Bernstein JT, Franco-Arellano B, L’Abbé MR. Comparison of global nutrient profiling systems for restricting the commercial marketing of foods and beverages of low nutritional quality to children in Canada. Submitted to the American Journal of Clinical Nutrition, revisions requested, July 24, 2017.
3. Advertising Standards Canada. Canadian Children’s Food and Beverage Advertising Initiative. Uniform Nutrition Criteria White Paper. September 2014. Internet: http://www.adstandards.com/en/childrensinitiative/CAIUniformNutritionCriteriaWhitePaper.pdf (Accessed: 19 September 2016).
4. Potvin Kent M, Smith JR, Pauzé E, L’Abbé M. The food and beverage industry's self-established Uniform Nutrition Criteria are not improving the healthfulness of food and beverage advertisements viewed by Canadian children on television. Under revision; submitted to International Journal of Behavioural Nutrition and Physical Activity, May/June, 2017.
5. Australian Government. ComLaw. Legislative Instrument Compilation. Australia New Zealand Food Standards Code - Standard 1.2.7 - Nutrition, Health and Related Claims - F2015C00967. December 7, 2015. Internet: https://www.comlaw.gov.au/Details/F2015C00967 (accessed 17 January, 2016).
6. Schermel A, Emrich TE, Arcand J, Wong CL, L'Abbe M R. Nutrition marketing on processed food packages in Canada: 2010 Food Label Information Program. Appl Physiol Nutr Metab 2013;38:666-72. doi: 10.1139/apnm-2012-0386.
112
UT - Office of Research Ethics – Application Form for Faculty Researchers 11 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
7. Government of Canada. Food and Drug Regulations (C.R.C., c. 870), Sections B.01.001, B.01.002A and D.01.001. Ottawa: Minister of Justice. 2016. Internet: http://laws-lois.justice.gc.ca/eng/regulations/c.r.c.,_c._870/page-165.html#h-353 (Accessed: 16 March 2016).
8. Scarborough P, Boxer A, Rayner M, Stockley L. Testing nutrient profile models using data from a survey of nutrition professionals. Public Health Nutr 2007;10:337-45. doi: 10.1017/s1368980007666671.
9. Scarborough P, Rayner M, Stockley L, Black A. Nutrition professionals' perception of the 'healthiness' of individual foods. Public Health Nutr 2007;10:346-53. doi: 10.1017/s1368980007666683.
(b) Attach a copy of all questionnaires, interview guides and/or any other instruments. (c) Include a list of appendices here for all additional materials submitted (e.g., Appendix A – Informed Consent; Appendix B – Interview Guide, etc.): Appendix A – List of Relevant Food Categories Appendix B – Recruitment Email Appendix C – Informed Consent Form Appendix D – Survey Questions 13. PARTICIPANTS, DATA AND/OR BIOLOGICAL MATERIALS (a) Describe the participants to be recruited list the eligibility criteria, and indicate the estimated sample size (i.e. min-max # of participants). Where applicable, please also provide a rationale for your choice in sample size and/or sample size calculation. Refer to Section 12.2.4.1. (b) Where the research involves extraction or collection of personally identifiable information, please describe the purpose, from whom the information will be obtained, what it will include, and how permission to access the data is being sought. (Strategies for recruitment are to be described in section #15.) No personally identifiable information will be collected or used from the participants in this study. (c) Is there any group or individual-level vulnerability related to the research that needs to be mitigated (for example, difficulties understanding informed consent, history of exploitation by researchers, power differential between the researcher and the potential participant)? If so, please provide further details below. As nutrition professionals will be recruited for this study, there is no group or individual-level vulnerability related to the research that needs to be mitigated. (d) If your research involves the collection and/or use of biological materials (e.g. blood, saliva, urine, teeth, etc.), please provide details below. Be sure to indicate how the samples will be collected and by whom. This study does not involve the collection and/or use of biological materials. 14. EXPERIENCE OF INVESTIGATORS WITH THIS TYPE OF RESEARCH (a) Please provide a brief description of previous experience by (i) the principal investigator/supervisor or sponsor, (ii) the research team and (iii) the people who will have direct contact with the participants. If there has not been previous experience with this type of research, please describe how the principal investigator/research team will be prepared.
113
UT - Office of Research Ethics – Application Form for Faculty Researchers 12 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
Principal Investigator: Dr. Mary L’Abbé is the Earle W. McHenry Professor and Chair, Department of Nutritional Sciences, University of Toronto and former Director of the Bureau Nutritional Sciences, Health Canada. She is internationally recognized for her leadership of nutrition policy for public health. She has extensive experience in nutrition research, including assessing the nutritional quality of the Canadian food supply; nutrition labelling policies for population health; consumer nutrition knowledge, attitudes and behaviours and development of an online food composition database (FLIP); and nutrient profiling. Dr. L’Abbé and her graduate students have conducted and published six previous consumer research studies using online panelists. Her group has successfully developed other nutrition web tools and apps (e.g., BigLife Salt Calculator and One Sweet App). Co-investigators: Theresa Poon is currently a MSc graduate student researching the validity of several global NP models for the assessment of the nutritional quality of foods in a Canadian context. She also contributed to the other NP research projects with Dr. L’Abbé, namely on the systematic review of global NP models and the use of NP models for marketing restrictions to children. In this study, she will provide expertise in evaluating the validity of the proposed NP approach. Dr. Marie-Eve Labonté was a CIHR Postdoctoral fellow conducting research on NP methods for public health nutrition and nutrition regulation. She specifically developed expertise in the use and adaptation of various NP systems for the evaluation of the nutritional quality of the Canadian food supply, overall and in association with characteristics that drive consumer choices (e.g., food prices). Dr. Labonté recently accepted a position as an Assistant Professor at Université Laval, Québec City, effective June 1st, 2017, but will continue to participate in this study. Jodi Bernstein is currently a PhD graduate student researching sugars in the Canadian prepackaged food supply. She has expertise in utilizing data from the FLIP database. In this study, she will provide assistance with the data collection and analyses. Database Manager: Alyssa Schermel is the Database Manager of FLIP. She obtained her MSc. in Nutritional Sciences at the University of Toronto under Dr. L’Abbé. In this study, she will provide assistance with respect to the FLIP database as needed. People who will have direct contact with the participants: None of the investigators will be in contact with the participants as the survey completed by the nutrition professionals will be administered online. 15. RECRUITMENT OF PARTICIPANTS Where there is recruitment, please describe how, by whom, and from where the participants will be recruited. Where participant observation is to be used, please explain the form of insertion of the researcher into the research setting (e.g., living in a community, visiting on a bi-weekly basis, attending organized functions). If relevant, describe any translation of recruitment materials, how this will occur and whether or not those people responsible for recruitment will speak the language of the participants. Refer to Section 12.2.4.1 and Appendix B.
Attach a copy of all posters, advertisements, flyers, letters, e-mail text, or telephone scripts to be used for recruitment as appendices.
114
UT - Office of Research Ethics – Application Form for Faculty Researchers 13 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
16. COMPENSATION Please see U of T’s Compensation and Reimbursement Guidelines. (a) Will participants receive compensation for participation? Financial Yes No In-kind Yes No Other Yes No (b) If Yes, please provide details and justification for the amount or the value of the compensation offered. Not applicable (c) If No, please explain why compensation is not possible or appropriate. This research will be conducted as part of the CIHR Project Grant, entitled “the IMPACT of food environment policies on the Canadian food supply, dietary intakes and health: evidence to inform policy action”. As no identifying information on participants is collected, we cannot compensate participants. Also, the professional associations consider the completion of this survey as public service, as we are not a commercial entity, and do not request payment for the distribution of this survey to their members. (d) Where there is a withdrawal clause in the research procedure, if participants choose to withdraw, how will compensation be affected? Not applicable SECTION C –DESCRIPTION OF THE RISKS AND BENEFITS OF THE PROPOSED RESEARCH 17. POSSIBLE RISKS (a) Please indicate all potential risks to participants as individuals or as members of a community that may arise from this research: (i) Physical risks (e.g., any bodily contact or administration of any substance): Yes No (ii) Psychological/emotional risks (e.g., feeling uncomfortable, embarrassed, or upset): Yes No (iii) Social risks (e.g., loss of status, privacy and/or reputation): Yes No (iv) Legal risks (e.g., apprehension or arrest, subpoena): Yes No (b) Please briefly describe each of the risks noted above and outline the steps that will be taken to manage and/or minimize them. There are no expected risks associated with this study. 18. POSSIBLE BENEFITS • Describe any potential direct benefits to participants from their involvement in the project • Describe any potential direct benefits to the community (e.g., capacity building)
115
UT - Office of Research Ethics – Application Form for Faculty Researchers 14 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
• Comment on the potential benefits to the scientific/scholarly community or society that would justify involvement of participants in this study
There will be no direct benefit to the participants for their involvement in this study. Information learned from this study may benefit nutrition professionals, consumers, and researchers, collectively, in the future. Participation in this study is important, as the results will provide understanding with regards to nutrition professionals’ perceptions on the healthfulness of foods and whether certain foods should be restricted from marketing to children. In addition, the results will be used to test the validity of the proposed NP model. The U of T will share the results of this study with Health Canada, and these results may be used to inform the use of the proposed nutrient profile model in the policy on restricting the marketing of unhealthy foods to children. SECTION D – INFORMED CONSENT 19. CONSENT PROCESS (a) Describe the process that will be used to obtain informed consent and explain how it will be recorded. Please note that it is the quality of the consent, not the form that is important. The goal is to ensure that potential participants understand to what they are consenting. Refer to Section 12.2.4.4 and Appendix C. (b) If the research involves extraction or collection of personally identifiable information from or about a research participant, please describe how consent from the individuals or authorization from the data custodian (e.g., medical records department, district school board) will be obtained. No personally identifiable information will be obtained from the participants in this study. 20. CONSENT DOCUMENTS (a) Attach an Information Letter/Consent Form For details about the required elements in the information letter and consent form, please refer to our informed consent guide (http://www.research.utoronto.ca/wp-content/uploads/documents/2014/10/GUIDE-FOR-INFORMED-CONSENT-V-Oct-2014.pdf) Additional documentation regarding consent should be provided such as:
- screening materials introductory letters, letters of administrative consent or authorization
(b) If any of the information collected in the screening process - prior to full informed consent to participate in the study - is to be retained from those who are later excluded or refuse to participate in the study, please state how potential participants will be informed of this course of action and whether they will have the right to refuse to allow this information to be kept. No information will be collected prior to obtaining full informed consent. 21. COMMUNITY AND/OR ORGANIZATIONAL CONSENT, OR CONSENT BY AN AUTHORIZED PARTY (a) If the research is taking place within a community or an organization which requires that formal consent be sought prior to the involvement of individual participants, describe how consent will be obtained and attach
116
UT - Office of Research Ethics – Application Form for Faculty Researchers 15 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
any relevant documentation. If consent will not be sought, please provide a justification and describe any alternative forms of consultation that may take place. Dr. L’Abbé spoke directly with the leadership of the professional associations to obtain their permission to distribute the survey to their members. Permission was granted by DC on July 11, 2017. Permission was granted verbally by the president and executive director of CNS on July 27, with final sign-off received from the board of directors at their meeting on August 23, 2017. Permission from OPDQ was received by email Sept 11, 2017. (b) If any or all of the participants are children and/or individuals that may lack the capacity to consent, describe the process by which capacity/competency will be assessed and/or, the proposed alternate source of consent. As nutrition professionals will be recruited for this study, it is anticipated that all participants will have the capacity to consent. (c) If an authorized third party will be used to obtain consent:
i) Submit a copy of the permission/information letter to be provided to the person(s) providing the alternative consent
ii) Describe the assent process for participants and attach the assent letter.
Not applicable 22. DEBRIEFING and DISSEMINATION (a) If deception or intentional non-disclosure will be used in the study, provide justification. Please consult the Guidelines for the Use of Deception and Debriefing in Research Deception or intentional non-disclosure will not be used in the study, as the objective of the study will be disclosed to the participants in the recruitment email and informed consent form (refer to Appendices C and D, respectively). (b) Please provide a copy of the written debriefing form, if applicable. (c) If participants and/or communities will be given the option of withdrawing their data following the debriefing, please describe this process. Not applicable (d) Please describe what information/feedback will be provided to participants and/or communities after their participation in the project is complete (e.g., report, poster presentation, pamphlet, etc.) and note how participants will be able to access this information. No feedback will be provided directly to participants after study participation. Results of the study will be disseminated through research publications and at scientific conferences. 23. PARTICIPANT WITHDRAWAL (a) Where applicable, please describe how participants will be informed of their right to withdraw from the project and outline the procedures that will be followed to allow them to exercise this right.
117
UT - Office of Research Ethics – Application Form for Faculty Researchers 16 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
Participants will be informed of their right to withdraw from the study in the Informed Consent Form (refer to Appendix X). They will be provided with the contact information of the primary investigator, research team, and Research Oversight and Compliance Office – Human Research Ethics Program. Participants will be able to decline participation in the study at any time by closing the webpage and not completing the survey. (b) Indicate what will be done with the participant’s data and any consequences which withdrawal may have on the participant. Any data collected from a participant that withdraws consent from the study will not be used in the analyses. There will be no consequences for participation withdrawal. (c) If participants will not have the right to withdraw from the project at all, or beyond a certain point, please explain. Ensure this information is included in the consent process and consent form. Participants will have the right to withdraw consent at any stage of the study. SECTION E – CONFIDENTIALITY AND PRIVACY 24. CONFIDENTIALITY Data security measures must be consistent with UT's Data Security Standards for Personally Identifiable and Other Confidential Data in Research. All identifiable electronic data that is being kept outside of a secure server environment must be encrypted. (a) Will the data be treated as confidential? Yes No (b) Describe the procedures to be used to protect the confidentiality of participants or informants, where applicable Data collected from each participant will be assigned a unique study identification number known only by a member of the research team tasked with assigning the numbers. No personally identifiable information will be obtained from the participants in this study. All data files will be encrypted and retained on a secure server. (c) Describe any limitations to protecting the confidentiality of participants whether due to the law, the methods used, or other reasons (e.g., a duty to report) Not applicable 25. DATA SECURITY, RETENTION AND ACCESS (a) Describe how data (including written records, video/audio recordings, artifacts and questionnaires) will be protected during the conduct of the research and dissemination of results. All data files, containing only the study subject identification numbers and no personally identifiable information, will be encrypted and retained on a secure server. Access to the data will be granted only to members of the research team. (b) Explain how long data or samples will be retained. (If applicable, referring to the standard data retention practice for your discipline) Provide details of their final disposal or storage. Provide a justification if you
118
UT - Office of Research Ethics – Application Form for Faculty Researchers 17 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
intend to store your data for an indefinite length of time. If the data may have archival value, discuss how participants will be informed of this possibility during the consent process. The encrypted electronic files will be stored for 10 years and deleted thereafter. Ten years is the standard retention practice for data collected from non-randomized controlled trials conducted at the Department of Nutritional Sciences, Faculty of Medicine. (c) If participant anonymity or confidentiality is not appropriate to this research project, please explain. Not applicable (d) If data will be shared with other researchers or users, please describe how and where the data will be stored and any restrictions that will be made regarding access. Once data analysis is completed, the research team will share with Health Canada the results in aggregate form via scientific publications, technical reports, or presentations. The raw data will be accessible only by Dr. L’Abbé and the research team. SECTION F – LEVEL OF RISK AND REVIEW TYPE See the Instructions for Ethics Review Submission Form for detailed information about the Risk Matrix. 26. RISK MATRIX: REVIEW TYPE BY GROUP VULNERABILITY and RESEARCH RISK (a) Indicate the Risk Level for this project by checking the intersecting box
______________________Research Risk____________________________ Group Vulnerability Low Medium High Low 1 1 2 Medium 1 2 3 High 2 3 3 (b) Explain/justify the level of research risk and group vulnerability reported above: There are no expected risks associated with this study. The nutrition professionals recruited for this study are expected to have a low level of vulnerability. (Please note that the final determination of Review Type and level of monitoring will be made by the reviewing University of Toronto REB) Based on the level of risk, these are the types of ethics review that an application may receive: Risk level = 1: Delegated Review; Risk level = 2 or 3: Full Board Review For both delegated and full reviews (SSH&E, HS, or HIV), please submit one electronic copy of your application and all appendices (e.g., recruitment, information/consent and debriefing materials, and study instruments) as a single Word document or a pdf. Do not submit your entire research proposal. Please ensure that the electronic signatures are in place and e-mail to [email protected]
119
UT - Office of Research Ethics – Application Form for Faculty Researchers 18 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
The deadline for delegated review (SSH&E or HS) is EVERY Monday, or first business day of the week, by 4 pm. Information about full REB meeting and submission due dates are posted on our website (SSH&E, HS or HIV). HIV REB reviews all applications at full board level but applies proportionate review based on the level of risk. All other submissions (e.g., amendments, adverse events, and continuing review submissions) should be sent to [email protected] SECTION G – SIGNATURES 27. PRIVACY REGULATIONS My signature as Principal Investigator, in Section G of this application form, confirms that I am aware of, understand, and will comply with all relevant laws governing the collection and use of personally identifiable information in research. I understand that for research involving extraction or collection of personally identifiable information, provincial, national and/or international laws may apply and that any apparent mishandling of personally identifiable information must be reported to the Office of Research Ethics. As the UofT Principal Investigator on this project, my signature confirms that I will ensure that all procedures performed will be conducted in accordance with all relevant University, provincial, national and international policies and regulations that govern research involving human participants. I understand that if there is any significant deviation from the project as originally approved I must submit an amendment to the Research Ethics Board for approval prior to implementing any change.
Signature of Principal Investigator:_____________________________________ Date: Dec 8, 2017
As the Departmental Chair/Dean, my signature confirms that I am aware of the requirements for scholarly review and that the ethics application for this research has received appropriate review prior to submission. In addition, my administrative unit will follow guidelines and procedures to ensure compliance with all relevant University, provincial, national or international policies and regulations that govern research involving human participants. My signature also reflects the willingness of the department, faculty or division to administer the research funds, if there are any, in accordance with University, regulatory agency and sponsor agency policies. Print Name of Departmental Chair/Dean (or designate): Signature of Departmental Chair/Dean: ___________________________ Date: (or authorized designate)
120
UT - Office of Research Ethics – Application Form for Faculty Researchers 19 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
APPENDIX A – LIST OF RELEVANT FOOD CATEGORIES
Health Canada’s Indicator Food Categories (n=21)1
Food Categories with Packaging Specifically Marketed to Children, as per Labonté et al. (2017 [submitted]) (n=16)2
Canadian Children’s Food and Beverage Advertising Initiative Uniform Criteria (n=8)3
Foods Specifically Marketed to Children on Television, as per Potvin Kent et al. (2017 [submitted]) (n=19)4
1. Vegetables, with added fat, sugars or sodium
1. Bakery products 1. Milk and Alternatives
1. Cakes (including pudding)
2. Fruit, with added fat, sugars or sodium
2. Beverages 2. Grains 2. Candy (without chocolate)
3. Fruit, without added fat, sugars or sodium
3. Cereals, other grain products
3. Soups 3. Cold cereals (i.e. ready-to-eat products marketed as breakfast foods)
4. Juice (100% pure) 4. Dairy products, substitutes
4. Meat and Alternatives
4. Cheese (excluding cottage cheese)
5. Breads, plain rice and plain pasta
5. Desserts 5. Fruit and Vegetables
5. Chocolate bars (excluding boxed chocolate and candy with chocolate)
6. Breakfast cereals (no sugar added)
6. Dessert toppings, fillings
6. Occasional Snacks
6. Compartment snacks and lunch kits (i.e. prepackaged products comprised of two or more ingredients in separate compartments sold as portable snacks or meals)
7. Breakfast cereals (sugar added)
7. Marine, fresh water animals
7. Mixed Dishes 7. Cookies
8. Sweet snacks, desserts, cookies, granola bars
8. Fruit, fruit juices 8. Meals on the Go
8. Ice cream (including frozen yogurt, sherbet, sorbet or frozen treats made from these foods)
9. Savory snacks 9. Meat, poultry, their products, substitutes
9. Pizza (not sold in restaurants)
10. Fluid milk or soy based beverages (unsweetened)
10. Miscellaneous 10. Portable snacks (i.e. cereal/protein/fruit bars or squares and fruit snacks)
11. Fluid milk (sweetened) 11. Combination dishes 11. Fast food restaurants
12. Yogourts with sugar added
12. Nuts, seeds 12. Non-fast food restaurants (i.e. dine-in restaurants)
13. Yogourts without sugar added
13. Potatoes, sweet potatoes, yams
13. Snack foods (e.g. chips, popcorn, pretzels, cheese puffs and meat-based snacks like jerky)
14. Cheeses 14. Snacks 14. Diet and regular soft drinks
15. Meat and alt 15. Soups 15. Energy drinks 16. Processed meats 16. Sugars, sweets 16. Sports drinks
121
UT - Office of Research Ethics – Application Form for Faculty Researchers 20 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
Health Canada’s Indicator Food Categories (n=21)1
Food Categories with Packaging Specifically Marketed to Children, as per Labonté et al. (2017 [submitted]) (n=16)2
Canadian Children’s Food and Beverage Advertising Initiative Uniform Criteria (n=8)3
Foods Specifically Marketed to Children on Television, as per Potvin Kent et al. (2017 [submitted]) (n=19)4
17. Composite dish (ready made meals, mixed dishes)
17. Juices
18. Confectionary 18. Drinks and nectars (excluding water, milk and alternatives, tea and coffee drinks, cocktail mixers, and alcoholic beverages)
19. Sugar and artificially sweetened beverages (soda, energy drinks, juice drinks) and vegetables juices with added sodium
19. Yogurt (including yogurt beverages)
20. Meals from fast food outlets
21. Miscellaneous 1 Canada H. Toward Restricting Unhealthy Food and Beverage Marketing to Children: Discussion paper for public consultation. June 2017. Internet: https://s3.ca-central-1.amazonaws.com/ehq-production-canada/documents/attachments/9bced5c3821050c708407be04b299ac6ad286e47/000/006/633/original/Restricting_Marketing_to_Children.pdf (Accessed: 17 June 2017). 2 Labonté ME, Poon T, Mulligan C, Bernstein JT, Franco-Arellano B, L’Abbé MR. Comparison of global nutrient profiling systems for restricting the commercial marketing of foods and beverages of low nutritional quality to children in Canada. Submitted to the American Journal of Clinical Nutrition, revisions requested, July 24, 2017. 3 Advertising Standards Canada. Canadian Children’s Food and Beverage Advertising Initiative. Uniform Nutrition Criteria White Paper. September 2014. Internet: http://www.adstandards.com/en/childrensinitiative/CAIUniformNutritionCriteriaWhitePaper.pdf (Accessed: 19 September 2016). 4 Potvin Kent M, Smith JR, Pauzé E, L’Abbé M. The food and beverage industry's self-established Uniform Nutrition Criteria are not improving the healthfulness of food and beverage advertisements viewed by Canadian children on television. Under revision; submitted to International Journal of Behavioural Nutrition and Physical Activity, May/June, 2017.
122
UT HREP – Application Form for Faculty Researchers 21 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
APPENDIX B – RECRUITMENT EMAIL1
Dear nutrition professional, We are requesting your participation in a research study involving nutrition professionals conducted by the University of Toronto with funding from the Canadian Institutes of Health Research. You have been contacted because you are a member of Dietitians of Canada, l’Ordre professionnel des diététistes du Québec, and/or the Canadian Nutrition Society. The study aims to assess the healthfulness of pre-packaged and restaurant foods and whether or not these foods should be allowed to be marketed to children. We will use the results to test the validity of a nutrient profile model that has been proposed for use in Canada. Your involvement will be important, as the results of this study may be used to inform the policy on restricting the marketing of unhealthy foods to children in Canada. Study participants will be asked to complete an online survey, which will involve classifying the healthfulness of a random set of 40 foods. If you are interested in participating or learning more about this study, the information letter and informed consent form is provided as preamble to the online survey, which is hosted by SurveyGizmo. The survey is available in English or French. Please access the survey here [hyperlink], or at the following weblink: [weblink]. The survey will be available for two weeks until [date]. If you have any questions regarding the study, please email our research team at with the subject line "survey on marketing foods to children". Thank you for your consideration in participating in this study. Mary R. L'Abbé, PhD Earle W. McHenry Professor and Chair, Department of Nutritional Sciences Faculty of Medicine, University of Toronto
Tel ; Cell Email: Web: Assistant to the Chair: Don Newton Tel: ; Email:
1 The recruitment email will also be presented in French.
123
UT HREP – Application Form for Faculty Researchers 22 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
APPENDIX C – INFORMED CONSENT FORM2 Information Letter and Informed Consent Form Principal Investigator: Dr. Mary L’Abbé Earle W. McHenry Professor and Chair, Department of Nutritional Sciences Faculty of Medicine, University of Toronto
Phone: Email: Title: Assessing the healthfulness of foods for restricting the marketing of unhealthy foods to children You are being asked to participate in a study to assess the healthfulness of pre-packaged foods and whether or not these foods should be allowed to be marketed to children. This research is conducted by the University of Toronto (U of T) with funding from the Canadian Institutes of Health Research. The U of T will use the results to test the validity of a nutrient profile model that has been proposed for use in Canada. The U of T will share the results of this study with Health Canada, which may be used to inform the policy on restricting the marketing of unhealthy foods and beverages to children. Before agreeing to participate in this study, it is important that you read and understand the following explanation of the study procedures. You should understand enough about the study risks and benefits to be able to make an informed decision before deciding whether you wish to participate. This is known as the informed consent process. Procedure We are inviting approximately 9,000 participants to take part in this study. Participants include nutrition professionals from Dietitians of Canada, l’Ordre professionnel des diététistes du Québec, and/or the Canadian Nutrition Society. This study takes place through an online survey delivered using SurveyGizmo. Participation includes:
1. Providing consent for us to use the data collected from the survey, including demographics (e.g., degrees acquired, years of nutrition experience, type of nutrition
2 The informed consent form will also be presented in French.
124
UT HREP – Application Form for Faculty Researchers 23 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
experience, experience working with children) and your classifications of foods. Please note that some of the nutrition information of the foods may be presented in English only.
2. Completing the survey, which involves reviewing 40 foods and should take approximately 20 minutes to complete.
Conditions for Participating Your participation in this study is voluntary, and you may withdraw from or refuse to participate in the study at any time. You may decline to answer any question in the survey without any negative consequences. Risks and Discomforts There are no expected risks or discomforts associated with this study. Benefits There is no direct benefit to you for your involvement in this study. Information learned from this study may benefit you and other nutrition professionals, consumers, and researchers in the future. Your participation in this study is important, as the results will help us understand nutrition professionals’ perceptions on the healthfulness of foods and whether certain foods should be restricted from marketing to children. The U of T will use the results to test the validity of a nutrient profile model that has been proposed for use in Canada. The U of T will share the results of this study with Health Canada, which may be used to inform the policy on restricting the marketing of unhealthy foods to children. Confidentiality All information obtained during the study will be held in strict confidence. You will be identified only by a study-specific identification number. No names or personal identifying information will be collected or used in any publication or presentations. No information identifying you will be transferred outside this study. Compensation Compensation will not be provided for participation in this study. In no way does agreeing to participate in this study waive your legal rights, nor does it relieve the investigators, sponsors, or involved institutions from their legal and professional responsibilities. Questions and Contact Information This study has been reviewed and received ethics approval from the U of T (Human Study REB# ). If you have any questions or concerns about your rights as a participant in this study, you can contact the Research Oversight and Compliance Office – Human Research Ethics Program at or . This contact is not involved with the research project in any way, and calling them will not affect your participation in the study.
125
UT HREP – Application Form for Faculty Researchers 24 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
If you have any questions regarding the study in general, please email our research team at with the subject line "survey on marketing foods to children". Consent By selecting the “I agree” option below, I consent to take part in the study with the understanding that I may withdraw at any time.
I agree. I do not agree.
126
UT HREP – Application Form for Faculty Researchers 25 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
APPENDIX D – SURVEY QUESTIONS3
Part A: Demographics
1. Which province or territory do you reside in? Alberta British Columbia Manitoba New Brunswick Newfoundland and Labrador Northwest Territories Nova Scotia Nunavut Ontario Prince Edward Island Quebec Saskatchewan Yukon Prefer not to say
2. What degree(s) and/or professional designations have you acquired? (Select all that apply.)
BSc MSc PhD Post-doctoral fellowship RD or DtP Other (please specify): [Open field] Prefer not to say
3. How many years of working/professional nutrition experience do you have? <1 year 1 to 3 years 4 to 6 years 7 to 9 years 10 or more years Prefer not to say
4. What type of nutrition experience do you have? (Select all that apply.) Clinical practice Public health Research Academia
3 This information will be presented in French in the French version of the survey.
127
UT HREP – Application Form for Faculty Researchers 26 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
Industry Other (please specify): [Open field] Prefer not to say
5. Are you a parent or guardian of a child or children aged 17 years or younger? Yes a. If yes, please list age(s) of child(ren): [Open field]
No Prefer not to say
6. Do you have experience working with children? Yes a. If yes, please list age(s) of child(ren): [Open field] b. If yes, please specify the context in which you worked with children. (Select all
that apply.) Education Clinical practice Community or public health Research Other (please specify): [Open field] Prefer not to say
No Prefer not to say
128
UT HREP – Application Form for Faculty Researchers 27 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
Part B: Classification of Foods [for the subset of responders who will be shown the generic product name, NFt, and ingredient list of the food]
You will be asked to classify a total of 40 foods. Note: Some of the nutrition information (names of the foods, ingredient lists) is presented in English only.
[The specific food shown below is an example and will vary for each question. The following will be repeated for questions 1 to 40 in Part B.]
Granola bar, cereal bar, oatmeal
1a. Should the food be allowed to be marketed to children? Yes No
1b. How would you rate the healthfulness of the food on a scale from “least healthy” to “healthiest”? Note: Rate the food compared to all foods, rather than foods from a similar category.
least healthy [visual analogue scale from 1 to 100] healthiest
INGREDIENTS: WHOLE GRAIN ROLLED OATS, WHOLE WHEAT FLOUR, SUGAR, CANOLA OIL, TAPIOCA SYRUP, GLYCERIN, CHICORY ROOT EXTRACT (INULIN FIBRE), DRIED EGG YOLK (CONTAINS SODIUM SILICOALUMINATE), MOLASSES, RAISIN JUICE CONCENTRATE, PALM AND PALM KERNEL OIL, BAKING SODA, DRIED EGG WHITE, SALT, MODIFIED MILK INGREDIENTS, CINNAMON, NATURAL FLAVOUR, MALTODEXTRIN, SOY LECITHIN, ROSEMARY EXTRACT, CITRIC ACID.
129
UT HREP – Application Form for Faculty Researchers 28 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
Part B: Classification of Foods [for the subset of responders who will be shown the PDP, NFt, and ingredient list of the food]
You will be asked to classify a total of 40 foods. Note: Some of the nutrition information (food packages, ingredient lists) is presented in English only.
[The specific food shown below is an example and will vary for each question. The following will be repeated for questions 1 to 40 in Part B.]
1a. Should the food be allowed to be marketed to children?
Yes No
1b. How would you rate the healthfulness of the food on a scale from “least healthy” to “healthiest”? Note: Rate the food compared to all foods, rather than foods from a similar category.
least healthy [visual analogue scale from 1 to 100] healthiest
INGREDIENTS: WHOLE GRAIN ROLLED OATS, WHOLE WHEAT FLOUR, SUGAR, CANOLA OIL, TAPIOCA SYRUP, GLYCERIN, CHICORY ROOT EXTRACT (INULIN FIBRE), DRIED EGG YOLK (CONTAINS SODIUM SILICOALUMINATE), MOLASSES, RAISIN JUICE CONCENTRATE, PALM AND PALM KERNEL OIL, BAKING SODA, DRIED EGG WHITE, SALT, MODIFIED MILK INGREDIENTS, CINNAMON, NATURAL FLAVOUR, MALTODEXTRIN, SOY LECITHIN, ROSEMARY EXTRACT, CITRIC ACID.
130
UT HREP – Application Form for Faculty Researchers 29 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
Part C: Final Questions [for the subset of responders who will be shown the generic product name, NFt, and ingredient list of the food]
1. What factor(s), if any, did you consider in deciding whether the food should or should not be marketed to children? (Select all that apply.)
[The response fields illustrated in the figure below will be presented in a multi-stage manner. For example, participants will be presented with the five responses in green initially. Depending on the green responses selected, the responses in blue will be presented thereafter, if applicable; and so on.]
2. Are you familiar with Health Canada’s consultation document on restricting the marketing of unhealthy foods and beverages to children that was published in June 2017?
Yes No
Name of the food
Nutrition Facts table
Nutrients to limit
Energy
Total fat
Saturated fat
Trans fat
Sodium
Total sugars
Cholesterol
Nutrients to encourage
Mono- or polyunsaturated fats
Protein
Fibre
Vitamins or minerals
Serving size Other (please specify)
Ingredient list
Number of ingredients listed
Order of ingredients
Fruits
Vegetables
Whole grains
Refined grains
Hydrogenated oils
Partially-hydrogenated oils
Added/free sugars
Low/no-calorie sweeteners
Other flavouring agents
Monosodium glutamate (MSG)
Preservatives
Food colouring
Other food additives
Allergens
Ingredients not recognized
Other (please specify)
Other (please specify)
None of the above
131
UT HREP – Application Form for Faculty Researchers 30 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor
Version Date: December 8, 2017
Part C: Final Questions [for the subset of responders who will be shown the generic product name, NFt, and ingredient list of the food]
1. What factor(s), if any, did you consider in deciding whether the food should or should not be marketed to children? (Select all that apply.)
[The response fields illustrated in the figure below will be presented in a multi-stage manner. For example, participants will be presented with the five responses in green initially. Depending on the green responses selected, the responses in blue will be presented thereafter, if applicable; and so on.]
2. Are you familiar with Health Canada’s consultation document on restricting the
marketing of unhealthy foods and beverages to children that was published in June 2017?
Yes No
Food package
Brand
Nutrient content claims
Health claims
Symbols or logos
Other (please specify)
Nutrition Facts table
Nutrients to limit
Energy
Total fat
Saturated fat
Trans fat
Sodium
Total sugars
Cholesterol
Nutrients to encourage
Mono- or polyunsaturated fats
Protein
Fibre
Vitamins or minerals
Serving size Other (please specify)
Ingredient list
Number of ingredients listed
Order of ingredients
Fruits
Vegetables
Whole grains
Refined grains
Hydrogenated oils
Partially-hydrogenated oils
Added/free sugars
Low/no-calorie sweeteners
Other flavouring agents
Monosodium glutamate (MSG)
Preservatives
Food colouring
Other food additives
Allergens
Ingredients not recognized
Other (please specify)
Other (please specify)
None of the above
132
133
Appendix C
Systematic Review of NP Models This systematic review was registered on PROSPERO(41). The manuscript was
submitted in January 2018 to Advances in Nutrition for publication: Labonté MÈ, Poon
T, Gladanac B, Ahmed M, Franco-Arellano B, Rayner M, L’Abbé MR (unpublished).
Nutrient profile models with applications in government-led nutrition policies aimed at
health promotion and noncommunicable disease prevention: a systematic review.
Although this research was not explicitly part of my MSc thesis, I had a significant role in
this research, as indicated in Table C-1. The manuscript of the systematic review is
provided as a supplemental file to this thesis.
Table C-1 Authors’ contributions to systematic review of NP models
Task Author
Prior to January 2016 when TP started MSc
Designed research study Labonté, L’Abbé Identified models from Rayner’s catalogue Labonté, Ahmed Conducted literature searches Labonté Screened title/abstracts of publications Labonté, Ahmed After January 2016 when TP started MSc
Screened full-text publications Labonté, Poon Screened reference lists Labonté, Poon Assessed eligibility of NP models Labonté, Poon Extracted data Gladanac (majority), Labonté, Poon, Franco-Arellano Verified extracted data Labonté, Poon Prepared tables and figures Labonté, Poon
Wrote manuscript Labonté Provided comments/revisions to draft manuscript Poon Provided comments/revisions to final manuscript Poon, Gladanac, Ahmed, Franco-Arellano, Rayner,
L’Abbé
134
Copyright Acknowledgements Appendix A is reproduced with permission from “Labonté MÈ, Poon T, Mulligan C,
Bernstein JT, Franco-Arellano B, L’Abbé MR. Comparison of global nutrient profiling
systems for restricting the commercial marketing of foods and beverages of low
nutritional quality to children in Canada. Am J Clin Nutr 106, 1471-1481.”