Recommendations on the content of a survey protocol for€¦ · 37 SCHEDULE FOR THE PROPOSED...

Working document QAS/14.590

July 2014

Document for comment

Recommendations on the content of a survey protocol for 1

surveys of the quality of medicines1 2

(July 2014) 3

DRAFT FOR COMMENT 4

5

6

7

8

9

10

11

12

© World Health Organization 2014 13

All rights reserved. 14

This draft is intended for a restricted audience only, i.e. the individuals and organizations having received this draft. The draft 15 may not be reviewed, abstracted, quoted, reproduced, transmitted, distributed, translated or adapted, in part or in whole, in 16 any form or by any means outside these individuals and organizations (including the organizations' concerned staff and 17 member organizations) without the permission of the World Health Organization. The draft should not be displayed on any 18 website. 19

Please send any request for permission to: 20

Dr Sabine Kopp, Group Lead, Medicines Quality Assurance, Technologies, Standards and Norms, Department of Essential 21 Medicines and Health Products, World Health Organization, CH-1211 Geneva 27, Switzerland. Fax: (41-22) 791 4730; 22 email: [email protected]. 23

The designations employed and the presentation of the material in this draft do not imply the expression of any opinion 24 whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or 25 of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate 26 border lines for which there may not yet be full agreement. 27

The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or 28 recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors 29 and omissions excepted, the names of proprietary products are distinguished by initial capital letters. 30

All reasonable precautions have been taken by the World Health Organization to verify the information contained in this 31 draft. However, the printed material is being distributed without warranty of any kind, either expressed or implied. The 32 responsibility for the interpretation and use of the material lies with the reader. In no event shall the World Health 33 Organization be liable for damages arising from its use. 34

This draft does not necessarily represent the decisions or the stated policy of the World Health Organization. 35

36

1

This was previously working document QAS/12.510: Proposal for a procedure on sampling and market

surveillance survey.

Should you have any comments on the attached revision, please send these to Dr S. Kopp, Group

Lead, Medicines Quality Assurance, Technologies, Standards and Norms ([email protected]) with a

copy to Ms Marie Gaspard ([email protected]) by 30 August 2014.

Our working documents will be sent out electronically only and will also be placed on the

Medicines website for comment under “Current projects”. If you do not already receive our

draft working documents please let us have your email address (to [email protected]) and we

will add it to our electronic mailing list.


page 2

SCHEDULE FOR THE PROPOSED ADOPTION PROCESS OF DOCUMENT QAS/14.590: 37


surveys of the quality of medicines 39

40

41

Survey and feedback from national organizations and

nongovernmental organizations

June–September 2011

Presentation to forty-sixth meeting of the WHO Expert

Committee on Specifications for Pharmaceutical

Preparations

9–12 October 2011

Expert Committee recommendation to develop sampling

procedure (WHO Technical Report Series, No. 970)

October 2011

Proposal for a procedure on sampling and market

surveillance

August 2012

Working document sent out for comment September 2012

Collation of comments September–October 2012

Presentation to forty-seventh meeting of the WHO Expert


Preparations

October 2012

Receipt of document on Recommendations on the

content of a Survey Protocol: Surveys of the quality of

essential medicines from Ms Patricia Tabernero,

Scientific Coordinator, Antimalarial Quality Module,

WorldWide Antimalarial Resistance Network, in close

collaboration with Ms Sue J. Lee, Ms Kasia Stepniewska

and Mr Paul N. Newton

June 2014

Working document sent out for comment July 2014

Collation of comments August–September 2014

Presentation to forty-ninth meeting of the WHO Expert


Preparations

October 2014

Any further action as necessary …


page 3

42

43


surveys of the quality of medicines 45

46

A survey protocol may follow chapters as suggested below or may be appropriately modified. 47

Explanatory notes are in italics. 48

49

BACKGROUND 50

Extract from the forty-eighth report of the WHO Expert Committee on Specifications for 51

Pharmaceutical Preparations: 52

8.2.2 Proposal for a procedure on sampling and market surveillance survey 53

Following the recommendation made by the Expert Committee at its forty-sixth 54

and forty-seventh meetings in 2011 and 2012 the Secretariat commissioned the 55

development of guidance for sampling procedures based on examples obtained 56

from many countries as feedback to the Secretariat's proposal for a procedure on 57

sampling and market surveillance and on a first proposal presented in 2012. This 58

first proposal was based on an existing survey protocol developed by the WHO 59

Prequalification Laboratory Programme, which had been extensively involved 60

in the establishment of survey protocols for major studies for antimalarial and 61

antituberculosis medicines. 62

The Expert Committee noted in 2012 that the document would be of 63

particular importance in monitoring and post-marketing surveillance and agreed 64

that it should be further developed as a general document to provide advice on 65

sampling for various groups of medicines. The Expert Committee also noted 66

the need for separate, specific guidance in relation to SFFC medical products. 67

A comprehensive draft working document, which would be prepared 68

for public consultation in due course, was presented to the Expert Committee 69

for comments. 70

The Expert Committee noted this information and thanked the Secretariat 71

for the draft working document. 72

73

This document is now being submitted as the first of two dealing with the monitoring and 74

postmarket surveillance of medicines and providing advice on survey protocols and sampling for 75

medicines. A second specific guidance is in preparation in relation to spurious/falsely-labelled/ 76

falsified/counterfeit (SFFC) medical products. 77


page 4

CONTENTS 78

page 79

1. Acknowledgements .............................................................................................................................. 6 80

2. Glossary of terms and abbreviations ..................................................................................................... 7 81

3. Executive summary .............................................................................................................................. 8 82

4. Introduction and background ................................................................................................................ 8 83

5. Initial planning ................................................................................................................................... 13 84

5.1 What is already known about medicine quality? ........................................................... 13 85

5.2 What registered and unregistered medicines are available in the survey area? ............... 13 86

5.3 What is the distribution system or supply chain of the medicines? ................................ 14 87

5.4 Maps of pharmaceutical outlets .................................................................................... 15 88

5.5 Information on health-seeking behaviour for the target pharmaceuticals? ..................... 15 89

6. Objectives .......................................................................................................................................... 15 90

6.1 Are there poor-quality drugs in this area, border checkpoint, etc.? ................................ 16 91

6.2 What are the proportion of poor-quality medicines being sold and/or proportion 92

of outlets selling poor-quality medicines in a particular geographical area? ................................ 17 93

6.3 Do the proportion of poor-quality medicines or the proportion of outlets selling 94

poor-quality medicines exceed a predetermined level? ............................................................... 17 95

6.4 Has medicine quality changed for the better or worse in an area or for a 96

medicine or medicine group? ..................................................................................................... 18 97

6.5 Areas to be sampled ..................................................................................................... 18 98

6.6 Products to be surveyed ................................................................................................ 20 99

6.7 Medicine preparation type ............................................................................................ 22 100

6.8 Timing of the survey .................................................................................................... 22 101

6.9 Types of sample collection sites ................................................................................... 22 102

6.10 Definitions used to determine quality ........................................................................... 25 103

7. Survey management and time frame ................................................................................................... 25 104

8. Methodology ...................................................................................................................................... 26 105

8.1 Terminology ................................................................................................................. 27 106

8.2 Sampling designs (convenience, random, LQAS, sentinel) ........................................... 29 107

Convenience sampling ............................................................................................................... 29 108

Random sampling ...................................................................................................................... 30 109

Lot quality assurance sampling .................................................................................................. 31 110

Sentinel site monitoring ............................................................................................................. 32 111

8.3 Number of dosage units to be collected: ....................................................................... 33 112

8.4 Sampling techniques for the dosage units within medicine samples .............................. 38 113

9. Sample collection ............................................................................................................................... 40 114

Overt sampling versus mystery shopper methodology ................................................................ 40 115

Survey team and procedures for collection ................................................................................. 42 116

10. Ethical considerations on quality of medicines surveys ..................................................................... 44 117

General ethical issues................................................................................................................. 45 118

Risks to the survey team members? ........................................................................................... 47 119


page 5

What is the risk to those surveyed? ............................................................................................ 47 120

Do medicine quality surveys require review by a research ethics committee? ............................. 48 121

Should outlet staff be told that they are being part of a study? .................................................... 50 122

What to do with the information once it has been gathered? ....................................................... 51 123

11. Storage and transport of samples ...................................................................................................... 52 124

12. Techniques to determine quality and laboratory facilities ................................................................. 53 125

Assays to be conducted .............................................................................................................. 54 126

Test methods and specifications ................................................................................................. 55 127

Receipt and testing of samples by a testing laboratory ................................................................ 56 128

13. Data management, reporting and publication.................................................................................... 56 129

14. Conclusions ..................................................................................................................................... 60 130

15. References: ...................................................................................................................................... 61 131

Annex 1. Sources of information of medicine quality 132

Annex 2. Methodology section 133

Annex 3. National sampling plan 134

Annex 4. Sample collection form 135

Annex 5. SOP for mystery shopper methodology 136

Annex 6. SOP for overt sampling methodology 137

Annex 7. Testing protocol 138

Annex 8. Content of the analytical test report 139

Appendix to survey protocol 140

Example based on WHO survey of the quality of antituberculosis medicines circulating in selected NIS of the former

Soviet Union:

1. Questionnaire

2. National sampling plan

3. Sample collection form

4. Testing protocol 5. Content of the analytical test report

141

142


page 6

ACKNOWLEDGEMENTS 143

144

This document was prepared by: 145

Patricia Tabernero (WorldWide Antimalarial Resistance Network (WWARN), Centre for 146

Tropical Medicine, Nuffield Department of Medicine, University of Oxford, Oxford, United 147

Kingdom); Sue J. Lee (Mahidol Oxford Tropical Medicine Research Unit, Faculty of Tropical 148

Medicine, Mahidol University, Bangkok, Thailand, Centre for Tropical Medicine, Nuffield 149

Department of Medicine, University of Oxford, Oxford, United Kingdom); Kasia 150

Stepniewska (Worldwide Antimalarial Resistance Network (WWARN), Centre for Tropical 151

Medicine, Nuffield Department of Medicine, University of Oxford, Oxford, United Kingdom); 152

Paul N. Newton (Worldwide Antimalarial Resistance Network (WWARN), Centre for 153

Tropical Medicine, Nuffield Department of Medicine, University of Oxford, Oxford, United 154

Kingdom). 155

156

We are very grateful to all those who have given advice and have helped in the completion of 157

this report: 158

Roger Bate, American Enterprise Institute, Washington, D.C., United States of America; 159

Facundo Fernández, Georgia Institute of Technology, Atlanta, Georgia, United States of 160

America; Michael Green, Division of Parasitic Diseases, CDC, Atlanta, Georgia, United 161

States of America; Philippe Guerin, WorldWide Antimalarial Resistance Network, Oxford 162

University, United Kingdom; Manuel Hetzel, Swiss Tropical and Public Health Institute, 163

Switzerland; Harparkash Kaur, London School of Hygiene and Tropical Medicine, United 164

Kingdom; Mayfong Mayxay, Laos–Oxford, Mahosot Hospital – Wellcome Trust Unit, 165

Vientiane, Lao People’s Democratic Republic; Michael Parker, Ethox Centre, Oxford 166

University, United Kingdom; Souly Phanouvong, United States Pharmacopeia, Rockville, 167

MD, United States of America; Raffaella Ravinetto, Institute of Tropical Medicine Antwerp, 168

Belgium; Shunmay Yeung, London School of Hygiene and Tropical Medicine, United 169

Kingdom. 170

171


page 7

GLOSSARY OF TERMS AND ABBREVIATIONS 172

173

ACT artemisinin-based combination therapy 174

API active pharmaceutical ingredient 175

AQ antimalarial quality 176

BP British Pharmacopoeia 177

CODFIN Counterfeit Drug Forensic Investigation Network 178

FDC fixed-dose combination 179

GDP good distribution practices 180

GMP good manufacturing practices 181

INN International Nonproprietary Name 182

INTERPOL International Criminal Police Organization 183

LIC/MIC low- and middle-income countries 184

LQAS lot quality assurance sampling 185

MEDQUARG Medicine Quality Assessment Reporting Guidelines 186

NDRA national drug regulatory authority 187

NGO nongovernmental organization 188

OOS out-of-specification products 189

Ph.Int. The International Pharmacopoeia 190

QAMSA Survey of the Quality of Selected Antimalarial Medicines 191

QCL quality control laboratory 192

REC Research Ethics Committee 193

SSFFC substandard/spurious/falsely-labelled/falsified/counterfeit medical 194

products 195

UNODC United Nations Office on Drugs and Crime 196

USP United States Pharmacopeia 197

WHO World Health Organization 198

WWARN WorldWide Antimalarial Resistance Network 199

200


page 8

EXECUTIVE SUMMARY 201

202

The quality of medicines and medical products has been a neglected factor in global attempts 203

to improve access to effective health care. Data on the quality of medicines, if properly 204

collected, interpreted and used, have much to contribute to improving the effectiveness of 205

interventions to improve the quality of medicines and medical products and advocacy for their 206

importance. The different methods by which data on the quality of medicines and medical 207

devices can be collected, and their potential pitfalls and advantages, are not yet common 208

knowledge among public health scientists, officials and pharmacy inspectors. 209

These guidelines set out the necessary steps to conduct medicine quality surveys with 210

discussion of statistically valid sampling methodologies, the different options to measure poor 211

quality medicine frequency and suggestions on how to assess and interpret the data obtained. 212

This report also discusses the advantages and disadvantages of each of the potential sampling 213

techniques and the ethical challenges to survey teams when collecting medicines in the field. 214

Accurate estimates of the frequency of poor-quality medicines in the market are urgently 215

needed as a requirement for valid comparisons between countries and regions and to measure 216

the impact of interventions. Convenience sampling is important and useful for detecting poor-217

quality medicines but cannot be used to give accurate estimates of frequency, for which some 218

form of random sampling is required. Transparent and consistent reporting can provide robust 219

evidence to assist in improving medicine quality by informing interventions. 220

221

INTRODUCTION AND BACKGROUND 222

223

In the last decade many studies have highlighted the public health impact of poor-quality 224

medicines and medical products, especially in low- and middle-income countries (LIC/MIC). 225

Poor-quality medicines, both falsified and substandard, of the majority of commonly used 226

medicines have recently been described (1–9). However, it is not a new problem with, for 227

example, reports of falsified Cinchona bark in the 1600s (10) and falsified quinine in the 228

1800s (11-13). Deaths from adulterated sulphonamide medicines in the United states of 229

America (USA) in the 1930s were responsible for the strengthening of the Food and Drug 230

Administration (14). Recent reports include falsified meningitis vaccines in Niger, where 231


page 9

60 000 inoculated people were unprotected (15) and poor-quality paracetamol syrups 232

containing the antifreeze diethylene glycol, resulting in hundreds of children suffering kidney 233

failure and death in south Asia, central America and Africa (16). Such problems continue, for 234

example, in 2012, ~125 people died in Pakistan, after taking isosorbide mononitrate that 235

contained toxic amounts of the antimalarial drug pyrimethamine (17) and fakes of the 236

anticancer drug bevacizumab (Avastin) circulating in ~22 US states (18). Deliberate 237

falsification of medicines is widespread, but poor manufacturing and insufficient regulation, 238

without deliberate attempt to deceive, are linked to poor-quality substandard medicines 239

entering the supply chain. These medicines may go unnoticed because of lack of acute 240

toxicity or obvious treatment failure but they can still cause avoidable morbidity and mortality 241

as they often contain subtherapeutic amounts of active pharmaceutical ingredients (APIs). 242

Despite having been a significant long-standing public health problem, the quality of global 243

medicine supply has received remarkably little attention. Much effort and finance have been 244

expended on trying to optimize the treatment of diseases and improving access, but such 245

investment is lost if the medicines patients take are of poor quality. Medicines are 246

fundamental to strategies for effective reduction of mortality and morbidity and good quality 247

is essential for ensuring their efficacy and safety. Therefore, quality assurance and quality 248

control of medicines throughout the manufacturing and distribution system, from 249

manufacturing of pharmaceutical ingredients to medicines use by patients, is essential to 250

ensure optimal treatment. Unfortunately, according to the World Health Organization (WHO) 251

30% of the world´s national drug regulatory authorities (NDRAs) do not have functional 252

capacity (19) and in low-income countries (LICs) the NDRAs often lack sufficient financial 253

and human resources to carry out such controls in a stringent and comprehensive way. 254

There has been much discussion of the use of the term substandard/spurious/falsely-255

labelled/falsified/counterfeit (SSFFC) medicines in the Member State Mechanism (20, 21). 256

This term conflates medicines that are deliberately and fraudulently mislabelled 257

(spurious/falsely-labelled/falsified/counterfeit) with those that arise from errors and 258

negligence in factory production (substandard). Here we classify poor-quality medicines into 259

two main categories, falsified and substandard, based on earlier WHO definitions (Box 1, 260

WHO definitions). Falsified medicines are made with the intention to deceive patients and 261

health workers and are, by definition, the work of criminals. Public health should be the prime 262

consideration when defining poor-quality medicines and the term falsified has the advantage 263


page 10

of being devoid of trademark and intellectual property considerations of the term counterfeit 264

(22, 23). In contrast, substandard drugs are medicines produced by registered manufacturers 265

that do not meet the quality specifications due to error and negligence and not deliberate fraud. 266

However, if the negligence was gross or systematic, depending on the legal system of the 267

country concerned, criminal negligence may – and should – be invoked. In addition, 268

inadequate post-production storage conditions may result in degradation leading to formation 269

of decomposition products and reduced APIs. Both generic and innovative medicines have 270

been falsified or produced as substandard products. 271

Poor-quality medicines have far-reaching consequences ranging from prolonged sickness, 272

treatment failure, side effects, economic hardship and death. In addition, public confidence on 273

medicines, health-care systems and manufacturers may be diminished as a result of using 274

poor-quality medicines, leading to societies suffering major economic losses. Medicines with 275

sub-therapeutic amounts of active ingredients may contribute to drug resistance hampering 276

disease control and harming patients (4). 277

278

Although there is evidence that a significant proportion of medicines consumed, especially in 279

LIC/middle-income countries (MICs), are of poor quality (4, 8, 24), the quality of the data 280

describing the epidemiology of poor-quality medicines is poor (9), in part because of diverse, 281

unstandardized methodologies followed when collecting medicines in the supply chain. 282

Accurate estimates of the prevalence of poor-quality medicines are sparse, making 283

comparison of the results through space and time very difficult, impeding evidence-based 284

policy decisions and evaluations. There are also very few data on the geographical 285

distribution of poor-quality medicines in relation to population density, borders, disease 286

epidemiology, public versus private health facilities, trade routes and socioeconomic status 287

(25). 288

289

There has been little discussion on the most appropriate sampling and reporting strategies (25, 290

26), in contrast to much progress in other branches of medical science (27). The majority of 291

objective evidence on medicine quality derives from surveys of antimalarial medicines. 292

However, we hope that this discussion will inform surveys to determine the quality of other 293

classes of essential medicines. 294

295


page 11

Surveys providing information on the quality of medicines have been conducted by different 296

NDRAs, research groups, international organizations and nongovernmental organizations 297

(NGOs) (28–35). They are helpful to give us snapshots of the medicine quality situation but 298

have important limitations. The accuracy, reliability and generalizability of the data obtained 299

will depend on study design and especially whether the medicine collectors, consciously or 300

subconsciously, prefer to find (e.g. if it might result in a publication or additional funding), or 301

not find, poor-quality medicines (e.g. if it might cause embarrassment, panic, danger, or is 302

part of a larger criminal, political or economic agenda). Such factors may result in 303

underestimating or overestimating the frequency of outlets selling poor-quality medicines and 304

surveys may especially be prone to bias if inspectors are known by pharmacists or other 305

sellers. Adequate systems for management and supervision of field personnel are necessary 306

for data collection, which may not be available in rural areas. Funding is another limitation, 307

restricting the number of samples to be collected in the field, the techniques to be used for 308

analysis or the number of staff available to conduct the survey and analysis. The restriction of 309

the techniques to be used for analysis may further put the accuracy of the results at stake and 310

may result in overlooking important quality problems, e.g. poor bioavailability, presence of 311

impurities, cross-contamination or inter-batch variability. Furthermore, publication and 312

dissemination of the results appears to be relatively difficult despite its obvious and 313

immediate relevance to public health. 314

A summary of the necessary steps to conduct quality surveys is presented in this document, 315

with discussion of different statistically valid sampling techniques for market surveillance. 316

We discuss their advantages and disadvantages, how they can be performed and give 317

examples and standard operating procedures (SOPs) that can be adapted. Accurate estimates 318

of the frequency of poor-quality medicines in the market are urgently needed as a requirement 319

for valid comparisons between countries and regions and to understand time trends and the 320

efficacy of interventions. Transparent and consistent reporting will provide robust evidence to 321

assist in improving medicine quality by informing interventions. As highlighted in this report 322

there remain some uncertainties as to the correct techniques and more research is urgently 323

needed to inform optimum methodology. 324

325

326


page 12

Box 1. WHO definition of medicine quality 327

(N.B. Consistent with a trend over the last two years we use the term falsified and not 328

counterfeit) 329

• Counterfeit medicines “A counterfeit medicine is one which is deliberately and fraudulently 330

mislabelled with respect to identity and/or source. Counterfeiting can apply to both branded 331

and generic products and counterfeit products may include products with the correct 332

ingredients, wrong ingredients, without active ingredients, with insufficient quantity of active 333

ingredient or with fake packaging.” 334

• Substandard medicines “Substandard medicines (also called out-of-specification (OOS) 335

products) are genuine medicines produced by manufacturers authorized by the NDRA which 336

do not meet quality specifications set for them by national standards. Normally each medicine 337

that a manufacturer produces has to comply with quality standards and specifications. These 338

are reviewed and assessed by the NMRA before the product is authorized for marketing.” 339

World Health Organization (2009) What are substandard medicines? Available at: 340

http://www.who.int/medicines/services/counterfeit/faqs/06/en/index.html 341

342


page 13

INITIAL PLANNING 343

344

When it has been decided that more needs to be known about the quality of essential 345

medicines in a region (whether country, group of countries or area or community within a 346

country) the following should be considered. 347

348

5.1 What is already known about medicine quality? 349

350

Data on medicine quality are scattered and rarely available at one source. A list of sources and 351

websites of different organizations working on medicine quality is given in Annex 1. In 352

addition to searches in PubMed and Google Scholar, key sources are: 353

- WHO (http://www.who.int/medicines/services/counterfeit/en/index.html) 354

- USP MQDB database (http://www.usp.org/around-world/pqm-uspusaid/medicines-355

quality-database-mqdb) 356

- WWARN AQ Surveyor for antimalarials 357

(http://www.wwarn.org/resistance/surveyors/antimalarial-quality) 358

- ReMed (http://www.remed.org/) 359

- QUAMED (http://www.quamed.org/en/home.aspx) 360

- NDRAs and regional groupings of NDRAs (such as EU that may hold unpublished 361

data, http://ec.europa.eu/health/human-use/falsified_medicines/index_en.htm) 362

363

These data, along with any alerts of medicine quality or pharmacovigilance reports, will assist 364

deciding which medicines should be investigated. For much of the world and for most 365

medicines, there is very little information in the public domain and for these 366

regions/medicines discussions with pharmacists and NDRAs and those involved in treating 367

major public health problems may help prioritize surveys. 368

369

5.2 What registered and unregistered medicines are available in the survey area? 370

371


page 14

Attempts should be made to obtain lists of registered medicines from the ministry of health 372

and NDRA in a particular country. A medicine is registered in the NDRA when it meets the 373

required standards of quality, safety and efficacy in product specific premarketing 374

assessments (good manufacturing practices (GMP) inspection and product’s dossier 375

assessment). However, in many countries this information can be hard to access. It can be 376

much harder to obtain lists of unregistered medicines as by definition these should not be 377

present. As these are outside the NDRA regulated system they may contain a higher 378

frequency of poor-quality medicines – hence it is vital to consider including them in the 379

survey. Information about unregistered medicines may be obtained from previous surveys 380

conducted in the same area of the study or during the survey. 381

382

5.3 What is the distribution system or supply chain of the medicines? 383

384

The supply chain of a pharmaceutical product is the system of organizations, activities and 385

resources involved in moving a medicine from manufacturers of raw materials, including the 386

API, to patients. A supply chain of one product can consist of multiple manufacturing sites, 387

market warehouses, distribution centres, national or international wholesalers, retailers and 388

public and private outlets. 389

In order to have a good market surveillance system in place it is important to understand how 390

medicines are supplied within the country of study and how they reach patients. There is little 391

information in the public domain about how distribution systems vary between countries (36, 392

37). For antimalarials the ACTWatch reports are an important source of information 393

(http://www.actwatch.info/) and have demonstrated great diversity in the distribution chains 394

between countries. 395

396


page 15

5.4 Maps of pharmaceutical outlets 397

398

Pharmaceutical outlets, licensed and unlicensed, are defined as any point of sale or provision 399

of medicines for individual patients or other medicine providers. They are often the first 400

points of treatment seeking in LIC/MICs. 401

NDRAs, ministries of health, provincial health departments and health centres may be able to 402

provide lists of outlets of the area of study that will facilitate the planning of the survey. In 403

addition, international organizations such as ACTwatch (http://www.actwatch.info/) and 404

Population Services International (PSI) (http://www.psi.org/) are mapping and identifying the 405

exact location of health facilities, private clinics and pharmacies in many countries. For 406

example, the Thai Pharmaceutical System Research & Development Foundation (PhaRed) 407

(http://www.phared.org/home/index.php?lang=en) also has internet maps of pharmacies in 408

Thailand. The collection of medicines from unlicensed outlets may be linked to a preceding 409

shop census or household survey, especially when information on treatment-seeking 410

behaviour is investigated. 411

412

5.5 Information on health-seeking behaviour for the target pharmaceuticals? 413

414

In many countries the medicine market is heavily segmented with different markets for people 415

of different spending power and ethnicity. For example, the wealthier people may go to 416

pharmacies or private clinics, whilst the poorest go to grocery shops or street peddlers and 417

people of middle income may go to hospitals. There will also be brands of the same product at 418

different prices aimed at different market segments. It is therefore fundamental for the 419

accuracy and generalizability of the survey to understand where different categories of 420

patients tend to buy their medicines and what kind of product they buy. 421

Bearing the above information in mind, the key decision to inform the methodology of the 422

study is the main objective or question being asked. 423

424

OBJECTIVES 425

426


page 16

It is essential to set detailed objectives at the start of planning because all the activities and 427

requirements for the survey are derived from the objectives. Clearly defined objectives are 428

essential for setting up conditions for sampling and testing, which are reflected in the protocol 429

of the survey. 430

431

In order to make clear objectives the question(s) to be addressed in the survey should be 432

clearly formulated. Examples of common questions that are asked are given below. 433

434

1.1 Are there poor quality drugs in this area, border checkpoint, etc.? 435

436

For example, one may be concerned that there may be a quality problem with medicine x in 437

area y or medicine x being imported through checkpoint y. This is both the simplest and most 438

complicated question to answer. Simple, because one could visit a variety of outlets or inspect 439

a variety of shipments looking for poor-quality medicines based on local intelligence or 440

“hunches”. Complicated, because it is difficult to interpret the results – if one finds a poor-441

quality medicine, is it common or rare and if one does not find a poor-quality medicine, is this 442

because one did not look in the correct place or at a sufficient number of samples? 443

444

However, such an approach is vital for NDRAs in order to take enforcement action against 445

falsified and substandard medicines, for alerting the public and pharmaceutical industry and 446

for prompting surveys. They are analogous to pharmacovigilance case reports of new adverse 447

effects of medicines. 448

449


page 17

1.2 What are the proportion of poor-quality medicines being sold and/or proportion of 450

outlets selling poor-quality medicines in a particular geographical area? 451

452

For example, an alert may have been received from a customer or from the above Objective 453

(6.1) but the investigators would like to know how common these poor quality “medicines” 454

are. There are two main approaches – convenience and random surveys. In the convenience 455

sampling approach, samples are collected without specific guidance as to which outlets to 456

sample. Results are dependent on the collector´s choices, which are predisposed to bias and 457

cannot be associated with confidence intervals. In contrast, random sampling of sufficient 458

power is a more objective approach that provides accurate estimates of poor-quality 459

medicines or the proportion of outlets selling them in the study area, and confidence intervals 460

of this estimate. Random sampling requires larger sample sizes, greater complexity and hence 461

higher survey costs but should be the preferred methodology if resources allow. 462

When investigating the proportion of poor-quality medicines sold to patients, outlet sale 463

volumes should be considered to provide an accurate estimate of the prevalence. 464

465

1.3 Do the proportion of poor quality medicines or the proportion of outlets selling 466

poor-quality medicines exceed a predetermined level? 467

468

A major problem of conducting random surveys is their complexity and expense with large 469

sample sizes required. An alternative, simpler and less expensive approach, needing smaller 470

sample sizes than formal random sampling, uses lot quality assurance sampling (LQAS). 471

LQAS methodology is useful to determine whether a lot of goods meet desired predefined 472

specifications without having to inspect the entire lot. This cannot give an accurate estimate 473

of the prevalence of poor-quality medicines or outlets selling falsified medicines, but will 474

allow determination as to whether the proportion is below a certain percentage, e.g. 10%, in 475

the area sampled. 476

477


page 18

1.4 Has medicine quality changed for the better or worse in an area or for a medicine 478

or medicine group? 479

480

Repeated random surveys and prospective sentinel monitoring sites may be able to estimate 481

changes in medicine quality through time and evaluate the impact of interventions. However, 482

such prospective studies require that the sampling design is consistent through time. Sentinel 483

surveys using the same sites every time, suffer the disadvantage that outlet staff and their 484

distributors may become aware of the survey being conducted and change their behaviour 485

accordingly. This is particularly important when inspectors are known by the pharmacists or 486

outlet providers. Convenience sampling cannot be used to answer this specific question as 487

results are not statistically comparable through space and time. 488

Other related questions include: 489

- what are the supply chains by which poor-quality medicines are distributed and the 490

market segments they serve? 491

- How does the proportion of poor-quality medicines vary at different levels of the 492

regulated and informal distribution chains? 493

- How does the proportion of poor-quality medicines vary by medicines produced 494

within country versus those imported from different countries? 495

- How does the proportion of poor-quality medicines vary by registered and non-496

registered products? 497

498

The detailed objectives for a quality survey should be formulated in a way which makes 499

possible to identify the following. 500

501

1.5 Areas to be sampled 502

503

Samples should not usually be collected only in the capital city as situations in rural and 504

suburban areas often differ. Different geographical areas should be sampled unless the survey 505

expressly justifiably targets one area. A distinction can be made between rural and urban 506

outlets but with the definitions of these two categories clearly stated. 507


page 19

Medicines can also be collected in an area where the population is at risk of a disease (38). 508

Locations can be classified by variables such as incidence/prevalence of the disease for which 509

the medicines are produced for, or by population density or by degree of urbanization. For 510

example, malaria is endemic in Thailand only at international borders and antimalarials 511

should be collected in these targeted areas. 512

In the design of a survey with any sampling method it is assumed that the 513

prevalence/incidence of a disease/income level or other risk factors are homogenous across 514

the geographical area to be studied. Small surveys focused on particular areas using the same 515

methodology will obtain comparable results. 516

517

For example, the areas to be sampled in a survey can be stratified by: 518

- income level of the population to be sampled; 519

- areas within countries at high risk of the disease for which the target medication is 520

indicated; 521

- areas within countries at high risk of poor-quality medicines; 522

- rural or urban areas or stratified by this variable; 523

- areas stratified by different distribution systems; 524

- countries or groups of countries. Sampling a diversity of countries in a region, 525

according to the same survey protocol, gives broader pictures of quality of medicines 526

in the region and enables comparisons between countries, but tend to be very 527

expensive, difficult to manage and unlikely to be representative of the situation. 528

529

530

Countries with political instability may not allow the team to sample some areas and study 531

sponsors may disallow such research on safety grounds. 532

533


page 20

1.6 Products to be surveyed 534

535

Public-health considerations and the potential public-health impact of different poor-quality 536

medicines should be key guides for which medicines to sample. In resource-poor settings 537

medicines sampled should be on the country’s essential medicine list or for key public-health 538

problems. The dosage form and strengths to be sampled should be prioritized using evidence 539

as to what patients frequently use, e.g. if patients overwhelmingly use doxycycline capsules 540

these should be prioritized rather than doxycycline tablets. The number of dosage units or 541

multidose packages of selected medicines to be collected should be agreed in advance to 542

allow: 543

- conducting the planned tests; 544

- confirmatory testing for those found to OOS; 545

- retention samples; 546

- forensic analysis if evidence of falsification is obtained. 547

548

It is reasonable to formulate the objectives and organize the survey in a way to focus on 549

medicines: 550

- for which inferior quality has documented actual or potential serious implications for 551

the health of patients, such as treatment failures; 552

- used in large volumes; 553

- susceptible to quality deterioration (unstable active ingredients, liquid dosage forms); 554

- for which quality problems have been described and more information on frequency, 555

distribution and time trends are required; 556

- for which resistance issues are pertinent. 557

558

As outlets may have more than one brand of a particular medicine available decisions should 559

be made before sampling as to which to request and, if a selection has to be made, this should 560

be done by random selection to avoid bias from seller or investigator. Mystery shoppers 561

requesting a very specific brand or product may alert sellers of the survey being conducted 562

and this should be taken into consideration. However, such an approach may be required if 563

evidence suggests that only one brand of an essential medicine is afflicted by falsification or 564

substandard production. If the objectives of the survey require a wide picture of the quality of 565


page 21

medicines available on the market, samples produced by as many manufacturers as possible 566

should be collected. However, in order to collect samples from more manufacturers it may be 567

necessary to visit more sampling sites. 568

In general, not more than five different types of products (identified by active ingredient/s and 569

dosage form) should usually be included in one survey; otherwise the project would be 570

difficult to manage, unless the study is sufficiently funded and staffed to allow a larger 571

number of types of product. The number of different types of products to be collected should 572

also be a function of the number of different brands that can be expected, which might inflate 573

the number of samples to be tested (less diversity would be expected in public-health facilities 574

than in the private sector). 575

Collectors should be mindful of the stock of sampled products in outlets, and potential 576

difficulties of replenishment of sampled medicines through the supply chain, so as not to 577

jeopardize the availability of these medicines to patients. If there is risk of product shortage 578

after sampling replacement of the sampled amount should be arranged, immediately after the 579

survey, or, less desirably, collection of that particular product in that outlet omitted. 580

Different categories of products to be surveyed may include: 581

- by active ingredients (e.g. artemisinin-based combination therapy (ACTs), sulfadoxine/ 582

pyrimethamine); 583

- by classification of medicines into therapeutic groups; 584

- by formulation for the same drug (e.g. combination therapy sold in loose formulation, 585

blister pack, fixed dose); 586

- by target groups (e.g. paediatric patients); 587

- by stated manufacturer or distributor; 588

- by specific programme under which they are supplied (e.g. a Global Fund grant, national 589

disease control programme); 590

- as commonly used; 591

- registered vs non-registered medicines. 592

593

Information on commonly used drugs may be derived from household surveys with 594

investigation of treatment-seeking behaviour. Collaborating with other sectors, such as 595

national disease control programmes, may help to identify if such questions have been 596

previously included in other surveys. 597


page 22

598

1.7 Medicine preparation type 599

600

Medicine quality surveys have especially focused on the quality of tablets/capsules 601

formulations and there is relatively little information about the quality of intravenous or 602

intramuscular dosage forms, syrups or fixed-dose combination therapies, such as for 603

tuberculosis and HIV. The study of diverse formulations is of great importance as different 604

dosage forms may be used according to the degree of severity of a disease or target groups, 605

such as children. For example, tablets are normally used for the treatment of uncomplicated 606

malaria whilst severe malaria is treated with parenteral formulations. There are very few data 607

on the quality of parenteral antimalarial formulations despite their extreme importance in the 608

treatment of severe malaria (9). 609

610

The dosage form of the medicines to be collected in a survey should be clearly specified in the 611

study plan. It is important to ensure that laboratory procedures for the different dosage forms 612

are available, especially for syrups, etc., which may be difficult to analyse. In addition, some 613

parenteral formulations and vaccines may require temperature-controlled storage and 614

transport conditions after collection. 615

616

1.8 Timing of the survey 617

618

It is important to take into consideration when the samples will be collected. There is no 619

information on seasonal changes in frequency of poor-quality medicines, but these may occur. 620

For example, it is possible that falsified antimalarials are more common during the malaria 621

season. Access to outlets in rural areas may be impaired in the rainy season due to factors 622

such as floods and landslides. 623

624

1.9 Types of sample collection sites 625

626

Outlets vary greatly in type both within and between countries, and may be classified by 627

NDRAs according to the countries’ medicine regulatory laws and by their mobility, number 628


page 23

and training of staff. To allow comparison between regions/countries with different outlet 629

classification terminologies, outlets could be classed as suggested in the Methodology section, 630

point 8.1, Terminology. 631

Samples collected close to the point of sale to patients in the supply chain (see section 5.3) 632

may be influenced by distribution and storage conditions, such as high temperatures and being 633

sold past expiry date. However, these will be closest in terms of quality to the medicines that 634

patients actually take. There is remarkably little information on degradation of medicines in 635

supply chains of variable quality in terms of good pharmaceutical practices (GPP) (39). If 636

medicine quality problems suggestive of degradation are found in the distal parts of supply 637

chains, collection of additional samples of the same product at higher levels of the chain, e.g. 638

in central wholesalers and medical stores, may help shed light on the importance of poor 639

supply chain management. 640

641

Sampling may be performed at the most proximal parts of the distribution chain such as at 642

manufacturers, wholesalers, importers and central medical stores. There is great diversity in 643

the structure and function of such supply chains between and within countries and 644

understanding of this will be vital in planning surveys (36). Samples collected at proximal 645

points of entry to the market should be less affected by the environment they encounter in 646

distribution and storage, but are relatively removed from the actual quality of medicines that 647

patients will have access to and take. Sampling at this point of the supply chain has the 648

advantage of detecting quality issues before the products reach patients and corrective actions 649

may be more easily put in place if the results are quickly available. The sampling process in 650

countries where there are only a few distributors from which all outlets obtain medicines will 651

be very different from the process in countries with multiple independent distributors selling 652

directly to small outlets. 653

Lists of the locations of the target outlets, e.g. licensed and/or unlicensed outlets, in the 654

selected geographically defined areas are required. It may not be possible to map the “territory” 655

of itinerant medicine sellers but an estimate of what proportion of the total number of sellers 656

are itinerant could be used to generate a sensitivity analysis. Another option would be to 657

include a list of the market places where itinerant vendors are known to buy their medicines. 658

Furthermore, sampling just the public sector outlets for medicine X will be of little use in 659

assessing the public health impact of medicine quality if patients in this community most 660


page 24

frequently obtain medicine X from private sector pharmacies or from informal mobile traders 661

(40). Sample collection in public health facilities would mean overt collection in almost all 662

the cases (see section 9). 663

NDRA personnel experienced in national distribution chain structure and function should be 664

asked to advise on sample distribution from the site, transport and storage conditions, 665

especially where quality deterioration could occur. 666

Procurement and treatment outlets in selected areas may be very diverse in terms of 667

regulatory status, size, stock, staffing and level within the distribution chain. The level of 668

sampling within the supply chain will determine the influence of distribution and storage 669

conditions on medicines’ quality which will be more important the more distal in the supply 670

chain is sampled. 671

Different types of collection sites may include: 672

- pharmacy outlets of different categories; 673

- public sector or public-health facilities (national) procurement centres, referral hospitals, 674

health centres or sub-health centres, district hospitals, health posts, community health 675

workers; 676

- private sector: private hospitals, clinics, polyclinics, cabinets, pharmacies; 677

- confessional health facilities or NGOs; 678

- informal providers: drug stores, general retailers, itinerant providers; 679

- wholesalers, at points of entry to the market; 680

- at a specific manufacturer. 681

682

Any type of sampling will suffer from biases especially when inspectors are known by 683

providers, for example, if: 684

- some of the locations to be sampled are not easily accessed by a normal shopper, i.e. 685

collection sites in the public sector, then buying medicines from such locations becomes 686

exceptional and may not be representative of the population; 687

- sampling from wholesalers or points of entry to the market; staff at these facilities may 688

have some knowledge about poor-quality medicines that may lead to bias; 689

- estimating the number of expired medicines in the market may only be possible if 690

medicines are collected using covert sampling methodology (see section 9). 691


page 25

692

1.10 Definitions used to determine quality 693

694

Different countries use different definitions for poor-quality medicines, hampering 695

comparison of the results between countries (22, 23, 41). It is important to specify which 696

definitions will be used to determine quality at the start of the survey. 697

698

SURVEY MANAGEMENT AND TIME FRAME 699

700

Ideally each country involved (ministry of health/NDRA) should agree with the survey plans 701

before it commences. Responsibilities and tasks of persons having key roles in survey 702

organization (e.g. survey coordinator, focal persons in individual countries) should be 703

identified at the beginning of the survey and should include those with responsibility for 704

monitoring the conduct of the survey, for performing chemical and packaging analysis, 705

processing results and preparation of the final report. 706

Issues such as the utilization of results and their public availability should be clearly 707

understood by responsible authorities and all the parties involved in the survey from the 708

beginning. A publication plan including authorship of any papers to be submitted for peer-709

reviewed publication and a distribution list of those to whom the report will be disseminated 710

should be agreed at the beginning of the survey. 711

It is vital to organize a meeting with participation of focal persons involved in sample 712

collection to explain and discuss the project, survey protocol and provide detailed instructions. 713

The survey plan with key milestones and organizations/persons responsible for individual 714

parts should be predefined together with an estimated timeframe (Table 1). It is necessary to 715

plan the financial resources expected for the whole survey before beginning. It usually costs 716

more than anticipated! 717

Table 1. Timeframe and responsible officers for the survey (as an example) 718

Activity Timeframe Responsible officers

Selection of geographical area and medicines to

be surveyed


page 26

Agreement with authority (ies)

Selection of testing laboratory (ies)

Preparation of testing protocol in agreement with

testing laboratory (ies)

Meeting held with focal points from the targeted

geographical area to discuss the survey protocol

Preparation of detailed national sampling plans

Preparation and pilot test of data collection tools

Training and supervision of personnel

Collection of samples and transport to testing

laboratory (ies)

Sample scanning and logging into databases

Testing of samples

Compilation of results

Data analysis

Report drafting

Meeting held with the participating countries to

discuss the results and the actions needed

Report finalization

Publication and distribution of the results

719

METHODOLOGY 720

721

The main purpose of statistically valid objective sampling is to collect samples of medicines 722

from outlets in various sectors to accurately reflect what consumers use in a given site and/or 723

geographic area (26). Objective sampling is important to draw inferences about the entire 724

"population", whether this refers to the "population of shops" or the "population of patients". 725

It is vital to clearly state the question or aim of the survey/study before designing the 726

methodology (see section 6). It is also critical to maintain the covert nature of the sampling, if 727

that has been decided as the methodology, in order to avoid bias when collecting the 728

medicines in the field. 729


page 27

A detailed sampling plan should be prepared prior to the collection of dosage units and it 730

should specify the types of medicines, types of outlets, the area to be covered and who will 731

collect them and how. In order to conduct all the necessary quality assays the number of 732

samples per product to be collected and analysed should be agreed with the quality control 733

laboratory. If possible this should be drafted in cooperation with the respective NDRA(s). The 734

sampling plan will identify: 735

- sampling frame; 736

- sampling unit; 737

- sample size; 738

- outlets; 739

- sample or package; 740

- dosage unit. 741

742

In the following sections the terminology and details of sampling techniques for outlets and 743

sampling techniques for medicines are discussed. For discussion of further methodological 744

considerations not covered in this chapter please refer to Annex 2.4. 745

746

8.1 Terminology 747

748

• Sampling plan: a detailed outline of how many dosage units will be collected from how 749

many medicines, and from what outlets and in which region. Table 3 lists specific 750

elements to be addressed in the sampling plan. 751

752

[Note from the Secretariat: The authors have been contacted regarding the table as to 753

whether it is missing or whether the text refers to Table 6 on page 39.] 754

755

• Sampling frame: the up-to-date list of outlets where the dosage units will be obtained and 756

the set of medicines from which the dosage units are to be collected. The sampling frame 757

should coincide with the “population” of interest. There may be cases where the sampling 758

frame may differ from the actual “population” depending on the objectives of the survey, 759

e.g., if only registered outlets are sampled then unregistered outlets will not be represented. 760


page 28

Results from surveys may not be comparable with other studies that sampled a different 761

population of interest. 762

763

• Sampling unit: the sampling units for analysis may be the outlets and/or the medicines 764

sold from them or both. The distinction is important as, for example, an area may have 765

one outlet selling 50% of the poor-quality medicine(s) or 10 outlets each selling 5% of the 766

poor-quality medicines. Weighting may be required based on the number of treatments 767

dispensed per outlet, which could be derived from household surveys or sales volumes 768

declared by the outlets. Surveys have usually estimated the proportion of poor-quality 769

medicines in outlets and not the proportion of shops selling poor-quality medicines. By 770

using the proportion of medicine outlets selling poor-quality essential medicines as the 771

unit of observation and a standardized, randomized sampling procedure of sufficient 772

sample size, it would be possible to map distribution of medicine quality and allow 773

comparisons through time. Such a procedure will require knowledge of the expected 774

underlying prevalence to calculate the sample size. If this knowledge is unavailable the 775

worst-case scenario must be assumed, necessitating larger sample size. Large sampling 776

sizes may alert providers to those procuring for the survey but samples of sufficient size 777

are vital to give reliable results. 778

779

• Sample size: the outlets selected from the sampling frame or the dosage units within the 780

medicines drawn from the population. A minimum sample size of outlets is needed with 781

medicines in stock to provide information of the changes in medicine quality. 782

783

• Outlets: a place where medicines are sold or distributed. Outlets vary greatly in type and 784

can be classed as public (government), registered private for profit (e.g. private 785

pharmacies, supermarkets), private not for profit (e.g. mission hospitals, NGOs) and 786

informal (e.g. kiosks, street vendors, grocery shops). Sampling should usually be 787

performed in both the public and private sectors as well as in the "informal market" (i.e. 788

outside the approved distribution chain), i.e include both licensed and unlicensed outlets. 789

790

• Sample: a sample means an individual medicine (active ingredient/s by International Non-791

Proprietary Names (INNs), dosage form, strength and brand) collected at one outlet. All 792

dosage units (or medicine units) of one sample must be of the same brand and batch – if 793


page 29

they are the same medicine brand but from different batches they should be regarded as 794

multiple samples. 795

796

• Dosage unit: a single individual unit of a medicine (e.g. a single tablet or a single package 797

containing powder or a vial of medicine) that is the smallest dosage unit that can be 798

analysed. Dosage units come in diverse forms, including tablets, capsules, oral 799

solutions/suspensions, powder for oral solution/suspension and injections or powders for 800

injections. Tablets and capsules may come packaged in a box and be presented in blister 801

packs (with batch number, manufacturer and expiration date) or in unlabelled plastic 802

(locally repacked) bags coming from large-sized boxes. Vials may also be sold 803

individually sometimes without the appropriate leaflet instructions or paired diluent. 804

Surveys may sample nationally-registered medicines and should help to find the presence 805

of unregistered ones. The number of dosage units per sample to be collected should be 806

specified at the beginning of the survey. 807

808

8.2 Sampling designs (convenience, random, lot quality assurance sampling (LQAS), 809

sentinel) 810

Convenience sampling 811

Convenience surveys, in which samples are collected without specific guidance as to which 812

outlets to sample, have been the predominant techniques used and may provide evidence to 813

support legal action in police and NDRA investigations. As the name implies convenience 814

surveys are simple and relatively inexpensive and do not require complete lists of outlets in 815

defined areas, which may be difficult to obtain, especially for unlicensed or mobile outlets. 816

However, they are inherently prone to biases and are potentially misleading and should only 817

be considered if other sampling methods cannot be implemented. The results of convenience 818

sampling are crucially dependent on the collector’s choice of outlets and any derived 819

prevalence estimates cannot be generalized to other areas, even within the same country. If 820

those conducting sampling were particularly interested in finding poor-quality or quality 821

medicines, this may influence their selection of outlets to sample. Changes in the prevalence 822

of poor-quality medicines, and the outlets selling them, cannot be reliably interpreted over 823

time if obtained from convenience sampling as changes may simply represent sampling 824

artifact. Convenience sampling techniques are now avoided in medical research because of 825

these inherent disadvantages. Nevertheless convenience surveys may provide the initial signal 826


page 30

of a pharmaceutical problem (analogous to case reports of adverse effects to a drug) and may 827

be useful during routine post-marketing medicine quality monitoring, particularly when 828

doubts are raised about the quality of a specific medicine in a particular area. Whenever 829

convenience sampling is used, how the sites were identified and the proportion of the outlets 830

this represents should be reported. If convenience sampling does indicate a drug quality 831

problem, more objective methods should be used in subsequent surveys. If convenience 832

surveys do not demonstrate a problem one should bear in mind that this may be a false 833

negative result. It is important to emphasize the limitations of this technique in reports and 834

scientific papers and when discussing with journalists. 835

836

Examples of convenience sampling include surveys conducted in Africa (24, 34, 42) 837

and South-East Asia (28, 43). 838

839

Random sampling 840

Random sampling is an objective technique that, with sufficient sample size, will give reliable 841

estimates of the prevalence of outlets selling poor-quality medicines with confidence intervals. 842

Stratified sampling to adjust for potential differences in income levels of the population of 843

interest or sales size of each location should be considered. It can also include geographical, 844

trade and socioeconomic variables, such as rural versus urban, private versus public outlets 845

and one geographic area versus another. Stratification requires adjustment of the sample size 846

calculation (see Annex 2.4 for more on further statistical issues related to methodology). 847

Sampling proportional to population size (or number of medicine outlets) will be more 848

efficient compared to simple random sampling. It is important that the randomization 849

procedure uses formal random number tables or simple statistical software and not 850

pseudorandom techniques. Comparisons with subsequent estimates using the sampling design 851

should be valid and will allow the evaluation of interventions. The disadvantages of random 852

sampling are the large sample sizes needed and the additional costs in labour and time. In 853

addition, it is important to recognize that a random survey will only produce reliable and 854

useful information if the sampling frame and actual within-outlet sampling is in concordance 855

with the primary aims of the study. For example, a random survey of the quality of medicine 856

in the private sector when most patients obtain this medicine in the public sector would not be 857

useful, nor would a random survey using overt shoppers for a medicine that the outlet staff 858

know they should not be selling. 859


page 31

An example of this technique is described in a stratified random sample of the quality of 860

antimalarial medicines in the Lao People’s Democratic Republic (44). The random sampling 861

of medicine outlets was accomplished by the collectors in the field making numbered lists of 862

outlets and telephoning a central location giving the total number of outlets found. The central 863

location staff then used random number tables to tell the field team which outlets to sample. 864

865

Other examples of random surveys are from Nigeria, Uganda and the United Republic of 866

Tanzania (29, 31, 32, 45). 867

868

Lot quality assurance sampling 869

LQAS can be used to determine whether the prevalence of outlets selling poor-quality 870

medicines exceeds a certain threshold and may be a useful methodology to be used in routine 871

surveillance. There has been almost no discussion as to what proportion of outlets selling 872

poor-quality medicines should be regarded as unacceptable (25). Ideally there should be zero-873

tolerance for outlets selling poor-quality medicines, as even a 1% prevalence of such 874

medicines for potentially fatal diseases, such as malaria, tuberculosis and HIV, is disastrous 875

for individual patients. 876

LQAS is designed to allow determination as to whether a batch, or lot, of goods meet desired 877

specifications without having to inspect the entire lot. Thus, the sample size in LQAS is 878

defined as the number of outlets (“goods”) that are selected for each site (“lot”) and the only 879

outcome is that the site is “acceptable” or “unacceptable”. Setting the level of risk taken by 880

not inspecting each and every item enables the investigator to accept or reject an entire lot 881

after inspecting a randomly selected sample of items. Therefore, the sample size in LQAS is 882

based on defined threshhold values that classify good and bad outcomes and the probability of 883

error that the investigators are willing to tolerate. For example, an area in which 10% or more 884

of the outlets sell poor-quality medicines may be considered a “bad” situation since the risk of 885

buying poor-quality medicines will be high, whereas 5% or less may be considered a “good” 886

situation since the risk of buying poor-quality medicines will be lower (25, 46). 887

Acceptable probabilities of error must be specified; the risk of accepting a “bad” lot 888

(“consumer risk”) and the risk of not accepting a “good” lot (“provider risk”). These risks are 889

commonly referred to as Type I (alpha) and Type II (beta) errors, respectively. The former is 890


page 32

often set to 0.05. This means that if the null hypothesis (that the site has fewer outlets selling 891

poor-quality medicines than the specified value) is true, there is a 5% chance that a site with 892

an unacceptable proportion of outlets selling poor-quality medicines will be “accepted” or go 893

undetected. In general, Type 1 risk is set lower than the Type 2 risk. 894

Once the threshold values and probabilities of error have been considered a sample size and 895

decision value can be obtained. The decision value is the number of outlets selling poor-896

quality medicines that need to be found before an area is considered unacceptable. LQAS still 897

requires random sampling and has the disadvantage that it does not estimate an exact 898

prevalence but the advantage of requiring smaller sample sizes (see example in Annex 2.1b). 899

Sampling can stop once the number of outlets selling poor-quality medicine is exceeded, 900

greatly reducing sampling time and costs (47). 901

Random-sampling LQAS is probably the most efficient and accurate initial sampling 902

procedure. As LQAS will only provide a binary result formal random sampling will be 903

required to examine longitudinal changes in the prevalence of poor-quality medicines 904

accurately. It can also be useful when the exact prevalence of poor-quality medicines is 905

known as a way to monitor the situation. There is only one published LQAS medicine 906

quality survey (46). More experience is needed so that comparisons of costs and human effort 907

needed against formal random sampling can be made. A LQAS toolkit is available from the 908

Liverpool School of Tropical Medicine (48). Other references for this topic are (49–52). 909

910

Sentinel site monitoring 911

Sentinel site monitoring, in which the quality of medicines in a particular locality is followed 912

through time, is used in many countries (53). There has been no consensus as to whether these 913

sites should be chosen on the basis of potentially important variables such as rural versus 914

urban and private versus public outlets, or using what sampling methodology (i.e. 915

convenience or random samples or LQAS). Although the power of sentinel site monitoring 916

resides in allowing longitudinal changes to be followed in one place, it suffers from the 917

disadvantage that shop owners may soon realize that they are being sampled, change their 918

behaviour accordingly and thus are no longer representative of the population. 919

Each sentinel site should arrange an appropriate schedule that takes into account the logistics 920

and availability of resources (26). 921


page 33

For examples of sample size calculation with LQAS and random sampling see Annex 2.1. 922

923

8.3 Number of dosage units to be collected 924

925

A problematic and unresolved issue is the number of dosage units that should be collected. 926

The uniformity of dosage units’ assay measures the individual content of the active ingredient 927

in each capsule or tablet and compares it with the amount of active ingredient claimed on the 928

label of the medicine. 929

Current recommendations from pharmacopoeias 930

- The United States Pharmacopeia (USP) recommends that 30 units of each medicine 931

should be collected. Unless otherwise specified in the individual monograph the 932

requirements for dosage uniformity are met if the amount of API in each of the 10 dosage 933

units as determined from their weight variation or their content uniformity method lies 934

within the range of 85.0% to 115.0% of the label claim and the relative standard deviation 935

is less than or equal to 6.0%. If one unit is outside the range of 85.0% to 115.0% of label 936

claim and no unit is outside the range of 75.0% to 125.0% of label claim, or if the relative 937

standard deviation is greater than 6.0%, or if both conditions prevail, USP recommends 938

testing 20 additional units. The requirements are then met if not more than one unit of the 939

30 is outside the range of 85.0% to 115.0% of label claim and no unit is outside the range 940

of 75.0% to 125.0% of label claim and the relative standard deviation of the 30 dosage 941

units does not exceed 7.8% (54). 942

943

- The procedures of The International Pharmacopoeia (Ph.Int.) (2006) recommend 944

determining the amount of API in each of 10 units using the analytical method specified 945

in the individual monograph. For uniformity of content for single-dose preparations assay 946

the Ph.Int. states: “This test is only to be applied where the declared quantity of active 947

ingredient is 5% or less of the total formulation. For tablets and powders for injection: 948

each single unit should contain within ±15% of the average amount of active ingredient. 949

However if one individual unit deviates by more than ±15% but is within ±25% of the 950

average amount of the active ingredient, examine further 20 units drawn from the same 951


page 34

original sample as the first 10 units. The preparation of the test complies only if the 952

amount of active ingredient found in no more than one out of 30 units deviates by more 953

than ±15% of the average amount. None should deviate by more than ±25% of the 954

average amount” (55). 955

956

The number of units needed per assay and the acceptance criteria for a sample to be compliant 957

for the Ph.Int. (55) is described in Table 4. 958

959

Table 4. International Pharmacopoeia specifications and acceptance criteria per assay. 960

Assay No. of units to

be tested

Acceptance criteria

Tablet and powder for injection

Q=Quantity

Capsules, oral powders,

suppositories

Visual inspection 20 Tablets are smooth, undamaged. Capsules are smooth,

undamaged and of uniform

colour.

Uniformity of

content for single

dose preparations

10 units initially

and 20 more if

one tablet does

not comply

One out of 30 units deviates more than

±15% of the API and none by more

than ±25% of the average amount

Three out of 30 units deviate

by more than ±15% and none

by more than ±25% of the

average amount

Uniformity of mass

for single-dose

preparations

20 - For less than 80 mg of average

mass: (minimum of 18 tablets ±

10% of deviation, maximum of 2

tablets ± 20% of the deviation)

- 80 mg to 250 mg of average mass:

(minimum of 18 tablets ± 7.5% of

deviation, maximum of 2 tablets ±

15% of the deviation)

- More than 250 mg of average

mass: (minimum of 18 tablets ±

5% of deviation, maximum of 2

tablets ± 10% of the deviation)

- The mass of each capsule

should be ± 10% of the

average mass

- On less than 300 mg of

net mass of capsule

content (minimum of 18

tablets ± 10% of

deviation, maximum of 2

tablets ± 20% of the

deviation)

- 300 mg and over:

(minimum of 18 tablets ±

7.5% of deviation,

maximum of 2 tablets ±

15% of the deviation)


page 35

961

Such a sample sizes gives enough dosage units to determine identity and accurate estimates of 962

content of active ingredients, dissolution and degradation. However, many outlets in the rural 963

tropics do not have 30 dosage units of a particular medicine (25), especially expensive brands, 964

and so a request for such a large quantity is likely to suggest to the outlet owner that the buyer 965

is not an ordinary shopper (56). In addition, outlet staff may dispense units of the same 966

medicine with different batch numbers, expired dates, fewer number of units or in a different 967

container to the original packaging. Ideally unopened packages should be collected. An 968

alternative would be to sample key medicines at the level of the wholesaler where requests for 969

large quantities would be routine. USP guidelines on dosage-unit sampling are designed for 970

analyses that would withstand examination in a court of law, such as would be needed by a 971

NDRA to press legal charges. However, scientific research studies have not included legal 972

chain of custody procedures and hence cannot usually be used as evidence, but may 973

precipitate NDRA legal investigations with appropriate protocols (57). 974

For mystery shopper surveys in which samples failing analysis would prompt NDRA 975

investigations smaller sample size of dosage units may be appropriate. Mystery shoppers 976

could normally collect one or two full treatment courses without raising too much attention 977

from shop keepers. Thirty units for tablet or capsule forms of single drug preparation would 978

allow assessment of identity, content of active ingredients, dissolution and degradation. When 979

fewer units are found (i.e. informal sector) samples with less than 30 units may be included 980

(Table 5). Newly deployed non-destructive techniques such as high throughput mass 981

Disintegration 6 - All six tablets should disintegrate

to pass the test

- All six capsules should

disintegrate to pass the

test

Dissolution 6 units initially

and 18 more if

one unit does not

comply

- 6 tablets tested and each unit is not

less than Q +5%

- Average of 12 units is equal to or

greater than Q and no unit is less

than Q-15%

- Average of 24 units is equal to or

greater than Q; not more than 2

units are less than Q-15%; no unit

is less than Q-25%


page 36

spectrometry, Raman and IR spectroscopy can perform assays for identity and active 982

ingredient content, followed by assays for disintegration and dissolution, allowing for smaller 983

sample sizes. 984

The amount of work in collecting and analysing medicine quality surveys is frequently 985

underestimated and discussions of what is realistic given human and financial resources and 986

the intrinsic difficulties of studies (e.g. remote travel, sample transport) are essential. The 987

maximum number of samples to be collected should be set before sample collection starts (see 988

Annex 2.3). If more countries are involved maximum number of samples should be set for 989

each country. 990

The following general rules can be used, if not justified otherwise. 991

992


page 37

Table 5. Minimum number of samples to be collected when conducting medicine quality 993

surveys. 994

Dosage form Packaging

(typical)

Number of dosage units or multidose packs per

batch

Overt sampling or

wholesaler sampling

Mystery shopper

methodology

Tablets and capsules

(immediate-/modified-release,

chewable, dispersible, etc.)

Blisters, co-blisters,

bottles, securitainers

Approx. 100 units

(e.g. 5 packs of 20 units

3 packs of 30 units

3 packs of 40 units

2 packs of 60 units

1 pack of 90 units and

above)

In case of co-packaged

products approx. 100 units

shall be collected from each

medicine.

30 units;

fewer units may be

collected from informal

sector (5–10 units)

Multidose oral

solutions/suspensions, powder for

oral solution/suspension and

injections or powders for injections

Multidose bottles and

vials

6 containers of 60 ml/

100 ml

3 containers of 240 ml

3 containers of 60 ml/

100 ml

2 containers of 240 ml

Single-dose powders for oral

solution/suspension and single-

dose injections or powders for

injections

Sachets and single-dose

bottles, vials and

ampoules

15 units (20 units if dose is

below 50 mg)

10 units

995

996

997


page 38

8.4 Sampling techniques for the dosage units within medicine samples 998

999

How many dosage units within a medicine sample need to be analysed is difficult and 1000

contentious with few data to inform discussions. There is very little evidence on the 1001

interdosage unit % API or dissolution variability within a sample of poor-quality medicines 1002

with only one publication, that we are aware of, that discusses this issue (58). We suggest 1003

that sampling strategies should assume that variability is high but more research is urgently 1004

needed to inform this question. 1005

The aim of sampling dosage units is to estimate the proportion of poor quality drugs in a 1006

specific outlet based on a relatively small subset of units being tested. The outlet is assessed 1007

based on sampling of dosage units from that outlet. Usually the package of units is obtained 1008

and then a number of units from that package are tested and a conclusion is drawn regarding 1009

the package and therefore the outlet. 1010

The assumption behind this approach is that a fixed but unknown proportion of the population 1011

(sampling frame) contains poor-quality drugs. Therefore the proportion of poor-quality units 1012

in a sample (a subset of units from the population) can estimate the proportion of poor-quality 1013

medicines for the whole population. 1014

The actual proportion of poor-quality medicines will vary between samples but is determined 1015

by a discrete probability distribution that can be used to calculate the size of the sample to be 1016

drawn so that meaningful inferences can be made about the whole population. The two 1017

distributions which describe a number of positive (poor-quality) units in a randomly drawn 1018

sample are the binomial and hypergeometric distributions and they are described in more 1019

detail in Annex 2.2 and 2.3. 1020

Other resources describing the representative medicine sampling can be found (59–61). 1021

A national sampling plan is included in Annex 3. 1022

1023


page 39

Table 6. [or Table 3?] Key decisions to be made in planning medicine quality survey 1024

Variable Possibilities Example

1 Country? Atlantis

2 Region(s) within

country?

e.g, province, district, city, rural

versus urban

- To estimate the proportion of poor-

quality medicines in a malaria endemic

district

3 Type(s) of outlet? Licensed and/or unlicensed;

private sector and/or public

sector; wholesalers; street vendors

- to compare pharmaceutical quality of

domestically produced medicines with

imported ones in order to adopt

appropriate regulatory actions and adjust

pharmaceutical policy in the country

4 Timing of the survey Seasonality of disease of concern Sampling should be conducted in the

malaria season

5 Medicine(s) to be

sampled

All essential medicines; registered

and/or unregistered; antibiotics;

products labelled as made by

company xxxx; antimalarials,

anti-TB

- Survey of the quality of antituberculosis

medicines circulating in selected newly

independent states of the former Soviet

Union

6 Medicine preparation

type

Tablets/capsules,

ampoules/syrups, unlabelled

plastic bags

Tablets; fixed-dose combinations and

ampoules

7 Sampling unit and

number of samples/unit

USP; Ph.Int.;

British Pharmacopoeia, European

Pharmacopoeia

Minimum of 217 outlets to be sampled

and 30 units per sample (USP 2013)

8 Sampling technique(s)

and sample size for

outlet

Convenience; random; LQAS;

sentinel;

Stratified cluster design

9 Sampling technique(s)

and sample size for units

Hypergeometric; binomial;

Bayesian, LQAS

Hypergeometric

10 Collection method(s) Sampling by identified NDRA or

research staff; mystery shopper

Mystery shopper vs overt sampling

methodology (interviews)

11 With NDRA There may not be a NDRA With malaria control programme, TB

control programme

12 Sample labelling Country of collection, year, serial number

13 Sample storage and

transport

Appropriate temperature,

transport with temp data loggers

Cold chain for specific medicines

14 Choice of laboratory WHO prequalified; external

quality assurance

National quality control laboratory

15 Choice of chemical

analysis technique

Identification, dissolution,

impurities

HPLC, Raman spectra, dissolution…

1025


page 40

16 Choice of packaging

analysis

Visual inspection, comparative packaging

analysis, non-comparative packaging

analysis, CD3 device , patient information

leaflet analysis

17 Contact point (as mentioned on pp 57 for

contacting during laboratory

analysis of dosage units)

1026

SAMPLE COLLECTION 1027

1028

Overt sampling versus mystery shopper methodology 1029

1030

Procedures for who should do the sampling will be dependent on the regulatory status of the 1031

medicine(s) in question, and what is known about the knowledge of sellers – whether he/she 1032

knows that the outlet is selling poor-quality medicines and understands the health, legal and 1033

ethical implications. Many outlets in countries with weak medicines regulation sell unlicensed 1034

medicines, which even if of good quality, may make outlet staff suspicious and anxious about 1035

investigations. If the seller knows or is concerned that his/her stock contains illegal or poor-1036

quality medicines and that the buyer is potentially linked to the NDRA, this is likely to greatly 1037

influence what medicines are offered for sale (62). However, if outlet staff are anxious to 1038

avoid poor-quality medicines, overt sampling with feedback would allow more data to be 1039

collected on poor-quality medicines and their risk factors and lead to direct improvement in 1040

the medicine supply. Overt sampling may be the only possible method in some circumstances, 1041

such as if samples are collected where people are seen first by clinicians. 1042

Mystery shoppers (63) are the appropriate collectors in most circumstances. Sampling should 1043

usually be performed by nationals of the country concerned, although there may be some 1044

situations, such as with concerns that migrant workers may take inferior medicines, where this 1045

would not be applicable. It may not be safe for people living in the same wider community to 1046

act as purchasers. On the contrary, in some remote rural locations, it would be difficult for 1047

someone who is not local to request medicines and it would cause suspicion. Mystery 1048

shoppers should try to mimic “normal shoppers” for the community in which the outlet is 1049

located and should dress and behave appropriately. For example, they should avoid signs or 1050

speech suggesting that they come from an urban elite if the sampling is conducted in rural 1051

areas and the aim of the survey is to target what medicines rural people obtain. They should 1052


page 41

use a standard scenario, stating, for example, that they are visiting from another part of the 1053

country and would like some medicines for disease X for reason Y for a stereotyped patient Z. 1054

If many dosage units of a large number of medicines are requested the seller may become 1055

suspicious and the medicines collected may not reflect what is actually available at those 1056

outlets (62). 1057

An additional concern is that in many resource-poor countries the medicine market is heavily 1058

segmented with different markets for people of different spending power and ethnicity. For 1059

example, the least poor may go to pharmacies or private clinics, whilst the poorest go to 1060

grocery shops or street peddlers and people of middle income may go to hospitals. Even 1061

within a single outlet there will often be several different brands of the same product at 1062

different prices aimed at different market segments with, for example, a “local” brand (least 1063

expensive), a non-local and non-European (middling expensive) and a “Western” brand 1064

(expensive). Therefore, what the mystery shopper will collect may depend partly on how 1065

wealthy the shopkeeper thinks the shopper is. This also emphasizes the importance of 1066

sampling guided by the volume consumed rather than that displayed. Any weighting that 1067

requires sales volumes cannot be done if mystery shoppers are used, unless these are preceded 1068

or followed by an outlet census. Expired drugs may be more likely to be sold to mystery 1069

shoppers than overt buyers. 1070

In overt sampling methodology data should be collected using a structured questionnaire and 1071

the location of each outlet recorded with a hand-held global positioning system (GPS) unit. 1072

The questionnaire should collect data on outlet identification, outlet characteristics such as 1073

their supply chains, provider knowledge, availability of medicines and stock outs. Opinions 1074

and sell-reported practice with regards to medicines or falsified medication should also be 1075

included. A standard reporting form should be used to record details of the medicines such as 1076

collection date and site, cost and type, location and type and size of outlet (Annex 4). 1077

Data collection tools should be translated in the language of the collectors, back-translated to 1078

English, piloted and revised before implementing the final version. 1079

If the samples are collected through the mystery shopper methodology, the interaction 1080

between the mystery shopper and outlet staff should be documented on a debriefing form. 1081

When the mystery shopper returns from each outlet the study coordinator should perform a 1082

debriefing and transcribe the reported interaction with translation as appropriate. Translations 1083

should use a meaning-based translation method rather than a literal or interpretative approach. 1084


page 42

The original text will remain in the document with the translation made beneath each section. 1085

Sections of text will be double-checked for accuracy of translation by other members of the 1086

field team. This part of the work may need to be conducted in collaboration with social 1087

scientists. 1088

Standard operating procedures (SOP) documents should be created for the methodology 1089

followed. An example of the SOPs for overt sampling and mystery shopper methodology can 1090

be found in Annex 5 and Annex 6, respectively. 1091

Examples of overt sampling include (35) and examples of mystery-shopper methodology 1092

include (44). 1093

1094

Survey team and procedures for collection 1095

1096

Ideally the survey team should be composed of a survey coordinator, inspectors or staff 1097

trained for collection of medicines, assistants and, if necessary, a driver. Cooperation with the 1098

WHO country office staff in the respective country may be useful. The focal person for each 1099

country will arrange for training of collectors to be familiar with the project, survey protocol, 1100

national sampling plan and instructions for collection of samples. Staff from the NDRA and 1101

the different national control programmes, such as the malaria or tuberculosis control 1102

programmes, may provide a useful insight into the survey planning. However, if the NDRA is 1103

highly politicized involving them may bias the entire process, since they may not accept a 1104

“bad” result from any sampling. 1105

The following specifications for sample collection should be included/adhered to: 1106

- the number of units per sample and number of batches to be collected from each 1107

collection site for each selected medicine as indicated in the sampling plan; 1108

- there should not be a mixed batch collection per sample; all units of one sample must 1109

be of the same batch number; 1110

- the medicine samples should not be taken out of the original primary packaging and 1111

outer containers (though removal from large secondary packs is appropriate). 1112

Containers such as bottles and vials should not be opened; 1113


page 43

- samples collected should have at least six months remaining to expiry to allow time 1114

for chemical analysis; 1115

- the medicine labels and package leaflets should not be removed or damaged; 1116

- individual samples should be recorded using the sample collection form for each 1117

sample (example in Annex 4). Whenever the required information is not available it 1118

should be indicated in the appropriate space on the sample collection form, where any 1119

abnormalities should also be recorded; 1120

- each sample will be identified by a unique sample code, defined on the sample 1121

collection form template and specified on all the original packages belonging to the 1122

respective sample. The sample collection form and all packages belonging to one 1123

sample will be kept together (e.g. blisters inserted in a dedicated zip log plastic bag 1124

marked with the appropriate sample code and trade name of the product); 1125

- manufacturer’s batch certificates of analysis will be collected with samples, if 1126

available, and kept with the sample collection form; 1127

- the samples should be collected and kept under controlled conditions. Samples should 1128

be stored in in zip lock plastic bags labelled with their unique identification codes 1129

marked with permanent markers. For large surveys bar-code systems would be very 1130

helpful and reduce errors. Samples should be kept in foam boxes or closed containers, 1131

protected from light, excessive moisture or dryness. Prepacked desiccants may be used 1132

for sample conservation. Storage temperatures should be 15 °C to 25 °C, unless 1133

otherwise indicated on the sample packaging. Samples should not be subjected to 1134

temperatures above 30 °C or frozen. Certain heat-sensitive medicines should be kept 1135

in between 4 °C and 8 °C (64); 1136

1137

- the time period, within which samples should be collected in the countries and the 1138

deadline for sending the last sample to the testing laboratory, should be clearly 1139

indicated and followed. 1140

1141


page 44

ETHICAL CONSIDERATIONS ON QUALITY OF MEDICINES SURVEYS 1142

1143

There are important ethical considerations in investigating medicine/medical product quality, 1144

usually important in other branches of medical research, which have received scant attention. 1145

We are aware of no specific opinion or consensus guidance (Tabernero et al (in preparation)). 1146

The production and trade in falsified medicines is a criminal activity with the details and 1147

penalties of the offence depending on the laws and regulations of the country concerned. The 1148

production and trade in substandard medicines may also be an offence, usually against 1149

national medicine regulatory legislation and may, in some jurisdictions, be criminal 1150

negligence, i.e. allowing otherwise avoidable poor-quality medicines to be accessible to 1151

patients due to negligence. In addition to the law we are also informally regulated by moral 1152

obligations as members of the community as well as by our profession (65, 66). Criminality 1153

and negligence are thankfully rarely the subjects of medical research, but they may arise in 1154

different research studies and thus raise important issues when they emerge, for legal and 1155

ethical codes. 1156

Here we raise some of the issues that those involved in collecting medicines for quality 1157

testing should consider. 1158

The World Medical Association developed the Declaration of Helsinki as a statement of 1159

ethical principles in medical research involving human subjects. The document has been 1160

revised a number of times since its first publication in 1964; at the time of writing the most 1161

recent revision was in October 2013 (67). Other important information on medical research 1162

ethics can be found in Nuffield Council on Bioethics (68); Belmont Report (69); CIOMS 1163

Guidelines (70); report from the UNICEF/UNDP/World Bank/WHO Special Programme for 1164

Research and Training in Tropical Diseases (TDR) (71); Emanuel et al. 2004 (72), the 1165

American Anthropological Association (66) and the American Sociological Association (65) 1166

websites. In addition, WHO good clinical practices (GCP) (73) and the International 1167

conference on Harmonisation (ICH) (74) translate such ethical principles in methodological 1168

good practices for clinical and medical research. These guidelines are inclusive to subjects 1169

working on data management that have direct impact on data quality and reliability of the 1170

results. Even if quality surveys do not involve the participation of human subjects in medical 1171

experiments their results may inform national policies and have a direct impact on individuals 1172


page 45

and their public health. Therefore, it is of paramount importance that the people involved in 1173

such studies comply with appropriate methodological and ethical standards. 1174

Medicines may be collected by diverse groups of investigators, from NDRA inspectors, 1175

agents of pharmaceutical companies, journalists and international organizations, to scientists 1176

from higher education institutes. All such groups should behave ethically and consider the 1177

ethical obligations and consequences of their work, especially towards the risk to those 1178

surveyed, those surveying and the public health of the population. This discussion focuses on 1179

academics conducting research but the principles also apply to other groups. 1180

1181

General ethical issues 1182

1183

The ethical issues related to medicine collection for quality testing will depend on the context 1184

and country in which they are to be carried out. It is important to carefully consider ethical 1185

issues in favour of carrying out the surveys as well as at the ethical “challenges”. The issues 1186

in favour of such work include: 1187

- individuals and communities are harmed by taking medicines that are ineffective or toxic, 1188

or both; 1189

- there are important public health benefits to be gained from having accurate information 1190

about the frequency of poor-quality medicines on the market in particular locations; 1191

- provision of information to NDRAs that will help identify those who sell and/or 1192

manufacture falsified medicines in subsequent NDRA/police investigations; 1193

- provide information on medicine quality problems that will alert NDRAs and 1194

manufacturers to problems and facilitate technical support for improvements or corrective 1195

actions in manufacturing medicine/medicinal product quality; 1196

- the benefits to communities and health professionals of raising awareness about poor-1197

quality medication/medical products and how to reduce their risk, in particular by 1198

allowing their identification and corrective actions before they reach the patients; 1199

- the prevention and/or reduction of the already scarce financial resources that would have 1200

been spent on poor-quality medicines to be used for other health-care benefits. 1201

1202


page 46

Nonetheless, despite the benefits the carrying out of such surveys can also present a number 1203

of important ethical challenges and risks. Risks to surveyors and the surveyed are discussed 1204

below. Another neglected issue is that if investigators, at small, remoter outlets, sample the 1205

entire stock of a particular medicine, the accessibility of medicines to patiemts may be 1206

reduced, e.g. if a survey removed all antimalarials in a malarious area in a distal limb of the 1207

supply chain this would be unethical. If this is likely, as a way of addressing this risk, the 1208

survey team should provide quality-assured medicines in exchange at the time of overt 1209

purchase or return soon after mystery-shopper sampling to provide these. Furthermore, health 1210

providers in the government sector may not be willing to give medicines to the survey team 1211

because their drugs are only used for the patients with evidence of having a certain disease 1212

(i.e. positive malaria blood smear) otherwise they may be in trouble if they are audited. 1213

In addition, the survey methodology and/or conduct may pose important ethical challenges 1214

due to insufficient sample size, incorrect sampling or the analysis of a partial set of samples. 1215

This may lead to inaccurate results and policy recommendations and eventually harm the 1216

community. Just as in other branches of medical research, scientific and methodological 1217

soundness is a fundamental prerequisite for the ethical soundness of medicine quality surveys. 1218

Careful methodological and ethical considerations should therefore guide the preparation, the 1219

conduct and the dissemination of results of each survey. 1220

If a survey identifies manufacturers/suppliers as sources of poor-quality medicines to the 1221

outlets, caution should be taken when disseminating this information publicly. On the one 1222

hand, the information may expose the research team to legal action from the concerned 1223

company(ies), whilst not going public will result in not protecting the community from 1224

preventable harm. Such difficulties may be especially difficult if sources are government or 1225

international agencies (5). Legal advice should be sought but the default position should be to 1226

release public health information. These potential situations should be envisaged in the initial 1227

protocol and strategies predefined in advance, when possible in strict collaboration with the 1228

NDRAs, other relevant national stakeholders (e.g. national malaria programme). 1229

1230

1231


page 47

Risks to the survey team members? 1232

1233

The production of and trade in falsified medicines is a criminal activity. Hence, the safety of 1234

those acting as mystery shoppers or involved in surveys of medicine quality must be 1235

considered. There are only a few examples in the public domain of people working on 1236

medicine quality being attacked or threatened. The most extreme was the apparent attempted 1237

murder of the head of the Nigerian Food and Drug Aministration, presumably related to her 1238

work (75). Formal overt inspection of the outlets by NDRA inspectors is likely to be the best 1239

option when the risk to the mystery shoppers is thought to be too significant. However, if the 1240

NDRA is poorly functioning or corrupt this will not be possible (76). 1241

The level of risk acceptable by the survey team, the responsibilities of institutions and 1242

supervisors and adequate training and support should be discussed with the NDRA, police and 1243

the collection team members and their institutions and a risk assessment performed. If 1244

mystery shoppers are members of the community this may present particular problems if they 1245

are recognized as participating in the survey. Therefore, appropriate risk assessments and 1246

discussion of training, confidentiality, security and insurance should be considered. Clear 1247

guidelines appropriate to local conditions need be developed and clearly explained to all those 1248

involved in this work. 1249

More widely, safety concerns of the research team will have to be addressed if it becomes 1250

known that surveys are being carried out by particular institutions or individuals – this 1251

information may be accessed within a few minutes on the internet. The publication of results 1252

may lead to reputational damages to specific individuals or groups (e.g. losing confidence in 1253

all the manufacturers from a given country, because poor-quality medicines from that given 1254

country have been found). Care should be taken in avoiding the identification of mystery 1255

shoppers in survey reports and publications. 1256

1257

What is the risk to those surveyed? 1258

1259

There is very little information as to what proportion of medicine providers sell poor-quality 1260

medicines, at different levels of the health-care system and how many know that the 1261

medicines that they are selling are poor quality (56, 77). Clearly they have a moral and 1262


page 48

ethical responsibility to ensure that what they sell is good quality in cases where they are able 1263

to assess the reliability of supply channels and tell the differences between a genuine and a 1264

falsified medicine. Unfortunately, in much of the world they do not have the training or 1265

facilities to allow this. Risks include: 1266

- if overt, especially NDRA/police, inspections are noted in the community and raise 1267

suspicions of the quality of medicines sold in an outlet; the outlets may lose income by the 1268

community shopping elsewhere even though the medicines supplied are good quality; 1269

- innocent outlet staff and owners selling poor-quality medicines (e.g. those who bought 1270

drugs in good faith from suppliers of poor-quality medicines) may risk harm. Suppliers, 1271

irrespective of their knowledge of the quality of their products, may be at risk of losing 1272

income (possibly losing the business and bankruptcy) and perhaps being attacked or their 1273

medicines being seized by the community, as has been described (78); 1274

- if staff in outlets selling falsified medicines inform or are thought to have informed on 1275

those trading in such medicines, to NDRA or research staff, they may be at risk. 1276

1277

For these reasons, data and samples in a research study should be anonymized when sent for 1278

analysis, devoid of linkage to named outlets until results are obtained and, after appropriate 1279

risk assessments, discussed with the NDRA. Caution should be exercised if there is suspicion 1280

of corruption and poor supervision of the supply chain. Ethical challenges should be discussed 1281

with the ethical committee concerned and, if appropriate, with local pharmacy associations. 1282

1283

Do medicine quality surveys require review by a research ethics committee? 1284

1285

There is no consensus as to whether ethical approval is required for surveys and different 1286

bodies have different opinions. The scope of the WHO Standards and Operational Guidance 1287

for Ethics Review of Health-Related Research with Human Participants (79) includes that “all 1288

health-related research ethics committees, whether they review biomedical, social science, 1289

epidemiological, operational, or health systems research”. This is consistent with the position 1290

that medicine quality surveys should be subject to careful ethical scrutiny, but does not state 1291

this. Where possible and realistic those who are conducting such work should have access to 1292

ethics support and training. 1293


page 49

It should also be noted that regulatory requirements for ethical clearance vary between 1294

different countries and that they must be carefully checked by the research team before 1295

planning each specific survey. 1296

The aim of the survey can be primarily an essential element of good public health practice, to 1297

improve programmes, reduce harm to patients and enforce drug quality standards or to 1298

generate new scientific knowledge. It may therefore represent routine surveillance and not 1299

research. We suggest that ethical scrutiny should be considered if (a) the survey goes beyond 1300

routine surveillance, (b) and/or if risk assessment suggests that there are significant risks to 1301

either surveyors or the surveyed. The advice of an independent medical ethicist may be 1302

valuable. The following steps should be considered: 1303

- ethical review should be considered before the survey, either with discussion of the need 1304

for ethical review with the Research Ethics Committees or a formal submission in the 1305

study country(ies) (80). Periodical and final study reports, should be submitted according 1306

to Research Ethics Committee requirements; 1307

1308

- there may be circumstances in which ethical review would not be appropriate, for example, 1309

if the Research Ethics Committees might be compromised or might feel that embarrassing 1310

data on poor-quality medicines should be suppressed. Should this be the case, the choice 1311

to proceed without ethical review should be documented and justified; 1312

- given the field nature of the surveys, they will usually be carried out in cooperation with 1313

the concerned NDRA(s). Should this not be appropriate (76) or possible, the decision to 1314

proceed without NDRAs involvement and approval should be documented and justified. 1315

1316

If risk assessment suggests that there are significant risks to either surveyors or the surveyed 1317

and/or if the results of the study are expected to be potentially sensitive, it is recommended 1318

that in addition to discussing the need, or otherwise, of applying for ethical clearance with the 1319

concerned Research Ethics Committee(s) in the country(ies) of study, a specially established 1320

independent ethics and legal advisory board where all the pertinent skills and expertise would 1321

be represented (e.g. analytical, legal, ethical, anthropological …) may be appointed to advise 1322

on the survey design and conduct, the communication of results and the management of any 1323

problems/incidents occurring at any stage of the research. The board should be selected with 1324

due consideration for expertise and for potential conflicts of interest/full independence, e.g. 1325


page 50

none of the members should feel that embarrassing data on poor-quality medicines should be 1326

suppressed and that public health should be the prime guide to decision making. 1327

Those conducting the work should have access to ethical support and training in research 1328

ethics. If resources are limited this should focus at least on skills required for specific tasks 1329

(e.g. privacy and confidentiality issues for those collecting the information). If the country has 1330

no available ethics committee, the Science and Ethical Review Group (SERG) at WHO (81, 1331

82) has developed guidelines for the establishment of scientific and ethical review bodies. 1332

1333

Should outlet staff be told that they are being part of a study? 1334

1335

There is no consensus as to whether it is necessary to tell outlet staff that they are being 1336

sampled or whether a covert, mystery-shopper, approach is more appropriate. On the one 1337

hand an overt approach may allow the investigator to learn more about the product sample, 1338

distribution systems for medicines and what the outlet staff knows. This might be appropriate 1339

if the survey is carried out as routine surveillance. If it is decided that outlet staff should be 1340

informed the aims, methods and source of funding of the research should be explained and 1341

consent requested. This is important as participants should understand the implications for 1342

enrolment in the study and the right to abstain and withdraw. This consent-seeking process 1343

should take into consideration the cultural context and community consultation or 1344

engagement prior to the course of the research. The researchers should also establish 1345

appropriate measures to ensure confidentiality of outlet staff. 1346

On the other hand, a mystery-shopper approach increases the probability that the samples 1347

obtained will represent what such a shopper would be sold in real life. This is clearly 1348

appropriate for NDRA investigations and should also be aimed for during research surveys, 1349

especially for unregistered medicines that may otherwise be concealed. If the seller knows or 1350

is concerned that his/her stock contains illegal or poor-quality medicines and that the buyer is 1351

potentially linked to the NDRA or a medicine quality survey, this may falsely improve the 1352

quality of the medicines offered for sale (63). However, if outlet staff is aware and anxious to 1353

avoid falsified medicines, open sampling with feedback would allow more data to be 1354

collected on the risk factors and facilitate direct improvement in the medicine supply by 1355

positively engaging with pharmaceutical retailers. 1356


page 51

A third compromise strategy is for outlets to be sampled by mystery shoppers after being 1357

informed on a prior visit that the survey will happen at some point in the future and requesting 1358

consent for this future undisclosed visit. However, this may influence the seller´s medicine 1359

selling behaviour, resulting in an inaccurate picture of the situation. Further research to 1360

examine the various methods for addressing these issues and to inform the development of 1361

models of good practice is needed. 1362

1363

What to do with the information once it has been gathered? 1364

1365

It is unethical for the details of the stated manufacturer and other sample details of poor-1366

quality medicines not to be reported to the NDRA and the WHO Rapid Alert System (83). 1367

There is no point in performing the survey if this is not done. We recommend that poor-1368

quality samples are reported in a timely manner to the NDRA for administrative and/or 1369

enforcement actions and should also be made available in the public domain, e.g. in an open 1370

access, peer-reviewed scientific paper and NDRA website. 1371

1372

There is no consensus on what should be done, how and when, in terms of release of public 1373

information if suspicious medicines/medical products are found. Clearly public health should 1374

be the primary priority, but there remains tension between commercial interests, the need to 1375

investigate (and investigator safety) and the need to act quickly to safeguard public health. It 1376

will also be important to ensure that public engagement is performed in a way to reduce the 1377

risk that patients stop taking genuine medicines and to reduce the risk of public loss of faith in 1378

medicines or the health-care system (Newton et al, submitted). 1379

1380

Results and findings should also be used for advocacy and educational purposes. When there 1381

is evidence of international illicit trade in poor-quality medicines, the WHO Rapid Alert 1382

System and INTERPOL are able to link countries NDRAs and police forces. 1383

1384

The antimalarial quality group at the WorldWide Antimalarial Resistance Network has been 1385

compiling reports of antimalarial quality globally and developed the antimalarial quality 1386

surveyor, an online visualization tool to try to understand the epidemiology of poor quality 1387

medicines (9, 84). 1388


page 52

Samples should be described with the term “stated” before the manufacturer’s name and 1389

address, etc., as for falsified medicines they, by definition, do not originate from the 1390

manufacturer stated on the packet. It would be important not to falsely accuse a genuine 1391

manufacturer, making good quality of products that has been targeted by counterfeiters, of 1392

counterfeiting. Investigators may need to request legal advice to ensure that samples are 1393

appropriately described. 1394

1395

STORAGE AND TRANSPORT OF SAMPLES 1396

1397

Storage and transport of the sample should be done according to the requirements set out in 1398

paragraph 2.3 of WHO Guidelines for Sampling of Pharmaceutical Products and related 1399

materials (85): 1400

- the samples should be kept in original packaging and under storage conditions as 1401

specified on the sample label; 1402

- for transport all samples should be packaged adequately and transported in such a way as 1403

to avoid breakage and contamination during transport. Any residual space in the 1404

container should be filled with a suitable material. Where required the cold chain should 1405

be retained during storage and transport; 1406

- data temperature loggers should be included within shipments to document adequate 1407

temperature in prolonged transit after data download; 1408

- a covering letter, the copies of a sample collection form and, if available, copies of the 1409

manufacturer’s batch certificate of analysis should accompany the samples; 1410

- samples with the accompanying documents should be sent to the assigned testing 1411

laboratory by a courier service. For each shipment it should be clearly indicated that 1412

samples are sent for laboratory testing purposes only, will not be used on humans or 1413

animals, have no commercial value and will not be placed on the market. The NDRA of 1414

the country of the laboratory may be able to assist with permissions for importation; 1415

- the laboratory should be informed of the shipment, with the tracking number as provided 1416

by the courier service, to be able to follow the shipment and collection as soon as 1417

possible; 1418


page 53

- copies of sample collection forms and, if available, copies of manufacturer’s batch 1419

certificates of analysis should also be sent to the survey coordinator/person preparing the 1420

report on the survey. 1421

1422

The USP developed the Rapid Assessment tool as a guideline for “Rapid Assessment of 1423

Medicines Quality Assurance and Medicines Quality Control” (86). 1424

1425

1426

TECHNIQUES TO DETERMINE QUALITY AND LABORATORY FACILITIES 1427

1428

An appropriate laboratory has to be selected for testing. Preferably a prequalified laboratory 1429

should be used (see the list of WHO-prequalified laboratories at www.who.int/prequal). 1430

Should such a laboratory not be available or should it not have sufficient capacity then 1431

another laboratory, for which evidence of reliability is available, should be chosen and the 1432

choice should be explained/justified in the study protocol, ethical/regulatory submissions, 1433

reports and publications (87). Many countries do not have a fully functioning quality control 1434

laboratory and should consider making arrangements with a laboratory abroad. 1435

The appropriate arrangement with the laboratory has to be made in advance, both to plan 1436

together that enough units and financial resources are available for carrying out all the tests 1437

needed to obtain meaningful and reliable results, and to give the laboratory time to be ready 1438

for testing (find the appropriate time slots, purchase necessary materials and standards …). 1439

The request for testing should be in line with WHO guideline Considerations for requesting 1440

analysis of drug samples (88), and no sample should be sent before such an arrangement is 1441

made. A contract with the laboratory should be concluded in advance to allow the laboratory 1442

to prepare for testing. 1443

A detailed address of the laboratory selected for the survey should be provided to the 1444

sampling organization. If there is more than one laboratory involved, clearly specify products 1445

which should be sent to each laboratory, for what assays and provide evidence that the 1446

laboratories will work according to the same standards, to make results comparable across 1447

different laboratories, products and countries. 1448

The laboratory will normally start testing only when all the samples are received. Therefore it 1449

is important to set and adhere to the deadline for sending to the testing laboratory. 1450


page 54

1451

Assays to be conducted 1452

1453

Laboratory testing of all collected samples will be performed according to the testing protocol, 1454

which is a part of the survey protocol and has to be agreed with the testing laboratory(ies) 1455

according to the pharmacopoeial monograph and specifications. Tests to be conducted depend 1456

on the objectives of the survey. For example of testing protocol see Annex 7. 1457

In principle, the following tests should be included in quality surveys of products on the 1458

market: 1459

• appearance; 1460

• identity; 1461

• assay; 1462

• related substances test; 1463

• dissolution or disintegration and uniformity of mass for (solid dosage forms, e.g. tablets, 1464

capsules); 1465

• pH value for liquid dosage forms (e.g. oral solutions, injections, powders for injection) 1466

• sterility of injectables; 1467

• packaging analysis of all packaging and labelling. 1468

1469

Some tests for injectable samples, such as for uniformity of content for single-dose dosage 1470

forms or sterility, are not normally included in quality surveys but we suggest that they should, 1471

being costly. The recent catastrophic fungal contamination of injectable methylprednisolone 1472

emphasizes the importance of being alert to this problem (2, 89). 1473

Screening methods do not provide a full picture of the quality of medicines and, as shown in 1474

Survey of the Quality of Selected Antimalarial Medicines study (QAMSA) (24), often 1475

underestimate non-compliant findings in comparison with laboratory testing. It is 1476

recommended to perform testing in a quality control laboratory, at the very least, for a random 1477

selection of those samples that pass screening assays and all those that fail. Simple screening 1478

methods are more suitable for screening of large number of samples in the field, e.g. to search 1479

for falsified medicines, but should always be validated by reference assays. Laboratory 1480


page 55

procedures should be blinded, i.e. packaging and labelling assessment should not be 1481

conducted by the same person who carries out the chemical tests. 1482

1483

Test methods and specifications 1484

1485

Test methods and specifications are in general selected according to the following rules: 1486

- preferably Ph.Int. monographs should be used, if available; 1487

- if no monograph exists in the Ph.Int. then BP or USP can be used; 1488

- if there is no pharmacopoeial monograph or the existing monographs do not provide for 1489

desired tests, a validated method of the laboratory or manufacturer's method, if available, 1490

should be used; 1491

- wherever possible the laboratory should perform internal and external quality assurance for 1492

the assays used 1493

1494

For the example of testing protocol used in the QAMSA study (24) for artemisinin-based 1495

combination therapy and sulfadoxine/pyrimethamine see Annex 7. 1496

When samples from different manufacturers are collected within a quality survey all samples 1497

containing the same combination of active ingredients are tested according to the same 1498

specification to enable comparison of samples from different manufacturers. This 1499

specification is then used to decide on compliance or non-compliance of tested samples for 1500

the purposes of this survey. It should be noted that individual manufacturers may use different 1501

specifications and different methods for testing of their products and these specifications and 1502

methods may be approved by regulatory authorities in different countries. Non-compliance 1503

with the specification selected for the survey does not necessarily imply non-compliance with 1504

the specifications approved in the country. But it indicates the need to look at the product and 1505

conditions of regulatory approval more closely and further actions should be considered by 1506

the respective NDRA. 1507

Medicine quality evaluation laboratories should have decision trees and flow diagrams for 1508

samples and those developed as part of the Counterfeit Drug Forensic Investigation Network 1509

project are available (90, 91). 1510


page 56

1511

Receipt and testing of samples by a testing laboratory 1512

1513

When samples are received, the testing laboratory will: 1514

- inspect each sample to ensure that the labelling is in conformance with the information 1515

contained in the sample collection form or test request; 1516

- store the samples according to the respective medicine requirement, including compliance 1517

with any cold chain requirements; 1518

- conduct quality testing in line with the testing protocol and in compliance with WHO 1519

standards recommended for quality control laboratories; 1520

- complete an analytical test report (Annex 8). If non-compliant results are found and 1521

confirmed after application of a laboratory OOS procedure, they have to be reported 1522

immediately to the study team or the contact point. The contact point should be specified in 1523

the sampling plan; 1524

- keep records of each sample, accompanying document/s and retention samples for at least 1525

5 five years with the analytical test requirements; 1526

- an electronic databank (e.g. scanned pictures or photographs of the medicines, such as of 1527

the tablets, packaging, and package leaflet) is recommended. 1528

1529

DATA MANAGEMENT, REPORTING AND PUBLICATION 1530

1531

Data obtained from the different collection tools such as sample forms, mystery-shopper 1532

debriefing forms or open sampling questionnaires should be double entered into a predesigned 1533

data entry database. Diverse software is available for data entry when conducting 1534

epidemiological studies e.g. Epiinfo (http://wwwn.cdc.gov/epiinfo/) or Epidata 1535

(http://www.epidata.dk/) are freely available from the internet, and should be preferred to 1536

Excel which is not a software specifically designed for data management. Preferably, and 1537

especially if the survey’s results may be sensitive, a password-controlled database with audit 1538

trail should be chosen. The information entered in the database should link the sample with 1539

subsequent analysis and interpretation data using the unique ID code. The database should be 1540

checked for coding errors and consistency. Two double-entry datasets should be compared 1541


page 57

and differences checked against the original data form. Data management and statistical 1542

analysis of the results can be conducted with, for example, STATA®

, (Stata Corp., College 1543

Station, TX, USA) or SPSS programmes. Personal identification of individuals who 1544

participated in the survey (buyers, sellers, etc.) should not be entered in the database with the 1545

use of codes only. 1546

The concerned NDRAs should be informed of any non-compliant results as soon as possible 1547

and investigations pursued, in line with regulatory practice and legislation, with the stated 1548

manufacturer. In addition the WHO Rapid Alert System should be informed (83) and if 1549

applicable batch recall procedures should be started. 1550

The analytical test reports of the testing laboratory(ies) will be provided to all NDRAs 1551

involved in the project. The outcomes of the project will be discussed by national authorities 1552

in a meeting and corrective actions, if necessary, will be recommended. Relevant regulatory 1553

measures in the different countries lie within the responsibility of the NDRA in collaboration 1554

with the police when applicable (falsified medicines or criminal negligence). Agreed 1555

outcomes and the report from the survey should be reviewed by the investigation team first to 1556

make it available for publication by WHO. The WHO alert system and INTERPOL (when 1557

applicable) should be informed when results are considered a public health emergency. The 1558

Medicine Quality Assessment Reporting Guidelines published in 2009 (25) provide an 1559

example checklist of items that should be included in reports of medicine quality (Table 7). 1560

1561


page 58

Table 7. Checklist of items suggested to be addressed in reports of surveys of medicine 1562

quality (adapted from (25)) 1563

Section and topic Item Description

Title/abstract/keywords 1 Identify the article as a study of medicine quality (recommended

MeSH headings “medicine quality, substandard, degraded,

counterfeit”)

Provide an abstract of what was done and what was found,

describing the main survey methods and chemical analysis

techniques used

Introduction 2 Summarize previous relevant drug quality information and

describe the drug regulatory environment

State specific objectives

Methods

Survey details 3 The timing and location of the survey; when samples collected and

when samples analysed

Definitions 4 The definitions of falsified, substandard and degraded medicines

used

Outlets 5 The type, including indices of size, of drug outlets sampled

Sampling design 6 Sampling design and sample size calculation

Type and number of dosage units purchased/outlet.

Samplers 7 Who carried out the sampling and methodology. What did the

collector say in buying the medicines? Definition of sampling

frame. Question of interest, assumptions, sampling method(s)

(including method of randomization if random sampling used)

Statistical methods 8 Describe the data analysis techniques used.

Ethical issues 9 Whether ethical/regulatory approval sought (if not, provide a

justification) and whether the study encountered any ethical issues.

Whether independent ethics advisory board/expert appointed. For

overt surveys whether/how informed consent was obtained. How

confidentiality/security of people was protected.


page 59

Packaging 10 Packaging examination and reference standards

Chemical analysis 11 Chemical analysis and dissolution testing SOPs and location(s) of

laboratory. Description of validation and reference standards used

Method validation 12 Details of laboratory method validation results, including but not

limited to certificate of analysis (COA) for reference standard,

within- and between-run repeatability (RSD% for n=5-8),

detection and quantitation limits, accuracy observed for reference

samples, linear range for all analytes, sample preparation recovery

studies, selectivity. Possibly validation against a reference method

or interlaboratory study.

Blinding 13 Whether chemistry was performed blinded to packaging and vice

versa

Results

Outlets 14 The details of the outlets actually sampled, (e.g. public, private for

profit, private not-for-profit, informal, itinerant)

Missing samples 15 The reasons why any outlets chosen for sampling did not furnish a

sample.

Packaging and chemistry results 16 Packaging and chemistry results and their relationship

Details of products sampled - how many, in what drug classes,

countries of origin, batch numbers, manufacture and expiry dates

Results for each analysis – packaging, % AI, dissolution

Additional information could be included in “Supplementary

material”

Category of poor-quality

medicine

17 A clear statement for each medicine sample detected whether the

investigators class it as genuine, falsified, substandard or degraded,

with an explanation as to why and whether the medicine was

registered with the government in the location(s) sampled. Other

key elements may be listed (e.g. is the product prequalified by

WHO?)

State company and address as

given on packaging

18 If the names of companies and addresses not given – to give a

reason as to why this information is not provided.


page 60

Sharing data with NDRA and

WHO

19 Whether the data shared with the appropriate NDRA and WHO

Rapid Alert System

Dissemination 20 Description of any non-covert packaging features that would allow

others to detect falsified medicines. If publication is not possible,

to consider disseminating via web-based “Supplementary material”

Discussion

Key results 21 Summarize key results with reference to study objectives

Limitations 22 Discussion of limitations of study, especially how robust the

estimates of prevalence are and how applicable they may be to

wider geographical areas. If not all tests were carried out (e.g.

sterility for injectables), list these as limitations. Discuss the

direction and extent of any potential bias

Interpretation 23 An interpretation of the results, in conjunction with prior studies,

in relation to public health

Intervention 24 Whether interventions are thought appropriate and, if so, what type

Other information

Conflict of interest 25 State any potential conflicts of interest

Funding 26 Give the source of funding and role of funders in the study

Supplementary information 27 Suggest to list important analytical methods and additional results

which would allow others to replicate the work and compare with

the reported study

1564

1565

CONCLUSIONS 1566

1567

Despite poor-quality medicines having been a long and significant public-health problem, the 1568

quality of the medicine supply has received remarkably little attention. Medicines are 1569

fundamental to strategies for effective reduction of mortality and morbidity and good quality 1570

is essential for ensuring their efficacy and safety. The quality of the data describing the 1571

epidemiology of poor-quality medicines are poor, in part because of the diverse 1572

methodologies followed when collecting medicines in the field. The majority of medicine 1573


page 61

quality surveys performed have not used rigorous sampling methods. Any sampling 1574

methodology is inherently prone to bias if it is conducted in such a way that it is different 1575

from the population of interest, i.e. in cases where the location of sample collection is not the 1576

normal access to treatment or if collectors are known in the area. Convenience sampling of 1577

outlets is unlikely to accurately reflect the quality of medicines that consumers actually use in 1578

a defined area, but it can give reports of the presence of poor-quality medicines and provide 1579

an approximate estimate of frequency, without confidence intervals, of unknown accuracy. 1580

All sampling techniques, except convenience sampling, require a sampling frame from which 1581

the samples are drawn. This sampling frame should be representative of the population to 1582

which the investigators wish to be able to generalize the results. Even though randomization 1583

has been considered the “gold standard” sampling technique it will not deal with problems of 1584

inspectors known to pharmacists or any internal knowledge by staff about poor-quality 1585

medicines at the facilities. Therefore it is critical to maintain the covert nature of sampling. 1586

Large sampling sizes or a very high number of dosage units collected will alert sellers to the 1587

aims of those collecting samples as well as asking for a specific brand or medicine for 1588

collection. Communication lines and methods should be agreed well in advance prior the 1589

survey and a plan for publication and dissemination should be developed and agreed at the 1590

planning stage of the survey. Accurate estimates of the frequency of poor-quality medicines in 1591

the market are urgently needed as a requirement for valid comparisons between countries and 1592

regions. Further discussion and consensus-building on the ethics of medicine quality surveys 1593

is required. 1594

REFERENCES 1595

1596

1. Newton, P.N., et al., Counterfeit anti-infective drugs. The Lancet Infectious Diseases, 1597

2006. 6(9): p. 602-613. 1598

2. Caudron, J.M., et al., Substandard medicines in resource-poor settings: a problem that 1599

can no longer be ignored. Tropical Medicine & International Health, 2008. 13(8): p. 1600

1062-1072. 1601

3. IOM, Countering the Problem of Falsified and Substandard Drugs, in National 1602

Academy of Sciences, L. Gostin, Editor. 2013, Institute of Medicine of the National 1603

Academies: http://www.iom.edu/Reports/2013/Countering-the-Problem-of-Falsified-1604

and-Substandard-Drugs.aspx. 1605

4. Newton, P., et al., Poor quality vital anti-malarials in Africa - an urgent neglected 1606

public health priority. Malaria Journal, 2011. 10(1): p. 352. 1607

5. Dorlo, T.P.C., et al., A Poor-Quality Generic Drug for the Treatment of Visceral 1608

Leishmaniasis: A Case Report and Appeal. PLoS Negl Trop Dis, 2012. 6(5): p. e1544. 1609

6. Keoluangkhot, V., et al., Impaired Clinical Response in a Patient with Uncomplicated 1610

Falciparum Malaria Who Received Poor-Quality and Underdosed Intramuscular 1611


page 62

Artemether. The American Journal of Tropical Medicine and Hygiene, 2008. 78(4): p. 1612

552-555. 1613

7. Leslie, T., et al., Epidemic of Plasmodium falciparum malaria involving substandard 1614

antimalarial drugs, Pakistan, 2003. Emerging Infectious Diseases, 2009. 15(11): p. pp 1615

1753-1759. 1616

8. United States Pharmacopeial Convention (USP). Medicines Quality Database 1617

(MQDB). Promoting the Quality of Medicines (PQM) Program 2014 [cited 2014 1618

April 14th ]; Available from: http://www.usp.org/global-health-impact-1619

programs/promoting-quality-medicines-pqmusaid/medicines-quality-database-mqdb. 1620

9. Tabernero, P., et al., Mind the gaps - the epidemiology of poor-quality anti-malarials 1621

in the malarious world - analysis of the WorldWide Antimalarial Resistance Network 1622

database. Malaria Journal, 2014. 13(1): p. 139. 1623

10. Saunders W, Observations on the superior efficacy of the Red Peruvian Bark. Letter 1624

from Dr Edward Rigby, Norwich, 8th September 1782. Vol. 4th edition. 1782, 1625

London, UK: J Johnson & J Murray. 1626

11. Heath WJ, America’s First Drug Regulation Regime: The Rise and Fall of the Import 1627

Drug Act of 1848. Food and Drug Law Journal, 2004. 59: p. 169-200. 1628

12. GP A, Drug Adulteration. Lancet 1826. 6(147): p. 413. 1629

13. Croft CJ, Adulteration Quinine. Lancet, 1838. 31(794): p. 292. 1630

14. Ballentine, C., Taste of Raspberries, Taste of Death. The 1937 Elixir Sulfanilamide 1631

Incident, in FDA Consumer magazine. 1981, U.S. Food and Drug Administration: 1632

http://www.fda.gov/aboutfda/whatwedo/history/productregulation/sulfanilamidedisast1633

er/default.htm. 1634

15. World Health Organization, WHO launches taskforce to fight counterfeit drugs, in 1635

Bulletin of the World Health Organisation 2006, WHO: 1636

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2627462/. p. 685-764. 1637

16. Hanif, M., et al., Fatal renal failure caused by diethylene glycol in paracetamol elixir: 1638

the Bangladesh epidemic. BMJ, 1995. 311(6997): p. 88-91. 1639

17. Nishtar, S., Pakistan's deadly cocktail of substandard drugs. Lancet, 2012. 379(9821): 1640

p. 1084-5. 1641

18. U.S. Food & Drug Administration (FDA). Counterfeit Version of Avastin in U.S. 1642

Distribution. 2012 [cited 2012 February, 14]; Available from: 1643

http://www.fda.gov/Drugs/DrugSafety/ucm291960.htm. 1644

19. WHO, World Malaria Report 2011. World Health Organisation, 2011. 1645

20. World Health Organization, Substandard/spurious/falsely-labelled/falsified/counterfeit 1646

medical products in Sixty-Fifth World Health Assembly. Agenda item 13.13. 2012, 1647

WHO: http://apps.who.int/gb/ebwha/pdf_files/WHA65/A65_R19-en.pdf. 1648

21. WHO, Substandard/Spurious/Falsely-labelled/Falsified/Counterfeit medical products. 1649

Situation report by the Secretariat, in First Meeting of the Member State Mechanism 1650

on Substandard/Spurious/Falsely-labelled/Falsified/Counterfeit Medical Products. 1651

Provisional Agenda item 4. 2012: 1652

http://apps.who.int/gb/ssffc/pdf_files/A_MSM1_INF1-en.pdf. 1653

22. Attaran, A., et al., How to achieve international action on falsified and substandard 1654

medicines. BMJ, 2012. 345. 1655

23. Newton, P.N., et al., The Primacy of Public Health Considerations in Defining Poor 1656

Quality Medicines. PLoS Med, 2011. 8(12): p. e1001139. 1657

24. World Health Organization, J. Sabartova, and A. Toumi, Survey of the quality of 1658

selected antimalarial medicines circulating in six countries of sub-Saharan Africa. 1659

2011: http://www.who.int/entity/medicines/publications/WHO_QAMSA_report.pdf. 1660

p. 118. 1661


page 63

25. Newton, P.N., et al., Guidelines for field surveys of the quality of medicines: A 1662

proposal. PLoS Medicine, 2009. 6((3)): p. pp 0252-0257. 1663

26. United States Pharmacopeia, Guidelines for Drug Sampling. USP DQI Drug Quality 1664

Monitoring Program. Use of the Basic Tests at the Peripheral Level, in The United 1665

States Pharmacopeial Convention. 2006, USP: 1666

http://www.usp.org/sites/default/files/usp_pdf/EN/dqi/samplingGuidelines.pdf. p. 20. 1667

27. Equator Network, Enhancing the QUAlity and Transparency Of health Research. 1668

2014: http://www.equator-network.org/. 1669

28. Newton, P., et al., Fake artesunate in southeast Asia. The Lancet, 2001. 357(9272): p. 1670

1948-1950. 1671

29. Kaur, H., et al., A Nationwide Survey of the Quality of Antimalarials in Retail Outlets 1672

in Tanzania. PLoS One, 2008. 3(10): p. e3403. 1673

30. Bate, R. and K. Hess, Anti-malarial drug quality in Lagos and Accra - a comparison 1674

of various quality assessments. 2010: Malaria journal. 9 (pp 157), 2010. Date of 1675

Publication: 2010. 1676

31. Onwujekwe, O., et al., Quality of anti-malarial drugs provided by public and private 1677

healthcare providers in south-east Nigeria. Malaria Journal, 2009. 8(1): p. 22. 1678

32. Taylor, R.B., et al., Pharmacopoeial quality of drugs supplied by Nigerian 1679

pharmacies. The Lancet, 2001. 357(9272): p. 1933-1936. 1680

33. Alfadl, A.A., et al., Quality of Antimalarials Drugs in Sudan: Results of Post-1681

marketing Surveillance. Sudanese Journal of Public Health, 2006. 1.1. 1682

34. USP, Survey of the Quality of Selected Antimalarial Medicines Circulating in 1683

Madagascar, Senegal and Uganda. USP, 2009. 1684

35. ACTwatch, Outlet Survey. Kingdom of Cambodia. 2011 Survey Report, in Outlet 1685

Survey Report (Endline), Cambodia, 06/11 – 08/11, P.S. Kathryn O’Connell, Editor. 1686

2011, ACTwatch. Evidence for Malaria Medicines Policy: 1687

http://www.actwatch.info/sites/default/files/content/outlet-1688

reports/ACTwatch%20Cambodia%20OS%20Endline_2011.pdf. p. 78. 1689

36. WHO. The World Medicines Situation 2011 Access to Essential Medicines as Part of 1690

the Right to Health 2011 2011 [cited 2013 August 20th,]; 14]. Available from: 1691

http://apps.who.int/medicinedocs/en/m/abstract/Js18772en/. 1692

37. Prashant Yadav, Helen Lega Tata, and Magali Babaley, World Medicines Situation 1693

2011 - Storage and Supply Chain Management, in Essential Medicines and Health 1694

Products Information Portal. 2011, World Health Organization: 1695

http://apps.who.int/medicinedocs/en/m/abstract/Js20054en/. 1696

38. White, L.J., et al., Defining Disease Heterogeneity to Guide the Empirical Treatment 1697

of Febrile Illness in Resource Poor Settings. PLoS ONE, 2012. 7(9): p. e44545. 1698

39. International Pharmaceutical Federation (FIP), Good Pharmacy Practice. 2014: 1699

http://www.fip.org/good_pharmacy_practice. 1700

40. Yeung, S., et al., Socially-marketed rapid diagnostic tests and ACT in the private 1701

sector: ten years of experience in Cambodia. Malaria Journal, 2011. 10(1): p. 243. 1702

41. World Health Organization. Preliminary Draft Survey on National Legislation on 1703

"Counterfeit Medicines" Feedback from Member States to the Circular Letter CL 1704

25.2009. Working document WHO/ACM/1 2010 May 2010; Available from: 1705

http://www.who.int/medicines/services/counterfeit/WHO_ACM_Report.pdf. 1706

42. Bate, R., et al., Pilot study of essential drug quality in two major cities in India. 2009: 1707

PloS one. 4 (6) (pp e6003), 2009. Date of Publication: 2009. 1708

43. Dondorp, A.M., et al., Fake antimalarials in Southeast Asia are a major impediment 1709

to malaria control: multinational cross-sectional survey on the prevalence of fake 1710

antimalarials. Tropical Medicine & International Health, 2004. 9(12): p. 1241-1246. 1711


page 64

44. Sengaloundeth, S., et al., A stratified random survey of the proportion of poor quality 1712

oral artesunate sold at medicine outlets in the Lao PDR - implications for therapeutic 1713

failure and drug resistance. Malaria Journal, 2009. 1714

45. Ogwal-Okeng, J.W., D.O. Okello, and O. Odyek, Quality of oral and parenteral 1715

chloroquine in Kampala. 1998: East African medical journal. 75 (12) (pp 692-694), 1716

1998. Date of Publication: Dec 1998. 1717

46. Hani M. J. Khojah, et al., Adherence of Community Pharmacies in Riyadh, Saudi 1718

Arabia, to Optimal Conditions for Keeping and Selling Good-Quality Medicines. 1719

Pharmacology & Pharmacy, 2013. 4(5): p. 431. 1720

47. Lemeshow, S. and S. Taber, Lot quality assurance sampling: single- and double-1721

sampling plans. World Health Stat Q, 1991. 44(3): p. 115-32. 1722

48. Liverpool School of Tropical Medicine LSTM, The LQAS Generic Toolkit. 2014: 1723

http://www.lstmliverpool.ac.uk/research/departments/international-public-1724

health/metre/lqas-generic-toolkit/. 1725

49. Reinke, W.A., Applicability of Industrial Sampling Techniques to Epidemiologic 1726

Investigations: Examination of an Underutilized Resource. American Journal of 1727

Epidemiology, 1991. 134(10): p. 1222-1232. 1728

50. Lanata, C.F. and R.E. Black, Lot quality assurance sampling techniques in health 1729

surveys in developing countries: advantages and current constraints. World Health 1730

Stat Q, 1991. 44(3): p. 133-9. 1731

51. Robertson, S.E., et al., The lot quality technique: a global review of applications in the 1732

assessment of health services and disease surveillance. World Health Stat Q, 1997. 1733

50(3-4): p. 199-209. 1734

52. Valadez, J.J., A Trainer’s Guide. Using LQAS for Baseline Surveys and Regular 1735

Monitoring, in Assessing Community Health Programs. 2003, Teaching-aids At Low 1736

Cost (TALC), London Illustrations by The Hesperian Foundation: 1737

http://www.coregroup.org/storage/documents/LQAS/Intro.pdf. 1738

53. Phanouvong, S., Mekong Malaria Initiative, Antimalarial Drug Quality Monitoring 1739

and Evaluation. Indicators, in USP. 2004: 1740

http://pdf.usaid.gov/pdf_docs/pnadh147.pdf. 1741

54. Pharmacopeia, U.S. Uniformity of dosage units. Chapter 905. Stage 6 Harmonization 1742

2011 [cited 2014 February 27th, ]; Available from: 1743

http://www.pharmacopeia.cn/v29240/usp29nf24s0_c905.html. 1744

55. WHO. The International Pharmacopeia Fourth Edition. Volume 2 2006; Available 1745

from: http://apps.who.int/phint/en/p/docf/. 1746

56. Syhakhang, L., et al., Knowledge and perceptions of drug quality among drug sellers 1747

and consumers in Lao PDR. Health Policy and Planning, 2004. 19((6)): p. 391-401. 1748

57. Newton, P.N., et al., A collaborative epidemiological investigation into the criminal 1749

fake artesunate trade in South East Asia. 2008: PLoS medicine. 5 (2) (pp e32), 2008. 1750

Date of Publication: Feb 2008. 1751

58. Venhuis, B.J., et al., Dose-to-dose variations with single packages of counterfeit 1752

medicines and adulterated dietary supplements as a potential source of false negatives 1753

and inaccurate health risk assessments. Journal of Pharmaceutical and Biomedical 1754

Analysis, 2014. 89(0): p. 158-165. 1755

59. United Nations Office on Drugs and Crime, Guidelines on Representative Drug 1756

Sampling, Laboratory and Scientific Section, Editor. 2009: 1757

http://www.unodc.org/documents/scientific/Drug_Sampling.pdf. 1758

60. Administration, U.S.D.o.J.D.E., Scientific working group for the analysis of seized 1759

drugs (SWGDRUG) Recommendations,. 2011: 1760

http://www.swgdrug.org/Documents/SWGDRUG%20Recommendations%206.pdf. 1761


page 65

61. Frank, R., C. Hoffman, and S. Hinkley, Representative Sampling of Drug Seizures in 1762

Multiple Containers. ASTM International, 1991. 36(2). 1763

62. SEARPharm and S.E.A.F.-W.F.o.P. Associations, Extent of Spurious (Counterfeit) 1764

Medicines in India. South East ASian FIP-WHO Forum of Pharmaceutical 1765

Associations, 2007. 1766

63. Madden, J.M., et al., Undercover careseekers: Simulated clients in the study of health 1767

provider behavior in developing countries. Social Science & Medicine, 1997. 45(10): 1768

p. 1465-1482. 1769

64. Pharmacopeia, U.S., Ensuring the Quality of Medicines in Resource-Limited 1770

Countries: An Operational Guide, in Drug Quality and Information Program. 2007, 1771

United States Pharmacopeial Convention: 1772

http://apps.who.int/medicinedocs/documents/s18424en/s18424en.pdf. 1773

65. American Sociological Association. ASA Code of Ethics. Ethics and the Code of 1774

Ethics [cited 2014 February 25th]. 1775

66. The American Anthropological Association (AAA). Code of Ethics of the American 1776

Anthropological Association. 1998 [cited 2014 February 25th]. 1777

67. The World Medical Association (WMA), WMA Declaration of Helsinki - Ethical 1778

Principles for Medical Research Involving Human Subjects. 2013: 1779

http://www.wma.net/en/30publications/10policies/b3/index.html. 1780

68. Nuffield Foundation. The Nuffield Council on Bioethics,. 1991 [cited 2014 February 1781

24]. 1782

69. The National Commission for the Protection of Human Subjects of Biomedical and 1783

Behavioral Research, The Belmont Report, in Ethical Principles and Guidelines for 1784

the Protection of Human Subjects of Research. 1979, Department of Health, 1785

Education, and Welfare: 1786

http://videocast.nih.gov/pdf/ohrp_appendix_belmont_report_vol_2.pdf. 1787

70. Council for International Organizations of Medical Sciences (CIOMS) in 1788

collaboration with the World Health Organization (WHO), International Ethical 1789

Guidelines for Biomedical Research Involving Human Subjects. 2002: 1790

http://www.cioms.ch/publications/layout_guide2002.pdf. 1791

71. World Health Organization, Ethical challenges in study design and informed consent 1792

for health research in resource-poor settings in TDR/SDR/SEB/ST/07.1 SPECIAL 1793

TOPICS No.5, Special Programme for Research & Training in Tropical Diseases 1794

(TDR) sponsored by UNICEF/ UNDP/ World Bank/ WHO, Editor.: 1795

http://whqlibdoc.who.int/publications/2007/9789241563383_eng.pdf. 1796

72. Emanuel, E.J., et al., What Makes Clinical Research in Developing Countries Ethical? 1797

The Benchmarks of Ethical Research. Journal of Infectious Diseases, 2004. 189(5): p. 1798

930-937. 1799

73. World Health Organization, Guidelines for good clinical practice (GCP) for trials on 1800

pharmaceutical products,, in WHO Technical Report Series. 1995: 1801

http://apps.who.int/medicinedocs/pdf/whozip13e/whozip13e.pdf. 1802

74. ICH, Guideline for good clinical practice in INTERNATIONAL CONFERENCE ON 1803

HARMONISATION OF TECHNICAL REQUIREMENTS FOR REGISTRATION OF 1804

PHARMACEUTICALS FOR HUMAN USE 1996: 1805

http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E61806

_R1/Step4/E6_R1__Guideline.pdf. 1807

75. Aldhous, P., Counterfeit pharmaceuticals: Murder by medicine. Nature, 2005. 1808

434(7030): p. 132-136. 1809

76. Francis PA, THE TIP OF AN ICEBERG. 2003, Pharmabiz.com: 1810

http://www.pharmabiz.com/article/detnews.asp?articleid=15677&sectionid=47. 1811


page 66

77. Khan, M., et al., Perceptions and practices of pharmaceutical wholesalers 1812

surrounding counterfeit medicines in a developing country: a baseline survey. BMC 1813

Health Services Research, 2011. 11(1): p. 306. 1814

78. Newton, P.N., et al., Manslaughter by Fake Artesunate in Asia-Will Africa Be Next? 1815

PLoS Med, 2006. 3(6): p. e197. 1816

79. WHO, Standards and Operational Guidance for Ethics Review of Health-Related 1817

Research with Human Participants. 2011: 1818

http://whqlibdoc.who.int/publications/2011/9789241502948_eng.pdf. 1819

80. Ravinetto, R., et al., Double ethical review of North–South collaborative clinical 1820

research: hidden paternalism or real partnership? Tropical Medicine & International 1821

Health, 2011. 16(4): p. 527-530. 1822

81. WHO, Ethical issues in patient safety research: interpreting existing guidance. 2013: 1823

http://apps.who.int/iris/bitstream/10665/85371/1/9789241505475_eng.pdf. 1824

82. WHO, Quality practices in basic biomedical research in Prepared for TDR by the 1825

Scientific Working Group on Quality Practices in Basic Biomedical Research, 1826

S.P.f.R.T.i.T.D.T.s.b.U.U.W.B. WHO, Editor.: 1827

http://www.who.int/tdr/publications/documents/quality_practices.pdf. 1828

83. WHO, WHO project for the surveillance and monitoring of SSFFC medical products, 1829

in WHO Drug Information. Quality and Safety of Medicines. 2013: 1830

http://www.who.int/medicines/publications/druginformation/issues/DrugInformation21831

013_Vol27-2/en/index.html. p. 115. 1832

84. Tabernero, P. and P.N. Newton, The WWARN Antimalarial Quality Surveyor. 1833

Pathogens and Global Health, 2012. 106(2): p. 77-78. 1834

85. World Health Organization, WHO guidelines for sampling of pharmaceutical products 1835

and related materials, in WHO Technical Report Series. 2005: 1836

http://apps.who.int/prequal/info_general/documents/TRS929/WHO_TRS_929-1837

Annex4.pdf. 1838

86. United States Pharmacopeia, Rapid Assessment of Medicines Quality Assurance and 1839

Medicines Quality Control. 2007, USP: 1840

http://www.usp.org/sites/default/files/usp_pdf/EN/dqi/rapidAssessmentTool.pdf. 1841

87. World Health Organization, WHO good practices for pharmaceutical quality control 1842

laboratories, in WHO Technical Report Series. 2010: 1843

http://apps.who.int/prequal/info_general/documents/TRS957/GPCL_TRS957_Annex11844

.pdf. 1845

88. Organization, W.H., Considerations for requesting analysis of drug samples, in WHO 1846

Technical Report Series,. 2002: 1847

http://apps.who.int/prequal/info_general/documents/TRS902/WHO_TRS_902-1848

Annex4.pdf. 1849

89. Smith, R.M., et al., Fungal Infections Associated with Contaminated 1850

Methylprednisolone Injections. New England Journal of Medicine, 2013. 369(17): p. 1851

1598-1609. 1852

90. Fernandez, F.M., et al., Poor quality drugs: grand challenges in high throughput 1853

detection, countrywide sampling, and forensics in developing countries. Analyst, 1854

2011. 136(15): p. 3073-82. 1855

91. Kovacs, S., et al., Technologies for Detecting Falsified and Substandard Drugs in Low 1856

and Middle-Income Countries. PLoS ONE, 2014. 9(3): p. e90601. 1857

1858


page 67

1859

ANNEX 1 1860

SOURCES OF INFORMATION OF MEDICINE QUALITY 1861

1862

International organizations and NGOs Examples of medicine

regulatory authorities and

national bodies

Pharmaceutical industry

- WHO Essential Medicines and Health Products - WHO Prequalification Programme

- International Medical Products Anti-Counterfeiting

Taskforce (IMPACT)

- International Conference of Drug Regulatory

Authorities

- ReMeD-Réseau Médicaments & Développement - Fondation Chirac

- Medical Products Counterfeiting and

Pharmaceutical Crime (MPCPC) Unit of

INTERPOL

- Permanent Forum on International Pharmaceutical

Crime

- Medicines Transparency Alliance

- Médecins Sans Frontières Campaign for Access to

Essential Medicines

- Third World Network

- Council of Europe-Medicrime convention

- European Alliance to safe meds

- United Nations Office for Drugs and Crime - Thai Pharmaceutical System Research and

Development Foundation (PhaRed)

- United States Pharmacopeial Convention (USP)

PQM: Promoting the Quality of Medicines in

Developing Countries - The Global Pharma Health Fund (GPHF)

- IRACM Institute of Research Against Counterfeit

Medicines

- Ghana Food and Drugs Board

- Kenya Pharmacy and Poisons

Board

- NAFDAC Nigeria

- Thailand Food and Drugs

Administration - Health Sciences Authority,

Government of Singapore

- Medicines and Healthcare

products Regulatory Agency,

UK Government

- US Food and Drug

Administration

- Centers for Disease Control and

Prevention, USA. Counterfeit

and Substandard Antimalarial

Drugs

- Securing Pharma

- Pharmaceutical Security Institute

- Reconnaissance International

- No to Fakes

- Sanofi

- Pfizer

Alert lists and systems

- WHO-WPRO Rapid Alert System

for Counterfeit Medicines

- E-drug

- E-med

- E-fármacos

- Partnership for safe medicines

- Mpedigree

- PharmaSecure

- Sproxil

- Safe Medicines Beta

- Pharmabiz

Academic/Research Initiatives

- ACT Consortium

- ACT Watch

- QUAMED, ITM, Antwerp

- Counterfeit Drug Forensic

Investigation Network

(CODFIN)

- Institut de Recherche sur l'Asie

du Sud-Est Contemporaine - Chatham House

- Africa fighting Malaria - The CONPHIRMER consortium

- ReAct

Newspaper websites with interest

in medicine quality

- www.ghanaweb.com

- www.modernghana.com

- www.allafrica.com

- www.afrol.com

- www.tribune.com.ng/index.php

- www.monitor.co.ug


page 68

ANNEX 2 1863

METHODOLOGY SECTION 1864

2.1 Random sampling and LQAS examples 1865

1866

(a) Random sampling 1867

For example, the investigators would like to determine the prevalence of outlets selling poor-1868

quality medicine of INN X in area Y. 1869

1870

We can estimate a sample size assuming a prevalence of poor-quality medicines of 50% (or p 1871

= 0.5). This choice of estimated prevalence will give us the most conservative (i.e. largest) 1872

sample size needed. To determine the actual prevalence of outlets selling falsified medicines 1873

with a precision of 5% (i.e. to within ±0.05; precision = 0.05 × 2 = 0.1) with 95% confidence 1874

intervals (z = 1.96) we would need a random sample size (n) of ~390 (n = 1875

4p(1−p)z2/precision2 = 4 × 0.5(1−0.5 ×(1.96)2/(0.1)2) (1). This means that purchases from 1876

390 different outlets selling medicine X would be required to obtain an objective estimate of 1877

the prevalence of those selling poor-quality medicine X at one time point in one area. Within 1878

each outlet the sampling of medicine units should follow one of strategies described in section 1879

8 and Annex 2.2 below. 1880

1881

Yamane (1967) (2) provides a simplified formula for random sampling that assumes a 95% 1882

confidence level and a prevalence of 50%: 1883

sample size = N/ (1+N (precision) 2

) 1884

where N = the total number of outlets selling medicine. 1885

1886

(b) LQAS 1887

For example, the investigators would like to determine whether the prevalence of outlets 1888

selling poor-quality medicine of INN X is less than 80% in area Y. 1889

1890

For LQAS sampling we may set our upper threshold to 95% and the lower threshold to 80%. 1891

This means that it is acceptable for 95% of outlets selling medicine X (the unit) in one district 1892

(the lot) in location Y to have good-quality medicines and unacceptable for less than 80% to 1893

have good-quality medicines. If we set the Type I error to 0.05 (i.e. there is a 5 in 100 chance 1894

that a district with 80% or fewer of the outlets selling good-quality drugs will go undetected) 1895


page 69

and the Type II risk to 0.10 (i.e. there is a 10 in 100 chance that we will inappropriately direct 1896

resources to a district in which 95% or more of the outlets are in fact selling good-quality 1897

drugs). Our sample size would be 38 randomly selected outlets and the district would be 1898

considered unacceptable if more than four outlets had poor-quality medicine X (calculated 1899

using sample LQ (3, 4)). In other words, the null hypothesis that the district has at least 80% 1900

of its outlets selling good medicine X would be rejected if more than four out of 38 outlets 1901

sold poor-quality medicine X in area Y. 1902

2.2 Sampling techniques for the dosage units within medicine samples 1903

A binomial distribution assumes that the probability of selecting a positive unit does not 1904

change during the sampling process; in other words, the proportion of poor-quality samples in 1905

the population remains constant. This certainly would not be true when a sample would be 1906

selected from a small population. For example, for a population of 10 units with 5 poor-1907

quality units, the initial probability of selecting a poor-quality unit is 5/10 but changes to 4/9, 1908

3/8, etc., after one, two, three poor-quality units are drawn. For the probability to remain 1909

constant, the drawn unit would need to be returned to the population before the next draw – 1910

this type of drawing is called “sampling with replacement” and may only be relevant to very 1911

large populations when removing a few units from the population will change the proportion 1912

of positive units only negligibly. The hypergeometric distribution, in contrast, describes 1913

sampling without replacement and takes into account that the proportion of positive units 1914

change throughout the sampling process, so can be used for any population. The binomial 1915

distribution is a good approximation to the hypergeometric when the relative size of the 1916

drawn sample is small compared to the population. In practice, for samples <50 units the 1917

hypergeometric distribution should always be used. 1918

Assuming that the population of N units consists of N1 positive units and N2 negative units, 1919

the probability distribution allows estimation of the probability α of observing x positive (in 1920

this case poor-quality medicine) units in a random sample of size n if the proportion of poor-1921

quality units in the whole population of size N is k = N1/N. A sample size of n units that have 1922

to be analysed can be calculated such that if all units in that sample are poor quality (x=1) we 1923

can conclude with (1−α) 100% confidence that overall at least k100% of the units are positive, 1924

i.e. are poor-quality medicines, using the relationship between α, x, n, k and N defined by the 1925

distribution function. Similarly, the calculation can be done when it is expected that one or 1926

two units in the sample will be negatives (i.e. do not contain poor-quality medicines). In this 1927


page 70

case, the number of units to be sampled will be significantly higher. To show that the 1928

population contains >5% of poor-quality units with 95% confidence, 3–5 units will need to be 1929

analysed for a population size 10–10 000, assuming they all test positive, while for >90% the 1930

required numbers of units to be sampled will vary between 8–29. When analysing essential 1931

medicines, if no poor-quality medicines are found it would be desirable to conclude that the 1932

proportion of poor antimalarials is sufficiently low, say below 1%. See calculation below. 1933

1934

Tables with the required number of samples to be tested for different scenarios and baset 1935

European Network of Forensic Sciences Institutes (ENFSI), for examples, see Appendix 2.6. 1936

The calculator for qualitative sampling of seize drugs is available at: 1937

http://www.enfsi.eu/documents/enfsi-dwg-calculator-qualitative-sampling-seized-drugs-2012 1938

1939

The choice of the numbers for α and k depend on laboratory guidelines, costs, legal 1940

requirements and the specific situation. The assumption that all sampled units contain poor- 1941

(or good-) quality medicines is often made based on previous surveys or experience in the 1942

field. 1943

1944

The LQAS methodology, which was described in the context of outlets sampling, can also be 1945

employed for dosage-unit sampling. The number of units to be sampled and analysed can be 1946

calculated in populations with a specified ≥ k100% poor-quality medicines with significance 1947

level α. This method ensures that populations with ≤ m100% will not be wrongly classified as 1948

populations with a high proportion of poor-quality medicines, with 80% power. 1949

1950

Rather than using a probability distribution to calculate the required number of units to be 1951

sampled a Bayesian approach may be used. The assumption of this approach is that the 1952

proportion of poor-quality medicines in a random sample is known and fixed but the 1953

proportion in the population is not known. A distinction can be made between sampling with 1954

replacement and sampling without replacement, as for the frequentist approach. Sampling 1955

with replacement is easier and can be used as an approximation for situations where the 1956

population size is at least 50 and the sample drawn is relatively small. Here an overestimate is 1957

not such a problem as with the binomial distribution. 1958

1959


page 71

In the Bayesian approach, although the population proportion is not known, there may be 1960

some ideas about the size of this proportion. These ideas are represented by a probability 1961

distribution p (θ), the so-called “prior distribution” which expresses the uncertainty about the 1962

exact value of the proportion or our belief about what the size of the proportion may be 1963

(which may be based on previous studies). This uncertain knowledge is combined with the 1964

information provided by the sample and a so-called “posterior distribution” of the proportions 1965

is provided. With this posterior distribution it is possible to calculate directly the probability 1966

that the proportion of bad-quality drugs is at least k (given the sample results) without using 1967

tests or confidence intervals. With this approach it is possible to incorporate some knowledge 1968

about the total sample that you may possibly have. Sample size calculations can be conducted 1969

for more than 50 samples or less than 50 samples. 1970

1971

2.3 Sample size calculation for the dosage units within medicine samples 1972

The sample size n to be selected from a population of size N is calculated by testing the null-1973

hypothesis that the number of positives in the population, N1 is less than K against the 1974

alternative hypothesis that the number of positives is at least K: 1975

1976

Null hypothesis H0 : N1 < K 1977

Alternative hypothesis H1 : N1 ≥ K. 1978

1979

The null-hypothesis will be rejected when the number of positives in the sample selected is 1980

larger than a certain number. If this number is taken as the number of positives expected in 1981

the sample x then n should be selected such that the probability of observing more than x 1982

positives in that sample under the assumption of N1 positives in the whole population is very 1983

low, i.e. 1984

P(X ≥ x | N1 <K) ≤ α (1) 1985

Where α is the probability that the null-hypothesis is rejected when it is actually true and it is 1986

usually selected as 0.05 or 0.01. 1987

1988

Inequality (1) can be expressed as: 1989

∑ P�X = �|�� − 1,�, �) ≤ α 1990

and solved for n using an appropriate formula for the probability P(X=i| K-1,N,n) that a 1991

sample of a given size contains a prespecified number of positives. Two such formulas are 1992


page 72

listed below: for the hypergeometric distribution, which should be used for sampling with 1993

replacement, and the binomial distribution, for sampling without replacement. 1994

The number of units to be tested n can be derived for different values of N, K and α and these 1995

are shown in tables in section 2.5 below. 1996

Hypergeometric distribution 1997

The probability that a sample of size n contains X positives (units containing poor-quality 1998

drugs), given that the population of size N contains N1 positives, is calculated as: 1999

�� = �|��, �, �) = ��

��

��

Where �� =�!

��)!�! and is equal to the number of ways of selecting b units from a 2000

population of a units. 2001

Binomial distribution 2002

The probability that a sample of size n contains X positives (units containing bad quality 2003

drugs), given that the population of size N contains a proportion of θ =N1/N poor quality drugs, 2004

is calculated as: 2005

2006

�� = �|��, �, �) = ��! θ��1 − θ)��

Where ��! = �!��)!�! as before. 2007

2008


page 73

2.4 Further statistical considerations 2009

2010

Data can be weighted at the analysis stage and survey settings can be incorporated to account 2011

for clustering of the outlets within the districts, provinces, regions. 2012

a) Booster sample: oversampling to ensure adequate representation of relatively rare but 2013

important outlet types (i.e. public health facilities operating in the district of the sampled 2014

area). 2015

b) Sampling weights: sample weights can be calculated for an outlet survey data to 2016

allow for: 2017

- differences in sampling probabilities due to variation in the size of strata; 2018

- the oversampling of the booster sample; 2019

c) the sampling strategy which involved a census of outlets in populations of varying 2020

size selected with probability proportional to size (PPS). Weights can be based on 2021

sampling probabilities (2). 2022

d) Clustering: sample can be clustered at the level of the district/province or communities. 2023

The calculation of standard errors takes this clustering into account because outlets in a 2024

given cluster are likely to be more similar to each other than to outlets in other clusters (5, 2025

6), i.e. sampling can be conducted using a one-stage probability proportion size (PPS) 2026

cluster design, with the measure of size being the relative commune population. Different 2027

zones can be calculated as different strata with approximately 10 000–15 000 inhabitants 2028

(7). 2029

e) Stratification: communities can be sampled separately in each stratum and this can also 2030

be adjusted for in the calculation of the standard error terms during analysis. The 2031

stratification of the sampling units by geographical, trade and socioeconomic variables 2032

can be conducted before the sample selection is conducted. Outlets vary greatly in type 2033

whether sampling should stratify the outlets in this way for analysis should be discussed 2034

during the survey planning. 2035

f) Lots: it may be necessary to specify lots (or defined areas) using predefined geographical 2036

boundaries (e.g. districts, provinces, communities, rural vs urban, etc.) and to vary 2037

thresholds to classify good and bad outcomes by area. While exact longitudinal changes 2038

in prevalence cannot be calculated strict thresholds may be useful for identifying areas 2039

with acceptable levels of good-quality medicines. 2040


page 74

2.5 Tables showing number of units to be tested for various K, and α for the binomial 2041

and hypergeometric distributions 2042

a. Binomial distribution 2043

2044

95% confidence 99% confidence

K=0.5 K=0.7 K=0.9 K=0.5 K=0.7 K=0.9

0 negatives 5 9 29 7 13 44

1 negative 8 14 46 11 20 64

2 negatives 11 19 61 14 25 81

2045

b. Hypergeometric 1 2046

2047

Population

Size


K=0.5 K=0.7 K=0.9 K=0.5 K=0.7 K=0.9

10 3 5 8 4 6 9

20 4 6 12 5 9 15

30 4 7 15 6 10 20

40 4 7 18 6 10 23

50 4 8 19 6 11 26

60 4 8 20 6 11 28

70 5 8 21 7 12 30

80 5 8 22 7 12 31

90 5 8 23 7 12 32

100 5 8 23 7 12 33

200 5 9 26 7 13 38

300 5 9 27 7 13 40

400 5 9 27 7 13 41

500 5 9 28 7 13 41

600 5 9 28 7 13 42

700 5 9 28 7 13 42

800 5 9 28 7 13 42

900 5 9 28 7 13 43

1.000 5 9 28 7 13 43

5.000 5 9 29 7 13 44

10.000 5 9 29 7 13 44

2048

2049


page 75

2050

c. Hypergeometric 2 2051

2052

Population

Size


K=0.5 K=0.7 K=0.9 K=0.5 K=0.7 K=0.9

1 neg 2 neg 1 neg 2 neg 1 neg 2 neg 1 neg 2 neg 1 neg 2 neg 1 neg 2 neg

10 5 7 7 9 10 – 6 7 8 9 10 –

20 6 8 10 13 17 20 8 10 12 14 19 20

30 7 9 11 14 22 27 8 11 14 17 25 29

40 7 9 12 15 26 32 9 11 15 18 30 35

50 7 10 12 16 29 36 9 12 16 20 34 41

60 7 10 12 16 31 39 9 12 16 20 38 45

70 7 10 13 17 32 41 10 12 17 21 40 48

80 7 10 13 17 34 43 10 12 17 21 42 51

90 7 10 13 17 35 45 10 13 17 21 44 54

100 7 10 13 17 36 46 10 13 17 22 46 56

200 8 10 14 18 40 53 10 13 18 24 54 67

300 8 10 14 19 42 55 10 13 19 24 57 71

400 8 11 14 19 43 57 10 13 19 24 58 74

500 8 11 14 19 44 58 10 14 19 24 59 75

600 8 11 14 19 44 58 10 14 19 25 60 76

700 8 11 14 19 44 59 11 14 19 25 61 77

800 8 11 14 19 44 59 11 14 19 25 61 77

900 8 11 14 19 45 59 11 14 19 25 61 78

1.000 8 11 14 19 45 59 11 14 19 25 62 78

5.000 8 11 14 19 46 61 11 14 20 25 64 81

10.000 8 11 14 19 46 61 11 14 20 25 64 81

2053

2054

2055

2.6 Calculator for qualitative sampling of seized drugs. Developed by The European 2056

Network of Forensic Sciences Institutes (ENFSI) and available at: 2057

http://www.enfsi.eu/documents/enfsi-dwg-calculator-qualitative-sampling-seized-drugs-2012 2058

2059


page 76

Software instructions

The software is a Microsoft Excel 2000 application. You will need to have the “Analysis

ToolPak” add-in installed (select: Tools/Add ins…/Analysis ToolPak). The “protection”

option (without a password) is enabled so that users may only enter data in specific required

cells. This protection option can be disabled if you wish to experiment with the package.

Older versions of Excel can only handle numbers up to about 1E+308. In such cases if a

number (either in a result or an intermediate calculation) exceeds this value then an overflow

error occurs and a “#NUM” error result is returned. Users must be aware of this when dealing

with large numbers. For example: (Excel 2000 version) 100 000 tablets, 0.99 confidence, k =

0.99, expected negatives = 2 will give an invalid result for the sample size. No laboratory

would ever use such unrealistic levels. However users should be aware of the limitations of

the software.

The graph is for display purposes only. The sample size scale is set from 1 to 100 as this

range will cover most results.

Hypergeometric sampling

1. The Excel sheet has five tabs at the bottom (Instructions, Hypergeometric, Bayesian,

Binomial and Estimation of Weight).

2. Select the Hypergeometric tab.

3. Enter the desired values for steps 1,2,3 & 4.

4. The required sample size will be given at step 5 (cell B5).

The Excel hypergeometric distribution function is used here as follows:

P = HYPGEOMDIST ((n-r), n, (N*k)-1, N)

This gives the probability of finding n-r positives in a sample of size n taken from a

population N containing (N*k)-1 positives.

In the case of 0 negatives expected (r = 0):


page 77

If P gives the probability of finding n positives, then 1-P gives the probability of not finding

this number of positives.

In other words 1-P gives the probability of finding at least one negative. A sample size n is

chosen to give a value for 1-P which exceeds the desired confidence level (1-a).

NOTE 1

It may happen that a number of samples have been taken, assuming that there would be zero

negatives; however, upon analysis one of the samples appeared to be negative. What can then

be said about the proportion of the seizure that is positive for drugs? The macro can also

calculate this proportion. (Note: depending on the setting of your Excel program, points or

commas must be used for decimals)

Scenario:

Seizure of 1000 tablets

Proportion of positives = 0.9

Expected negatives = 0

Confidence level = 0.95

This requires a sample size of 28.

Suppose you have analysed these 28 tablets and found one negative. What proportion of the

seizure can you still be 0.95

confident to contain positives?

Step 1: Start at the begin position. Scroll down until sample size 28 is visible on the screen

(this has a current probability value of 0.951419384)

Step 2: Change the “expected negatives” value from 0 to 1.

(this will reduce the probability value for sample size 28 to 0.793866654)

Step 3: Reduce the value for “proportion of positives” continuously until the probability for

sample size 28 reaches

or exceeds 0.95 again (this happens when k=0.84)

Therefore we can be 95% confident that 84 % of the seizure contains positives.


page 78

NOTE 2

The HPD calculations are based on whole numbers only. Therefore, if certain entries (or the

result of some intermediate calculations) are not whole numbers the software will round the

values down to the nearest whole number. This may produce some anomalies in sample size,

especially with low population numbers.

e.g. for a small population of, say 12 tablets (with k = 0.5 and 1-α = 0.99), the calculated

sample size is 5 whereas if the population increases to 13, the sample size reduces to 4. In the

latter case (with k = 0.5) the number of positives in the population is calculated by the

software to be 13 * 0.5 = 6.5 tablets. Obviously HPD cannot use the value of 6.5 tablets in its

calculations and therefore the value is rounded down to 6. This is why the sample size is

reduced here because HPD is calculating the probability of finding at least one negative when

there are only 6 positive tablets (instead of 6.5) in a population of 13 (in reality the rounding

process in this case has changed the value of k from 0.5 to 0.46).

Bayesian sampling

1. Select the Bayesian tab.

2. Enter the desired values for steps 1, 2, 3, 4, 5 and 6.

Note 1: Although population size is not used in the calculations for the beta distribution, it

is necessary to enter

the population size so that the software can decide whether to use beta or beta-binomial

distribution in the calculations.

Note 2: The values selected for steps 2 and 3 (a & b values) will depend on the analyst’s

prior knowledge or assumptions about θ.


N >= 50

The Excel beta distribution function is used here as follows:

P(θ>k)=BETADIST (k, a+(n-r), b+r, lower limit for k, upper limit for k).

N <50


page 79

The Γ(x) function can be calculated in Excel by using a combination of the EXP and

GAMMALN worksheet

functions as follows:

GAMMALN(x) = LN(Γ(x))

The EXP function is the inverse of the LN function therefore:

EXP(GAMMALN(x)) = Γ(x)

This function is incorporated into the beta-binomial distribution equation as follows:

P(Y>=y)=

( EXP(GAMMALN(n+a+b))*COMBIN(N-n,y) * EXP(GAMMALN(y+x+a)) *

EXP(GAMMALN(N-x-y+b) )

/ ( EXP(GAMMALN(x+a) * EXP(GAMMALN(n-x+b)) * EXP(GAMMALN(N+a+b) )

Binomial sampling

1. Select the Binomial tab.

2. Enter the desired values for steps 1,2 &3.


The Excel binomial distribution function is used here as follows:

P = BINOMDIST (n-r, n, k, FALSE)

Estimation of weights

1. Select the Estimation of Weight tab.

2. Enter the required values for steps 1 to 6.

3. The confidence interval is given in cells B12: D12

The confidence interval is calculated as follows:

C.I. = mean weight ± t*s/ √(n-r)


page 80

In the event of any negatives being detected in the samples, a correction factor (n-r)/n is

applied as follows:

C.I. = (mean weight) * (n-r)/n ± (t*s/ √(n-r)) * (n-r)/n

For smaller populations where n/N > 0.1 a further correction factor √((N-n)/N) is

applied giving:

C.I. = (mean weight) * (n-r)/n ± (t*s/ √n-r) * (n-r)/n* √((N-n)/N)

Estimation of tablet numbers

1. Select the Estimation of Tablets tab.

2. Enter the required values for steps 1 to 5.

3. The estimated number of tablets is given in cell B9

2060

2061


page 81

a. Hypergeometric 2062

Step 1: Enter Population Size

(N): 30

Step 2: Enter Proportion of

Positives in Population

(k):

0,9

Step 3: Enter Expected Number

of Negatives (r): 2

Step 4: Enter Confidence level

(1-�� 0,95

Step 5:

Sample size (n)

which gives the

required confidence

level

27

Sample Size (n) P(Sample Negatives>r)

1 0

2 0 3 0,000985222

4 0,003831418

5 0,009304871

6 0,018062397

7 0,030651341 8 0,047509579

9 0,068965517

10 0,095238095

11 0,126436782

12 0,162561576 13 0,20350301

14 0,249042146

15 0,298850575

16 0,352490421

17 0,40941434 18 0,468965517

19 0,530377668

20 0,592775041

21 0,655172414

22 0,716475096 23 0,775478927

24 0,830870279

25 0,881226054

26 0,925013684

0.75

0.8

0.85

0.9

0.95

1

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

P

R

P(Sample Negatives) > r

Confidence Level


page 82

2063

2064

2065

2066

2067

2068

2069

b. Bayesian 2070

Step 1: Enter Population Size

(N): 1000

Step 2: Enter Value

a: 1

Step 3: Enter Value

b: 1 Step 4:

Enter Proportion of

Positives in

Population (k):

0,75

Step 5: Enter Expected

Number

of Negatives (r):

0

Step 6: Enter Confidence

level

(1-a)

0,95

Step 7:

Sample size (n)

which gives the

required confidence

level (using BETA

disribution for

N>=50)

10

Sample Size (n) R(q>k) 1 0,4375

2 0,578125

3 0,68359375

4 0,762695313

5 0,822021484 6 0,866516113

7 0,899887085

27 0,960591133

28 0,986206897 29 1

30 1

0.75

0.8

0.85

0.9

0.95

1

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

P

R

P

Confidence Level


page 83

8 0,924915314

9 0,943686485

10 0,957764864 11 0,968323648

12 0,976242736

13 0,982182052

14 0,986636539

15 0,989977404 16 0,992483053

17 0,99436229

18 0,995771717

19 0,996828788

20 0,997621591 21 0,998216193

22 0,998662145

23 0,998996609

24 0,999247457

25 0,999435592 26 0,999576694

27 0,999682521

28 0,999761891

29 0,999821418

30 0,999866063 31 0,999899548

32 0,999924661

33 0,999943496

34 0,999957622

35 0,999968216 36 0,999976162

37 0,999982122

38 0,999986591

39 0,999989943

40 0,999992458 41 0,999994343

42 0,999995757

43 0,999996818

44 0,999997614

45 0,99999821 46 0,999998658

47 0,999998993

48 0,999999245

49 0,999999434

50 0,999999575 51 0,999999681

52 0,999999761

53 0,999999821


page 84

54 0,999999866

55 0,999999899 56 0,999999924

57 0,999999943

58 0,999999957

59 0,999999968

60 0,999999976 61 0,999999982

62 0,999999987

63 0,99999999

64 0,999999992

65 0,999999994 66 0,999999996

67 0,999999997

68 0,999999998

69 0,999999998

70 0,999999999 71 0,999999999

72 0,999999999

73 0,999999999

74 1

75 1 2071

2072

2073

2074

c. Binomial 2075

Step 1: Enter Proportion of

Positives in

Population (k):

0,9

Step 2: Enter Expected

Number

of Negatives (r):

0

Step 3: Enter Confidence

level

(1-a)

0,95

Step 4:

Sample size (n)

which gives the

required confidence

level

29

Sample Size (n) P(Sample Negatives>r)

1 0,1

0.75

0.8

0.85

0.9

0.95

1

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

P

R

P(Sample Negatives) > rConfidence Level


page 85

2 0,19

3 0,271

4 0,3439 5 0,40951

6 0,468559

7 0,5217031

8 0,56953279

9 0,612579511

10 0,65132156 11 0,686189404 12 0,717570464

13 0,745813417

14 0,771232075

15 0,794108868

16 0,814697981

17 0,833228183 18 0,849905365

19 0,864914828

20 0,878423345

21 0,890581011

22 0,90152291

23 0,911370619

24 0,920233557 25 0,928210201

26 0,935389181

27 0,941850263

28 0,947665237

29 0,952898713

30 0,957608842 31 0,961847958 32 0,965663162

33 0,969096846

34 0,972187161

35 0,974968445

36 0,9774716

37 0,97972444 38 0,981751996 39 0,983576797

40 0,985219117

41 0,986697205

42 0,988027485

43 0,989224736

44 0,990302263 45 0,991272036

46 0,992144833

47 0,99293035

48 0,993637315

49 0,994273583


page 86

50 0,994846225

51 0,995361602

52 0,995825442 53 0,996242898

54 0,996618608

55 0,996956747

56 0,997261073

57 0,997534965

58 0,997781469 59 0,998003322 60 0,99820299

61 0,998382691

62 0,998544422

63 0,998689979

64 0,998820982

65 0,998938883 66 0,999044995

67 0,999140496

68 0,999226446

69 0,999303801

70 0,999373421

71 0,999436079

72 0,999492471 73 0,999543224

74 0,999588902

75 0,999630012

76 0,99966701

77 0,999700309

78 0,999730278 79 0,999757251 2076

2077

2078


page 87

2079

d. Estimation of weight 2080

Step 1: Enter Confidence level (1-a) 0,95 a= 0,05

Step 2: Enter Population (N) 100 t= 2,086

Step 3: Enter Sample Size (n) 23 n/N= 0,230

Step 4: Enter Mean Weight (x-bar) 0,265 Q= 0,877

Step 5: Enter Sample Standard

Deviation (s) 0,023 rsd(%) = 8,7

Step 6: Enter Number of Negative

Results in Sample ( r ) 2 P= 0,913043478

Confidence Interval = 0,24196 ± 0,008

Estimate of Weight = 24,196 ± 0,839

2081

2082

e. Estimation of tablets 2083

Step 1: Enter Total Weight of Tablets

(grams): 1027,1

Step 2: Enter Number of Individual

Weighings (n): 10

Step 3: Enter the Number of Tablets per

Individual Weighing: 50

Step 4: Enter Mean Weight

of Individual Weighings (grams): 14,772

Estimated Number of Tablets = 3476 Tablets

2084

2085


page 88

REFERENCES 2086

2087

1. Machin David, et al., Sample Size Tables for Clinical Studies, Second Edition. 1997: 2088

Blackwell Science, Oxford, UK. 2089

2. Phanouvong, S., The Quality of Antimalarial Medicines in Western Cambodia: A Case 2090

Study Along the Thai-Cambodian Border Southeast Asian J Trop Med Public Health, 2091

2013. 44(3): p. 349-362. 2092

3. Brixton Health, Sample LQ. 2008: http://www.brixtonhealth.com/samplelq.html. 2093

4. Valadez, J.J., Assessing child survival programs in developing countries : testing lot 2094

quality assurance sampling, Population and International Health Harvard School of 2095

Public Health, Editor. 1991, Harvard University Press,: 2096

http://trove.nla.gov.au/work/34476731?selectedversion=NBD8721131. 2097

5. North Carolina Center for Public Health Preparedness, Two-Stage Cluster Sampling: 2098

General Guidance for Use in Public Heath Assessments, in Introduction to Cluster 2099

Sampling, P.H.R.S.T. 5, Editor.: http://nciph.sph.unc.edu/cha-learning-2100

congress/TwoStageClusterSampling.pdf. 2101

6. World Health Organization, Cluster sampling to assess immunization coverage: a 2102

review of experience with a simplified sampling method, in Bulletin of the World 2103

Health Organization. 1982: http://whqlibdoc.who.int/bulletin/1982/Vol60-2104

No2/bulletin_1982_60%282%29_253-260.pdf. p. 25-260. 2105

7. ACTwatch, Outlet Survey. Kingdom of Cambodia. 2011 Survey Report, in Outlet 2106

Survey Report (Endline), Cambodia, 06/11 – 08/11, P.S. Kathryn O’Connell, Editor. 2107

2011, ACTwatch. Evidence for Malaria Medicines Policy: 2108

http://www.actwatch.info/sites/default/files/content/outlet-2109

reports/ACTwatch%20Cambodia%20OS%20Endline_2011.pdf. p. 78. 2110

2111

2112


page 89

ANNEX 3 2113

NATIONAL SAMPLING PLAN 2114

2115

Survey of the quality of antimalarial medicines …………… 2116

2117

National sampling plan 2118

2119

Country: Focal person: 2120

2121

MEDICINES TO BE COLLECTED: 2122

• INN(s), dosage form, strength 2123

• 2124

• 2125

• 2126

2127

NUMBER OF UNITS TO BE COLLECTED PER SAMPLE: 2128

• Approx. 100 units for tablets/capsules 2129

• 2130

• 2131

2132

TOTAL NUMBER OF SAMPLES PER COUNTRY: 2133

2134

NAMES AND ADDRESSES OF THE SITES, WHERE SAMPLE SHALL BE 2135

COLLECTED: 2136

Facility name Address Facility type

1. (private/public;

2. level 1/level 2;

3. wholesaler/retailer/treatment

centre/…)

1.

2.

3.

4.

5.

2137

2138


page 90

INSTRUCTIONS FOR COLLECTORS 2139

• The amount of the selected products defined above will be sampled from the identified 2140

sites. All these samples are inclusive of the samples needed for the OOS investigations 2141

and retention samples. 2142

• An item collected from each presentation at the same collection site will be called a 2143

sample. All units (tablets, capsules, vials) of one sample must be of the same batch; 2144

there should not be a mix up with batches. In the case that in a collection site the required 2145

number of packages of the same batch is not available sample of that particular medicine 2146

is not collected. 2147

• Samples collected shall have at least six months remaining to expiry. 2148

• One batch of each product will be collected from each collection site and only unopened 2149

original packages shall be collected. 2150

• The medicine samples should not be taken out of the original primary packaging and 2151

outer containers (though removal of blisters from large secondary packs is appropriate). 2152

Containers such as bottles and vials should not be opened. 2153

• The medicine labels and package leaflets should not be removed or damaged. 2154

• Sampling will be recorded using the sample collection form. Whenever the required 2155

information is not available it should be indicated in the appropriate space on the sample 2156

collection form, where also any abnormalities should be recorded. 2157

• In order to avoid confusion, each sample will be identified by a unique sample code (for 2158

coding system see the sample collection form) specified in the sample collection form as 2159

well as on all the original packages belonging to the respective sample. Packages 2160

belonging to one sample and sample collection form will be kept together (e.g. blisters 2161

inserted in a dedicated envelope marked with the appropriate sample code and trade name 2162

of the product). 2163

• Manufacturer’s batch certificates of analysis will be collected with samples, if available, 2164

and kept with the sample collection form. 2165

• The samples should be collected and kept under controlled conditions as per label 2166

requirement. The cold chain should be maintained where required. 2167

• Samples should be collected in all the countries involved within the period ……… and 2168

the deadline for sending the last sample to the testing laboratory is ………. 2169

2170

2171


page 91

ANNEX 4 2172

SAMPLE COLLECTION FORM 2173

2174

Survey of the quality of medicines …………… 2175

Sample collection form* 2176

Country: Sample code: 2177

(Country code/product abbreviation/sequence 2178

number/sampling date ddmmyy)** 2179

Name of location/place where sample was taken: 2180

2181

Address (with telephone, fax number and email address, if applicable): 2182

2183

2184

Organization and names of people who took samples: 2185

1. 2186

2. 2187

Product name of the sample: 2188

Name of active pharmaceutical ingredient(s) (INN) with strength: 2189

2190

Dosage form (tablet, capsule, powder for injection, etc.): 2191

Package size, type and packaging material of the container: 2192

2193

Batch/lot number: 2194

Date of manufacture: Expiry date: 2195

Regulatory status in the country, registration number, if applicable: 2196

2197

Name and address of the manufacturer: 2198

2199

* This sample collection form should always be kept with the sample collected. Proper sampling procedures

should be followed.

** Product abbreviations: …….. . Sample code system can be extended to be appropriate for a particular country

collection system.


page 92

Quantity collected (number of sample units or of multidose containers taken): 2200

2201

Initialize first page: Product name: Sample code: 2202

Storage/climatic conditions at sampling site/point (temperature and humidity, indication of 2203

conditions during daytime only is acceptable, comments on suitability of premises where 2204

products are stored at the particular site for the NDRA information): 2205

2206

2207

2208

Abnormalities, remarks or observations that may be considered relevant, if any: 2209

2210

2211

2212

2213

2214

2215

2216

2217

2218

Date: 2219

2220

Signature of person(s) taking Signature of representative of the 2221

samples establishment where sample(s) was 2222

taken (optional) 2223

1. .............................................................. 2224

.............................................................. 2225

2. .............................................................. 2226

2227

Note: Samples collected must remain in their original containers, intact and unopened 2228


page 93

ANNEX 5 2229

SOP FOR MYSTERY SHOPPER METHODOLOGY 2230

2231

SOP. No. 1 Mystery shopper

Date of issue

Prepared by: Review by: Approved by:

Version no.

2232

I. Objective 2233

Investigate the quality of medicines……………………………………. in the private and/or public sectors in 2234

location …………………………sold to patients. 2235

2236

Key outcomes will be: 2237

1. What medicines were offered to mystery shoppers at registered and unregistered outlets? 2238

2. Proportion of outlets selling monotherapy? 2239

3. What is the chemical quality of these medicines? 2240

4. Has the prevalence of counterfeit and substandard medicines changed since the …………. year survey? 2241

2242

II. Rationale 2243

The study ……………………. funded quality study is planned to be conducted 2244

in ……………………………………..provinces in the country……………………………..starting 2245

on ………………………………... One urban and one rural district will be selected by random number tables in 2246

each of the ………………………..study provinces. Collection of medicines will be conducted through separate 2247

mystery-shopper (MS) methodology. 2248

2249

III. Requirements 2250

Equipments Consumables Documents/ register

- Phone

- one laptop computer with

- Microsoft Office and Epidata

- Hard boxes to keep the drugs

- Safe box for field money

- Fuel

- Phone credit

- Zip bags to collect

drugs

- Notebook

- Pens

- Marker pen

- Provincial health

Department

- Diary/scheduling

sheet

- Printed SOPs

2251

2252


page 94

IV. Procedure 2253

The team will be composed of a coordinator, ………………senior staff, ………………….. assistant 2254

and ………………… driver. The assistants will purchase the anti-infective medicines from all outlets identified 2255

in the selected districts. Lists of pharmacies and shops in the selected areas will be provided by government 2256

authorities. Provincial and district staff will assist with updating the lists of pharmacies and shops that sold 2257

medicines. 2258

Mystery shopper 2259 2260

A MS, dressed as a worker, will visit the outlets. He/she will state: “I would like to buy some drugs for my 2261

friend – we are travelling and work in construction – may I see which ones you have so I can choose?” 2262

When the MS is offered medicine(s), the MS will ask “Do you have any other makes, please ?” 2263

Twenty tablets of each preparation and five vials of parenteral medicines will be requested. After visiting each 2264

outlet the research coordinator will debrief the MS, using a debriefing form including feedback about his/her 2265

experience and challenges with the study methodology and how these were overcome and could be improved. 2266

List of medicines 2267

Artemether-lumefantrine (tabs) 2268

Artesunate (tabs) 2269

Artesunate (iv/im) 2270

Artemether (im) 2271

Chloroquine (tablet or capsule, syrup) 2272

Chloroquine (im) 2273

Mefloquine (tabs) 2274

Quinine (tablet or capsule) 2275

Quinine (iv/im) 2276

Sulphadoxine-pyrimethamine (tabs) 2277

Dihydroartemisinin-piperaquine (tabs) 2278

2279

V. Sample labelling 2280

Samples will stored in zip lock plastic bags and labelled with the date of collection, the place of collection 2281

(provider/outlet identified by a unified ID code) and the name of the collector. Samples will be kept in a foam 2282

box and sent to ………………………………as early as possible for storage at + 4C in a refrigerator. 2283

2284

VI. Data analysis 2285

All information will be double entered in a pre-established Epi-Data database. It will be analysed in Stata. 2286

2287

VII. Conducting MS 2288


page 95

1. Ensure that the MS has the necessary equipment: credit in his/her mobile phone and cash to pay for the 2289 medicines. 2290

2. If more than one shopkeeper is present at the shop, the MS should wait for the first shopkeeper 2291 available. 2292

3. If the shopkeeper/health-care provider asks you who you are, state your “identity” according to the 2293 scenario 2294

4. If the shopkeeper health-care provider guesses that you are a “fake”, do not deny it but explain the 2295 reason for your visit. Specify that there will be no consequences for them or any other provider as this 2296 research is for study purposes only. Ask them not to reveal your identity to other providers in the area as 2297 you will be going to them as well. 2298

5. At the end of every visit go to a discreet place where you can record the price you paid for the drugs, 2299 and note the name of the provider/outlet on a plastic bag in which you will keep the drugs you were sold. 2300 Also take some notes in your notebook on the interaction: how many people where there, how long it 2301 took you, was it easy to convince the provider to give you drugs without a test. 2302

5. Before leaving the village call the research coordinator to announce your return. Debrief with the 2303

research coordinator as soon as possible and give him/her the medicines bought. When the MS returns 2304

the research coordinator should perform the debriefing. Transcribe the reported interaction 2305

in…………….language. Translate in English using a meaning-based translation method (rather than 2306

literal or interpretative approach). The original text will remain in the document with the translation 2307

made beneath each section. Sections of text will be double-checked for accuracy of translation by other 2308

members of the field team. Enter data into Epidata. 2309

2310

VIII. Precaution-Safety guidelines 2311

1. All team members will carry the phone numbers of all other team members. 2312

2. All MS team members will provide an emergency contact number to the senior staff member and 2313

Research Coordinator prior to departure in the field. 2314

3. Locally-hired actors will be provided with an information sheet written in …………..detailing the 2315

objective of the research and giving the contact details for one of the team members 2316

speaking………………... They will be told to show this paper in case they have problems with the 2317

surveyed health providers during the course of the research. 2318

4. Confidentiality will be maintained at all times. This includes restricting access to all data to the 2319

investigators and data entry clerks. Individual identification numbers and not names will be used in 2320

databases. 2321

2322

IX. Timeline 2323

Training: ……week 2324

MS field data collection in five provinces: …… months (approx.) 2325

Data entry and scanning: …….weeks 2326

Contingency: …… weeks 2327


page 96

ANNEX 6 2328

SOP FOR OVERT SAMPLING METHODOLOGY 2329

2330

SOP. No. 2 Overt sampling

Date of issue:

Prepared by: Review by: Approved by:

Version no.

2331

I. Objective 2332

To undertake ………………..medicine sampling in all private providers of the ………………study site 2333

in ……………………country 2334

The overt medicine survey will cover all private providers of medicines within the study catchment areas, 2335

including registered providers in …………..district and ……………………..town, village health volunteers 2336

in………………district villages and unregistered outlets. This will include private clinics, pharmacies, general 2337

shops, drug sellers and mobile providers, excluding providers using only traditional drugs and diagnostics. 2338

Key outcomes will be: 2339

1. What medicines are offered to overt collectors at registered, unregistered outlets and by VHV? 2340 2. Proportion of outlets selling monotherapy. 2341 3. What is the chemical quality of these antimalarials? 2342

2343

II. Rationale 2344

2345

The purpose of this study is to investigate the quality of ……………………………………. in 2346

the ………………..study sites in country…………………………... 2347

The study will be conducted in one phase were an overt sampling will be conducted in the providers of the study 2348

area. 2349

III. Requirements 2350

2351

Equipment Consumables Documents/register

- GPS

- Car

- Phone

- Identification badge

- Bag for documents

- Watch or clock

- Fuel

- Phone credit

- Notebook

- Pens/Pencil

- Clipboard

- Batteries for GPS

- Ziplock bag

- Labels

- Information letter for

VHV

- Diary/scheduling sheet

- 25 copies of overt

sampling forms and

SOPS

- Information sheets and

consent form

2352


page 97

IV. Procedure 2353

2354

1. Outlet survey. This section is designed to identify the type of each outlet in the survey. Doing so will 2355

provide an assessment of the numbers and types of outlet in which people procure antimalarials. 2356

2. Obtaining consent form. This section ensures that providers understand the purpose of this study and 2357

their role in it, should they voluntarily consent to be part of the study. It also screens outlets that are not 2358

eligible to be surveyed out. 2359

2360

V. Inclusion criteria 2361

2362

Providers selling ……………………………….drugs in ………………………… or …………………………. 2363

town. 2364

2365

VI. Data collection tools 2366

2367

Data collection will be done using the …………………….. form 2368

2369

VII. Data entry 2370

2371

Data will be double entered into a predesigned database that will be linked to subsequent data using the unique 2372

ID code. 2373

2374

VIII. Conducting private providers census 2375

2376

1. The primary sampling unit is ……………………. town and …………………… province. Prepare a 2377

scheduling sheet where survey will take place. 2378

2. Obtain list of known outlets. This will be supplemented by additional information when you are in the 2379

field. 2380

3. Ensure you have necessary equipment, consumables and documents. 2381

4. Coordinator and supervisors should identify a number of potential key informants who can guide them 2382

to the other unregistered and mobile providers, including local health officials, community leaders, 2383

villagers and other key informants. 2384

5. Through discussion with these key informants, the supervisor should write a full list of potential 2385

medicine providers in their area. 2386

6. Each member of the study team should then be allocated an area for sampling. 2387

2388

2389


page 98

Conducting the census 2390

1. Before going to each first shop, fill in the details in Section A. The first outlet code should be “01” , the 2391

next one “02”, etc. Remember to fill in the ID code at the top of the form. 2392

2. Start walking! 2393

3. At the shop the surveyor should him/ herself, followed by a brief summary of what the project is all 2394

about and why the private provider is asked to participate in the study. Explain that you are involved in 2395

the ……………………………… project which is about improving the usage ……………… in private 2396

providers in ………………………. Explain that you are collecting information from all private 2397

providers in that area. Ask the screening questions. 2398

4. If there is more than one person in the shop introduce yourself to the one who serves customers first and 2399

then ask for the owner or any other person selling medicines. 2400

5. If the shop is eligible for inclusion give the information sheet and ask for consent and begin the 2401

questionnaire. 2402

6. Also ask about other private providers in the vicinity to identify any health facilities that are not on the 2403

original list. This is called a snowball sampling method and will be particularly useful for identifying 2404

other private providers. 2405

7. Record information of closed outlets. If outlets are not permanently “closed down” interviewers will try 2406

to make call-backs to these outlets 2407

8. If the informants work both in a health centre and a private outlet conduct the interview in the private 2408

outlet. 2409

9. At the end of every visit fieldworkers will position every outlet using a global positioning system (GPS). 2410

One longitudinal and latitudinal reading will be taken. Turn on the GPS and check it is showing 2411

coordinates. Record GPS coordinates and altitude of the facility in the census form. Turn off the GPS. 2412

10. Move to the next provider and repeat steps 1–10. 2413

11. Participants will be able to drop out at any time during the survey and do not have to reply to questions 2414

if they do not wish to. Reasons for not participating or not answering questions will be obtained. 2415

12. Ensure that all the information collected is input into the database. 2416

2417

IX. Precaution-Safety guidelines 2418

2419

1. Check GPS battery to ensure it is fully charged before taking off 2420

2. Choose a suitable position to ensure that GPS has a high resolution. Ideally it must be in an 2421

open space away from trees and any roof. 2422

3. Do not dispose of used batteries into fire or near fire because of risk of explosion. 2423

4. Always carry an extra pack of batteries 2424

2425


page 99

X. Quality assurance 2426

2427

1. Appropriate and rigorous selection and training of fieldworkers. 2428

2. Daily supervision of data collection in the field for the survey, checking completed census 2429

forms, ensuring that all potentially eligible outlets are visited. 2430

3. Confidentiality will be maintained at all times. This includes restricting access to all data to 2431

the investigators and data-entry clerks when needed, as well as using individual identification 2432

numbers on software programs for data analysis. 2433

4. Spot check of a random sample of 5% of outlets censused which met the screening criteria and 2434

5% that did not. For those meeting the screening criteria this does not need to entail a full 2435

repeat of the questionnaire, but rather is a quality control visit to assess whether the interview 2436

was completed, whether a few key questions were answered appropriately. For those not 2437

meeting the screening criteria the spot check should verify that the screening criteria where 2438

appropriate were completed. This is to ensure that interviewers do not report “No” to these 2439

questions just to skip the outlet and reduce their workload. 2440

2441

XI. Limitation of the procedure 2442

2443

1. Cloud cover may affect the resolution of the GPS. 2444

2. Some attendants may not give contact information in the absence of their chief. 2445

3. Some people may not accept to be part of the study. 2446

4. Some private providers who work in a health centre might refuse to receive the team in their 2447

private clinic 2448

2449

XII. Additional notes 2450

2451

1. All survey team members should be polite and friendly, carry your authorization documents, 2452

put on identification badges and work according to procedure stipulated in SOP. 2453

2. Before inviting participants all survey team members must make sure they meet the eligibility 2454

criteria set out in the study protocol for each target group. 2455

3. Thank the participant at the end of every visit. 2456

4. No payment or gift will be given to providers who participate to this study. 2457

2458

2459


page 100

ANNEX 7 2460

TESTING PROTOCOL 2461

2462

Survey of the quality of antimalarial medicines …………… 2463

2464

TESTING PROTOCOL 2465

2466

Product Tests to be performed and specifications for testing

Artemether/

Lumefantrine tablets

FDC

International Pharmacopoeia monograph

• Appearance - general requirements of Ph.Int. for tablets

• Uniformity of mass – Ph.Int.

• Identity – HPLC as for assay

• Assay – Ph.Int. – HPLC

• Artemether related substances – Ph.Int. – TLC

• Dissolution – laboratory-validated method similar to the method used for

Lumefantrine and Artemether tablets in USP Non–US Monograph (authorized

1.3.2009);

Requirements: lumefantrine – not less than 60% (Q) in 45 min; artemether –

not less than 40% (Q) in 1 hour and not less than 60% (Q) in 3 hours.

Artesunate /

Amodiaquine tablets

Co-blistered

Artesunate tablets:

International Pharmacopoeia monograph

• Appearance – general requirements of Ph.Int. for tablets



• Assay – Ph.Int. – HPLC

• Artesunate related substances – Ph.Int. – HPLC

• Dissolution – Ph.Int.

Amodiaquine tablets:

US Pharmacopeia monograph

• Appearance – - general requirements of Ph.Int. for tablets

• Uniformity of mass – PhInt

• Identity – USP

• Assay – USP – UV spectrophotometry

• Dissolution – USP

Artesunate /

Amodiaquine tablets

FDC

Laboratory-validated methods




• Assay – HPLC; 90–110%

• Artesunate-related substances – tested without specifications


page 101

• Dissolution – not less than 75% of each API at 30 minutes

Sulfadoxine/

Pyrimethamine tablets

FDC

US Pharmacopeia monograph



• Identity – HPLC

• Assay – USP – HPLC

• Dissolution – USP

Sulfamethoxypyrazine/

Pyrimethamine tablets

FDC

Manufacturer-validated methods

• Appearance – package leaflet


• Identity global positioning system HPLC as for assay

• Assay – HPLC 90–110 %

• Dissolution – not less than 80% (Q) in 30 min

2467

2468


page 102

ANNEX 8 2469

CONTENT OF THE ANALYTICAL TEST REPORT 2470

2471

Content of the analytical test report 2472

Analytical test report2 2473

An analytical test report usually includes a description of the test procedure(s) employed, results 2474

of the analysis, discussion and conclusions and/or recommendations for one or more samples 2475

submitted for testing. 2476

The analytical test report shall in accordance with the good practices for pharmaceutical quality 2477

control laboratories provide the following information: 2478

1. Name and address of the laboratory performing the sample testing. 2479

2. Number/code of the analytical test report. 2480

3. Name and address of the originator of the request for testing. 2481

4. Laboratory registration number of the sample. 2482

5. Sample code from the sample collection form. 2483

6. Date on which the sample was received. 2484

7. Name of the country where the sample was collected. 2485

8. Sample product name, dosage form, active ingredients, strength, package size, type and 2486

packaging material of primary container. 2487

9. Description of the sample (both product and container). 2488

10. Batch number of the sample, expiry date and manufacturing date, if available. 2489

11. Name and address of the manufacturer. 2490

12. Reference to the specifications used for testing the sample, including the limits. 2491

13. Reference to the reference standards used for quantitative determinations. 2492

14. Detailed results of all the tests performed (numerical results, if applicable), including any 2493

observations made during analysis. 2494

15. Conclusion whether or not the sample was found to be within the limits of the specifications 2495

used. 2496

16. Discussion of the results obtained. 2497

17. Date on which the test was completed. 2498

18. Signature of the head of the laboratory or authorized person. 2499

2500

2 World Health Organization. WHO Technical Report Series , No. 957 (2010). Annex 1: Good practices for

pharmaceutical quality control laboratories;

http://www.who.int/medicines/areas/quality_safety/quality_assurance/GoodpracticesPharmaceuticalQualityCont

rolLaboratoriesTRS957Annex1.pdf?ua=1.


page 103

EXAMPLE OF SURVEY PROTOCOL 2501

based on 2502

WHO survey of the quality of antituberculosis medicines circulating in selected newly 2503

independent states of the former Soviet Union 2504

Version: Final (May 2009) 2505

1. GENERAL OUTLINE 2506

1. Introduction 2507

WHO estimates that nearly half a million multidrug-resistant tuberculosis (MDR-TB)3 cases emerge 2508

each year, as a result of inadequate or poorly-administered treatment regimens, insufficient supply or 2509

quality of anti-TB medicine, and transmission of drug-resistant strains. Newly independent states of 2510

the former Soviet Union (NIS) have some of the highest prevalence rates of MDR-TB, with 2511

proportions of MDR-TB among new and previously treated TB cases have been reported as high as 2512

28.3% and 61.6%, respectively.4 2513

It has been hypothesized that one of the most important factors for the resurgence of TB, and the high 2514

rates of MDR-TB, in the NIS was the socioeconomic crisis that followed the disintegration of the 2515

Soviet Union in 1991.5,6,7

This crisis resulted in interruptions in medicines supply and overall 2516

deterioration of the health sector, which had an impact on the transmission of and susceptibility to TB 2517

and MDR-TB. The lack of standardized treatment regimens in many countries is also likely to have 2518

contributed to the development of drug resistance. 2519

Limited research has been conducted into the factors contributing to drug resistance in this region and 2520

to the marked regional and national differences in drug resistance rates. In particular there has been 2521

little consideration of the extent to which substandard, spurious, falsely-labelled, falsified and 2522

counterfeit (SSFFC) anti-TB medicines might circulate in this region. This survey aims to investigate 2523

the quality of anti-TB medicines in use in selected NIS. 2524

2525

3 MDR-TB is defined as resistance to at least rifampicin and isoniazid, the two most powerful anti-TB

medicines.

4 WHO/IUATLD Global Project on Anti-tuberculosis Drug Resistance Surveillance. Anti-tuberculosis drug

resistance in the world: fourth global report. Geneva, World Health Organization, 2008.

5 Raviglione MC et al. Tuberculosis trends in eastern Europe and the former USSR. Tubercle and Lung

Disease, 1994 Dec, 75(6):400-16.

6 Shilova MV, Dye C. The resurgence of tuberculosis in Russia. Philosophical Transactions of the Royal

Society London B: Biological Sciences, 2001 Jul 29, 356(1411):1069-75.

7 Global tuberculosis control: a short update to the 2009 report. Geneva, World Health Organization, 2009

(available at: http://whqlibdoc.who.int/publications/2009/9789241598866_eng.pdf, accessed 23 June 2011).


page 104

2. Definitions 2526

Country code for the purposes of this project means a 2-digit code used for the country in an 2527

email address. 2528

Delivery centre for the purpose of this project means a point, where a medicine enters the 2529

country, central stores and stores, where a medicine is kept during the in-2530

country distribution. 2531

Product abbreviations (for the purposes of coding samples): 2532

Isoniazid = H 2533

Rifampicin = R 2534

Isoniazid + rifampicin = H-R 2535

Kanamycin = Km 2536

Ofloxacin = Ofx 2537

Sample for the purposes of this project means an item collected from each 2538

presentation at the same collection site. That means that a product of the same 2539

name, content of APIs, the same dosage form, strength, batch and from the 2540

same manufacturer collected in two different sites represents two samples. 2541

Treatment centre for the purpose of this project means the final site, where a medicine is 2542

delivered and where it is provided to a patient. 2543

3. Objectives 2544

The WHO Stop TB and Essential Medicines and Health Products Departments, and counterparts in the 2545

WHO Regional Office for Europe, are collaborating with NMRAs to study the quality of first- and 2546

second-line anti-TB medicines circulating in the countries with the highest MDR-TB and XDR-TB 2547

rates. The project will commence in countries of Eastern Europe and the NIS as indicated in this study 2548

protocol and will subsequently expand to China and India. 2549

The aim of this project is therefore to evaluate the pharmaceutical quality of widely used anti-TB 2550

medicines (first- and second-line) obtained at public and private sector procurement and treatment 2551

centres in selected countries of Eastern Europe and the NIS. The following questions will be addressed: 2552

• Which anti-TB medicines are mostly used? 2553

• What proportion of anti-TB medicines samples, including fixed-dose combination (FDC) 2554

products, collected at approved procurement and treatment centres fail quality testing? 2555

• Which specific quality tests do the samples fail, if any? 2556

• Are any of the deficiencies critical, i.e. could most likely affect treatment efficacy and/or 2557

cause harm to the patient? 2558

The results of this study are expected to assist responsible authorities in the countries surveyed to 2559

adopt regulatory actions, if necessary, and to develop appropriate quality assurance strategies for anti-2560

TB medicines. They will also provide information for WHO to adapt its prequalification procedures. 2561

Finally, they will be of use in awareness and advocacy programmes on quality issues in anti-TB 2562

medicines in general. 2563


page 105

Limitations of the study 2564

This quality survey cannot completely solve the problem of bioavailability which may be of special 2565

relevance in case of rifampicin. 2566

Appropriate bioavailability must be assured by complex regulatory measures which include 2567

compliance of manufacture with GMP standards, appropriate quality specifications for both active 2568

ingredients and finished product and proof of bioequivalence with proper comparator. These 2569

parameters should be assessed during registration and followed up in the post-registration period. 2570

From anti-TB medicines included in this survey only rifampicin strictly requires bioequivalence 2571

testing in vivo, for other appropriate bioavailability may be judged upon based on in vitro testing. 2572

However, organization of bioequivalence studies for rifampicin-containing products cannot be applied 2573

as a quality control method without proper understanding of GMP compliance and registration 2574

conditions and as a control method for the given purpose is considered unethical. Instead of that 2575

comparative dissolution is performed to respond at least partially to the issue of bioavailability. 2576

It is obvious that study findings are relevant only to tested samples and extrapolation to individual 2577

batches and products is limited. 2578

4. Methodology 2579

4.1 Participating countries 2580

The study should involve some six countries where anti-TB medicines are not produced or are 2581

produced on a small scale. 2582

The following nine countries were approached before the selection of countries was made: 2583

• Armenia 2584

• Azerbaijan 2585

• Belarus 2586

• Estonia 2587

• Kazakhstan 2588

• Latvia 2589

• Moldova 2590

• Ukraine 2591

• Uzbekistan 2592

The countries where samples should be collected were selected as below. 2593

1. A questionnaire (Annex 1) was sent to the NMRA in the nine above-mentioned countries. The 2594

questionnaire asked for first- and second-line anti-TB medicines, including FDCs, that were in 2595

2008 used in the country in both public and private sectors, the volumes used, the 2596

manufacturers of these medicines and also which institutions were involved in importation and 2597

distribution of these medicines (to identify sampling locations). 2598

2. Following the compilation of the results of the questionnaire six relevant countries were 2599

selected, focusing on those where the widest choice of medicines selected for this study was in 2600

use. 2601


page 106

3. A certain amount of medicines is procured through the Global Drug Facility (GDF), including 2602

the Green Light Committee. These medicines should also be tested. However, it is important 2603

to select countries using other sources than GDF. 2604

4. An official WHO letter was sent to the ministries of health of the six selected countries. The 2605

letter described the project and asked for the willingness of the ministry of health to 2606

collaborate on this project. Without the consent of the ministry of health the country was not 2607

included in the project. 2608

Based on the results of the questionnaire and taking into account the above mentioned aspects the 2609

following countries were selected for sampling: 2610

• Armenia 2611

• Azerbaijan 2612

• Belarus 2613

• Kazakhstan 2614

• Uzbekistan 2615

• Ukraine 2616

The ministries of health of the above-mentioned six countries also nominated focal persons for this 2617

project. 2618

4.2 Anti-tuberculosis medicines surveyed 2619

Based on the information on medicines used in individual countries the final medicines selection was 2620

made. Apart from availability volumes and sources of medicines used in individual countries, the 2621

susceptibility of medicines to quality deterioration such as low stability was taken into account. 2622

Based on these considerations the following medicines were selected to be surveyed: 2623

• Isoniazid tablets 300 mg, 100 mg, injection 10% (5 mL) 2624

• Rifampicin capsules 300 mg, 150 mg 2625

• Isoniazid/rifampicin tablets 150 mg/300 mg, 150/150 mg, 75/150 mg, 60/60 mg, 30/60 mg 2626

• Kanamycin powder for injection 1 g 2627

• Ofloxacin tablets/capsules 200 mg, 400 mg, solution for injection 0.2% (200 mL) 2628

2629

4.3 Study period 2630

The study period should last from summer 2008 to beginning 2010 and as indicated in Table 2. 2631

Table 2. Timeframe and responsible officers for the survey of the quality of anti-TB medicines 2632

circulating in selected countries of Eastern Europe and the NIS. 2633

Activity Timeframe Responsible officers

Anti-TB medicines quality survey

questionnaire sent to the nine countries

June 2008 EURO Pharmaceutical

Programme

Compilation of results of the questionnaire October 2008 EURO Pharmaceutical

Programme

Selection of countries and anti-TB

medicines to be surveyed

November 2008 EURO Pharmaceutical

Programme and HQ TB and

EMP/QMS departments

Letters sent to ministers of health of six December 2008 EURO Pharmaceutical


page 107

countries for collaboration in this project Programme

Laboratory(ies) selected for the

performance of the tests

January 2009 HQ EMP/QMS

Preparations of contracts with NMRA to

cover for national expenditures

May 2009 EURO Pharmaceutical


EMP/QMS departments

Meeting held with selected countries and

pharmaceutical experts

May 2009 EURO Pharmaceutical

Programme and HQ EMP/QMS

Collection of samples by NMRAs June–September

2009

NMRAs in close collaboration

with WHO country, region and

HQ staff

Analysis of medicines quality by selected

laboratories

October 2009–April

2010

HQ EMP/QMS

Compilation of results April–June 2010 EURO Pharmaceutical


EMP/QMS

Meeting held with the participating

countries (ministries of health and NMRA

staff) and WHO pharmaceutical and TB

experts to discuss the final results and the

actions needed

June 2010 EURO Pharmaceutical


EMP/QMS departments

2634

4.4 Selection of sample collection sites 2635

Samples should be collected from the following levels of distribution chain: 2636

• level 1 – delivery centres (private and/or public); as the aim of the survey is to assess the 2637

quality of medicines available to patients samples should be collected at the manufacturing 2638

sites only in cases where it is not possible to collect appropriate samples at the other sites; 2639

• level 2 – wholesalers, regulated retailers including dispensing facilities and treatment centres, 2640

in both private and public sectors. 2641

A meeting should be held with participation of focal persons from individual countries and WHO 2642

representatives from the EURO Pharmaceutical Programme and HQ TB and EMP/QMS Departments 2643

to explain the project, to provide detailed instructions and to identify names and addresses of sample 2644

collection sites in each country involved. 2645

Samples will be collected by the staff of the NMRAs in cooperation with the WHO country offices’ 2646

staff in the respective country and with backup support from the EURO Pharmaceutical Programme 2647

and HQ EMP/QMS Department. 2648

4.5 Sample collection 2649

For the purposes of this project a sample means an item collected from each presentation at the same 2650

collection site. That means that a product of the same name, content of APIs, the same dosage form, 2651

strength, batch and from the same manufacturer collected in two different sites represents two samples. 2652


page 108

A detailed national sampling plan will be prepared for each country by the focal person in the NMRA 2653

in cooperation with WHO pharmaceutical experts (Annex 2). The focal person in each country will 2654

arrange for training of collectors to be familiar with the national sampling plan and instructions. 2655

In general the following information shall be included in the national sampling plan: 2656

• identification of the country and the person responsible for sampling; 2657

• names and addresses of the sites, where samples shall be collected; 2658

• identification of medicines to be collected (active ingredients by INNs, dosage form, 2659

strength, manufacturer, number of batches to be collected in each site and number of units 2660

to be collected per batch of each medicine); 2661

• maximum number of samples collected per country; 2662

• detailed instructions for collecting samples (see below). 2663

2664

Number of dosage units or multidose packages of selected medicines to be collected should allow for: 2665

• conducting the agreed tests; 2666

• possible confirmative testing due to OOS investigations; 2667

• retention samples. 2668

The following general rules are used, if not justified otherwise: 2669

Dosage form Packaging

(typical)

Number of dosage units or

multidose packs per batch

Tablets and capsules

(immediate-/modified-release,

chewable, dispersible, etc. )

Blisters, co-blisters,

bottles, securitainers

Approx.100 units

(e.g. 5 packs of 20 units

3 packs of 30 units

3 packs of 40 units

2 packs of 60 units

1 pack of 90 units and above)

In case of co-packaged FPPs

approx.100 units shall be collected

from each medicine.

Multidose oral

solutions/suspensions, powder for

oral solution/suspension and


Multidose bottles and

vials

6 containers of 60 mL /100 mL

3 containers of 240 mL

Single-dose powders for oral

solution/suspension and single-dose


Sachets and single-dose

bottles, vials and

ampoules

Unless otherwise specified, 15 units

(20 units if dose is below 50 mg)

2670


page 109

Instructions for sample collection: 2671

• the time period, within which samples should be collected in the countries and the deadline for 2672

sending the last sample to the testing laboratory, should be clearly indicated and followed; 2673

• the minimum quantity of sample per batch and number of batches to be collected from each 2674

collection site for each selected medicine as indicated in the sampling plan shall be followed. 2675

Note that there should not be a mix up with batches, all units of one sample must be of the same 2676

batch. In the case that in a collection site the required number of packages of the same batch is 2677

not available, sample of that particular medicine is not collected; 2678

• samples collected should have at least six months remaining to expiry; 2679

• only unopened original packages shall be collected; 2680

• the medicine samples should not be taken out of the original primary packaging and outer 2681

containers (though removal from large secondary packs is appropriate). Containers such as bottles 2682

and vials should not be opened; 2683

• the medicine labels and package leaflets should not be removed or damaged; 2684

• sampling will be recorded using the sample collection form (Annex 3). Whenever the required 2685

information is not available, it should be indicated in the appropriate space on the sample 2686

collection form where also any abnormalities should be recorded; 2687

• in order to avoid confusion each sample will be identified by a unique sample code (for coding 2688

system see the sample collection form, Annex 3) specified in the sample collection form as well 2689

as on all the original packages belonging to the respective sample. Packages belonging to one 2690

sample and the sample collection form will be kept together (e.g. blisters inserted in a dedicated 2691

envelope marked with the appropriate sample code and trade name of the product); 2692

• manufacturer’s batch certificates of analysis will be collected with samples, if available, and kept 2693

with the sample collection form; 2694

• the samples should be collected and kept under controlled conditions, as per label requirement. 2695

The cold chain should be maintained, where required. 2696

If needed, the appropriate arrangements shall be made with treatment centres to ensure that there is no 2697

shortage due to collection of samples (e.g. requesting for replacements of medicines). 2698

4.6 Storage and dispatch of samples 2699

Storage and transport of the sample should be done according to the requirements set out in paragraph 2700

2.3 of WHO’s Guidelines for Sampling of Pharmaceutical Products and related materials:8 2701

• the samples should be kept in original packaging and under storage conditions specified on 2702

the label; 2703

• for transport all samples should be packaged adequately and transported in such a way as to 2704

avoid breakage and contamination during transport. Any residual space in the container 2705

should be filled with a suitable material. Where required the cold chain should be retained 2706

during storage and transport; 2707

• a covering letter, the copy sample collection form and, if available, copy of manufacturer’s 2708

batch certificates of analysis should accompany the samples; 2709

8 WHO guidelines for sampling of pharmaceutical products and related materials. In: WHO Expert Committee

on Specifications for Pharmaceutical Preparations. Thirty-ninth report. Geneva, World Health Organization,

2005, Annex 4 (WHO Technical Report Series, No.929).

http://whqlibdoc.who.int/trs/WHO_TRS_929_eng.pdf#page68


page 110

• samples with the accompanying documents should be sent straight away to the assigned 2710

testing laboratory by a courier service. For each shipment it should be clearly indicated that 2711

samples are sent for laboratory testing purposes only, will not be used on humans or animals, 2712

have no commercial value and will not be placed on the market. Low price just for customs 2713

purposes should be indicated to avoid problems with the customs clearance; 2714

• the laboratory and WHO contact point9 should be informed about the shipment and the 2715

tracking number as provided by the courier service; 2716

• copies of all sample collection forms and, if available, copies of manufacturer’s batch 2717

certificates of analysis should be sent to the WHO contact point7 after dispatch of samples. 2718

4.7 Testing laboratory 2719

An appropriate laboratory has to be selected for testing. Preferably a prequalified laboratory should be 2720

used. Should such a laboratory not be available or should it not have sufficient capacity then another 2721

laboratory should be chosen, where evidence of reliability is available. 2722

The appropriate arrangement with the laboratory has to be made. The request for testing should be in 2723

line with WHO’s guideline: Considerations for requesting analysis of drug samples10

and no sample 2724

will be sent before such an arrangement is made. An agreement for performance of work between 2725

WHO and the laboratory should be prepared and agreed upon by both parties. 2726

For this project the following laboratories have been selected: 2727

Name Address Products to be tested

AGES - PharmMed - Austrian

Agency for Health and Food

Safety

in cooperation with Laboratoire

National de Santé, Luxembourg

(AGES will be responsible for

the logistics and all the mono-

component isoniazid samples will

be sent to Austria)

Zimmermanngasse 3

A-1090 Vienna

AUSTRIA

• Isoniazid tablets, injection

COUNCIL OF EUROPE

European Directorate for the

Quality of Medicines &

HealthCare

(European Pharmacopoeia)

7 allée Kastner (entrance on rue

de la Carpe Haute) - CS 30026

F-67081 Strasbourg

FRANCE

• Kanamycin powder for

injection

• Ofloxacin tablets/capsules,

solution for injection

9 Dr Jitka Sabartova - Phone: +41 22 7913376, Fax: +41 22 7914730, E-mail: sabartovaj@who. int

World Health Organization, HSS/EMP/QSM, Prequalification Team –Laboratories, 20 Avenue Appia, CH-

1211 Geneva 27, Switzerland.

10 Considerations for requesting analysis of drug samples. In: WHO Expert Committee on Specifications for

Pharmaceutical Preparations. Thirty-sixth report. Geneva, World Health Organization, 2002, Annex 4

(WHO Technical Report Series, No. 902):

http://whqlibdoc. who. int/trs/WHO_TRS_902. pdf#page69.


page 111

Name Address Products to be tested

SGS Lab Simon S. A. Vieux Chemin du Poète 10

B-1301 Wavre

BELGIUM

Testing for identity, assay,

related substances and

uniformity of mass:

• Rifampicin capsules

• Isoniazid + rifampicin

tablets

J. W. Goethe University

Institute of Pharmaceutical

Technology

Biocenter

Max-von-Laue-Str.9

D-60438 Frankfurt am Main

GERMANY

Comparative dissolution

study:

• Rifampicin capsules

• Isoniazid + rifampicin

tablets

4.8 Tests conducted 2728

Laboratory testing of all collected samples will be performed according to the testing protocol agreed 2729

with the testing laboratories. In principle the following tests will be included: 2730

• appearance 2731

• identity 2732

• assay 2733

• related substances test 2734

• dissolution and uniformity of mass for tablets and capsules 2735

• pH value for injections and powders for injection 2736

• sterility, bacterial endotoxins tests for parenteral products. 2737

In the light of known problems with bioavailability of rifampicin contradictory outcomes of studies 2738

evaluating correlation between bioavailability and dissolution in vitro, and no clear conclusion on the 2739

recommended dissolution methodology in the literature, it has been decided to conduct comparative 2740

dissolution studies of collected products containing rifampicin. 2741

4.9 Test methods and specifications 2742

Tests methods and specifications are in general selected according to the following rules: 2743

• preferably Ph.Int. monographs should be used, if available; 2744

• if no monograph exists in the Ph.Int., then BP or USP can be used; 2745

• if there is no pharmacopoeial monograph or the existing monographs do not provide for desired 2746

tests a validated method of the laboratory or manufacturer's method, if available, should be used. 2747

2748

Dissolution methodology from various pharmacopoeias and literature was compared and conditions 2749

for comparative dissolution study of products containing rifampicin were outlined in cooperation with 2750

experts involved in the Prequalification Team and specializing on quality of anti-TB medicines, 2751

pharmaceutical technology and waivers of in vivo bioequivalence testing. 2752

For the agreed testing protocol for products selected for this survey see Annex 4. 2753


page 112

4.10 Receipt and testing of samples by a testing laboratory 2754

• Inspect each sample to ensure that the labelling is in conformance with the information 2755

contained in the sample collection form or test request. 2756

• Store the samples according to the respective medicine requirements. If appropriate ensure 2757

compliance with the cold chain. 2758

• Conduct quality testing in line with this protocol and in compliance with WHO standards 2759

recommended for quality control laboratories.11

2760

• Complete an Analytical Test Report (Annex 5). In the case that non-compliant results are 2761

found and confirmed after application of a laboratory out-of-specification procedure, report 2762

them immediately to WHO contact point7. 2763

• Keep records of each sample, accompanying document/s and retention samples for at least 2764

six months if the sample complied with the analytical test requirements, or for at least one 2765

year or until the expiry date (whichever is longer) if it did not comply. 2766

• An electronic databank (e.g. photos of medicine such as tablets, packaging, package leaflet) 2767

is recommended. 2768

5. Data management, analysis and publication 2769

Any non-compliant result found in the survey will be investigated with the respective manufacturer 2770

and NMRA. 2771

A data analyst/statistician will be hired to compile and analyse the laboratory test results. 2772

The analytical test reports of the testing laboratories will be provided to all NMRA involved in the 2773

project. The outcomes of the project will be discussed by national authorities and WHO in a meeting, 2774

and corrective actions, if necessary, will be recommended. To take the relevant measures in countries 2775

lies within the responsibility of the NMRA. 2776

Agreed outcomes and the report from the project will be published by WHO. 2777

11

World Health Organization. WHO Technical Report Series , No. 957 (2010). Annex 1: Good practices for

pharmaceutical quality control laboratories;

http://www.who.int/medicines/areas/quality_safety/quality_assurance/GoodpracticesPharmaceuticalQualityC

ontrolLaboratoriesTRS957Annex1.pdf?ua=1.


page 113

EXAMPLE OF SURVEY PROTOCOL

based on

WHO survey of the quality of anti-tuberculosis medicines circulating in selected NIS of the former Soviet Union

2. QUESTIONNAIRE

please fill in ONLY for products which are actually available in July 2008

Name and address of the site

Responsible person(s) and contact details

(phone, mobile, e-mail)

Type of site

what organization is best to approach for sampling, i.e. which one

has the longest list of TB products available.

Hospital

pharmacy

Warehouse Retail

pharmacy

Please mark with X in the box under the correct answer

Finished pharmaceutical product and

dosage form

Strength Quantity

(packs)

Manufacturer name

and manufacturing site

address

Country

of origin

Not used in

the country

Single ingredient first-line antituberculosis medicines

Ethambutol, tablet 400 mg

Isoniazid, tablet 300 mg

Pyrazinamide, tablet 400 mg


page 114

Rifampicin, capsule 150 mg;

300 mg

Streptomycin, powder for injection (vial) 1 g

Other single ingredient antituberculosis

medicines used for the first-line treatment

(please specify in the same format below)

FDC products of first-line antituberculosis medicines

Isoniazid + rifampicin, tablet 75 mg + 150 mg

150 mg + 150 mg

Ethambutol + isoniazid, tablet 400 mg + 150 mg

Ethambutol + isoniazid + rifampicin, tablet 275 mg + 75 mg

+ 150 mg

Ethambutol + isoniazid + pyrazinamide +

rifampicin, tablet

275 mg + 75 mg

+ 400 mg + 150

mg


medicines used for the first-line treatment



page 115

Single ingredient second-line antituberculosis medicines

Amikacin, solution for injections (vial 2 mL, 4

mL)

250 mg/mL

Amikacin, powder for injection (vial) 1 g

Capreomycin, powder for injection (vial) 1 g

Cycloserine, capsule 250 mg

Ethionamide, coated tablet 125 mg

Ethionamide, coated tablet 250 mg

Kanamycin, powder for injection (vial) 1 g

Levofloxacin, tablet 250 mg

Moxifloxacin, tablet 400 mg

Ofloxacin, tablet 200 mg

Ofloxacin, tablet 400 mg

Prothionamide, coated tablet 250 mg

P-aminosalicylic acid, granules 4 g

P-aminosalicylic sodium, granules 100 g


medicines used for the second-line treatment



page 116

Scored solid dosage formulations for children, preferably dispersible

Ethambutol, tablet 100 mg





Isoniazid + pyrazinamide + rifampicin, tablet 30 mg + 150 mg

+ 60 mg

Pyrazinamide, tablet 150 mg

Other scored solid dosage formulations for

children, preferably dispersible, used for TB

treatment (please specify in the same format

below)


page 117

Instructions to fill the questionnaire

1. Before completion of the questionnaire please select the SITES which will have products actually available for sampling in July 2008. The site must have

the longest list of the medicines and amounts available for sampling.

2. Complete the form only for PRODUCTS that would be available at the selected sites for sampling in July 2008.

2. If the site has more that one product in the line, e.g. Ethambutol, tablet, 400mg, please insert the row below the specified product name and complete it

with all requested information.

4. Please complete the form in electronic format (as it is in MS Excel file) and send it by email to Olexandr Polishchuk, WHO/EURO at [email protected]


July 2014

Draft document for comment

118


based on 2

WHO survey of the quality of antituberculosis medicines circulating in selected 3

NIS of the former Soviet Union 4

5

3. NATIONAL SAMPLING PLAN 6

Country: 7

Focal Person: 8

MEDICINES TO BE COLLECTED 9

• Isoniazid tablets 300 mg, 100 mg, injection 10% (5 mL) 10

• Rifampicin capsules 300mg, 150mg 11

• Isoniazid/rifampicin tablets 150/300 mg, 150/150 mg, 75/150 mg, 60/60 mg, 30/60 mg 12

• Kanamycin 1 g powder for injection 13

• Ofloxacin tablets/capsules 200 mg, 400 mg, solution for injection 0.2% (200 mL) 14

15

NUMBER OF UNITS TO BE COLLECTED PER SAMPLE 16

• Approx. 100 units for tablets/capsules 17

o In case of rifampicin capsules and isoniazid/rifampicin tablets collect for one sample per 18

each strength and each manufacturer at least 24 additional units (in intact original primary 19

packaging) for comparative dissolution study, which will be carried out by the different 20

laboratory than the other tests. Collection of smaller pack sizes are preferable in order to 21

divide samples for dispatch and not interfere with the primary packaging. 22

• 5 bottles of 200 mL 23

• 15 ampoules of 5 mL 24

• 6 vials for powder for injection 25

26

NUMBER OF BATCHES TO BE COLLECTED PER PRODUCT IN EACH SITE 27

• Maximum 3 batches per product at one collection site; if the same product (same 28

manufacturer, same dosage from, same strength) is collected in more sites, please select 29

different batches, if possible. 30

31


page 119

TOTAL NUMBER OF SAMPLES PER COUNTRY 32

60 samples (12 samples per product preferably produced by different manufacturers) 33

NAMES AND ADDRESSES OF THE SITES WHERE SAMPLE SHALL BE COLLECTED 34

Facility name Address Facility type

4. (private/public;

5. level 1/level 2;

6. wholesaler/retailer/

treatment centre/…)

6.

7.

8.

9.

10.

35

SELECTION OF PRODUCTS 36

Isoniazid 37

Strength/

Dosage

form

Pack

size

Manu-

facturer

Sampling

site

Batch

no.

Sample

code

No. of

units per

sample

Additional

24 units

needed

38

Rifampicin 39

40

Isoniazid/rifampicin 41

42

43


page 120

120

Kanamycin 44

Ofloxacin 45

46

INSTRUCTIONS FOR COLLECTORS 47

• The amount of the selected products defined above will be sampled from the identified sites. All 48

these samples are inclusive of the samples needed for the OOS investigations and retention samples. 49

• An item collected from each presentation at the same collection site will be called a sample. All units 50

(tablets, capsules, vials) of one sample must be of the same batch, there should not be a mix up 51

with batches. In the case that in a collection site the required number of packages of the same batch 52

is not available a sample of that particular medicine is not collected. 53

• Samples collected shall have at least six months remaining to expiry. Products with a shorter 54

period remaining to expiry date are not collected. 55

• One batch of each product will be collected from each collection site and only unopened original 56

packages shall be collected. 57

• The medicine samples should not be taken out of the original primary packaging and outer 58

containers (though removal of blisters from large secondary packs is appropriate). Containers such 59

as bottles and vials should not be opened. 60

• The medicine labels and package leaflets should not be removed or damaged. 61

• Sampling will be recorded using the sample collection form (Annex 3). Whenever the required 62

information is not available it should be indicated in the appropriate space on the sample collection 63

form, where also any abnormalities should be recorded. 64

• In order to avoid confusion each sample will be identified by a unique sample code (for coding 65

system see the sample collection form, Annex 3) specified in the sample collection form as well as 66

on all the original packages belonging to the respective sample. Packages belonging to one sample 67

and sample collection form will be kept together (e.g. blisters inserted in a dedicated envelope 68

marked with the appropriate sample code and trade name of the product). 69

• Manufacturer’s batch certificates of analysis will be collected with samples, if available, and kept 70

with the sample collection form. 71

• The samples should be collected and kept under controlled conditions as per label requirement. 72

The cold chain should be maintained where required. 73

• Samples should be collected in all the countries involved during XXX and the deadline for 74

sending the last sample to the testing laboratory is XXX 75


page 121


based on 77

WHO survey of the quality of antituberculosis medicines circulating in selected NIS of the 78

former Soviet Union 79

80

4. SAMPLE COLLECTION FORM 81 * 82

Country: Sample code: 83

(Country code/product abbreviation/sequence number/sampling 84

date ddmmyy)**

85

Name of location/place where sample was taken: 86

87

Address (with telephone, fax number and email address, if applicable): 88

89

90

Organization and names of people who took samples: 91

1. 92

2. 93

Product name of the sample: 94

Name of active pharmaceutical ingredient(s) (INN) with strength: 95

96

Dosage form (tablet, capsule, powder for injection, etc.): 97

Package size, type and packaging material of the container: 98

99

Batch/lot number: 100

* This sample collection form should always be kept with the sample collected. Proper sampling procedures

should be followed.

** Product abbreviations: Isoniazid = H, Rifampicin = R, Isoniazid + rifampicin = H-R, Kanamycin = Km,

Ofloxacin = Ofx. Sample code system can be extended to be appropriate for a particular country collection

system.


page 122

122

Date of manufacture: Expiry date: 101

Regulatory status in the country, registration number, if applicable: 102

103

Name and address of the manufacturer: 104

105

Quantity collected (number of sample units or of multidose containers taken): 106

107

Initialize first page 108

Product name: Sample code: 109

Storage/climatic conditions at sampling site/point (temperature and humidity, indication of conditions 110

during daytime only is acceptable, comments on suitability of premises where products are stored at the 111

particular site for the NMRA information): 112

113

114

115

Abnormalities, remarks or observations that may be considered relevant, if any: 116

117

118

119

120

121

122

Date: 123

Signature of person(s) taking Signature of representative of the 124

samples establishment where sample(s) was 125

taken (optional) 126


page 123

1. .............................................................................................................................................................. 127

128

2. ............................................................................................................................................................. 129

130

Note: Samples collected must remain in their original containers, intact and unopened. 131

132


page 124

124


based on 134

WHO survey of the quality of antituberculosis medicines circulating in selected NIS of the 135

former Soviet Union 136

137

5. TESTING PROTOCOL 138

139

Product Tests to be performed and specifications for

testing

Reference substances

1. Isoniazid tablets

100 mg, 300 mg

USP



• Uniformity of mass

• Assay – HPLC

• Dissolution – UV

USP:

Isoniazid – 1349706, 200 mg

2. Isoniazid injection

10% (5 mL)

USP


• Visual inspection – clear and free from visible

particulate matter

• Volume in container/Extractable volume


• Assay – HPLC

• pH – 6.0 – 7.0

• Sterility (to be performed for one sample per

batch/manufacturer)

USP:

Isoniazid – 1349706, 200 mg

3. Rifampicin capsules

150 mg, 300 mg

Ph.Int.




• Assay – HPLC

• Related substances – HPLC

• Comparative dissolution study – 2 sets of

comparison – 1 using Ph.Int. conditions, the

other USP/BP conditions, 7 points up to 60

min

Ph.Int. :

9930409 – Rifampicin, 300 mg

9930410 – Rifampicin quinine,

200 mg

BP:

627 – 3-formylrifamycin, 25 mg

4. Isoniazid/rifampicin

tablets

150/300 mg, 150/150

mg, 75/150 mg, 60/60

mg, 30/60 mg

Ph.Int.




• Assay – 2x HPLC

• Related substances – for rifampicin only –

HPLC

• Comparative dissolution study – 1 set of

Ph.Int. :

9930331 – Isoniazid, 100 mg

9930409 – Rifampicin, 300 mg

9930410 – Rifampicin quinone,

200 mg

BP:

627 – 3-formylrifamycin, 25 mg


page 125

Product Tests to be performed and specifications for

testing

Reference substances

comparison using laboratory method based on

Ph.Int. conditions for Rifampicin tablets, 7

points up to 60 min.

5. Kanamycin powder

for injection

1 g (vial)

USP monograph for kanamycin injection



• Assay – HPLC with amperometric detection

• pH – 3.5– 5.0


manufacturer)

USP:

Kanamycin Sulfate – 1355006,

200 mg

Amikacin – 1019508 (for system

suitability), 300 mg

6. Ofloxacin

tablets/capsules

200 mg, 400 mg

USP monograph for ofloxacin tablets




• Assay – HPLC

• Related substances – HPLC

• Dissolution – UV

USP:

Ofloxacin – 478108, 200 mg

7. Ofloxacin solution

for infusion 0.2%

(200 mL)

USP monograph for tablets

• Appearance – manufacturers' specification:

clear, light yellow liquid

• Visual inspection – clear and free from visible

particulate matter

• Volume in container/Extractable volume – not

less than the nominal volume


• Assay – HPLC with USP limits for tablets

90.0–110.0%

• Related substances – HPLC with USP limits

for tablets

• pH – manufacturers' specification

• Bacterial endotoxins (to be performed for one

sample per batch/manufacturer)

The manufacturer uses test for pyrogens.

Limits for BE in products containing ofloxacin

found in:

o Brazilian Pharmacopoeia (max.5 E. U. /mg

of ofloxacin for ofloxacin injection) and

o Chinese Pharmacopoeia (less than 0.5 E.

U. /mL for ofloxacin and sodium chloride

injection and less than 0.75 E. U. /mg for

ofloxacin substance)


manufacturer and in case a positive BE result is

found)

USP:

Ofloxacin – 478108, 200 mg

140


page 126

126


6. CONTENT OF THE ANALYTICAL TEST REPORT 142

143

Analytical test report 144

The report of the results, including the final conclusion of the analysis of a sample which has been 145

submitted by a laboratory in another country or in the field not having appropriate facilities to 146

perform certain tests, and issued by the official pharmaceutical control laboratory that performed the 147

test. This is often in the same style as a certificate of analysis. 148

The analytical test report shall in accordance with the good practices for national pharmaceutical control 149

laboratories provide the following information: 150

7. name and address of the laboratory performing the sample testing; 151

8. number/code of the analytical test report; 152

9. name and address of the originator of the request for testing; 153

10. sample code from the sample collection form; 154

11. date on which the sample was received; 155

12. name of the country where the sample was collected; 156

13. sample product name, dosage form, active ingredients, strength, package size, type and packaging 157

material of primary container; 158

14. description of the sample (both product and container); 159

15. batch number of the sample, expiry date and manufacturing date, if available; 160

16. name and address of the manufacturer; 161

17. reference to the specifications used for testing the sample, including the limits; 162

18. reference to the reference standards used for quantitative determinations; 163

19. results of all the tests performed (numerical results, if applicable); 164

20. conclusion whether or not the sample was found to be within the limits of the specifications used; 165

21. date on which the test was performed; 166

22. signature of the head of the laboratory or authorized person. 167

168

*** 169

170

171

Recommendations on the content of a survey protocol for€¦ · 37 SCHEDULE FOR THE PROPOSED...

Documents

Transcript of Recommendations on the content of a survey protocol for€¦ · 37 SCHEDULE FOR THE PROPOSED...