Coming in from the ColdBenchmarking and Pharmacovigilance

Chris HolmesWorld Class International, Denmead, Hants, UK

COMMENTARY Int J Pharm Med 2003; 17 (2): 65-681364-9027/03/0002-0065/$30.00/0

© Adis Data Information BV 2003. All rights reserved.

Pharmaceutical companies today must maintain a pharma-covigilance operation to protect patients, to satisfy the rising de-mands of regulators and to protect themselves. Numerous well-publicised incidents have made this inevitable. However, ourexperience of facilitating the Pharmacovigilance BenchmarkingClub suggests that recognising pharmacovigilance as an activitycrucial to business performance, a source of added value, willreap richer rewards.

1. Satisfaction or Benefit?

Some recent incidents have provided regulators with strongpublic safety grounds for casting their nets both wider and deeper.But how should the pharmacovigilance operation be organised inresponse? There are two ways to look at regulatory compliancein reporting adverse events and completing Periodic Safety Up-date Reports (PSURs).

First, pharmaceutical companies can regard compliance asan end in itself, separate from its other activities – a burden thathas to be borne. These pharmaceutical companies will aim purelyat satisfying the regulators, organising for meticulous, on-timecompletion; the focus on form rather than content – a pharma-covigilance process that is marginal and essentially administra-tive.

A second view is that, “if we’ve got to do it, let’s get posi-tive”. The focus is on the business utility of the pharmacovigil-ance output. This, and not regulatory compliance, drives the op-eration; data collection and use of the data, with focus on essentialquality and best pharmacovigilance. The outputs that satisfy theregulators emerge automatically as a by-product of a basic busi-ness performance improvement process.

This second (added-value) route is fast becoming an imper-ative, not an option. Benchmarking, done thoroughly, providesan excellent set of ‘starting blocks’ for setting off along that pos-itive route in the expectation of success (figure 1).

2. What is Benchmarking?

Benchmarking looks at real performance. It seeks to estab-lish what is done and how well it is done in terms of measuresthat are consistent across locations and companies so that com-parisons are valid, and realistic targets for improvement can beset. Targets that are demanding but achievable – someone hasalready achieved them! Their best practice means that it is notnecessary to re-invent the wheel.

A pharmacovigilance benchmark covers the full range ofactivities from single-case handling to PSURs, from labelchanges to issues management. Besides the well-defined meas-ures such as compliance, it is also possible to benchmark thepeople side of the business. World Class International’s (WCI)project work indicates that skills, roles and responsibilities, andculture are some of the key aspects of the pharmacovigilanceprocess; aspects that are seldom accorded the importance theydeserve, but which we have found susceptible to measurement.

3. Not Just ‘What’ but ‘How’

The processes that are benchmarked need key measures thatstand up as numbers. They must be well-defined, tracked over anumber of years, identify influences and present comparative ef-fectiveness.

Most benchmarks focus on the numbers without providingenough information for the useful interpretation of the data. Thisis unhelpful – “these are the targets, how you achieve them is upto you”. But are the targets even comparable? Are costs beingcalculated in the same way? Or productivity per person? Therehas to be enough information to ensure that like is being com-pared with like, and to understand how the performance wasachieved.

Good benchmarking is not only about comparing measuresand attainments on those measures between companies or depart-ments. The numbers give a good indication of the capabilities or‘best practices’. What they do not explain are the processes thatmade this performance possible.

So the benchmarking measures must also afford a detailedinsight into a process that explains differences, clearly identifies‘best practice’ processes and allows for cross-industry sharing.The ‘learning organisation’ is not achieved with numbers only.It’s the ‘how to’ that matters.

4. Look Both Outwards and Inwards

Most companies look first to benchmark themselves againstother companies. But an internal benchmark can also providehigh-quality data. For instance, a Drug Safety Group with officesin different countries often runs different processes inside eachcountry. Harmonising these internal processes, taking the bestpractices that can be identified, and establishing an internalbenchmark, will take the business a huge step forward.

Also, external benchmarking does not need to be restrictedto companies in one’s own sector. For example, if the objectiveof a benchmark is to identify best practice data entry, then it mightusefully include a bank because that is a major activity for them;their techniques and their levels of speed and accuracy of dataentry should provide some useful insights.

5. An Industry Initiative: the Benchmarking Club

It is clear that a global, industry-wide benchmark is notsomething one company can develop for itself. The good news isthat ten companies have joined in a forum over the last 2 years.

With WCI as facilitator and creator of the benchmark ques-tions, they have benchmarked the single-case handling, signalmanagement, adverse event reporting and PSUR processes. We

are already discussing ways of improving and expanding the

benchmark. Figure 2 shows the overall concept. The presentation

of results in open forum preserves anonymity and confidentiality.

Identity, data and practice information is shared only under rules

set by the members themselves.

6. A Measured Approach

We have defined various measures that together form the

basis for a comparison of performance, structure and process.Some examples:

• adverse event volume, sources, classification;• compliance;• lead times from country to headquarters to country to author-

ities;• staff levels, skills and productivity;• PSUR compliance, process times, costs.

It is necessary to consider all the results to get the total pic-

ture; for example, the number of adverse events will be a function

of seriousness or difficulty or perhaps of source (spontaneous

reporting or clinical trials). The newness of a drug will also havean effect so it is important to get a view of the drug profile. Cost

may be a function of roles and responsibilities. The picture is

inevitably complex, but the process of benchmarking was never

going to be easy; the dangers of ‘measuring’ what can be mea-

sured rather than what needs to be measured are well-known, as

is the spurious comparability of ‘league tables’.

Clinical trial process NDAs

Regulatoryreporting

Approval

Label

Label

Issues managementprocess

Signalsprocess

Label change process

Medical info.

Changerequests

Sales andmarketing process

Adverse events

Country Centre Country

Global database

Manufacturing

Variationsubmission

PSUR process

Label implementation

Fig. 1. The pharmacovigilance process. NDAs = new drug applications; PSUR = periodic safety update reports.

66 Holmes

© Adis Data Information BV 2003. All rights reserved. Int J Pharm Med 2003; 17 (2)

7. Encouraging Results

Figure 3 shows results at a summary level. The benchmark

is a composite of results from all the companies. In each aspect,

it has been achieved by one company somewhere. The chart in-

dicates that there are both industry-wide deficiencies and indi-

vidual variance. Where everyone is below standard, processes

need to be re-thought across the board. Where there are wide

discrepancies between companies, the benchmark helps not just

with targets but with process enhancement.

Some of the immediate benefits are:

• best practice is shared;• assumptions about global pharmacovigilance are chal-

lenged;• the business case for good pharmacovigilance is strength-

ened;• a framework for effective and efficient ways of working is

established;• companies have both targets and a route-map.

In working on the benchmark, the very fact of having taken

a different look at pharmacovigilance and had the conversation

that inevitably follows, has opened up the pharmacovigilance

process and suggested where to converge in both measurement

dimensions and in practice.

8. Added Value

Pharmaceutical companies need to meet the regulations in apredictable and controlled way and obtain, as part of that system-atic pharmacovigilance process, some essential business-relatedoutputs, for example:• an evidential basis for planning and for justifying resources;• clarity about what are the ‘best practice’ metrics for global

pharmacovigilance process performance;• robust evidence of our performance, relative to others, in

terms of those metrics;• a sound basis for a way forward – how the industry should

go about setting future ‘best practice’ targets;• strong pointers to action we should take and the benefits we

can expect from it.Underlying these desirable outputs is the notion that phar-

macovigilance is a business-critical process. Traditional excusesfor inaction such as “we’re special” or “our industry is different”are clearly untenable. The Benchmarking Club’s work has al-ready shown that pharmacovigilance processes can, after all, besuccessfully and usefully benchmarked.

9. Conclusion

Benchmarking pharmacovigilance activities, based on clearand consistent definitions and measures, sets demanding but re-

Annual benchmark

audit

Continuous improvementprocess

Capacity and resourceplanning

Interactive working practices

Benchmarking data

Bridging performancegap (actual vs ideal)

Member companies

Industry(restrictedaccess)

Member companies

Member companies

Industry(by

invitation)

Forum

Benchmarking data

Current issues

Regulatory updates

Chaired by executive(from member companies)

Share best practices

Website

Benchmarking data

Pharmacovigilance Benchmarking Club

Bulletin board,Q&A tool, surveys

Magazine of drug safetyprojects and ideas

Monitoring of Internetdrug safety information

Who’s who guide ofmembers

Fig. 2. The Pharmacovigilance Benchmarking Club.

Commentary 67

© Adis Data Information BV 2003. All rights reserved. Int J Pharm Med 2003; 17 (2)

alistic and attainable targets and provides supporting detail thatpermits the correct interpretation of benchmark data and so setsthe company on the right improvement track.

Good pharmacovigilance data, integrated into the businessprocess, can:• provide a new framework to assure effective and efficient

ways of working in which best practice is shared;• provide the hard data that strengthen the business case for

deployment of appropriate resources;• improve practice and efficiency;• sharpen response to problem events, improving labelling and

advice;• aid development.

It may feel novel, but in pharmacovigilance, as in clinicalpractice, it is in everyone’s interest to find the best and replicateit.

Acknowledgements

The author has provided no information on sources of funding or onconflicts of interest directly relevant to the content of this review.

HighestMedianLowest

40

60

80

100

Leadership and strategy

Measurementsand targets

Externalpartnerships

Business processmanagement

Competitive services

Operational systemsand control

Sources ofinformation

Peopleempowerment

0

20

Fig. 3. Summary of qualitative results.

68 Holmes

© Adis Data Information BV 2003. All rights reserved. Int J Pharm Med 2003; 17 (2)