Data Management and IntegrityLaboratory Practices

Matt Davis and Gaye CammSenior Inspectors, Manufacturing Quality BranchMedical Devices and Product Quality Division, TGA

21 November 2019

What is data integrity?• Data Integrity is the extent to which data is:

– Complete – Consistent – Accurate

• Throughout the Data lifecycle:– Initial generation and recording– Processing– Use– Retention, archiving, retrieval and destruction

• (PIC/S Good Practices for Data Management and Integrity PI 041)

1

Creating the right environment• Data management controls embedded in PQS

– System design to ensure good DI practices– QRM approach to data integrity– Ongoing risk review data criticality/risk– Self Inspection

• Clear understanding of importance of data integrity at alllevels of the organisation

• Internal reporting is encouraged• Mature, open management approach to data integrity

Rationalisation

2

Risk management approach to data integrityData

Criticality Data Risk• Data Criticality

– CQA Batch release data > cleaning records

– Data relating to product quality/safety• Data Risk

– Vulnerability of data to alteration, deletion, recreation, loss or deliberate falsification

• Outcome - Effective control strategy to manage identified risks

3

ALCOA+ Principles

Attri

buta

ble • Clearly

indicates who recorded the data or performed the activity

• Signed / dated

• Who wrote it / when

Legi

ble • It must be

possible to read or interpret the data after it is recorded

• Permanent• No

unexplained hieroglyphics

• Properly corrected if necessary

Con

tem

pora

neou

s • Data must be recorded at the time it was generated

• Close proximity to occurrence

Orig

inal • Data must

be preserved in its unaltered state

• If not, why not

• Certified copies

Accu

rate

• Data must correctly reflect the action / observation made

• Data checked where necessary

• Modifications explained if not self-evident

Plus

• Complete• Consistent• Enduring• Available

4

Designing paper systems which reduce opportunities for falsification

Attributable

Legible / Permanent

Contemporaneous

OriginalAccurate

System design: documents in right place at right time,

clocks on wall, control of blank forms

Signatures, Aliases;

signature logs

Workbooks, forms controlled, verified ‘true copy’ scans

Reflective of the observation; Data checking, raw data

verification

No pencil, white-out, soluble ink,

SOP for corrections and archiving

5

Designing electronic systems which reduce opportunities for falsification

Attributable

Legible / Permanent

Contemporaneous

OriginalAccurate

Auto-saving; step-wise recording; System clock

synchronisation

User access control; e-signatures; metadata

Metadata which permits

reconstruction

Data capture; manual data entry;

source data & audit trail review

Data security, audit trails; back-up; sys. validation

6

Overarching DMDI policy• Procedures• Audit Trails• Secure data retention• Time/date stamps• Traceable• Correct movement of data• Employee adherence• Period review• Security• Detecting non-compliance

• Validation• Accurate and complete data• Unique user names/password/biometrics• Prevent unauthorised changes• Independent review• Detecting wrongful acts• Training• Control of outsourced activities• Corrective actions

7

Analytical laboratories – Common concerns & controls

• DI expectations:– Lab electronic systems– Data review processes– Audit review processes– Manual integration– Analyst training– Spreadsheet management– Test Injections

8

Laboratory electronic systems

E-signatures

Audit trail review

External calculation tools

Raw data verification

Data review SOPs

Data management

System administrator

Defined user privileges

Individual user access

SOPs for user access control

User Access

Test method configuration

Data back-up/archiving

OS security

Audit Trails

Configuration

Periodic system review

Change management

Configuration management

Hardware qualification

Software validation

Validation

9

Data review processes• Consider electronic and paper-based records

• Clear SOP required for data review

– frequency, roles, responsibilities and approach to risk-based review of data (and metadata, including audit trails as relevant)

– Ensure that the entire set of data is considered in the reported data, should include checks of all locations where data may have been stored,including locations where voided, deleted, invalid or rejected data may have been stored

• Who-When-What-How:

– Who collected/when was it collected/what was collected/how was the data collected?

– Who, when, what, how……data processed?

– Who, when, what, how…...data reviewed?

– Who, when, what, how…....data reported? 10

Audit trail review processes

• Technical Controls to aid secondary review of Audit Trails– Identifying data that has been changed or modified

– Review by exception – only look for anomalous or unauthorized activities

– By limiting permissions to change recipes/methods/parameters, or locking access to specific parameters or whole recipes/methods may negate the need to examine associated audit trails in detail as changes can be easily observed, restricted or prohibited

– Whatever activities are left open to modification need to be checked

11

Audit trail reviews - Common issuesReasons for change or deletion of GMP-relevant data not documented Annex 11 §9: …for change or deletion of GMP-relevant data the reason should be documented• Method to record legitimate changes to data that needs to be considered

when doing audit trail review, or covered by SOP i.e. allowable changes to methods.

• Explanation for ALL data recorded (complete data) including results that aren’t reported

• Deviations from standard procedures or atypical results should be investigated

12

Reasons for changes to data…May be valid reason for invalidating data and repeating acquisition e.g. equipment malfunction, incorrect sample/standard preparation, power failure

• Need to be recorded, investigated and potentially implement CAPA for invalid runs, failures, repeats and other atypical data

• All data should be included in the dataset unless there is a documented scientific explanation for their exclusion

• Possibly reviewed for trends at some timepoint

• May need a new SOP for laboratories in addition to standard OOS/OOT procedures

13

Audit trail challengesUse of paper copies for review of electronic systems

Many laboratories create paper copies of electronic records in the form of reports and rely on conduct the audit trail review. These reports can be huge (> 80 pages per chromatographic run when audit trails are included) and the risk that critical notifications in the sea of data may be lost is a significant risk.

14

Recording of audit trail reviewHow do manufacturers demonstrate they have reviewed audit trails?

• Documentation of audit trail reviews should be performed in a similar way to documenting any review process. Typically done by signing the results as ‘reviewed’ or ‘approved’, following a data review SOP.

• For electronic records, this is typically signified by electronically signing the electronic data set that has been reviewed and approved

15

Recording of audit trail reviewWhat about manufacturers who don’t have electronic signatures available?• Hybrid approach, which is not the preferred approach, using

paper printouts of original electronic records

– Requirements for original electronic records must be met.

– To rely upon these printed summaries of results for future decision-making, a second person would have to review the original electronic data and any relevant metadata such as audit trails, to verify that the printed summary is representative.

• It seems unreasonable to require specific evidence of exactly which records and metadata were looked at or opened (this would constitute an audit trail of the audit trail review)

16

Manual integration controls• Automatic integration should be default manual only

where absolutely required• SOP for integration required

– Define methods that can and cannot be adjusted– Document which actions are permissible, or restrict access to only

allow the desirable actions– Document both original and manually integrated chromatograms– Electronic signatures/audit trail for manually integrated peaks

• Review results to ensure compliance– Review reported data against electronic raw data (including audit

trails)

17

Staff Training• Essential that reviewers have a good knowledge

of how the computer system or software works, andhow the audit trail is designed and works together with the data

• This may require specific training in evaluating the configuration settings and reviewing electronic data and metadata, such as audit trails, for individual computerized systems used in the generation, processing and reporting of data.

• Include training on system vulnerabilities such as overwritten or obscured through the use of hidden fields or data annotation tools

18

Excel spreadsheetsSpreadsheets for managing and presenting data, and their versatility and ease of use has led to wide application. When data contained within the spreadsheet cannot be reconstructed elsewhere and is essential to GMP activity then the data governance measures need to be rigorous. Issues• If the spreadsheet has multiple users it may be impossible to ascertain who (Attributable)

made an entry, whether entries have been over-written and replaced (permanent), and when the data entries had been made (Contemporaneous).

• If the spreadsheet is not version-controlled and managed as a controlled document, then there may be different versions in use (Original).

• Where formulae and other functions are used there is potential for these to be corrupted without being detected (Accurate).

19

Test injections• All chromatographic systems need to equilibrate before they are ready for analysis. The time taken will typically

depend on factors such as the complexity of the analysis, the age and condition of the column, and detector lamp warm-up time.

• Generally there will be an idea of how long this will be from the method development/validation/verification/ transfer work performed in the laboratory and this should be documented in the analytical procedure.

• Prepare an independent reference solution of analyte(s) that will be used for the sole purpose of system evaluation. The solution container label needs to be documented to GMP standards and clearly identified for the explicit purpose of evaluating if a chromatography system is ready for a specific analysis.

• The analytical procedure needs to allow the use of system evaluation injections. Staff need to be trained in the procedure.

• Inject one aliquot from the evaluation solution and compare with the SST criteria. Clearly label the vial in the sequence file as a system evaluation injection. If the SST criteria are met then the system is ready for the analysis.

• Upon completion of the analysis, document the number of system evaluation injections as part of the analytical report for the run.

20

Microbiological laboratories – common concerns and controls

• General issues• DI expectations

– Note: All previous comments regardingcomputerised systems apply

– Microbiology may present a greater DI risk

21

General issues observed

Competence/supervision Lack of effective controls Secondary Checks Computerised system configuration

Organisational Culture / resources

Manipulation of data

No testing conducted

Not counting all colonies

OOL data not being investigated

Resampling/retesting without justification

Incomplete Testing

Samples not taken or “lost” in transit

No reconciliation of samples

Incubation conditions incorrect

Using unvalidated test methods

Poor test records

Not recording all key test data

Worksheets ripped up and replaced

No reconciliation of forms used

Lack of proper computerisedsystem security

Colony morphology not matching identification results

Contributing causes

22

DI controls – Manual test methods

Sampling ProceduresSampling schedule/plansTraining of techniciansSample formsDetailed collection methods Identity of sampler recorded

Test methodsTest volumes/weights recordedCalibrated equipment usedReference to all reagentsReference to validated methods/dilution factorsSamples processed under clean conditions, e.g. LAFNegative controls for processed samplesIdentity of tester/equipment recorded

IncubationIncubation records maintainedMin/max incubation time defined and validatedAll transfers/sub-culturing recordedAll incubated samples tagged and identified

Reading resultsTechnicians trained in detection, enumeration and morphology – clear SOPs, photosControlled environment for reading, light, magnificationCounting device used for coloniesClear acceptance criteria/limitsOOL & ID policy for manual recordingAll samples reconciledResults recordedCalculations applied correctlySecond checks and verification in accordance with quality risk management

23

Summary• Review DMDI guidance and TGA policy• Develop DMDI policy for your organisation• Risk based approach to systems and data

– Data criticality– Review capabilities of electronic systems

• Incorporate DMDI controls into QMS (and review!)

24

Where are you now?•Do not know about the issue

and unaware of the gap

UNCONSCIOUS INCOMPETENCE •Aware of the gap but not

yet able to deal with it

CONSCIOUS INCOMPETENCE

•Getting a handle on the problem but only with effort

CONSCIOUS COMPETENCE •Good practice becomes

automatic

UNCONSCIOUS COMPETENCE

25