Critical Clinical Alert

Management

Disaster Recovery for

Medication Management

Systems

Adam Stormont – Senior Pharmacist, New Technology and Systems Development

Critical Clinical Alert

Management

The Monash Health Experience

Background

• MerlinMAP electronic prescribing software introduced in June 2012

• Recording function for adverse drug reactions

• Data coded – Snomed and AMT

• Decision support for adverse drug reactions

• Repository function – database

Clinical Incident – September

2012

• Patient admitted to Emergency Department

• Experienced anaphylactic reaction to Ceftriaxone, intubated and sent to ICU

• Root Cause Analysis investigation revealed poor ADR history taking

• Alert History buried in previous SMR admissions

• Not checked by staff involved

• Change in alerts recording process – required!

National Standards – Accreditation

• Standard 4 – Medication Safety

• 4.7 The clinical workforce documenting the patient’s previously known adverse drug reactions on initial presentation and updating this if an adverse reaction to a medicine occurs during a episode of care

• 4.7.1 Known medication allergies and adverse drug reactions are documented in the patient clinical record

• 4.7.2 Action is taken to reduce the risk of adverse reactions

Issues with Alert Management

• No existing source of truth chosen

• Alerts being recorded in SMR, Symphony, Merlin, iPM

• Security issues – anyone with access can record alerts

• Different kind of alerts – clinical, administrative etc.

Governance

• Critical Clinical Alerts Committee was set up to provide the overarching governance for the issue of alerts.

• Committee was tasked with making a decision around how critical clinical alerts would be managed to ensure patient care not compromised

• Chaired by Allergist/Immunologist

• Membership from Medical, Nursing, Pharmacy, IT, HIS.

• Work was very much cut out for committee!!

Considerations

• Need one source of truth

• Issues with current alert display in SMR

• SMR – required upgrade to be able to display alerts in more useable fashion- $$$$

• ED – uses Symphony and users record alerts there – doesn’t talk to anything

• Merlin – robust process for recording clinical drug alerts but not used hospital wide

• iPM – Dirty alerts

Decisions

• Merlin – Source of truth

• Some modification required to information recorded

• Database expanded to cover certain foods and environmental allergies deemed critical to know in the hospital environment

• MerlinMAP to be made available for all doctors/pharmacists to record alerts in

• Discussions underway for Nursing/Allied Health staff to record alerts – requires education

MerlinMAP – ADR recording

Scanned Medical Record – How

will alerts be displayed?

Scanned Medical Record – How

will alerts be displayed?

Scanned Medical Record – How

will alerts be displayed?

Further issues to consider

• Education about how to take an Adverse Drug Reaction History

• Best possible Adverse Drug Reaction History

• Alert fatigue

• Risk of missing a serious reaction vs. Risk of over-reporting trivial or false reactions

• Review processes – resource intensive for pharmacists – but pharmacists do it very well!!

• Medicolegal issues

• System integration – Reduce duplication

Disaster Recovery for

Medication Management

Systems

The Monash Health Experience

What is System Downtime?

• Any time that a clinical IT system cannot be used

• Scheduled vs. Unscheduled outages

• Full vs. Partial Outage (e.g. Read only mode)

• It’s about process – and how the organisation responds

Current Position – Merlin and

MerlinMAP E-Prescribing

• Merlin is currently on a single VMWare Virtual Server.

• Significant advantages over a physical box

• Removes hardware as a system failure reason

• Pause of server every morning at 4am to complete backup to tape – 45 minutes

Downsides to current position

• Downtime for backup impacts Emergency Department use of e-prescribing

• If Merlin server became completely unusable during day recovery would be from previous nights back up position – all other transactions lost

• Substantial IT/Vendor intervention required.

• Potential for significant outage – several hours, days.

• Loss of data – impacts on revenue

What were we trying to achieve?

• Remove system downtime – make Merlin/MerlinMAP available 24/7

• Minimise data loss by having real-time copy of the primary server to an alternate server so that it can be used as the primary server in the event of an irrecoverable failure

• Maximise the percentage of system uptime

Proposed Solution

• Primary and Secondary Servers

• Secondary server in separate physical location

• Secondary server would run in a “subscriber mode” and receive updates from primary server

• Back-up of system will run from the secondary server - will remove need for downtime on primary server

• MerlinMAP and Merlin would be available 24/7 - 365

Proposed Solution

• Unidata Replication introduced in “stand-by” mode

• Disaster Recovery solution that does not provide for automatic failover

• Requires purchase of same number of licenses as primary server – but in “stand-by” mode so not as costly

• Pharmhos (vendor) did not recommend automatic failover

Proposed Solution

Why a manual decision to

failover?

• Hardware Fix

• User Session Connections

• PDE devices

• Interfaces

Procedure for Failover

• Verify primary server is irrecoverable (IT)

• Verify secondary server is still viable and users able to connect (IT)

• IT to advise Pharmhos of the requirement to failover

• Pharmhos or IT to login to the secondary server to confirm that the server has finished applying the replication logs to the Unidata & Postgres databases using the tools provided.

• Pharmhos or IT to switch the mode of operation of Unidata & Postgres Replication from standby to primary operation

Procedure for Failover

• IT to switch the IP address of shdanphndb01 in the DNS to the IP address of the failed primary server – this will point users that log onto shdanphndb01 to the secondary server.

• Pharmhos or IT to log into Merlin and start the inbound & outbound interfaces

• Pharmhos or IT at a later time can reset this server to replicate to another nominated server to ensure there is no data loss should the secondary server also fail & require a failover.

Timeliness of response

• The procedures detailed require a response from IT and vendor

• Time to make decision to failover or not

• Need a downtime procedure – script pads, notification of users.

Other Factors

• For IT staff to evoke the failover procedures they require training in operation of replication tools and method of swapping to secondary server

• Issues with training staff

• Support agreement with vendor – currently only 9am-5pm Monday - Friday.

To the future

• Once dealing with inpatient medication management esp. in acute settings will need a dedicated High Availability solution

• Requires a change in infrastructure – significant cost and investment required by both vendor and health service

Thankyou!!

• Questions??????