Role Prediction Using Electronic Medical Record System Audits Wen Zhang 1, Carl Gunter 3, David...
-
Upload
bryce-hood -
Category
Documents
-
view
219 -
download
1
Transcript of Role Prediction Using Electronic Medical Record System Audits Wen Zhang 1, Carl Gunter 3, David...
Role Prediction Using Electronic Medical Record System Audits
Wen Zhang1, Carl Gunter3, David Liebovitz4, Jian Tian1 , Bradley Malin1,2
1Dept. of Electrical Engineering & Computer Science, Vanderbilt University2Dept. of Biomedical Informatics, Vanderbilt University
3Dept. of Computer Science, University of Illinois at Urbana Champaign4Dept. of Medicine, Northwestern University
1
Misuse of EMR Systems is Real
• Medical center employees misuse medical record systems to breach privacy
When Where Who
2007 Palisades Medical Center George Clooney
2011 UCLA Various Celebrities
2
• HIPAA Security Rule Access to EMRs should be limited
• The problem is not limited to celebrity snooping
• But how?
Challenges to Security in EMRs
• Basic security principle:– Least privilege– Separation of duty
• Access control technologies have been around since the 1970’s
• Information systems often provide role-based access control (RBAC) capability[1]
– Privileges mapped roles
– Users mapped to privileges
• Roles are hard to define, so EMR systems often provide broad access rights
3[1] R.Sandhu, E.Coyne, H.Feinstein and C.Youman. IEEE computer. 1996.
In “Rare” Cases – Break the Glass
• A user may not sufficient access rights to perform job
• This model allows users to temporarily escalate privilege
• Access is logged and reviewed by administrator
• May require user to specify “reason” for access
4
Rare Cases?• Central Norway Health Region enabled break the glass
• 53,000 of 99,000 patients (54.5%) broken glass
• 5,000 of 12,000 users (42.7%) broke the glass
• Over 295,000 logged breakage events in one month
Role Users Invoked Glass Breaks in Past Month
Nurse 5633 36%
Doctor 2927 52%
Health Secretary 1876 52%
Physiotherapist 382 56%
Psychologist 194 58%
5[3] L. Røstad and N. Øystein. Proceedings of the 2nd International Conference on Availability, Reliability and Security (ARES)
Idea! Refine Access ControlBased on Behavior
• Experience-based Access Management (EBAM)
• Combine static knowledge (RBAC)
with actual actions (access logs) and organizational knowledge for feedback control
6
RBAC
EMR Access Logs
Medical Center Knowledge
Experience-Based Access Management [2]
[2] C.Gunter, D.Liebovitz, B.Malin. IEEE Security and Privacy Magazine. 2011.
• Use audit logs to predict if a user is associated with a role
• Goals:– Determine if expert-defined job titles are reasonable– Provide administrators with a better idea of how to refine roles
The Role Prediction Problem for EBAM
Doctor
NurseRole
Classifier
Biller
….
7
Access Reason
Medical Service
Location of Patient
User Patient Time Service User Position (Role) Reason Locationu1 p1 8/4/10 OBSTETRICS NMH Physician Office - CPOE Attending Phys/Prov Ward Au2 p2 12/14/10 OBSTETRICS NMH Physician - CPOE Patient Care Ward Au23 p3 12/14/10 PEDIATRICS Unit Secretary 2 Unit Secretary Orders Ward B
Evaluation with Cerner EMR of Northwestern Memorial Hospital
• Represent users as <Service, Reason, Location> vectors
• Statistics
8
Users Roles Reasons Services Locations
8095 140 143 43 58
• Example audit logs
• To assist in role management, we worked with organization experts to build a hierarchy (specialized to Northwestern)
• Optimization Tradeoff:• Goal 1: Accuracy (should increase as we step up in hierarchy)• Goal 2: Separation of Duty (will increase as we step down)
Leveraging Role Hierarchies
Employee
DoctorSpecific Clinician
Dietitian
Junior Dietitian
Senior Dietitian
Physician Nurse
…
… …
… … …
General (62 roles)
Conceptual (5 roles)
Specific (140 roles)
9
Basis of a “Role-Up” Algorithm
• General idea: Audit roles at different levels of the hierarchy
1. Score each role in conceptual position & general position
2. Select role with the highest score & generalize its children
3. Repeat 1 & 2 until a threshold score is reached
10
• Allow administrators to balance between the prediction accuracy and separation of duties (number of roles)
Balanced Scoring Function
• R measures the extent to which specificity could be kept by the node
• A measures the extent to which predictablity could be achieved by the node
11
Employee
DoctorSpecific Clinician
Dietary
Junior Dietician
Senior Dietician
Physician Nurse
Nurse 1 Nurse 2Physician
2Physician
1
0.4760.224 0.410
0.453 0.0441
α = 0.5, Threshold = 0.4
12
Employee
DoctorSpecific Clinician
Dietary
Junior Dietician
Senior Dietician
Physician Nurse
Nurse 1 Nurse 2
0.224 0.410
0.4530.0441
13
α = 0.5, Threshold = 0.4
Employee
DoctorSpecific Clinician
DietaryNurse
Nurse 1 Nurse 2
After one iteration, the role set is{Doctor, Nurse 1, Nurse 2, Dietary}
14
α = 0.5, Threshold = 0.4
Training & Testing at the Same Level of the Role Hierarchy
Employee
Specific Clinician
Nurse
Nurse 1
15
Conceptual
General
Specific
82.38%
52.45%
51.34%
AccuracyLevel
Distribution of Accuracy Over the Role Hierarchy
16
Rank Role Accuracy Users1 (tie) AP-Technologist 100% 541 (tie) ED Assistant 100% 261 (tie) ED NMH Physician-CPOE 100% 43
1 (tie) NMH Resident/Fellow ID Clinic-CPOE 100% 10
1 (tie) Patient Care Staff Nurse – Lactation 100% 14
17
Most Predictable Roles
Least Predictable Roles
Rank Role Accuracy Users140 Patient Care Staff Nurse 7.6% 1554139 Rehab OT 14.3% 28138 Transfer 20.0% 20137 View Only PC 3 21.4% 14136 Patient Care Staff Nurse (Pilot) 22.1% 217
18
Number of Users in the Role Can Influence Accuracy
19
Case Study: Most Likely Mispredictions for Patient Care Staff Nurse
Predicted Role PredictionPatient Care Staff Nurse - Lactation 19.6%View Only PC 1 14.3%Radiology – Nurse 14.0%Patient Care Staff Nurse (Pilot) 10.4%SN-RN/Customer Service 5.8%
20
Original Role Predicted Role ProbabilityRehab OT Rehab PT 85.7%Patient Care Staff Nurse - Agency
Patient Care Staff Nurse - Lactation
75.0%
Rehab PT Rehab OT 60.0%
View Only PC 3Patient Care Staff Nurse - Lactation
50.0%
Medical Records - Scanner
Medical Records 47.4%
21
Most Likely Mispredictions
Parameter Bias Trades Between Accuracy and Separation of Duty
• Biased toward Accuracy:• number of roles is small (27)• accuracy is highest (63%)
22
0.1 … 0.8 0.9Number of RolesRecommended
27 … 60 64
Accuracy ofRole Predictions
63.3% … 51.8% 51.3%
• Biased toward Specificity:• number of roles is high (60)• accuracy is lower (52%)
Conclusion and Future Plans
23
• EHR audit logs can be analyzed to determine if the users’ behaviors are consistent with their designated job titles
• Role hierarchies enable automatic discovery of appropriate levels of role management
• Plan to expand Role-“up” to allow for Role-“down” and Role-“over”
• Need to evaluate Role-up with real hospital administrators, to assess its usability and acceptance of results
Acknowledgements
• National Science Foundation– CCF-024422– CNS-0964063
• National Library of Medicine– R01-LM010207
• Office of the National Coordinator for HIT– SHARPS (sharps.org)
24