Items to be Discussed Report Definition Characteristics Types Importance Flow.
NIST Medical Device Communication Testing€¦ · Definition of the “Works”[from IEEE/NIST...
Transcript of NIST Medical Device Communication Testing€¦ · Definition of the “Works”[from IEEE/NIST...
NISTMedical Device Communication Testing
Semantic interoperability of Medical Devices
Test Tool Update
HL7 Healthcare Devices WG / IEEE x73 Upper Layers
John J. Garguilo
National Institute of Standards and Technology
8 May, 2014 – Phoenix, AZ
Contact: [email protected], 301-975-5248
2
NIST MDC Testing Staff
• John J. Garguilo
301-975-5248
• Nicolas Crouzier(GR) – RTMMS, HL7 V2 Tools
• Art Griesser + Michael Faughn (Contractor -
Prometheus Computing, LLC ) – DIM Modeling +
“MyDevice” (next gen ICSGenerator plus)
• New GR (half time) starting June ‘14
Semantic Interoperability of Medical Devices
3
Work Areas
• HL7 V2 (2.6) IHE-PCD Validation Tools
– Enhancements
“Off-cycle” added to Pre-Con tool (i.e., between Cycles 8 and 9)
– Cycle 9 CP updates being code for 2014-15;
Fall/Winter Pre-Connectathon + Virtual Testing,
January/February 2015 Connectathon
– IHE-PCD Pre-Connectathon, “Isolated Environment”
– IHE-PCD Connectathon, “Instance Environment”
• Rosetta Terminology Mapping Management System (RTMMS)
Overview + Update
– RTMMS Deployment Update and service status
• -10101a + IEEE Agreement to make available the “Works” + latest numbers…
• IGAMT – Implementation Guide Authoring Management Tool
– Continue to develop PCD-01 (ORU^R01)… targeting Sept WG mtgs
4
Work Areas, continued
• ISO/IEEE 11073 ‘Domain Information Model Editor’ -> “MyDevice”
– Progress/Update/Status (Michael Faughn + Art Griesser)
– UML Defined x73 DIM (-10201) basis for tooling
– DIM Editor / ‘MyDevice’ pilot tool demo
– Auto producing ISO/IEEE documentation (to meet SDO’s templates)
HL7 V2.6 Validation ToolingIHE-PCD Pre-Connectathon Cycle 9
Semantic interoperability of Medical Devices
Test Tool Update
Joint HL7/IEEE 11073 Healthcare Devices Working Group
National Institute of Standards and Technology
8 May, 2014, Phoenix AZ
Contact: [email protected], 301-975-5248
6
2013-14 IHE-PCD Pre-Connectathon Tool
Status
• IHE-PCD: NIST “Isolated Environment” Test Tools required
– HL7 V2.6
– Supports 6 Integration Profiles (number of test cases)
• DEC – Device Enterprise Communication (10)
– SPD – Filter (4)
– POI – Pulse Ox (3)
• PIV – Patient Infusion Verification (11)
• IPEC – Infusion Pump Event Communication (6)
• IDCO – Implantable Device Cardiac Observation (4)
• ACM – Alarm Communication Management (2)
• RDQ – Retrospective Data Query (1)
• OMS, RTLS/CMS white papers/WGs
• 30+ Test Cases
• 60+ existing User Accounts
– Contributing Orgs, SDO (leads/Co-chairs), Reviewers
– In total since tool made publically available (2011)
7
2013-14 IHE-PCD
Pre-Connectathon + Connectathon Status, Continued
• Cycle 9 - Test Cases/demographics reviewed used as basis
for tooling at 2015 NA Connectathon
• IHE-PCD Spring F2F– week of April 1st in Cleveland at new HIMSS Interoperability Center
• CPs documented and submitted (see PCD wiki)– Note: One withdrawal (adding an “alarm consumer”), others balloted and in process of
comment resolution (official ballot closed last Friday), CP usage change to NK1
Segment in ballot
– 9 CPs received and processed: http://wiki.ihe.net/index.php?title=PCD_CP_grid
• Roadmap of activities updated… see PCD wiki
http://wiki.ihe.net/index.php?title=Patient_Care_Devices
8
IHE-PCD 2013/14 Pre-Connectathon + Connectathon
NIST Testing Support
The NIST V2 (2.6) Tools perform the following validation:
• Syntax and Semantic Content Validation
– Against HL7 conformance profile
– Against IHE-PCD Technical Frameworks/Supplements constraints
(e.g., PCD-01: ORU^R01, Communicate Device Data)
– Against HL7 and/or user [local] provided tables
• Example of user provided table is RTM for Ref_IDs, Units, body sites, etc.
– Against ‘validation context’, including specific values
• Defined in XML (e.g., specific test case values)
9
Testing Environments
• Instance Testing– Conformance (e.g., against HL7 2.x or CDA)
• Test object conforms to specification on which it is based
• IHE Model: ~Virtual and Pre-Connectathon
• NIST IHE-PCD v2 Message Validation Test Tool
• Isolated System Testing– Includes Instance Testing Activities
– Protocol Conformance
– Functional Behavior Conformance
• Features and operational behavior correspond to specifications
• IHE Model: ~Virtual and Pre-Connectathon
• NIST IHE-PCD v2 Test Tool
• Peer-to-Peer System Testing– Includes Isolated System Testing Activities
– Interoperability Testing
• Testing complete application environment
• May include interacting with Database, using Network Communications, or interacting with
other hardware, applications, or systems if appropriate
• IHE Model: ~Connectathon
10
NIST V2 HL7 IHE-PCD Test Tool: Operational Process
END-USER
(VENDOR)
SYSTEM UNDER TEST
(SUT)
NIST IHE-PCD
HL7 v2/v3 TEST TOOL
SPECIFICATIONS
(test material that defines test assertions)
INTERACTION/REPORTS
MESSAGES (TEST OBJECTS)
MANUAL OR AUTOMATED SUT
Web Application
Interface
(via the communication protocol
currently only MLLP)
V3 – Future Work
STIMULUS OR RESPONSE (MESSAGES)
11
HL7 V2 Tool Updates – New/In Progress
• Reference to standard/date for every MDC terms (e.g. 10101 or
10101a) – To add column once completed
• Working on code generation taking into account discriminators with
the help of Paul Schluter.
• Adding Alerts attributes in RTMMS (work in progress with
Monroe Patillo)
• Various test cases corrections (messages examples, validation
contexts…)
• IPEC containment tree validation is in testing phase; targeting
ready for this summer (2014)
• Pre-Connectathon tool, fixed test case scenarios and validation
files for DOC-DOR-DOF test cases.
• Currently implementing PCD CPs (97-102 from
http://wiki.ihe.net/index.php?title=PCD_CP_grid)
12
HL7 V2 Tool Updates, continued - update
• Documentation Tab (see coming slides)
– Conformance Profile Tab
– Patient Demographics
– IDCO Patient Demographics
– PIV Drugs
– Other Resources
– Cycle 9: Looking into capability to upload libraries +
demographics
• incorporated [automatically] into validation context files
used by tooling
13
New Tool Updates: Test Event Selection – Out of cycle
• Top Right Corner – Out of Cycle to enable separation
(historical) of test event results (for/by paticipants, last
being Cycle 8 (Fall – Winter 2013-14)
Default Test Event
14
New tool updates – Profile Viewer – updating to reflect CPs
15
HL7 V2 Tool Updates (Started with Cycle 8 test events)
• Current Version / Release Notes – new CPs not yet integrated
– ‘About’ Tab (yellow ribbon)
16
HL7 V2 Tool Updates (Starting with Cycle 8 test events)
• Documentation Tab
– Conformance Profiles
17
HL7 V2 Tool Updates (Starting with Cycle 8 test events)
• Documentation Tab
– Patient Demographics
18
HL7 V2 Tool Updates (Starting with Cycle 8 test events)
• Documentation Tab
– IDCO Patient Demographics
19
HL7 V2 Tool Updates (Starting with Cycle 8 test events)
• Documentation Tab
– PIV Drugs
20
Cycle 8 Organizations using NIST Pre-Connection Tooling
# of Organizations: Total 21, Unique 17
Organization Name Organization Name
USST Hospira
True Process Inc. Draeger Medical
*Carefusion Cardiopulmonary Corp.
B Braun Baxter
Smiths Medical *West Health
iSirona BIT Computer Co. Ltd.
Covidien Masimo
*GE Healthcare *Philips Healthcare
Fukuda Denshi*Multiple Users from same Org
21
IHE-PCD Cycle 8 2013-14 Pre-Connectathon
Test Result Statistics – Organizations and Tests Executed
• Total Organizations (by User Accounts): 21
(Cycle 7: 19)
• Total Organizations (by *unique Company Name): 17
(Cycle 7: 14)
Registered (stored) at NIST site (over Pre-Con 2013-14 Cycle 8)
• Total number of Tests Executed:116
(Cycle 7: 82)
• Total Number of Tests Passed: 108
(Cycle 7: 81)
• Total Number of Tests Failed: 8
(Cycle 7: 1)
22
IHE-PCD Cycle 8 2013-14 Pre-Connectathon
Test Result Statistics – by Actor
Cycle 7
Actor Total Pass Fail
DEC Reporter 18 17 1 17
DEC Consumer 2 2 1
DEC Filter 1 1 1
PIV Programmer 1 1 0
PIV Consumer 4 4 5
ACM Reporter 11 10 1 6
ACM Manager 2 2 1
IDCO Reporter 1 1 0
IDCO Consumer 1 1 0
IPEC Reporter 5 5 4
IPEC Consumer 1 1 0
RDQ Consumer 1 1 0
RDQ Responder 0 0 0
Cycle 8 213-2014 Cycle 8
Actors: 13
48 Total:
46 Pass
2 Fail
Cycle 7:
Actors: 11
35 Total
Rosetta Terminology Mapping Management System - RTMMS
Semantic interoperability of Medical Devices
Test Tool Update
Joint HL7/IEEE 11073 Healthcare Devices Working Group
National Institute of Standards and Technology
21 January, 2014 – San Antonio, TX
Contact: [email protected], 301-975-5248
24
RTMMS Overview
• A web application* that allows vendors and reviewers access,
retrieval, and reporting of Rosetta Tables over the internet in
conformance to IHE-PCD RTM Profile
• An electronic resource/tool providing the capability of saving
data in xml format (as defined by RTM Profile)
• Aids the harmonization process by:
– Identifying missing terms
– Automatic generation of the “Harmonized Rosetta Table”
– Providing latest up-to-date view of hRTM table
• Facilitates the proposal of New Terms to IEEE 11073
Nomenclature standard
• Facilitates Conformance Tooling
– Message verification and conformance (syntax and semantics)
– Leading to interoperability…
– *developed by and currently hosted at NIST
– Integrated with ICSGenerator Tool (hRTM also imported into DIM Editor)
25
Rosetta Terminology Mapping Data Base
• Rosetta Table
– Maps vendor supported observations, units and enumerations to
ISO/IEEE x73 nomenclature
• Units Table
– Defines allowed units-of-measure
– Defines groups of related units-of-measure
• Enumerations Table
– Defines groups of enumerated values
• hRTM Table
– Generated from the original Rosetta
X73 Nomenclature DB
• Linked to RTMMS
Security DB - Stores users information
RTMMS Key Tables
26
• Access to NIST IEEE 11073 Nomenclature database
– Appendix A terms (from ISO/IEEE 11073 10101: Nomenclature)
– Appendix B terms (from ISO/IEEE 11073 10101: Nomenclature)
– IDCO terms (ISO/IEEE 11073-10301 – implantable device cardiac)
– aECG (annotated Electrocardiography)
– PHD terms (personal health domain)
• Access to RTM database
• Ability to propose terms in Rosetta
• hRTM, units, and enumeration download-able in XML format
• User registration
– Email confirmation, approval process… controlled through ‘admin’
• Filtering based on regular expressions
• Rosetta validation against hRTM
• Management capabilities for SDO users
Integrated w/ ICSGenerator / exported to ‘DIM Editor’
RTMMS Key Features
27
• General user
– Views Rosetta Tables
• Reviewer
– Participates in discussions
• Contributing Organizations
– Vendor/Contributer ‘sandbox’
– Modifies Vendor Rosetta Table
– Suggests new terms
– Modifies Units and Enumerations Table
• SDO (Standard Development Organization)
– Modifies Units and Enumerations Table
– Register new terms
• Admin
– Manages User Accounts
RTMMS Users
28
RTMMS by the numbers (as of 8 May 2014)
(since Jan 2014)
• 1619 (+4) Rosetta terms
• 671 (+1) hRTM terms
• 224 (0) Groups
• 8 (0) Unit Groups
– 445 (0) Units (MDC)
– 429 (0) Units (UCUM)
• 28 (0) Enumeration Groups
– 273 (+12) Enumerations
• 46 (+4) Contributing
Organizations now
Registered!
29
RTMMS Terminology Access and Views
Agreement finalized between IEEE and NIST (Dec 2012)• Allows users free and public access of terminology attributes that were
formally ‘pay-for’ via the NIST tool• The “Works”
Definition of the “Works” [from IEEE/NIST Agreement]
The following data items within the approved standards , existing and future versions, currently
designated as:
1 – IEEE 11073-10xxx
2 – IEEE 11073-20xxx
Limited to the following data items within, currently designated as:
1 – Ref ID
2 – Code
3 – Description
4 – Systematic Name
5 – Common Term – to be added to agreement (working w/ Kathryn Bennett/IEEE to amend)
30
RTMMS Primary Updates – Signed Agreement 12/20/12
31
RTMMS Updates, cont.
• User can download (an XML, HTML, and CSV) version of:– Rosetta – “hRTM” – Units – Term Approval::“Proposed terms” [192]– Term Approval::”Ready Terms” [286]– Note: Basis from prior WG discussions - this version enables greater flexibility
to work with when working outside the tool
• “Standard Table”– A new column “Standard Tbl” is now present in all Rosetta, hRTM and review
tables – Provides association of the term with corresponding table in the base
standard – Only the terms given by Jan W. (275) REFIDs are currently linked– NIST can add more if community provides the/additional mapped terms.
• Term Approval Updates (see subsequent slides)
32
Term Approval
Revised “term approval” tables/tab“Proposed terms” tab (192 rows)
Re-groups all the newly proposed terms with “MDCX_” prefixThe ‘Assign REFID’ button allows to change the REFID from an “MDCX_” term
to a “MDC_ term” (or anything else)
33
• “Ready to review terms” tab: 286 rows:– Terms with MDC_ prefix and null or 0 term code
Term being worked on by SDOs/MDC experts These processed terms would become / be added to the harmonized
Rosetta terms (hRTM) upon approval by SDO user and balloting
Term Approval, cont.
34
• “Ready to review terms” tab: 286 rows:– The ‘Edit RefID’ button allows approved user to edit a term and it’s attributes
– Edit a REFID, Standard Table, Block, Partition, Code(base 10), Systematic Name, Common Term, Acronym, and Description to the x73 database
– Pull down menus available (where enumerations provided/exist)
Term Approval, cont.
35
Terms being vetted (by the numbers)
• “Proposed Terms”: 192 rows• “Ready Terms Table”: 286 rows• “Mapped Terms”: 27 rows• “Proposed Units”: 101 rows• “Proposed Enumerations”: 133 rows
Term Approval, Proposed cont.
36
NIST MDC Testing Project
Web Sites• Project Web site: www.nist.gov/medicaldevices
• NIST HL7 V2 Test Tooling Web sites:
IHE-PCD Pre-Connectathon:http://hit-testing.nist.gov:13100/PCD-HL7WebPreCon/
IHE-PCD Connectathon: http://hit-testing.nist.gov:13100/PCD-HL7WebCon/
• NIST Medical Device Terminology Service:
Rosetta Terminology Mapping Management System (RTMMS):
http://hit-testing.nist.gov:13110/rtmms/
• NIST Implementation Conformance Statement
Generator (ICSGenerator):
– http://hit-testing.nist.gov/medicaldevices/ICSGenerator/ics_download.html
• DIM Work
– http://mdcx73-working.wikispaces.com/_NIST-ArtG
Semantic Interoperability of Medical Devices36
Domain Information ModelUML / DIM Editor, Progress Update
Semantic interoperability of Medical Devices
Test Tool Update – Michael Faughn
Prometheus Computing
Joint HL7/IEEE 11073 Healthcare Devices Working Group
National Institute of Standards and Technology
8 May 2014– Phoenix, AZ