1
Is that an ADaM dataset on the Janus wall?
The Humpty-Dumpty challenge of modeling study data with HL7-RIM
Dana Soloff
Director, Statistical Programming, Genzyme
September 25, 2009
BACUN – Boston Area CDISC Users Network
or SDTMV
2
Caveats
This presentation represents my thoughts, and not necessarily those of Genzyme.
3
Outline
Background
Our ADaM-HL7 Pilot What we did
Our motivation
What we learned
What do we (pharma) do next?
Our final analysis
4
CDISC-HL7 is coming!
“The FDA has embraced the HL7-RIM…
We envision the CDISC content to
be sent to FDA as XML messages based
on the HL7-RIM…
SDTM will evolve from a
submission standard to an analysis view…”BEHRMAN 2008
6*PDUFA: Prescription Drug User Fee Act. Oliva, March 2009.
balloted
9/2009
When is it coming?!!
7
Genzyme ADaM-HL7 Pilot
We hired consultants!
We trained on HL7 V3!
We tried to model ADaM to HL7!
We gave up!
(But Genzyme is trying again!)
8
What are the CDISC-HL7 Messages?
Study Design* What will be done?
Study Participation* Who is involved?
Subject Data What was observed? Includes analysis data Isn’t a message anymore
*Has published Domain Analyisis Model (DAM) (Think CDISC I.G.) and passed DSTU ballot 9/09.
Study Data: Now “CDA”No DAM – BRIDG?
It just says “Document””
10
Why? We were curious.
What is “CDISC-HL7”?
Why is the FDA doing this?
Will this impact statistical reviews?
How will we get our SDTM and ADaM data into CDISC-HL7?
Does this change our vision for data standards architecture?
11
SDTM & ADaM: Traditional datasets Two dimensional rows and columns
Keys relate datasets
Requires human readable metadata
Relationships between values often implicit
12
What is the impact of “implicit relationships?”
Analysis Dataset
One observation row Value for concomitant medication
Value for adverse event
Did the conmed cause the stroke?
Or was the conmed administered because of the stroke?
13
Specifications Match Structure of Datasets SDTMIG 3.1.2
SDTM 3.1.2
14
CDISC-HL7
Break each dataset up by variables and values
Elements floating in multi-dimensional space
Wrapped in little pods of metadata called attributes
Explicitly modeled relationships
Protocol Representation Model
BRIDG 2.2
What does it mean to map SDTM to BRIDG? (apologies to Diane Wold)
Define.xmlDM AE
LB
SDTMsPerson
Subject
Plays
Race
BirthdtEthnic
Usubjid
17
What is CDISC-HL7?
It’s not just SDTM reformatted
It’s a very big change
We have a lot to learn
The standards require further development
Complexity and our inexperience constrains our effective participation
18
Our questions
What is “CDISC-HL7”?
Why is the FDA doing this?
Will this impact statistical reviews?
How will we get our SDTM and ADaM data into CDISC-HL7?
Does this change our vision for data standards architecture?
19
The FDA wants more information on relationships between data
This adverse event was the result of this concomitant medication administered by this investigator on this date in response to this lab value…
Great for medical review!
Can be modeled in web browser and no limit to instant clicking around to understand relationships!
21
Pharma needs to join healthcare
EHR (electronic health record) is HL7 based
Efficiencies, reduced development time using same source data
Potential to combine sponsor clinical trial data with subject’s healthcare record data
Personalized Medicine – Genzyme!
Why analysis data in HL7?
Combine statistical results with “point of care” statistical results? No…Combine complex study-specific derived values across sponsors? No…Statistical analysis is performed on groups of observationsHealthcare is performed on individualsLinking the two is tough!
23
What is the main reason provided in the HL7 subject data use cases? Transparency.
Reviewers want more transparency between collected data and results
They want derived data and collected data together
They want to be able to easily identify which observations we imputed, excluded, etc.
24
Why not put analysis data on SDTM?There’s no place to put it.
Findings Interventions
Events
Response = Mean weekly lab > 5 units, no rescue therapies, no adverse events of interest over the
evaluation period
25
HL7 could theoretically solve this.
Response
Findings
InterventionsEvents
26
Except there are interim calculations
Data handling and algorithms applied at every step Imputations, selected observations
based on values or time windows
Often comparisons to other variables before choosing or calculating value Last follow up date could come from
AE, EOS, LB, etc.
RESPONSE
Rate of change
P
AP
Mean Lab
ATC Codes Transfusion
Adverse Events
Selected
28
The FDA also wants to compare actual to planned!
Protocol was amended four times! A lot of unplanned things happened
New drugs came on the market Sick people didn’t make it to scheduled
visits Trials weren’t executed perfectly Bizarre data values happened Some samples were incorrectly
analyzed
29
And then we get busy…
Perform same calculations on different populations ITT, Per Protocol
And by different imputation methods LOCF, WOCF
We may plan to use an observation in one analyses and not another…
30
Analysis data is different than collected data!
Real surgery doesn’t have do-overs!ITT, Per Protocol, Safety
More complexity and diversity in modeling statistics
Entities, acts, etc. don’t always make sense
31
Can we model ADaM in HL7?
Certainly not today! It might be possible in the future There will always be considerable
room for error Is there too MUCH information? Is HL7 the best way to provide more
transparency to reviewers? Is the cost-benefit ratio acceptable?
32
Our questions
What is “CDISC-HL7”?
Why is the FDA doing this?
Will this impact statistical reviews?
How will we get our SDTM and ADaM into CDISC-HL7?
Does this change our vision for data standards architecture?
33
The data are submitted and the fun begins!
FDA receives HL7 messages
Janus generates views of SDTM and ADaM that match
ours
additional analysis views with both collected and derived data
34
Would the FDA’s views of SDTM and ADaM match ours?
SDTM and ADaM allow flexibility in modeling
How can one model from the specific to the general without a human or rules?
One will never have standard messages defined to cover all cases
35
Will the FDA reviewer use our datasets or theirs?
SDTM is a collected data standard Original error was assuming that SDTM could ever
be basis for statistical review If reviewers are unhappy that there is no analysis
data on SDTM… And sponsors are required to model data in HL7
because SDTM is inadequate… And reviewers have access to another view than
SDTM that includes analysis data created from HL7…
Why would they use SDTM? Other than for WebSDM, iReview
36
Will views be reassembled correctly? The Humpty-Dumpty Problem!
The data and the metadata are in pieces! Some is part of HL7 attributes
The rest is in our black box
How will they put together an accurate view of the analysis data?
Our “metadata” – define.xml won’t document their view
Will derived variables be used incorrectly when used out of dataset context?
37
There will be challenges for the FDA and sponsor communication
If we have different input datasets
Or the Janus generated views are not accurate
How will this promote transparency with regard to statistical review?
It might help if FDA reviewers provide sponsors with their analysis data views We need define.xml!
And ODM format!
38
Is all that really better than this?
Selection criteria described in define.xml
39
Our questions
What is “CDISC-HL7”?
Why is the FDA doing this?
Will this impact statistical reviews?
How will we get our SDTM and ADaM data into CDISC-HL7?
Does this change our vision for data standards architecture?
40
Who prepares the submission?
Study, Data &
Analysis SME
CDISC-HL7SME
41
Understanding trial, data content and analyses key to correct modeling
Relationships between observed and calculated not all captured as data Until we have structured protocol &
SAP, a fairly complete set of robust messages, and maybe even then…
We need SMEs to model data
They don’t have the HL7 expertise
42
How do we QC the result?
Is this double work?
Complex specs for ADaM datasets
Complex specs for HL7
Complex specs to reassemble ADaM from HL7
Double-program pre-HL7 ADaM with post-HL7?
43
Our questions
What is “CDISC-HL7”?
Why is the FDA doing this?
Will this impact statistical reviews?
How will we get our SDTM and ADaM data into CDISC-HL7?
Does this change our vision for data standards architecture?
44
Almost everything we’ve done has been valuable! All are plans are usable!
Governance
End-to-end metadata driven data standards roadmap
Metadata Repository
Structured Protocol
Central Lab Standard
CDASH based collection standards
45
Where should we go from here?ADaM may be better than HL7 in providing transparency
Be realistic about what SDTM can do It’s fine as a collected data standard But not as a base for FDA analysis review
Implement ADaM 2.1 and ADaMIG 1.0! Improve our metadata (define.xml) Err on the side of traceability!
Inclusion of SDTM data a priority Intermediate datasets when helpful Provide FDA multiple “views” of the same data Provide FDA helper variables for analysis
46
What can we learn from our SDTM experience?
Less time spent developing model More time testing actual data! Engage FDA to understand and
develop joint vision Collaborate with each other on tool
development and share costs Pitch into fund to hire HL7
“technical lobbyist”
47
What else should sponsors do?
Buy a really big color printer!
Buy really big paper!
Buy a really big magnifying glass!
Buy a big HL7 warehouse! (eventually)
CDISC-HL7: Current sentiment heard around town…
HL7 may make sense for collected data- but we don’t like it!
- and we still need SDTM as a base for ADaM
ODM makes more sense for analysis data
- until we have proof that HL7 satisfies the use cases
49
Is HL7 TOO MUCH information? The world is round and we do
need a jet for collected data…
But are you sure we should take our jet to the ADaM grocery store?
Let’s give pharma a chance to upgrade to more robust ADaM!
HL7 is Coming!
Nor is
this
.
There are open questions.We all must work together to succeed
Repud
iatio
n
Mak
e da
rtboa
rd o
ut o
f UM
LOpe
n-m
inde
d,
scie
nce-
base
d,
prob
lem
-sol
ving
Lear
n UM
L
Blind
Faith
Get
HL7
Tat
too
Igno
re it
…
HL7 response meter
53
Thank you!
Questions?
Top Related