Post on 11-Nov-2014
description
“Modules of Workforce Development: Pragmatic Approaches to Knowledge Acquisition, Application, and Information Dissemination”
Enterprise Access to Evidence-Based Literature: A Dynamic Digital Library for Public Health
Enterprise Access to Evidence-Based Literature: A Dynamic Digital Library for Public Health
Karen Dahlen, Project Consultant Elaine Martin, Director
New England Region NN/LM Lamar Soutter LibraryUniversity of Massachusetts Medical School
This project is currently funded by the NLM via contract through the NN/LM, New England Region
Enterprise Library Approach to Knowledge Acquisition, Application & Information Dissemination
Measure Importance of Trusted Information Access
Ensure Direct Access from Intranet-Based Site
Access (Without Name/Password Required)
Support Core Competencies/Training
Use Survey Instruments Adapted to the Location
Relate Cost Efficiencies to Enterprise Licensing/Creative Partnerships
Document Institutional Change in a Dynamic EnvironmentEvaluate Over Time (Illustrate Efficiencies)
Shifting the Culture to Evidence-Based Practice
• Culture of evidence-based practice was affected when funding for public health libraries was eliminated.
• State Libraries (in many states) were mandated to support state agencies, but had no collections to support public health.
• Streamlined access to resources through the Digital Library Platform-- along with trust in new relationships--prompted resurgence in use of resources and has facilitated this cultural shift.
• Training stimulated access to resources reviving evidence-based interest and relationship to practice.
• Project has the ability to leverage government resources (NLM, CDC, NAL) to improve evidence-based practice.
ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH
Background (Why this Project)
Project Rationale
Partnerships
Council on Linkages: Academia & Public Health Practice
Public Health Competencies
• Analytic/Assessment Skills– References sources of public health data and information.– Information technology to collect, store, retrieve data.– Utilizes data to address scientific, political, ethical, and
social public health issues.
• Policy Development/Program Planning Skills– Analyzes information relevant to specific public health
policy issues
• Public Health Sciences Skills – Conducts a comprehensive review of the scientific
evidence related to : a public health issue, concern, or intervention.
– Retrieves scientific evidence from a variety of text and electronic sources.
• Knowledge Informatics– Access trusted resources directly from the
desktop.– Describe relationships and functionality of e-
resources to core public health interests.– Introduce specific types of resources: coding
and classification tools.• Informatics Skills
– Train on interoperable features that support public health work, including connectivity to trusted sites; advanced search filters, use of topical alerts and information mgt software.
Competencies: Council on Linkages Project Tied to Competencies
Obstacles to Getting Information
Gathering and Analyzing EB Information Rapid Detection of Epidemics
Detecting an anthrax
epidemic one day earlier
would save $1-7 billion
CDC Presentation by
C. Safran citing:
http://www.cdc.gov/ncidod/eid/vol3no2/kaufman.htm
Kaufmann AF, EID, V3, N2
ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH
Library Without Signature Building
Goal of the Project
Who ParticipatesObjectives & Data Sets
Development of Digital Library
PHIA: Innovative Business ModelNot Tied to Traditional Building
Madison, WI Atlanta, GA
Goal of PHIAPublic Health Information Access
To determine what resources are core, useful, evidence-based to advance public health practice.
Full text directly available through DL
Full text available through LP
Core to public health
Unique Aspects of Digital Library
• Resides on PHD Intranet (outside a signature building) without need for id and password.
• Relies on collaboration, interoperability & funding.• Move toward analysis; less time on access• Skills & training support personalized, interactive styles tied
to continuum of learning and institutional-specific priorities and projects.
• Collection of evidence occurs in many aspects of Brownson wheel not just at the literature review stage.
• Fundamental change in the way resources are distributed and utilized.
Public Health Departments& Library Partners
State Public Heath Department Library Partners AK Alaska Division of Public Health (AKDH) Univ of Alaska (Anchorage) Medical LibraryAR Arkansas Department of Health (ADH) Univ of Alabama Medical Sciences LibraryCO Colorado Dept of Public Health & Environment Poudre Valley Health System Library
CT Connecticut Public Health Department University of Conn Maynard Stowe Library HI Hawaii Department of Health University of Hawaii Medical LibraryIN Indiana State Department of Health Ruth Lily Medical Library, Indiana UniversityKY Kentucky Department for Health University of Kentucky Medical Library
ME Maine CDC Maine Medical Center Library MA Boston Public Health Commission* University of Massachusetts Soutter Library MA Massachusetts Public Health Department Lemuel Shattuck Hospital Library MD Maryland Dept of Health & Mental Hygiene University of Maryland HS LibraryNH New Hampshire DHHS New Hampshire DHHS Library RI Rhode Island Department of Health (RIDH) Rhode Island Hospital/LifespanVT Vermont Department of Health (VDH) UVM Dana Medical Library WI Wisconsin Department of Health Services Ebling Library of the Health Sciences WV West Virginia Bureau of Public Health West Virginia University LibrariesWA Washington State Department of Health University of Washington HS Library
PHIA Evaluation ProcessTied to Data Model
To advance access to evidence-based resources in support of improved public health practice.
What resources are high use?
What technical issues arise?
What resources are missing? How can training be improved?
What project components lead to improved efficiencies?
What cost efficiencies are involved with an enterprise approach to digital library implementation, and training.
• Logic Model– Resources, activities, outputs, outcomes
(minutes, visits, trainings, people lists, IP management, levels of troubleshooting, RML reports.
• Data Tools– Overarching questions– Journal SurveyMonkey– Pre-Post Training Surveys– Just-in-time Information Checks– Formal Evaluation (interviews & focus
group sessions
Data Sets & Sources
• Resource Metrics Collected• Journal Survey Monkey Identifies
Perceived Need & Library Use.• Document IT technical issues. • Licensing Issues Related to
Journals and Databases. • Training Surveys Collected• Preliminary Meetings/Updates Held.• Trainings Held• Year End Evaluation
--Interviews & Focus Groups
View of PHIA Digital Libraries
Connecticut Digital Library Arkansas Digital Library
ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH
Resources on Digital Library
Use of Resources
Article Delivery Use & Costs
Databases & Major Publishers of Resources
PubMed , Other NLM DatabasesNAL, CDC & Other Gov’t Resources
CLSI (Clinical Laboratory Standards) Global HealthCochrane Library
– Systematic ReviewsStat!Ref—E Books (35)
-- Coding & ClassificationASABE –Health Linked to Agriculture
Identifying Core, Useful, & Evidence-Based Journals
Publisher Total Use Journal Title UseMass Medical Society (1) 14,136 New England Journal of Medicine 14,136
AAP(1) 9,888 Pediatrics 9888
Oxford University Press (20) 23,369 Clinical Infectious Diseases 3147
Springer (18) 2, 485 Maternal & Child Health 474
ASM Journals (12) 3,518 Journal of Clinical Microbiology 1729
Annual Reviews (9) 697 Annual Review Public Health 394
Sage (11) 939 The Diabetes Educator 135
BMJ (9) 2932 British Medical Journal 1832
Mary Ann Liebert (11) 1798 Breast Feeding Medicine 355
University of Chicago Press (6) 955 Infection Control & Hospital Epidemiology 749
Lippincott/Ovid (6) 122 J Public Health Mgt Practice 55
NEJM Use 2010-2013
0
1000
2000
3000
4000
5000
6000
7000
4864
1772
982554 314 527 337 311
890308
1183 1178494 422
2010=2087 2011=11,194 2012=14,022 2013=14,136
ISDHM
DPH
MDHM
H
CDPHE
Alaska DH
KYCPH
NH DHHS
WID
HSRID
HVDH
ADH
HI DPH
Maine CDC
CT PHDBPHC
298 287233
175135
94 90 89 82 78 7142 32 18 5
ASM: Journal of Clinical Microbiology
Total Use 2013=1729
Oxford University Press UseJan-Dec 2013
Total Use by Each PHDN=13,827 High Use Journal Titles
Journal Title Total UseClinical Infectious Disease 3147American Journal of Epidemiology 941Schizophrenia Bulletin 771The Journal of Infectious Diseases 627
ICES: Journal of Marine Science 474JNCI: Journal of the National Cancer Institute 434
Journal of Antimicrobial Chemotherapy 297
International Journal of Epidemiology 288Alcohol and Alcoholism 274Journal of Analytical Toxicology 262Health Promotion International 252Rheumatology 246Age and Ageing 240
PHD TotalAlaska DPH 1245Arkansas DH 1483BPHC 182CDPHE 608CT DPH 1234HI SDH 638ISDH 700KYDPH 1134Maine CDC 757MDHMH 1427MDPH 1817NH DHHS 157RIDH 620
VDH 709WI DHS 1116
STAT!Ref E-Books/Tools Provide Basic Understanding of Topic
Filters & Tools– Codes[5]
• CPT with RVUs Data File, INGENIX® (2011)[1]
• ICD-9+CM - Volumes 1, 2 & 3 (2012)– Titles By Discipline[344]
Searchable Alerts (by topic)
– Functionality with NLM– Chapters link to PubMed– TOXNET linked to Search– Related Concepts/Meta…
Functionality with CDC– MMWR, Community Guide;
STAT!Ref E- Books for Public Health
Titles Containing PH ContentE-Book Use
ACP PIER, Journal Club & AHFS DI Essentials (2013) 482Oxford Textbook of Public Health - 5th Ed. (2009) 407
Manual of Clinical Microbiology - 10th Ed. (2011) 403
AHFS Drug Information (2013) 305Red Book: 2012 Report of the Committee on Infectious Diseases - 29th Ed. (2012) 241
Emerging Infections Series (2008 - 2010) 216
Textbook of Modern Toxicology, A - 4th Ed. (2010) 209
Coding & Classification BooksE-Book Use
ICD-9-CM - Volumes 1, 2 & 3 (2013) 881
CPT with RVUs Data File, INGENIX (2013) 474
ICD-10-PCS: Procedure Coding System (2014) 12
ICD-10-CM: Clinical Modification (2014) 15
Diagnostic and Statistical Manual of Mental Disorders 4th Ed. (DSM-IV-TR, 2000) 107
CLSI (Clinical & Laboratory Standards)20 Documents Used 790 Times
April 21, 2014Doc # Document Name #M100-S24
M100-S24: Performance Standards for Antimicrobial Susceptibility Testing; Twenty-Fourth Informational Supplement
248
EP09-A3
Measurement Procedure Comparison and Bias Estimation Using Patient Samples; Approved Guideline—Third Ed.
102
EP12-A2
User Protocol for Evaluation of Qualitative Test Performance
55
MM09-A2
Nucleic Acid Sequencing Methods in Diagnostic Laboratory Medicine; Approved Guideline—Second Edition
54
Calculations Related to Enterprise Licensing
State PHD FTE 10% RuleADH 2781/1390 139AKDPH 525 52BPHC* 1100/550 55CDPHE 1200/600 60CT DPH 780 78Hawaii DPH 3000/1500
150ISDH 780 78Kentucky BPH 400 40MDHMH 8000/4000 400Maine CDC 395 39MDPH 3000/1500 150NH DHHS 279 30RIDH 400 40VDH 300 35WDPHS 400 40VW DHS 780 78Washington
1600/800 80Total PHIA FTE 10,999 1546
Enterprise Licensing is based on 10% of the Total FTE level to capture baseline information and measure use/interest of resources.
Article Delivery Costs Via Library Partnerships
May 2011 – April 2012; Cost=$18,384 May 2012 --- April 2013; Cost=$12,241
STATE ILL DD Total
Connecticut 0 83 83
Maine 560 364 924
Massachusetts (BPHC) 8 3 11
Massachusetts (MDPH) 1 1 2
New Hampshire 0 0 0
Rhode Island 47 36 83
Vermont 21 191 212
Colorado 323 156 479
Total 960 834 1794
STATE ILL DD Total
Connecticut 2 20 22
Maine 154 90 244Massachusetts (BPHC) 9 10 19Massachusetts (MDPH) 7 6 13New Hampshire 0 0 0
Rhode Island 82 79 161
Vermont 54 243 297
Colorado 112 40 152
Arkansas 32 73 105
Kentucky 8 3 11
Wisconsin 0 0 0
Total 462 593 1055
Training Topics Embrace National and State Interests
Maryland DHMH Health Initiatives• Health Disparities(sexual minorities) • Preventive Services ACA* HIV Screening• Immigrant Health
Undocumented individuals under ACA• Drug Resistant Disease which impacts
Impact on TB, STI (plus others) • Health Care Reform and Interpersonal
Violence/Domestic Violence• Infectious Diseases; Hepatitis C Virus
HIV Infections --HIV and HCV Co-Infection
Number of Trainings/Diversity of WorkforceSTATE PHD Trainings Attendance
RI RIDH 2 30
VT VDH 3 46
MEMAINE CDC* 3
52
MA BPHC* 4 58
MA MPHD* 3 32
NH NH DHHS 2 24
CO CDPHE* 5 100
CT CT DPH* 5 101
KY KDPH 2 46
AR ADH 2 31
WI WI DPH 2 33
AK AK DPH 2 50
HI HI DPH 2 68
IN ISDH 2 60
MD MDHMH 2 38
TOTAL 15 41 769
Maryland Indiana Hawaii
Nutrition Specialist (2)
State Epidemiologist Education Coordinator (2)
Research Statistician Supervisor – Labs + (3) Asthma Coordinators(2
Policy Analyst (2) Injury Prevention Epidemiologist
Legislative Liaison
Program Mgr (2) Staff Attorney Epidemiologist
Epidemiologist Microbiologist ( 2) Research Analyst (2)
Medicare Specialist/Advisor (2)
Regional Program Director
Program Coordinator, SAPB
Policy Advisor (2) Director of Prog Evaluation
Epidemiologist (3)
Psychiatric VT Director of Prog Development
Genetic Counselor
Executive Ass’t Field Epi Director Tobacco Prevent Mgr
DeputySecretary
Access Services Mgr Informatics Analyst
Director VSA IT Supervisor Nutritionist
ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH
Enterprise EfficienciesUnique Aspects of the Business ModelEvaluation Processes
Cost Efficiencies: Article Analysis
Direct Access to Articles Alternative Article Delivery
Contract Year Total # Amount $
May 2013-March 2014 1038 10,102
May 2012-Apr 2013 1055 12,241
May 2011-Apr 2012 1794 18,385
May 2010-Apr 2011 672 8,263
Contract Year Total # Amount $
May 2013 –April 2014 17+ 285,409
May 2012-Apr 2013 15 200,352
May 2011-Apr 2012 11 156,280
May 2010-Apr 2011 9 96,252
Enterprise Licensing Efficiencies: Single TitlesContract Year May 2012-April 2013
Journal Title # Uses Cost Per Use
Replacement Cost
Cost of License
Am J Tropical Med & Hygiene 410 $1.58 $4,510 $ 650
Am J Respiratory & Critical Care Med 189 $14.28 $2,079 $2700
Health Affairs 2038 1.81 $22,418 $3690
Infection Control & Hospital Epidemiology
749 1.50 $8,239 $1125
Public Health Reports 658 4.69 $7,238 $3085
Intern’l J of Tuberculosis & Lung Diseases
192 5.83 $2112 $1120
Annual Review Series 595 12.60 $6545 $7500
Pediatrics 7846 .68 $86,306 $5300
PHIA: Innovative Business Model
• Enterprise licensing saves time, money, and effort.• Central management of IP addresses ensures quick turn-around.• Ability to understand change (System, personnel, environment).• Resources are identified via benchmarking,metrics of direct use through
Digital Library, and requests from PHDs.• Journals are identified via “Article Delivery” on Library Partner side.• Alternative delivery of resources strengthens state relationships with
immediate access (within 4-24 hours).
EVALUATION PROCESSES Levels of Data Collection
• Journal SurveyMonkey (baseline data for PHIA and PHD)• Enterprise Licensing (cost effectiveness over time)• Vendor statistical reports (validates “use of resources”) • Monthly reports from “library partners” (measures use and identify
resources to develop Digital Libraries). • Ongoing capture of suggested e-books, journals, and databases to
enhance collection• Pre-Post training links
– (data related to knowledge of resources)• Feedback from hands-on training• Interviews with leaders and workforce• Focus group sessions (after one year; subsequently) • Quarterly/Annual Reports (submitted to NLM)
Results
• Digital libraries will be in place in 17 PHDs by July.• Full-text access to more than 150 e-journals, 5
databases, reports and more.• Introductory sessions have introduced PHD leadership to
the project and explained obligations.• More resources have been added to “digital library.”• Structured training has been held in 15 PHDs.• More than 600 people have been trained with backup
models currently in test stage.
NACCHO Award Brings Recognition to Project
• March 2012 and 2013, the PHIA PHD Digital Library Project received a “promising practice award” from NACCHO.
• Presentations have been held at national and local meetings, e.g., APHA in Washington, DC in 2011 and Boston in November 2013.
• Article was published in AJPH in January 2014. Project was cited in August 2012 EID (CDPHE).
• Presentations have been made at national public health meetings, national and regional library meetings.
Testimonial
“Great recognition for this very valuable initiative. Kudos and thank you.”
Jewel Mullen, MD, MPH, MPA, Commissioner, Connecticut Department of Public Health
Discussion & OutcomeHow Do We?
• Expand PHIA to all 50 states.• Enhance partnerships & relationships.• Continuously train PH Workforce to improve
competencies (knowledge and skills) given staff turnover.
• Build a sustainable model including cost structure and project management.