Medical Writing Certification: Where Are We Now? Thomas Gegeny, MS, ELS (Chair) Karen Potvin Klein,...

Medical Writing Certification:Where Are We Now?

Thomas Gegeny, MS, ELS (Chair)Karen Potvin Klein, MA, ELS (Chair-Elect)

2012 Annual Conference Open Session #38 – Medical Writing Certification: Where are we now? 1

2

• Provide a brief history of AMWA’s certification-related activities since the 1990s

• Describe recent and ongoing activities by AMWA to develop a medical writing certification program

• Provide information on the current processes and planned timelines

• Review results from AMWA’s recent Job Analysis Survey, with data from more than 1,000 respondents

• Share next steps and opportunities to contribute

• Conduct question-and-answer session with panelists2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?

Today’s Objectives

3

Brief History of AMWA’s Initiative on Medical Writing Certification

2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?


Moving Toward Certification

1990s 2000s 2010s

1996: AMWA Certification Task Force • Issued preliminary report • Recommended to continue research

2009: Medical Writing Certification Task Force• Included AMWA members who also represented BELS, DIA, and ISMPP• Researched certification programs of allied groups (BELS, ACCME, ISMPP, CSE,

others); survey of employers; AMWA membership survey (2010)• Recommended that AMWA pursue certification

2007−2008: Long-Range Planning Committee Report• Noted perceived weakness for AMWA as “lack of certification in a competitive market”• Recommended development of credentialing process be added to AMWA’s professional

development offerings

2011: Certification Commission established by AMWA Board of Directors


2010 AMWA Membership Survey

Is professional certification with a competency examinationdesirable for the medical communication profession?

Yes (65.6%)

No (34.4%)

N = 1339

6

Certification CertificateGranted after an assessment process Granted after an educational process

Usually requires experience Open to individuals regardless of experience

Granted by an independent, standard-setting body

Granted by associations or institutions

Indicates knowledge and/or competency through an examination

Indicates completion of course work

Certification vs Certificate

Certification is a credential granted to an individual after the individual has met established criteria for competency, usually through an examination.

A certificate is a document attesting to an individual’s participation in an educational program designed to help him or her fulfill specific learning objectives.

Note: Table based in part on table available at http://www.cfre.org/pdf/Certificate-vs-%20Certification.pdf. Accessed Sept10, 2012.

AMWA offers an educational certificate

http://www.cfre.org/pdf/Certificate-vs-%20Certification.pdf


Education & Certification

• AMWA’s educational programming is core to its mission and will continue to be a cornerstone of its activities. AMWA continues to invest in its educational programs and resources.

• Medical Writing Certification will be a separate activity with dedicated budget and resources. It will not be directly tied to AMWA educational certificates (though these can be one means of preparing for an examination).

8

• Overview– Established to initiate, evaluate, maintain, and oversee the credentialing program for

medical writers – Seeks to represent the diversity that exists within the profession and serve as a voice for

stakeholders who have an interest in maintaining high standards in medical writing

• Authority and Responsibility– Charged with overseeing matters pertaining to certification of medical writers– Makes recommendations on certification policy and quality improvement– Oversees the application, examination, recertification, and appeals processes

• Mission To maintain a quality certification program for the medical writing profession by:

1. Overseeing the certification process and monitoring policies 2. Developing and revising the certification examination through exam assembly

committees and item development


Medical Writing Certification Commission

9

Chairperson(2-year term)

Chairperson-Elect

At-Large Members (4+)*

AMWA Executive Director

(ex officio)

Representative from AMWA Leadership(ex officio)


Current Composition of the Certification Commission

*Number may change over time; serve 2-year staggered terms

10

How Will the Commission Fit Into AMWA’s Organizational Structure?

AMWA

AMWA Leadership(Officers, Board of

Directors)

Chapters

Departments and Committees

Certification Commission

Associated Committees

• Fiscal oversight by AMWA• Separate budget (revenue, expenses)• Infrastructural resources (eg, technology)• Certification-specific staff support• Outside governance structure of AMWA

Commission and subcommitteemembership will include but not be limited to AMWA members

• Fulfill organizational mission(education and professional development)

• Oversight of AMWA business activities(finances, legal, etc)

• Full staff support (programs,administration, etc.

Note: Once the certification process is in place,Commission member candidates will be elected annually, with staggered 2-year terms.

11

What Has the Commission Accomplished So Far?

Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct

2011 2012

First Telecon

CommissionFormed:• B. Bonk• D. Clemow• B. Gastel• T. Gegeny• S. Hudson• K. Klein• S. Krug*• M. Mallia• B. Snyder*

*ex officio

Vendor contracted

Profession-related materials and references gathered to develop a draft listing of knowledge, skills, and abilities (KSAs) for competent medical writers: • Job descriptions • Educational curricula

(including AMWA wkshps)• Competency model (DIA)• Articles, other publications • Guideline documents

Job Analysis Panel meeting (2 days):Six commission members(DC, BG, TG, KK, SK, MM)and L. Alexander, N. Taylor, F. Witte

Job Analysis Panel formed to finalize KSA list and design survey

Draft KSA review (volunteers)

KSA list review (volunteers)

Final KSA review (commission, panel, & volunteers)

Review & BETA test of online survey

Survey

Job Analysis Panel teleconference:• Review survey results with vendor• Give input for final report• Verify next steps to communicate data• Plan for exam development

Annual Conference Open Session (OS-38)

KSA Survey Data Reportfinalization

12

What Lies Ahead?

Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct

2012 2013

New members joinCertification Commission(staggered terms)

• Call for volunteers (Sept 2012)• Enlisting of Subject Matter Experts (SMEs) to

assist with item writing for exam

Item Writing

• Draft test form• Map items to

content outline

Test form review and QC

Development of policies and procedures:• Eligibility and application for exam• Governance for commissioning body• Resources and exam preparation materials 1st exam administration:

Late 2013/Early 2014• Item analysis• Cut score study• Raw score determination• Final score roster

Article for AMWA Journal, other venues

13

So…Where Are We Now?



Why Was A Job AnalysisSurvey Done?

• A job analysis survey is a part of the certification process that helps define medical writing practices – Provides uniform standard– Identifies those who meet the standard– Ensures minimum level of competence

• Specifically, the job analysis establishes a source of evidence from input gathered across a large number of survey participants– Defines the profession and is defensible – Establishes foundation of testing program

• Establishes assessment content• Provides link between assessment and practice• Provides legal defensibility

– Determines test design

2012 Annual Conference Open Session – Medical Writing Certification: Where are we now? 15

Who is the certification vendor and how were they chosen?

• A competitive Request for Proposals (RFP) process was conducted in 2011

• Schroeder Management Technologies (SMT) was selected from among 3 potential vendors (considering price, technology, and experience)– SMT was founded in 1995 by testing industry leader Dr. Lee Schroeder

– SMT employs more than 30 testing industry professionals with specialized affiliation that include American Statistical Association (ASA), the Association of Test Publishers (ATP), the American Educational Research Association (AERA), the National Council on Measurement in Education (NCME), the Council on Licensure Enforcement and Regulation (CLEAR), and the Institute for Credentialing Excellence (ICE)

• SMT offers a variety of special exam development and implementation services including Job Analysis Item Writing Exam Assembly & Administration

Item Banking Item Analysis Standard Setting & Scoring


Job Analysis Survey

• Provides information related to the expertise and expectations of a (minimally) competent medical writing professional

• To be included, a task or KSA must be weighed with the following considerations:– Is it important to competency across all medical writing settings?– Is it performed by “practitioners” regularly?– Is it clearly written?– Are there errors or omissions?– Is it assigned to the appropriate area (“domain”) within the outline?– Is it testable?


Establishing Domains: Looking to AMWA’s Definition of the Profession• AMWA’s definition of a medical communicator

( http://www.amwa.org/default.asp?id=420 )

Medical communicators write, edit, or develop materials about medicine and health. They do this by gathering, organizing, interpreting, and presenting information in a manner appropriate for the target audience.

Materials prepared by medical communicators (collaborating with others, or working independently) include the following:

– Patient education brochures, news articles, Web content, and books for the general public– Journal articles and continuing education monographs for health care professionals– Regulatory documents for government agencies– Grant proposals for research scientists and institutions– Sales training and marketing materials for the pharmaceutical industry

Professional medical communicators have communication expertise, awareness of ethical standards, and health care knowledge.

http://www.amwa.org/default.asp?id=420

http://www.amwa.org/default.asp?id=420

18

Five Medical Writing Domains

Evaluating

Interpreting

Gathering

Presenting

Organizing

ethics


Medical Writing Knowledge, Skills, & Abilities (KSAs) by Domain (1)


Do The Domains/KSAs Apply Across the Spectrum of Medical Writing Careers?

Example of Medical Writing Environments

Domain/KSA Patient Info/Education

Pharmaceutical, Regulatory

Clinical or Basic Research

Continuing Education (CME, etc)

Gathering:Elicit information from collaborators, stakeholders

interviews with clinical experts, patient educators, dieticians, patients, etc.

medical reviewers, statistical support, legal/regulatory reviewers

principal investigators, co-authors, biostatisticians

authors/contributors, review committees (content/activity accreditation)

Evaluating:Evaluate collected information with regard to content, audience, context

ensure level of information is suitable and useful for patients and/or caregivers

evaluate content for appropriate level of evidence and suitability for intended context

ensure collected references are relevant to line of evidence and support logic

confirm content meets educational objectives for intended health care audience

Organizing:Apply templates and guidelines to documents

factsheet, Q&A, news article format;health literacy recommendations

clinical trial report, common technical document, product label; FDA

IMRAD paper, case report, poster; ICMJE, HIPAA, CONSORT

needs assessment, learning objectives, slide kits, test questions; ACCME

Interpreting:Synthesize and integrate information

summarize key information on treatment/condition; inform patients

summarize key safety and efficacy conclusions

communicate how study hypothesis was or was not supported by data

integrate clinical guidelines and current evidence to address educational needs

Presenting: Develop clear, concise prose A L W A Y S !

25

Medical Writing Job Analysis: Survey Results Summary

July 2012


26

Job Analysis Survey

April 2012 – Call for participation sent directly* to approximately 14,200 potential respondents:

• AMWA (5283 current, 4109 previous members)

• Drug Information Association, DIA (2687 members, specifically themedical writing special interest area)

• Board of Editors in the Life Sciences, BELS (988 members)

• Society for Technical Communication, STC (207 members)

• International Society for Medical Publication Professionals, ISMPP (930 members)

Available online for approximately 6 weeks (May 28-July 8, 2012).

A total of 1,177 individuals completed the survey. (Return rate = 8.0%, an average return rate for an unsolicited survey.)

*Note: Survey was also announced via social media, e-mail lists, and personal networks.

Survey Distribution and Response

27

Job Analysis Survey

The following performance and importance frequency rating scales were used for the job domains section of the survey:

Performance:0 = Not Performed

Importance:1 = Of No Importance2 = Of Little Importance3 = Moderately Important4 = Very Important5 = Extremely Important


Survey Rating Scales

28

Job Analysis Survey


Demographics: Experience

0-5 6-14 15-24 25+0

5

10

15

20

25

30

35

40

27.6

38.0

21.3

13.1

Years of Experience

Perc

enta

ge

29

Job Analysis Survey


Demographics: Region

North South Midwest West0

5

10

15

20

25

30

35

40

45

50

44.1

20.918.7

16.3

U.S. Region

Perc

enta

ge

Job Analysis Survey

2012 Annual Conference Open Session – Medical Writing Certification: Where are we now? 30

Demographics: Outside US

U.S. Africa Asia Cntrl. or So. Am.

Europe Middle East

No. Am. (non-U.S.)

Oceania Other*0

10

20

30

40

50

60

70

80

90

79.22

0.343.07

0.17

11.84

0.433.49

1.11 0.34

All Regions

Perc

enta

ge

* Four "Other" respondents provided: China, Entire US (independent writer), India, Many US Clients, several in UK, Japan, Eastern Europe

31

Job Analysis Survey


Demographics: Work Setting

Ad agency, P

R firm, o

r medica

l writi

ng /communica

tion agency

Contract

rese

arch org

anization (C

RO)

Govern

ment agency

or contra

ctor

Hospita

l, clin

ic, or o

ther health

care

facil

ity

Medica

l devic

e company

Nonprofit o

rganiza

tion or rese

arch in

stitute

Pharmace

utical/b

iotechnology c

ompany

Self-

employed/fr

eelance

Universi

ty or m

edical s

chool

Other*0

5

10

15

20

25

30

35

13.4

8.7

2.6 4.2 3.2 4.4

22.8

28.7

5.07.1

Work Setting

Perc

enta

ge

32

Job Analysis Survey


Demographics: Primary Role

Writer Editor Writer and editor

Researcher Supervisor or manager

Other*0

10

20

30

40

50

36.1

11.8

32.9

3.2

11.1

4.9

Role

Perc

enta

ge

33

Job Analysis Survey


Demographics: Level of Education

Some college

Associate's d

egree

Bachelor's degree

Master's

degree

Doctoral degree (PhD or e

quivalent)

Medical degree

Pharmacy degree

Nursing degree

Other*0

10

20

30

40

50

0.8 0.3

21.4

30.7

37.1

4.3 3.10.4

1.9

Education

Perc

enta

ge

Data from 2010 AMWA Membership survey:

Bachelor’s: 27.0% Master’s: 29.8% PhD: 26.5% MD or DO: 3.4% PharmD: 3.6%

34

Job Analysis Survey


Demographics: Field of Study

Engli

sh or o

ther h

umaniti

es

Journ

alism

or communica

tion

Medica

l, tech

nical, o

r pro

fessi

onal writi

ng

Health ca

re pro

fessi

ons

Life sc

iences

Other n

atural

science

s (e.g., c

hemistry, p

hysics

)

Other s

ocial s

cience

sOth

er0

10

20

30

40

50

12.2

5.1 5.1

13.5

41.7

6.44.0

11.9

Field of Study

Perc

enta

ge

35

Job Analysis Survey


Demographics: Professional Organizations

AMW

AAHCJ

AMW

A (Austr

alasian)

BELS CSEDIA

EMW

AISM

PP

NASWRAPS

STCTIPPA

WAM

E0

10

20

30

40

50

60

70

8074.0

1.3 0.3

11.6

5.6

14.510.4 8.1

3.1 3.3 3.1 1.9 0.5

Memberships

Perc

enta

ge

Note: More than one choice was allowed

36

Job Analysis Survey


Demographics: Professional Designations

CHES CMPP CPTC ELS ELS(D) RAC Other0

10

20

0.5

4.8

0.2

12.6

0.31.1

11.5

Designation

Perc

enta

ge

37

Job Analysis Survey


Demographics: Sex

Male; 20.1%

Female; 79.9%


Female: 84.4% Male: 15.6%

38

Job Analysis Survey


Demographics: Age

20s 30s 40s 50s 60s 70+0

10

20

30

40

5.1

24.0

28.129.8

11.6

1.4

Age Range

Perc

enta

ge


Median age: 47 years

39

Job Analysis Survey


Demographics: Ethnicity

0

10

20

30

40

50

60

70

80

90

10087.3

1.7 1.67.0

1.2 1.1

Ethnicity

Perc

enta

ge

40

Job Analysis Survey

More than 93% of the respondents indicated that the survey either completely or adequately described the critical tasks required of competent practice of the medical writer.


Survey Coverage of Medical Writing

Inadequately; 6.4%

Adequately; 61.4%

Completely; 32.2%

41

Job Analysis Survey

Cronbach’s coefficient alpha reliability estimate was calculated to evaluate the internal consistency of the instrument (Crocker & Algina, 1986).

This statistic is bound between 0 and 1, with values closer to 1. This indicates higher instrument reliability, showing that the tasks work well together to measure the knowledge, skills, and abilities requisite in the medical writing field.

The coefficient alpha was calculated using the SPSS© statistical program, version 17.0.

For this survey, the coefficient alpha reliability estimate was 0.97 for the importance statements. This estimate indicates that the survey is a reliable instrument.


Scale and Respondent Reliability

42

Job Analysis Survey


Mean Scoring of Item Importance

Rating Scale1 Of No Importance2 Of Little Importance3 Moderately Important4 Very Important5 Extremely Important

43

Job Analysis Survey



Lowest-ranking items still fallbetween moderately to very important

44

Job Analysis Survey



53 out of 69 line items were ranked at least “very important”

45

Job Analysis Survey



46

Job Analysis Survey


Subgroup analyses

Subgroup analyses were run based on the demographic questions related to Years Experience, Geographical Region, and Work Setting. Significant differences were identified at P=0.05 and P=0.01 levels.

Subgroup Category

# of Significant Differences

# of Elements within Sign. Diff.

Lowest Mean Rating within Sign. Diff.

Years Experience 26 20 3.82Geographic Location 6 5 3.71Work Setting 53 24 3.59

These findings indicate that although the differences may be significant, the lowest average rating of 3.59 is within the “Moderate Importance.”

While these data help clarify the relative importance of items to different subgroups, overall ranked importance of items is still high enough to warrant inclusion in examination design.

47

Job Analysis Survey

• As part of an initiative to pursue and establish a certification credential for professional medical writers, AMWA conducted a survey to assess the importance of knowledge, skills, and abilities (KSAs) for those who practice in the field.

• The results of this survey will be used to help develop a content outline and relative weighting for an examination. These results also further characterize the type of knowledge and skills used by medical writers across the spectrum of the profession. More detailed results will be published in the December 2012 issue of the AMWA Journal.

• The examination will test minimal competency (not aptitude or skill level) and may accompany the submission of written materials as part of the application process or other criteria such as time spent in field. 2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?

Summary (1)

48

Job Analysis Survey

• AMWA’s Certification Commission is working according to a timeline, with the aim to have the first examination developed by early 2014. The commission will maintain responsibility over certification as a semi-independent governing body.

• As opportunities to contribute expand over time, volunteers are encouraged to become involved and offer any subject matter expertise (whether in areas of medical writing or in exam development).

• AMWA’s leadership in this area helps further define the medical writing profession and establishes both a roadmap for those entering the profession and a benchmark for baseline competency expectations, regardless of area of specialization.


Summary (2)

49

Job Analysis Survey

Thanks to the following individuals who reviewed this presentation for content, clarity, and editorial accuracy:

• Lori Alexander, MTPW, ELS

• David Clemow, PhD

• Barbara Gastel, MD, MPH

• Doug Haneline, PhD

• Sue Hudson

• Marianne Mallia, ELS

• Barbara Snyder, MA

• Nancy Taylor, PhD, ELS2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?

Thanks & Acknowledgment

50

Job Analysis Survey


Q&A

Questions?

Additional PanelistsBarbara Gastel, MD, MPH

Texas A&M University

Marianne Mallia, ELSScientific Publications, Texas Heart Institute

Medical Writing Certification: Where Are We Now? Thomas Gegeny, MS, ELS (Chair) Karen Potvin Klein,...

Documents

Transcript of Medical Writing Certification: Where Are We Now? Thomas Gegeny, MS, ELS (Chair) Karen Potvin Klein,...