CESA 2017 Survey Results - American Psychological Association · CESA 2017 Survey Results APA’s...
Transcript of CESA 2017 Survey Results - American Psychological Association · CESA 2017 Survey Results APA’s...
CESA 2017 Survey Results APA’s Office of Continuing Education Sponsor Approval
Purpose
• To evaluate needs of Continuing Education (CE) sponsors
• To improve effectiveness and performance of Office of CE Sponsor Approval
• To measure level of satisfaction with new online application system (CESA OAS)
• To enhance available resources for CE Sponsors
Overview
I. Sponsor Demographics
II. Program Offerings
III. Application Process
IV. Annual Reporting Process
V. CESA OAS
A. Application
B. Annual Report
C. General Features
VI. Resources: Marketing & Communication
VII. Reflections: Strengths & Anticipations
A. Strengths
B. Anticipations and Next Steps
Sponsor Demographics
Response Rate
• 166 Total Respondents1
– 126 complete
– 40 partial2
– 0 disqualified3
Complete
76%
Partial
24%
1Total response rate was 20.47% and represents 166 out of 811 who received the survey for completion.2Partial=submission page not reached3Disqualified=response eliminated by survey logic or quota
Approval Status of Current Sponsors
Newly Approved
15%
Renewing Approval
85%
Approval Status
23%
18%
29%
10%
20%
Duration of Approval Status
0 to 3 years 4 to 8 years 9 to 14 years
15 to 19 years 20+ years
n=157 n=154
Lapses in Sponsorship
33%
22%
11%
6%
28%
Unsuccessful
application
Missed
application
deadline
Insufficient
resources
Did not want
to re-apply
Other
Reasons for Lapse in Sponsorship
Yes
13%
No
82%
Unsure
5%
n=134 n=18
CE Program Offerings
Current CE Program Offerings of Sponsors
Currently offer CE
95%
Do not currently
offer CE
5%
63%
25%
13%
Newly approved
Lack of
interest/audience
Insufficient resources
Why Not?
n=151 n=8
Types of Sponsor Applications
64%6%
24%
7%
CESA only (live CE offerings)
Homestudy only (any CE program where there is no live
interaction with presenter)
CESA and Homestudy Supplemental (simultaneous
applications)
CESA and Homestudy Supplemental (Homestudy
Supplemental added after CESA approval)
96%
11% 7% 6% 9%
Main Reasons for Type(s) of Application(s)
n=151 n=71
Homestudy Offerings of CE Sponsors
57% 57%
39%
32%
11%7%
Online course Pre-recorded
event (e.g.,
workshop,
webinar)
Video on-
demand (i.e.,
digital video
accompanied by
PowerPoint)
Publication (e.g.,
book, magazine,
journal, article)
Audio-only
recording (e.g.,
CD, podcast)
Other
n=110
Target Audiences of CE Programs
99%
83%
74%
60%
48%
46%
30%
21%
18%
15%
Psychologists
Social workers and/or human services workers
Mental health counselors
Marriage and family therapists
Graduate or doctoral students
Psychiatrists
Physicians, nurses, and other medical personnel
Educators
Undergraduate students
Othern=143
1Examples of “other” include: occupational therapists, speech pathologists, attorneys, first responders, court reporters, and juvenile justice workers
1
Application Process
Types of Applications Previously Completed by Sponsors
54%
14%
13%
11%
5% 4%
1%
CESA OAS
(2016 online
system)
Paper Paper + CESAS
(2011 system) +
CESA OAS
(2016 system)
Paper + CESA
OAS (2016
system)
Paper + CESAS
(2011 system)
CESAS (2011
online system)
CESAS (2011
system) + CESA
OAS (2016
system) n=141
Time Spent on OAS Applications
65%13%
23%
Time Spent on OAS Application vs Other
Formats
Less time
More time
The same amount
of time
n=31
15%
25%
14%
8%
38%
Time Spent on Most Recent (OAS)
Application
0-5 hours
6-10 hours
11-15 hours
16-19 hours
20+ hours
n=138
Satisfaction with Application Process via OAS
2%
n=109
9%
18%
53%
17%
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
Desired Additional Resources in Application Process
n=138
48%
41%38%
34%31%
10%
20%
Sample
promotional
materials
Flowchart or
timeline of
approval
process
Online
tutorials
Educational
and
Technical
Assistance
Glossary of
CE terms
None Other
Annual Reporting Process
Annual Report (AR) Experience
Yes
76%
No
21%
Don't know 4%
Sponsors Who Completed ARs
Less time
51%
More
time 12%
The same
amount
of time
19%
N/A 18%
Time Spent on AR Submission via
OAS vs Previous Submissions
n=139n=104
Satisfaction with Annual Report Process via OAS
2%
n=104
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
10%
23%
48%
17%
Efficiency of Annual Reporting (AR) Process
2%
7%
20%
36%
19%
16%
76%
4%
1%
19%
Is the current AR format effective?
Keep as is
Include
portfolio/other
materialsInclude site
visits
Other
n=129
n=97
1Examples of “other” include: requiring fewer submissions, collecting samples of promotional materials, requiring submission of new activities only, and improving uploading process
1
Is the AR an effective quality assessment?
CESA OAS
Introduction to CESA OAS
This section reflects CE sponsors’ experiences using the online application system (OAS). The chart below represents the processes that sponsors have conducted via the OAS.
Completed an
application for
sponsorship
approval 27%
Completed
an annual
report 28%
Both
(application
and annual
report) 46%
n=131
CESA OAS: Application
This section highlights features of the OAS specifically related to the application for sponsor approval.
Ease of Application Submission
n=93
3%
15%
25%
40%
17%
Very difficult Difficult Neutral Easy Very easy
Helpfulness of OAS Features Related to Application
3% 4%7% 6%
13% 14%
26%
21%
42%39%
24%
39%
25%
21%
10%
19%
2% 2%4% 3%
14%
19%
29%
12%
Hyperlinked
appendices for
relevant standards
(located in top left-
hand corner under
standard)
Hyperlinked
Educational and
Technical Assistance
for each standard
(located in top left-
hand corner under
standard)
"Saving vs Validating"
video
"CESA OAS
Application Tips and
Information" section
(located in "Info" tab)
1 (Not at all helpful)
2 (Not helpful)
3 (Neutral)
4 (Helpful)
5 (Very helpful)
N/A
Did not know about
n=91
n=90
n=90
n=90
CESA OAS Application Pre-Checklist
Yes
49.60%
No
50.40%
Use of App Pre-Checklist
Neutral
13%
Helpful
63%
Very
helpful
24%
Helpfulness of App Pre-Checklist
n=127 n=63
1Of those who used the checklist, none ranked it below neutral (on 5-point scale from “not at all helpful” to “very helpful”)
CESA OAS: Annual Report
This section highlights features of the OAS specifically related to the annual reporting process.
Ease of Annual Report Submission
n=95
1%
6%
23%
51%
19%
Very difficult Difficult Neutral Easy Very easy
Helpfulness of OAS Features Related to Annual Reports
n=91
n=92 n=90n=91
1% 1% 2% 3%3% 2%4%
7%
25%
21%23%
18%
43%
47%
34%
44%
20%
15%18%
23%
2% 2% 3%1%
5%
11%
15%
3%
Hyperlinked "CESA
Instructions for Annual
Reporting" document
(located in top left-hand
corner)
"Report Quick Info"
section
Option to view legacy
attachments
Option to manage
attachments
1 (Not at all helfpul)
2 (Not helfpul)
3 (Neutral)
4 (Helpful)
5 (Very helpful)
N/A
Did not know about
CESA OAS: General Features
This section highlights general features of the OAS.
Ease of Logging Into OAS
0%
8% 7%
46%
40%
Very difficult Difficult Neutral Easy Very Easy
n=131
Ease of Navigation through OAS Tabs
1% 1% 1%2%
7%5%
3%5% 4%
12%
17% 16%
19%17% 16%
42%39%
34%32%
33%
42%43%
33%31%
39%
43%
36%
1%
4%
13%
7%
2%1%
Profile Applications Annual reports Certificates Payments Overall
navigability
1 (Very difficult) 2 (Difficult) 3 (Neutral) 4 (Easy) 5 (Very Easy) N/A
n=129 n=128 n=126 n=127
n=128
n=128
Helpfulness of General Features
1% 1% 1% 1% 1% 1%
6%2%
2%3% 2% 2%
25%21%
18%
27%
19%
12%
43% 42%
37% 37%
47%
40%
12%
19%
26%
15%
25%
39%
1% 1% 2% 2% 0% 1%
13% 14% 14% 15%
6% 6%
CESA OAS
updates
"Quick Tips"
sections
"Notice" sections Action Center "General
Information and
Guidelines" page
Ability to review
applications and
reports as read-
only documents
after submission
1 (Not at all helpful) 2 (Not helpful) 3 (Neutral) 4 (Helpful) 5 (Very Helpful) N/A Did not know about
n=126n=126
n=127 n=126
n=125
n=125
Submission of Payments
1%
2%
25%
73%
Wire Transfer
Electronic funds
transfer (EFT)
Check
Credit card
Preferred Method of Submission
36%39%
19%
4%2%
Very easy Easy Neutral Difficult Very
difficult
Ease of Submission
n=130 n=131
Resources: Marketing & Communication
Resources: Marketing
Yes
49%
No
51%
Use of APA Approval Icon
17%
35%
48%
Website Promotional
materials
Both (website and
promotional
materials)
Placement of Icon
n=123n=60
Resources: Communication
48%
26%
23%
3%
Go to the website
Call
Other
Sponsors’ Preferred Methods
n=122
5%
41%
9%
39%
6%
Weekly Monthly Several
times a
month
Annually Less than
annually
Frequency of Website Visits
n=122
Mechanisms to Ensure Adherence to Deadlines
76%
43%
29%
15%
12%
0.8%
2%
CESA OAS email reminder
Personal calendar reminder
CESA website
Contact Office of CESA
CESA OAS Action Center
None
Other
n=123 (respondents chose all that applied)
Topics that Bring Sponsors to CESA Website
71%
59%56%
44%41%
35%
3%
Application
deadline
dates
FAQ sections Resources for
CE sponsor
approval
Updates CESA contact
information
CESA OAS
information
Other
n=117 (respondents chose all that applied)
Use of CESA Resources
88%
77%
75%
66%
48%
37%
31%
Policies and Procedures or Standards and Criteria
Annual report instructions
Sample documents
OAS login link
Estimated timeline for application cycles
Reference guides
List of current APA-approved Sponsors
n=65 (respondents chose all that applied)
Most Frequently Asked Questions by Topic Area
62% 59% 56% 53%43%
29%
18%12%
8%3%
n=120 (respondents chose all that applied)
Reflections: Strengths & Anticipations
StrengthsThis section discusses several strengths of the CESA Office and OAS that were highlighted by the survey findings.
Strengths
• 70.6% of respondents were satisfied or very satisfied with the application process via OAS (n=109)1, compared with 16% (n=56) and 25% (n=28) of respondents who were equally as satisfied with the previous paper and CESAS processes, respectively.
• 65.4% of respondents were satisfied or very satisfied with the AR process via OAS (n=104)2
Sponsor levels of satisfaction with
application and AR formats
• 64.5% of respondents spent less time on the application via OAS compared to other methods (n=31)3
• 51% of respondents spent less time on the annual report via OAS compared to other methods (n=104)4
Decreased time spent on applications and annual
reports via OAS
• 57% of respondents rated application submission as easy or very easy (n=93)5
• 69.5% of respondents rated AR submission as easy or very easy (n=95)6
Ease of application and annual report submission
via OAS
1See Slide 16 2See Slide 21 3See Slide 15 4See Slide 20 5See Slide 26 6See Slide 30
Helpfulness of OAS Features
• The OAS offers a collection of general features to assist sponsors in the application and AR processes.
• These features include: OAS updates, “Quick Tips” sections, a “Notice” section, the Action Center, the “General Information and Guidelines” page, and the ability to review applications and reports as read-only document.
• All features were rated as helpful or very helpful by the majority of respondents1.
General Features of OAS
• The OAS offers a collection of features specifically related to the application process designed to aid sponsors as they complete their applications.
• These features include: hyperlinked appendices for relevant standards, hyperlinked Educational and Technical Assistance for each standard, the “Saving vs Validating” video, and the “CESA OAS Application Tips and Information” section.
• Most features were rated as helpful or very helpful by the majority of respondents2.
OAS Features Related to
Application
• The OAS offers a collection of features specifically related to the AR process designed to aid sponsors as they complete their ARs.
• These features include: hyperlinked “CESA Instructions for Annual Reporting,” the “Report Quick Info” section, the option to view legacy attachments, and the option to manage attachments.
• All features were rated as helpful or very helpful by the majority of respondents3.
OAS Features Related to Annual Reports
1See Slide 35, responses of 4 or 5 on 5-pt. scale of helpfulness counted2See Slide 27, responses of 4 or 5 on 5-pt. scale of helpfulness counted
3See Slide 31, responses of 4 or 5 on 5-pt. scale of helpfulness counted
Resources for CE SponsorsSeveral resources preferred
or requested by sponsors are currently available on the
CESA website
These resources include:
Flowchart/timeline of approval process
Requested by 41.3% of respondents1
Educational and Technical Assistance
Requested by 34.1% of respondents1
Glossary of CE terms
Requested by 31.2% of respondents1
Other requested resources are currently in development (See Slides
49-52).
1See Slide 18
Anticipations and Next Steps
This section discusses anticipated future actions of the CESA Office to better support and provide guidance to CE Sponsors.
Key Areas for Development: Awareness of OAS Features and CESA Resources Application Pre-Checklist1
• 50.4% of respondents have not used the application pre-checklist.
• Out of 49.6% of respondents who have used the checklist, 87.3% rated it as helpful or very helpful
• The remaining 12.7% rated it as neutral; no respondents rated it as unhelpful or very unhelpful.
Educational and Technical Assistance (ETA)2
• 18.89% of respondents were unaware of the ETA offered with each standard (hyperlinked under standard title and included in Standards and Criteria for approval of Sponsors of Continuing Education for Psychologists).
• Out of 78.89% of respondents who rated the helpfulness of ETA, 60% rated it as helpful or very helpful.
Option to view legacy attachments on AR3
• 15.22% of respondents were unaware of the ability to view legacy attachments on ARs.
• Out of 81.52% of respondents who rated the helpfulness of this option, 52.17% rated it as helpful or very helpful.
OAS Action Center4
• 15.1% of respondents were unaware of OAS Action Center (found on profile page of OAS).
• Out of 83.3% of respondents who rated the helpfulness of the Action Center, 52.4% rated it as helpful or very helpful.
1See Slide 28 2See Slide 27 3See Slide 31 4See Slide 35
Key Areas for Development: Visibility, Accessibility, and Redundancy
Increase visibility and accessibility of CESA resources
• Sign-posting sponsors to improve visibility and accessibility of all existing resources
Build in redundancies for adhering to deadlines
• Sponsors cite OAS email reminder as the most-utilized mechanism for adhering to CESA deadlines (used by 76% of respondents)1.
• CESA Office strongly recommends that all sponsors create additional reminders independent of CESA Office and OAS, such as personal calendar reminders (currently used by 43.1% of respondents)1.
1See Slide 40
Next StepsOngoing
review of and updates to
online application
process based on sponsor feedback:
First application updates will go live on November 1st
Ongoing review of
resources for CE sponsors:
Inventory and development of general resources as well as
those focusing on the application and annual
reporting processes
Development of online tutorial video series is underway
Thank you!