Qualtrics Internship Reporting University of Minnesota Crookston

Steve Hannah

Center for Teaching, Learning, and Technology

The Problem

• At the end of an internship the employer is requested to

complete an evaluation form and the student completes the

same evaluation as a self-evaluation

• Previously - the form was a Word document the student

printed and gave to the employer

• There was no easy method of aggregating the data across

all students (paper only)

• The possibility existed for a student to complete the

employer evaluation and submit it

Terms

• Internship coordinator – University faculty

assigned to oversee internships

• Internship supervisor – employer who

supervises an intern

Advanced Qualtrics Features Used

• Display Logic

• Boolean Logic

• Email Triggers

• Piped Text

Internship Reporting Process with Qualtrics

• Process involves three Qualtrics forms

– Student self-evaluation (linked in Moodle)

– Request for Employer intern evaluation

– Employer intern evaluation

• Student submits end of Internship Self Evaluation

• Student completes Request for Employer Evaluation

• Employer receives email requesting evaluation of intern

• Employer completes Employer Intern Evaluation and

submits

How the Request for Employer Evaluation Works

• Request for Employer Evaluation form is linked in Moodle Internship Site

• Upon completion of the internship, the student submits the Request for

Employer Evaluation, the form uses email triggers to send an email to the

Internship supervisor and University coordinator

– The email includes the following information piped into the email message

• The employer’s email address is piped into the To: of the email

• The student’s name is piped into the subject line, i.e. “Employer Internship

Evaluation Request for John Doe”

• The employer’s first name is piped into the email message, ie “Dear Susan”

• The student’s name is piped into the email message, ie “John Doe, a student

intern from the University of Minnesota Crookston has identified you as their

internship supervisor…

Student completes

Request for Employer

Evaluation form

Email Trigger

configuration with

Piped Text and

Boolean Logic

Employer Notification Email

Dear Samuel,

Steve Hannah, a student at the University of Minnesota Crookston, has identified you as the student's

internship supervisor. At the completion of the internship we ask employers to complete a short evaluation

of the student. This evaluation is considered as part of the student's grade.

Your response is appreciated. You may complete the evaluation at

https://umn.qualtrics.com/SE/?SID=SV_xxxxxxxx

Thank you,

Phil Baird

pbaird@umn.edu

Internship Coordinator, Natural Resources

University of Minnesota

Crookston, MN 56716

Piped Text

Employer Evaluation Email Sent to Coordinator

(not real names) upon Submission of Evaluation

Shane Brown, Internship Supervisor for Chad Larson, has submitted an evaluation for the student's internship.

Recipient Data:

Time Finished: 2015-08-14 22:03:42

IP: 68.115.99.240

ResponseID: R_7VEEdjxNyjhCGd7

Link to View Results: Click Here

URL to View Results: http://umncrookston.ut1.qualtrics.com/CP/Report.php?SID=SV_9zd29TtUVVXXXXXXXX

Response Summary:

Date of evaluation (mm/dd/yyyy)

08/14/2015

Name of intern being evaluated

Chad Larson

Employer company or business name

MN Sports Institute / MN Vikings

Position held by intern

Marketing and Sports Management Internship

Piped Text

How the Request for Employer Evaluation Works

• There is an optional check box “My employer requires a paper

evaluation form” for certain employers who cannot access the Internet

from work, ie, Walmart.

• Checking the box invokes Boolean Display logic in the form, IF the

box is checked, THEN display additional required fields for the

employer mailing address.

• Checking the box also invokes a different email trigger, sending a

notice of the request to the employer (if an email address is included),

the internship coordinator, and the Department administrative assistant,

who mails the paper version of the evaluation to the internship

supervisor with an addressed envelope.

Positive Benefits

• Greatly reduced paperwork, record filing for internship

coordinators

• Central repository of internship evaluations

• Aggregated data

– Analysis of evaluations across all interns

– Analysis and comparisons of student self-evaluations to employer

evaluations

• Since 2011, the system has collected over 700 employer

internship evaluations (Natural Resources, Agriculture, and

Business)

3

3.2

3.4

3.6

3.8

4

4.2

4.4

4.6

4.8

5

Axi

s Ti

tle

Intern Attributes (Scale 1-Poor to 5-Exceptional)

Employer Evaluations n=247, Student Self-Evaluations n=367 (2012-2014)

Employer Student

3.00

3.20

3.40

3.60

3.80

4.00

4.20

4.40

4.60

4.80

5.00

Intern Self Evaluations - Mid Internship and End of Internship n=347 mid-internship end-internship

Questions?

Please feel free to contact me

Steve Hannah

Center for Teaching, Learning, and Technology

University of Minnesota Crookston

218.281.8382

shannah@umn.edu

Qualtrics in Student Life

What was your problem?

• Four separate hiring processes

– Women Mean Business

– GopherBusiness

– Carlson Crew

– Student Ambassadors

4 in 1 Application

Interview Selection

Using Quotas

Using Quotas

Using Quotas

Using Quotas

Setting Display Logic

Display Logic

Email Trigger

What did you learn/changes/advice do you have for

others about using Qualtrics?

• Streamlined Approach

• Personalization

• Using Display Logic more

– Transfer students

– Returning leaders

Caitlin Dow & Angela Murray Carlson School of Management

Undergraduate Program

cmdow@umn.edu

murr0132@umn.edu

Questions??

Qualtrics + APLUS = Career Action Plan

Eric Perrino, perr0273@umn.edu Office of Application Development, CLA Undergraduate Education Maggie Kubak, kubak001@umn.edu Assistant Director, Career and Internship Services (CCE, CDES, CFANS)

Welcome

Introductions IT?

Academic Advisors or Career Counselors?

Faculty or Instructors?

Others?

Come one, come all, to your Sophomore Career Appointment!

No one came.

So we created

(and required) the

Career Action Plan.

Why Qualtrics?

• Industry standard for surveys

• Easy to customize

• Plays well with others (APLUS, Shibboleth)

The Career Action Plan

• Qualtrics survey

1. One question about preparedness on each of four topics (career understanding, networking, resume, job experience)

2. Menu of action items based on students’ competency ratings on 4 topics

3. Student select five items to create their individualized Career Action Plan

• Plan is accessible to advisors in APLUS

Student Experience

1. Sept ‘14: Email including explanation and link

2. Two weeks later: reminder sent

3. Student takes survey

• receives automatic email of her Career Action Plan

• CAP is imported into APLUS for advisors/counselors

4. Jan. ‘15: email inquiry about Plan progress

5. March ‘15: Two-question survey about progress

What is APLUS?

• Advising and early alerts system used on campus • Notifies advisers of potential issues with their

assigned students • Captures contacts between advisers and

students

APLUS Integration

• Qualtrics APIs to capture the student CAPs

• Alert the advisers of students who haven’t taken the CAP.

Qualtrics Shibboleth Integration

https://wiki.umn.edu/ShibAuth/AttributeList

Qualtrics JavaScript API

•Problem getting ranked Action Plan Items from the REST API.

•Solution: Make Qualtrics do the work and store it in the ‘ActionPlan’ variable.

Qualtrics REST API

• Wrote script that periodically checks Qualtrics for new responses.

• Pull responses from Qualtrics using ‘getLegacyResponseData’ API call.

• Convert returned ‘InternetID’ into a Student ID. • Store returned ‘ActionPlan’ as a contact in APLUS

and associate with the Student ID.

getLegacyResponseData

• User: user#umn

• Token: (API token)

• Format: JSON

• Version: 2.5

• SurveyID: (SV_XXXXXX)

JSON Result

Thank you! Questions?

Eric Perrino, perr0273@umn.edu

Office of Application Development, CLA Undergraduate Education

Maggie Kubak, kubak001@umn.edu

Assistant Director, Career and Internship Services (CCE, CDES, CFANS)

Noah Holm, noahholm@umn.edu

CFANS-OIT, Academic Technologist