College of Public Health University of South Florida

Syllabus Page 1

Department of Community and Family Health

Syllabus Course Name: Seminar: Foundations of Evaluation and Research Prefix & Number: PHC 4759 HSC 4942 Semester: Fall, 2015

Course Description:

This course shows students the basic concepts essential to planning a research study or evaluation project.

Credit hours: 3

Pre-Requisites: None

Co-Requisites: None

Location: Online

Instructor Information:

Instructors

Dr. Kay M. Perrin

Office: COPH-1039

Office hours: By appointment only.

Phone: 813-974-6704

E-mail: kperrin@health.usf.edu

Preferred method of contact: The best and fastest way to reach me is by e-mail at kperrin@health.usf.edu; please do not contact me through Canvas.

Reply Policy: Please do not assume your e-mail or voicemail message has been received until I reply. Allow up to 24 hours for replies Sun-Thurs; 36-48 hours on Fri-Sat.

Required Materials:

Perrin, K.M. (2015) Principles of Evaluation and Research for Health Care Programs. Jones and Bartlett Learning ISBN: 978-1-284-03896-5 NOTE: Several of the assignments required in this class come straight from this text. Though you may be able to complete assignments in other classes without purchasing the course text book that is not the case with this course. You CANNOT effectively complete these assignments or the majority of the coursework in this class without this text. There is one copy of the text book on reserve at the USF Library.

Recommended Materials:

The following are supplemental (and not required reading materials): 1. CDC Framework for Program Evaluation in Public Health (1999):

ftp://ftp.cdc.gov/pub/Publications/mmwr/rr/rr4811.pdf 2. Frechtling, J. (2002). The 2002 User-friendly handbook for project

evaluation. National Science Foundation: http://www.nsf.gov/pubs/2002/nsf02057/start.htm

3. Grembowski, D. (2001). The practice of health program evaluation. Thousand Oaks, CA: Sage.

College of Public Health University of South Florida

Syllabus Page 2

4. Joint Committee on Standards for Educational Evaluation. (1994). The program evaluation standards: How to assess evaluations of educational programs (2nd ed.). Thousand Oaks, CA: Sage.

5. Oakes, J. M. (2002). Risks and wrongs in social science research: An evaluator’s guide to the IRB. Evaluation Review, 26(5), 443-479.

6. Patton, M. Q. (1997). Utilization-focused evaluation: The new century text (3rd ed.). Thousand Oaks, CA: Sage.

7. Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach (7th ed.). Thousand Oaks, CA: Sage Publications.

8. Scriven, M. (1991). Evaluation thesaurus (4th ed.). Newbury Park, CA: Sage Publications.

9. Shadish, W. R., Cook, T. D., & Leviton, L. C. (1991). Foundations of program evaluation: Theories of practice. Newbury Park, CA: Sage Publications.

10. W. K. Kellogg Foundation. (2004). Evaluation handbook. Battle Creek, MI: W. K. Kellogg Foundation.

Course Format: This course meets online.

College of Public Health University of South Florida

Syllabus Page 3

Learning Objectives

The course objectives are to ensure that students master the course subject matter and to develop students’ ability to use course concepts in thinking and problem solving.

Upon completion of this course, students will be able to:

1. Define program evaluation and research.

2. Describe the role of the literature review in research and evaluation planning.

3. Differentiate between program evaluation and basic research strategies.

4. Explain ethical issues involved with research and evaluation and use of human subjects in research.

5. Understand differences between quantitative and qualitative research methods.

6. Describe the most appropriate methods to use including the advantages and disadvantages in public health research design.

7. Compare and contrast quasi-experimental and experimental research designs.

8. Identify the strengths and weaknesses of different data gathering techniques.

9. Understand basic analytic strategies for data analysis and when it is necessary to consult with a statistician for additional assistance.

10. Describe a variety of techniques that may be used to communicate research results.

College of Public Health University of South Florida

Syllabus Page 4

Assessment Strategies

Note: This is a writing intensive course. As an upper-level public health course, you will be expected to turn in a written assignment almost every week. In addition to specific style and format requirements (see further assignment instructions) you will need to turn all assignments in on time. Please watch the calendar closely. Plan ahead and make time for slow computers. Do not submit an assignment five minutes before it is due, because uploading documents in Canvas may be slow and thus your assignment will be late.

LATE Assignments: Assignments are considered late at 5:01 pm on the day that they are due. Late assignments will be docked 2 points for every day (24 hour period) they are late past the 5pm due date.

Quizzes: There are three quizzes. Please watch the calendar closely, so you do not miss any quizzes.

Case Studies: The case studies are located at the end of each chapter. You are asked to respond to questions for three assigned case studies. Each case student assignment should be 300-350 words in length and fully answer all questions.

SEE-I Assignments: SEE-I Assignments allow for students to look at a research and evaluation concept and apply it to their own grant research topic.

SEE-I stands for State – Elaborate – Exemplify – Illustrate

State: How is the concept defined?

Elaborate: Why is this concept important for research?

Exemplify: Write an example of how this concept applies to your research topic.

Illustrate: Write an analogy for the concept that shows you understand it.

The SEE-I assignment should be less than 300 words. Below is an example of how an adequate SEE-I may be accomplished.

State: The concept is data cleaning. Data cleaning is defined after the data are collected and entered into a spreadsheet, the researcher examines the spreadsheet to determine if any data are missing or if any data were entered incorrectly, such as if the range is 5 (agree), 4 (moderately agree), 3 (neutral), 2 (moderately disagree) and 1 (disagree) and the spreadsheet reveals 11 as a data entry error. The 11 would be corrected to read 1.

Elaborate: Data cleaning is essential prior to analyzing the data. If the data have not been cleaned correctly, the data errors would result in incorrect data analysis results.

Exemplify: For example, in my research, if I did not change 11 to 1 during the data cleaning process, the mean score would be miscalculated and thus the results would be incorrect. This could mean that I make wrong assumptions with my results.

College of Public Health University of South Florida

Syllabus Page 5

Illustrate: Data cleaning is like removing debris before stitching a wound so healing can take place.

You may select three concepts from the following list for the SEE-I assignments:

Logic models Operational definitions

Social determinants of health Categorical data

SWOT analysis Two-tailed tests

Random errors Type I errors

Validity Paired samples t-test

Reliability Cost-Utility analysis

Grant Proposal Assignments: (Look on Canvas for specific grading rubrics) At the end of the semester you will write a ‘mini’ grant proposal to fund a research project. This will prepare you to conceptualize a small research project, design important research elements, and write up the proposal under strict grant-writing specifications. During the semester, you will write a draft of each section of the grant (these WILL be graded) and then you will synthesize and merge the sections into one 5-page document prior to the final submission. PLEASE NOTE, you should take these assignments seriously, providing adequate time and attention to them before you turn them in, as they account for over 1/3 of your total final grade: Topic Proposal: Email Dr. Perrin at kperrin@health.usf.edu for approval of your topic before starting work on your grant assignments. You are able to write your research proposal surrounding a public health topic of your choosing, provided that it meets a number of criteria. First, your topic must have a public health focus, and must be fairly narrowly defined and measurable. Your topic must relate to health among 100 college students at USF. You may look at a health issue that affects all students at USF, or one that affects a smaller population at USF (For example, women, men, adult students, students with children, faculty, staff, etc.). You must clearly define your health issue, its public health importance on a college campus, and identify your specific population. You will need to appropriately cite your references. Research Questions: You will need to come up with 2 research questions that you want to explore within your overarching public health topic. These research questions must be able to be answered through survey questions that are asked of participants in your research study. Your research questions ARE NOT the same as survey questions. See your text book as well as further instructions on Canvas regarding the difference between research questions and survey questions. Note: You must include your topic with the proposal with your research questions. Remember that you will use a behavior change theory or framework to help guide your research. Therefore, you may want to take a look at Chapter 4 to familiarize yourself with how theories can help shape your research questions. Literature Review: The literature review provides the background regarding your topic. In the literature review you must define your topic and population, provide current epidemiological information about the health issue in the target population (i.e. incidence and prevalence, health disparities, etc.). What are the personal and public health consequences of this health issue. You should also include how your research proposal will add to knowledge about this health topic and population.

College of Public Health University of South Florida

Syllabus Page 6

Theories and Models: For this assignment you will discuss why using theories and models are important to public health research and evaluation. You will then choose a theory or model that you will use to guide your research proposal. You need to discuss why this theory is the most appropriate for your research questions and research population. Which parts/constructs will you use? Which of your research questions do these constructs relate correlate too? How will your chosen theory/model help guide the creation of your survey? Survey Questions: For this assignment you will create a draft survey for your research project. You must have AT LEAST10 questions. Four of these questions should collect demographic information, and six of the questions should directly address one or more of your research questions. You will need to abide by the principles of good survey development within Chapter 8 of the text book. Methods and Data Analysis: For the methods section you will explain to us what it is exactly you will be doing to answer your research questions. You will need to discuss your research design and your specific data collection procedures. As part of your methods you will draw up a timeline of your research activities. For your data analysis plan, you will be filling out a worksheet. More information will be posted on Canvas. Budget and Budget Justification: ($30,000 with 4% indirect costs; 2 pages – no citations). A template will be given to you to fill out for your specific research project. This will require you to do a little bit of investigative work to determine what your needs for your project will be. More information will be posted on Canvas. EXCEL Data Worksheets: There is a practice dataset posted. You will have a quiz using another dataset and answering questions based on EXCEL calculations. It is highly recommended that you practice using EXCEL for data analysis prior to completion of this assignment. Refer to your textbook for specific directions using EXCEL. If you have a MAC computer, you will need to do this quiz on a PC, as the Data Analysis Plug-in can sometimes not work well with a Mac. In addition, sometimes it shows slightly different views than are discussed in the text. Final Grant: You will submit a 5-page a grant proposal: (Single-spaced, 11 point font, APA style). (Any grant that exceeds 5-pages will not be graded.) Abstract (approximately 1/2 page) Introduction, Statement of Problem and Research Questions (approximately 1/2 page) Literature Review (approximately 1 page) Theory or Model (approximately 1/2 page) Methods: (approximately 1½ pages) Data Collection, Survey, Sample size, Data analysis Budget and Budget Justification ($30,000 with 4% indirect costs) (approximately 1 page) References (not counted in the 5-page limit) No appendices are permitted.

College of Public Health University of South Florida

Syllabus Page 7

Grading Scale and Criteria:

Areas of Evaluation

Points/% Final Grade

Quizzes (10 points @ 3 quizzes) 30

A = 90-100% B = 80-89% C = 70-79% D = 60-69%

F = 59% or below

*Please note that the plus/minus

grading system will NOT be used in this

course.

Topic (15 points)

Research Questions (15 points)

Literature Review (15 points)

Theories and Models (15 points)

Survey (15 points)

Methods and Data Analysis (15 points)

Budget and Budget Justification (15 points) 105

Case Study (3 @ 10 points) 30

SEE-I Assignments (3@10 points) 30

Final Grant 65

Total points 260

Grading Policies: No plus or minus grades will be awarded. Please contact me if you are concerned about your performance prior to the end of the semester.

No extra credit is available for this course.

COURSE POLICIES

Attendance and Participation:

I expect regular participation in this course. I will acknowledge any religious holidays or observances for students that fall on class days during the semester as excused reasons for absence from class. Students should identify these observances to me in advance and arrange for securing any relevant work, assignments, or handouts from those days.

See Institutional Policies section for Emergency Preparedness for Academic Continuity.

Permission to Use Lectures:

All unauthorized distribution of course material is prohibited.

Incomplete Policy: COPH policy: http://publichealth.usf.edu/academicaffairs/academic_procedures.html

Upon completion of this course, the following checked objectives will be achieved by the students:

Identify and articulate the core functions of public health:

College of Public Health University of South Florida

Syllabus Page 8

Explains the basic principles of epidemiology

Assess social and behavioral interventions to improve health of populations X

Identify the impact of the environment and communicable diseases on health

X

Explain the role that public health plays in disaster prevention and management and evaluate public policy issues with respect to access, quality and cost when understanding health disparities with vulnerable populations

Exhibit critical thinking and analytical abilities, including the capacities to engage in inductive and deductive thinking and quantitative reason and to construct sound arguments: Identify topics pertaining to public health research X

Generate research questions, analyze and present data, and interpret and discuss findings

Demonstrate awareness about current public health topics including an analysis of the societal attitudes that generate differences on current public health topics

X

Communicate using effective skills: Demonstrate an ability to contribute effectively to group discussion and

presentations

X

Apply effective public speaking skills during classroom presentations X

Develop effective written presentations: Demonstrate the use of information literacy skills such as locating and

evaluating pertinent public health information

X

Demonstrate the ability to use library resources and scientific databases

Exhibit proper referencing secondary materials in APA format.

College of Public Health University of South Florida

Syllabus Page 9

Class Calendar

Date Topic Assignments are to be submitted on Canvas by

5:00 p.m. on Thursday.

8/27 Purpose: Chapter 1 Syllabus Quiz Topic

9/3 Ethics: Chapter 2 Chapter 2 Case Study Research questions

9/10 Determinants of Health: Chapter 3

Quiz 1: Chapters 1, 2, and 3 Chapter 3 Case study

9/17 Theories and Models: Chapter 4

Literature Review

9/24 Reliability and Validity: Chapter 5

Theory and Model Selection

10/1 Qualitative Data: Chapter 6

SEE-I (1)

10/8 Elements of Research: Chapter 7

Quiz 2: Chapters 4, 5, 6 and 7

10/15 Surveys: Chapter 8 SEE-I (2) Survey questions

10/22 Data Tools: Chapter 9 Chapter 9 Case Study

10/29 Populations and Samples: Chapter 10 Methods and Data Analysis

11/5 Inferential Statistics: Chapter 11

SEE-I (3) Quiz 3: Chapters 8, 9, 10, 11

11/12 Budgets and Cost Analyses: Chapter 12 Budget and Budget Justification

11/19 Working Week Working Week

11/26 Thanksgiving

12/3 Final Grant Final Grant

Additional Course Information

Course Evaluation: Informal feedback throughout the course is appreciated. The instructor will periodically survey students about the quality and usefulness of the assigned readings through the course Web site. There will also be an anonymous mid-course evaluation that will benefit current students, as well as a final evaluation to benefit future students.

Useful Web Resources:

College of Public Health University of South Florida

Syllabus Page 10

American Evaluation Association – http://www.eval.org/

The Evaluation Center, Western Michigan University – http://www.wmich.edu/evalctr/

Evaluation Tipsheets (Cooperative Extension and Outreach, College of Agricultural Sciences, Penn State University) – http://www.extension.psu.edu/Evaluation/titles.html

Resources for Methods in Evaluation and Social Research – http://gsociology.icaap.org/methods/

The Web Center for Social Research Methods – http://www.socialresearchmethods.net/

INSTITUTIONAL POLICIES

The most recent version of the Institutional Policies information can be found on the Academic Affairs Forms

page at http://health.usf.edu/publichealth/forms.html (Fall 2011)

Student Handbook: http://www.sa.usf.edu/dean/docs/full handbook.pdf

Student Conduct: USF Student Rights/Responsibilities: http://www.sa.usf.edu/srr/page.asp?id=81

USF Student Code of Conduct: http://www.sa.usf.edu/srr/page.asp?id=88

Disruption of Academic Process/Academic Integrity of Students:

Disruption of the academic process and violations of the policies regarding academic integrity will not be tolerated. Review USF policies on Disruption of the Academic Process and the Academic Integrity of Students at:

http://generalcounsel.usf.edu/regulations/pdfs/regulation-usf3.025.pdf

Academic Dishonesty/ Plagiarism:

Plagiarism will not be tolerated and is grounds for failure. Review USF Academic Dishonesty and Disruption of Academic Process Policy at:

Undergraduate:

http://www.ugs.usf.edu/pdf/cat1112/20112012.pdf#page=62

Graduate:

http://www.grad.usf.edu/inc/linked-files/USF_Grad_Catalog_2011-2012.pdf#page=39

The University of South Florida has an account with an automated plagiarism detection service (SafeAssign), which allows instructors and students to submit student assignments to be checked for plagiarism. I (the instructor) reserve the right to 1) request that assignments be submitted as electronic files and 2) submit students’ assignments to SafeAssign, or 3) request students to submit their assignments to SafeAssign through myUSF. Assignments are compared automatically with a database of journal articles, web articles, the internet and previously submitted papers. The instructor receives a report showing exactly how a student’s paper was plagiarized.

NOTE: An institution may not release a paper to a plagiarism detection software without the student’s prior consent unless all personally identifiable information has been removed, such as a student’s name, social security number, student number, etc.. Note that a paper/essay is considered an educational record and an institution may not ask a student to waive their rights under FERPA for the purpose of submitting papers to a plagiarism detection software.

For more information about Plagiarism and SafeAssign, visit:

Plagiarism tutorial: http://www.cte.usf.edu/plagiarism/plag.html

College of Public Health University of South Florida

Syllabus Page 11

SafeAssign: http://media.c21te.usf.edu/pdf/student/bbstud_subsafeassgn.pdf

Cheating Statement: The USF College of Public Health expects students to maintain academic honesty in all courses. By virtue of being registered in an public health course, students agree to refrain from cheating. If cheating in any form (academic dishonesty) is detected, appropriate action will be taken. (Refer to USF Academic Dishonesty Policy).

Undergraduate:

http://www.ugs.usf.edu/pdf/cat1112/20112012.pdf#page=67

Graduate:

http://www.grad.usf.edu/inc/linked-files/USF_Grad_Catalog_2011-2012.pdf#page=39

Undergraduate Academic Policies and Procedures:

http://www.ugs.usf.edu/pdf/cat1112/08acapol.pdf

Special Accommodations:

Students in need of academic accommodations for a disability may consult with the office of Services for Students with Disabilities to arrange appropriate accommodations. Students are required to give reasonable notice (typically 5 working days) prior to requesting an accommodation.

Students with Disabilities Services: http://www.sds.usf.edu/

Students: http://www.sds.usf.edu/students.asp

Faculty: http://www.sds.usf.edu/faculty.asp

Holidays and Religious Observances:

http://generalcounsel.usf.edu/policies-and-procedures/pdfs/policy-10-045.pdf

Emergency Preparedness:

In the event of an emergency, it may be necessary for USF to suspend normal operations. During this time, USF may opt to continue delivery of instruction through methods that include but are not limited to: Canvas, Elluminate, Skype, and email messaging and/or an alternate schedule. It’s the responsibility of the student to monitor the Canvas site for each class for course specific communication, and the main USF, College, and department websites, emails, and MoBull messages for important general information.

Student Grievance Procedure:

Review USF Academic Grievance Policy at:

http://generalcounsel.usf.edu/policies-and-procedures/pdfs/policy-10-002.pdf

Undergraduate:

http://www.ugs.usf.edu/pdf/cat1112/20112012.pdf#page=62

Graduate

http://www.grad.usf.edu/inc/linked-files/USF_Grad_Catalog_2011-2012.pdf#page=48

Student assistance is provided by Division of Student Affairs, Office of the Student Ombudsman.

College of Public Health University of South Florida

Syllabus Page 12

http://www.sa.usf.edu/ombudsman

RESOURCES FOR STUDENTS

Library Resources: USF Library Resources and Services: http://www.lib.usf.edu/

Shimberg Health Sciences Library: http://library.hsc.usf.edu/

Shimberg Health Sciences Library Tutorials: http://library.hsc.usf.edu/ (follow links under ‘Instructional Services’ section)

Creating Citations & Using Refworks:

http://guides.lib.usf.edu/CitingSources

Netiquette (online communication etiquette for online courses):

http://eta.health.usf.edu/publichealth/standards/syllabus/Online_Netiquette.pdf

Plagiarism & Safe Assign:

See Academic Dishonesty/Plagiarism Section

USF Email Accounts: http://health.usf.edu/publichealth/eta/pdf/MyUSF_Email.pdf

Elluminate Live Tutorials: (for online courses)

http://media.c21te.usf.edu/elluminatestudents.html