09-10 - Southeastern Oklahoma State University

SOUTHEASTERN OKLAHOMA STATE UNIVERSITY DEPARTMENT OF COMMUNICATION & THEATRE

OUTCOMES ASSESSMENT REPORT

COMMUNICATION PROGRAM Academic Year 2009-2010

Authors of Report Mr. Dell McLain, Department Chair

Dr. Lacinda Brese Mr. Ray Gaskin

Dr. Faye Mangrum Dr. Shannon McCraw

Report Submitted September 7,2010

1

TABLE OF CONTENTS

MISSION STATEMENT 3

PROGRAM OPTIONS 3

PROGRAM GOALS 3

PROGRAM OBJECTIVES AND OUTCOMES 3

SUMMARY OF DESCRIPTIVE, QUANTITATIVE AND QUALITATIVE DATA 4

TYPE OF ASSESSMENT USED 4-20

COMMUNICATION INTERNSHIP COMM 4940---ASSESSMENT 1 4-5

COMMUNICATION THEORY COMM 3113---ASSESSMENT 2 5-7

SURVEY OF COMMUNICATION COMM 1013---ASSESSMENT 2 7-9

INTERPERSONAL COMMUNICATION COMM 1223---ASSESSMENT 3 9-12

BUSINESS AND PROFESSIONAL SPEAKING COMM 2213-ASSESSMENT 3 12-15

RESEARCH IN COMMUNICATION COMM 4950---ASSESSMENT 4 15-16

EXTERNAL COMPETITIONS---ASSESSMENT 5 16-19

IETV / WEB-BASED 19

COMPARATIVE GROUP DATA 19

FACULTYINVOLVMENT 19

APPENDICES 20-26

2

DEGREE IN COMMUNICATION Bachelor of Arts

MISSION STATEMENT The mission of the communication program is to challenge students to develop and enhance oral and written communication skills, research competence, and media production skills. The communication program equips students with theoretical, critical, and practical knowledge to prepare students for entry into graduate school, professional school, and the workforce.

PROGRAM OPTIONS Media Studies

Advertising/Public Relations Emphasis Broadcasting Emphasis Journalism Emphasis

Organizational and Strategic Communication

PROGRAM GOALS MEDIA STUDIES: The Media Studies Option of the Communication Program includes three emphases. The advertising/public relations emphasis uses an integrated approach, which reflects the most modern perspectives of the industry. It is designed to prepare students by using mass media, problem solving techniques and creative endeavors for career preparation in advertising/public relations firms, institutions, business, government and other organizations. The broadcasting emphasis prepares students for careers in the electronic media industry. It is designed to provide students with a theoretical foundation along with practical experiences in the fields of electronic audio and video editing and production. The journalism emphasis prepares students prepares students for careers in all aspects of journalism. It is designed to give students practical experiences combined with a theoretical foundation.

ORGANIZATIONAL and STRATEGIC COMMUNICATION: This option blends theory, quantitative and qualitative research techniques and experience-based instruction to facilitate active learning vital to the contemporary organization. It is designed to prepare students to develop critical communication skills, learn how communication functions in organizational structures and develop rhetorical strategies characteristic of various organizational units. This option prepares students for professional careers and administrative positions in business, educational, health, legal and political organizations units.

PROGRAM OB.IECTIVES AND LEARNING OUTCOMES All Communication majors must successfully complete Business and Professional Speaking, Survey of Communication, Communication Theory, and Research in Communication. Each student will:

1) Recognize the dominant communication, rhetorical, and mass communication theories; 2) Be able to identify, describe, and employ communication methodologies; 3) Be able to exhibit communication competence through effective oral and written

presentations; 4) Be able to apply critical and creative thinking to problem solving; 5) Acquire a basic understanding of mediated communication through the use of new

communication technologies; 6) Gain an awareness of, understanding of, and respect of cultural diversity in human

interaction; 7) Validate and synthesize classroom learning by having the opportunity to gain practical

experience by participating in an internship or gain communication research experience by conducting a capstone research project. Specifically, students will be encouraged to participate in scholarly conferences and external competitions.

3

SUMMARY OF DESCRIPTIVE, QUANTITATIVE AND QUALITATIVE DATA USED TO ASSESS THE PROGRAM

TOTAL NUMBER OF STUDENTS ASSESSED BY ALL ASSESSMENTS: 772

TYPES OF ASSESSMENT USED THAT MEETS PROGRAM OBJECTIVES AND OUTCOMES AY2009-2010:

Assessment 1: Internship Supervisor Evaluation, Internship Weekly Reports, Internship Mid-Term Reports, Internship Final Reports, and Internship On-Site Visits [Objective(s) Measured: 4, 5, 6, 7/ Mid-Level and EXit].

Assessment 2: PretesUposttest of communication theories course required of core communication required for majors and minors (Communication Theory and Survey of Communication). [Objective(s) Measured: 1,2,3,4, 5/ Mid-level].

Assessment 3: PretesUposttest of introductory communication definitions, concepts, and theories in required entry-level courses (Interpersonal Communication and Business and Professional Speaking). [Objective(s) Measured: 1, 2, 3, 4, 5, 6/ Entry].

Assessment 4: Performance in research course [Objective(s) Measured: 1, 2, 3, 4, 5, 6 Entry/Mid-Level].

Assessment 5: External competitions [Objective(s) Measured: 3, 5, 6, 7/ Mid-Level].

COMMUNICATION INTERNSHIP COMM 4943

Program Objectives and Outcomes Measured by the Communication Internship

4) Be able to apply critical and creative thinking to problem solving; 5) Acquire a basic understanding of mediated communication through the use of new




Assessment 1 Internship Supervisor Evaluation, Internship Weekly Reports. Internship Mid-Term Reports. Internship Final Reports. and Internship On-Site Visits [Objectives Measured: 4, 5, 6, and 7

Assessment 1 Number of Students Assessed with Internship: 19 students

Assessment 1 Summary of the Descriptive, Quantitative and Qualitative Data and Sample The internship program continues to be one of the main assessments for the department. It is our primary assessment tool for gaining feedback from our constituents and stakeholders who provide feedback to the student and the instructor. While the internship program is an elective for organizational and strategic communication majors, it is required for communication majors with the media studies option. The sample of students for this assessment included all students

4

enrolled in the summer 2010 Internship Class. Every student enrolled was a communication major and a senior.

Work for this assessment included the internship supervisor evaluation, internship weekly reports, internship mid-term reports, internship final reports, and internship on-site visits. Relevant constituents and stakeholders were utilized by providing evaluations and reports to the professor leading the internship class. The assessment data were then shared with the constituents and stakeholders. Student internships provide opportunities for students to gain experience in their field and help them determine if they have an interest in a particular career and create a network of contacts. The feedback from our constituents help our students judge their progress and potential in a particular career.

The criteria for evaluating interns are included in the Internship Packet that is given to the student and respective supervisor/employer (See Internship Packet in Appendix A, page 20). Nineteen students successfully completed their internships, and 16 of those received a grade of "A" from their employers. Ninety-nine percent of supervisor's evaluation forms indicate that the employers would hire the interns in a regular position if they had a position open (Appendix A, Internship Packet). Ninety-nine percent of those employers said they would like to have another intern from our program in the future. This percentage represents a 1% decrease from the AY 2008-09. As in the past, several interns have been or have promise of being employed by the companies where they did their internship, depending graduation.

Assessment 1 - Strength's) and Weakness(e)s of the Groups as Indicated by the Data Based on the above data, the strength of our internship program indicates that our students are consistently rated highly by their supervisors. In fact, 99% of supervisor's evaluation forms rated the students as "superior." This represents a 1% decrease from AY 2008-09. When asked if the supervisor would hire each respective intern if a position was open, 99% of supervisors stated that they would hire the intern(s). All communication with the employers/constituent was extremely favorable. The major weakness of this program has been that some interns did not file reports on time, despite promise of grade reduction.

Assessment 1 - Effectiveness of Modification Recommended in 2008-09 Assessment Report An electronic format was created during the AY 2009-2010. Both students and supervisors were pleased to receive the packet in electronic format, and indicated the ease of use associated with electronic materials. Despite the promise of grade reduction for tardy reports, and the ability to fax or email reports, some students still failed to promptly submit their performance reports. Ninety percent of the students submitted all reports on time, as set forth in the internship requirements. This represents an increase from AY 2008-09.

Assessment 1 - Recommended Modification for AY 2010-2011 Recommendations for the summer 2011 are having a meeting before the summer semester begins and creating agreement pages in the packet. A well-publicized meeting with all the students, prior to the start of internships, will serve as a reminder of the importance of prompt performance reports. The addition of an agreement page to the packet, stating that the student will abide by all deadlines, will remind students that prompt performance is indeed a top priority.

Communication Theory and Survey of Communication

Program Objectives and Outcomes Measured by Communication Theory and Survey of Communication Courses


presentations; 4) Be able to apply critical and creative thinking to problem solVing;

5

5) Acquire a basic understanding of mediated communication through the use of new communication technologies;

6) Gain an awareness of, understanding of, and respect of cultural diversity in human interaction;

Assessment 2: Pretest tPosttest of core courses required of all communication majors and minors(Communication Theory and Survey of Communication [Objectives Measured: 1, 2, 3, 4, 5, 6t Mid-level (Shown Above)].

COMMUNICATION THEORYCOMM 3113

Assessment 2 Number of Students Assessed: 57 students

Assessment 2 Summary of the Descriptive. Quantitative and Qualitative Data and Sample The Pretest/Posttest for the communication theory course fulfill the requirements for the communication program goals and objectives 1, 4, 5, and 6 (shown above). The pretest/posttest for the communication theories course are required of all communication majors and minors. They designed to measure the student's overall knowledge of communication theories at the beginning of the course and whether there was a measurable increase in knowledge at the conclusion of the course. The examination consists of questions covering interpersonal communication, small group, organizational communication, rhetorical, mass communication, intercultural and gender theories. There are 25 objective questions on the test. The sample of students for this assessment included all students enrolled in the spring 2010 Communication Class. Students enrolled in this course were both communication majors and minors.

Assessment 2 Strength(s) and Weakness(e)s of the Groups as Indicated by the Data Analysis of Pretest/Posttest Thirty-four students in a face-to-face communication theory were assessed during the spring 2010 semester with the pretest/posttest. Another 23 students in an on-line section of the communication theory class during the summer 2010 semester were assessed, but this data is not available. Results from the past six years are shown in Table 1, indicating a continuous increase on the posttest scores. Table 2 reveals the mean score on the posttest increased 28 points during the AY2009-2010.

Table 1: H' t . I D t f C . fIS orlca a a or ommunlca Ion Theorv

Pretest Average

36.93

Posttest Increase Number of students

2009-2010 75.64 38.71% 34 2008-2009 38.8 68.9% 30.1% 45 2007-2008 35.34 74.355 39.015% 65 06-07 60.74 71.91 11.17% 35 students 05-06 Unavailable Unavailable Unavailable Unavailable 2004-2005 42.05% 60.1% 18.05% Unavailable 2003-2004 56.71 % 68.27% 11.56% Unavailable

Table 2:

Test Results for Communication Theory ... . .. .... ... . ." ,-,.

Score

Pretest (Sp 2010) 36.93

Posttest (Sp 2010) 75.64

Standard Deviation

12.47

18.81

High Score Low Score

68 16

96 0

6

Assessment 2 - Effectiveness of Modification Recommended in 2008-09 Assessment Report A recommendation was made in the 2008-2009 report that all departmental faculty review the assessment instrument for the purpose of updating the material in keeping with any changes in course content, resources, technology, textbooks, and/or method of delivery. The review of the pretest/posttest used in the communication theory reveals that test questions are fair and reflective of every context of communication. Six questions covered interpersonal communication and the students averaged 77.5% on them. Eight questions covered group, organizational, and public communication and the students averaged 81.9% on these questions. Six questions covered mass communication theory and students averaged 74.467% on these questions. See table 3. The review also reviewed that one of the mass communication questions may have the wrong answer. With the elimination of this question, the average is 83.72% on the mass communication questions. Four questions covered intercultural and gender communication and the students averaged 64.06% on them. One question covered computer-mediated communication and the students averaged 75% it. This modification to review the test questions was effective in that it revealed which context of communication needs more attention in the communication theory class. For example, students scored the lowest on intercultural communication questions. This may indicate more attention should be given to this topic. Also, only one question about computer-mediated communication is included on the quiz. Since this is becoming an important area in the study of communication, the quiz may need to be revised to include more on this topic.

Table 3:

Review of Pretest / Posttest for Communication Theory . ' ... ..... .. .. ....." " "

Context Average Answered Correctly

Interpersonal Communication 77.5%

Group, Organizational, and Public Communication

81.9%

74.467%Mass Communication

Intercultural Communication 64.058%

Computer-mediated Communication

75%

# of Questions on test

6

8

6

4

1

I

Assessment 2 - Recommended Modification for AY2010-2011

Results of pretest/posttest data should be collected each semester instead waiting until the semester when the assessment report should be written. This should present a loss of data. Perhaps a central location in the departmental office could be used to collect and organize the data from the pretest/posttest and distribute the information to the faculty for evaluation after each semester. One recommendation is that the faculty create a rubric for as a checklist for collecting the data. (See rubric in the Appendix 8, page 25).

SURVEY OF COMMUNICATION (COMM 1013)

Assessment 2 -- Number of Students Assessed: 53 students

Assessment 2 -- Summary of the Descriptive. Quantitative and Qualitative Data and Sample The pre-test/post-test in the survey of communication course is designed to measure the student's overall knowledge of the communication discipline at the beginning of the course and whether there was a measurable increase in knOWledge gained by the conclusion of the course. The exam consists of 50 questions in the areas of interpersonal, organizational, small group, rhetorical, and mass communication. The score for the test is calculated by multiplying each correct answer by 2. For example, the mean score of 63.58 is based on a possible 100 points. The sample of students for this assessment included all students enrolled in the fall 2009, spring

7

2010 and summer 2010 Survey of Communication classes. Students enrolled in these course were both majors and minors.

One section of the survey of communication course offered during the fall 2009 semester was a face-to-face course and two sections taught during the spring and summer 2010 semesters were online course. For the fall 2009 face-to-face section, a comparison of the pre-test and post-test scores indicates that student's knowledge levels did increase during this semester. The pretest average was 65.4% whereas the class average for the posttest was 74.6%. This was an increase of 9.2 points. For the summer 2010 internet section, a comparison of the pretest and posttest scores indicates that student's knowledge levels also increased during the summer semester. The pre-test average was 61.75% whereas the post-test average was 72.87%. This was an increase of 11.12 points. Data for the spring 2010 semester is not available. Data from the face-to-face and internet classes combined reveal a 10.18 point increase in the posttest scores. The difference between the face-to-face and internet classes is a 1.91 point increase on the posttest administered in the internet class. Data reveals there is no significant difference based on the instructional delivery modes. See tables 4 and 5 for summaries of the comparisons.

Table 4 Test Results for Survey of Communication Classes

Pretest Mean Score

Post test Mean Score

Increase High Score on Pretest

High Score on Posttest

# of Students

Face-to-Face Classes and Internet Classes Combined

AY 2009-2010

63.58 73.758 10.178 88 92 53

Face-to-Face Classes Fall 2009

65.43 74.64 9.21 88 92 29

Internet Classes Spring 2010

N/A N/A N/A N/A N/A N/A

Internet Classes Summer 2010

61.75 72.87 11.12 86 90 24

Table 5 Companson 0 e Ivery M d es for Survey 0fCf InstructlonaID I' o om mUnlcatlon I Face-to-face Scores Internet Scores

Pretest Mean Score 65.428 61.75 Posttest Mean Scores 74.64 72.87 Difference in Scores +9.21 +11.12

Assessment 2 -- Strengths and Weaknesses of the Groups as Indicated by the Data The pretest and posttest indicate the students improved slightly over the course of the semester in the introductory level survey of communication class. A weakness is that there has been decline in test scores for the past two years. See table 6 for this comparison.

Table 6: Istonca ata or urvey 0 ommUnicatlonH' . I D f S fC

Pretest Average Posttest Increase Number of students

63.582009-2010 73.758 10.18 53 2008-2009 71.1 86.61 15.51 31 2007-2008 54.97 79.65 16.07 40

8

Assessment 2 -- Effectiveness of Modification Recommended in 08/09 Assessment Report Consideration to drop students from the class or place their final grade on hold if they did not complete both the pretest and posttest was recommended by the faculty during the last academic year. This recommendation was not enacted. The concern for accurate data remains to be a problem for the department. Another approach should be investigated by the faculty.

Assessment 2 -- Recommended Modification for AY201 0-2011 Because the historical data reveals a decline in test scores, effort should be made to increase students' scores on the posttest.

INTERPERSONAL COMMUNICATION AND BUSINESS AND PROFESSIONAL SPEAKING

Program Objectives and Outcomes Measured by Interpersonal Communication and Business and Professional Speaking courses.

1) Recognize the dominant communication, rhetorical, and mass communication theories; 3) Be able to exhibit communication competence through effective oral and written

presentations; 6) Gain an awareness of, understanding of, and respect of cultural diversity in human

interaction;

Assessment 3 Pretest/posttest of introductory communication definitions, concepts, and theories in interpersonal communication and business and professional speaking courses required by communication majors. minors and are communication options for general education courses for the university [Objectives Measured: 1, 3.61 Midlevel (Shown Above)].

INTERPERSONAL COMMUNICATION

Assessment 3 - Number of Students Assessed: 266 students

Assessment 3 - Summary of the Descriptive Data, Quantitative and Qualitative Data and Sample The pretest/posttest instrument for the interpersonal communication course is comprised of 25 multiple-choice questions. The pretest/posttest assessed information regarding verbal messages, nonverbal behaviors, perception, self-concepts, critical listening, relationship development, conflict management, and interpersonal theories. The sample of students for this assessment included all students enrolled in the fall 2009, spring 2010, and summer 2010 interpersonal communication classes. Because this class is fulfills one of two options for general education requirements, students were from any major. For both face-to-face and internet sections of interpersonal communication, the data show students' mean posttest scores were 16.28 points higher than the mean pretest scores. The highest score on the posttest was 60 points of the possible 100, whereas the highest score on the pretest was 44 points of the possible 100 (see table 7).

9

Table 7 Test Results for Interpersonal Communication Classes .....

Face-to-Face Classes and Internet Classes

AY 2009-2010

Pretest Mean Score 34.47

Post test Mean Score 50.75

Increase

16.28

High Score on Pretest

44


96

# of Students

266

I

Face-to-Face Classes AY 2009-2010

28.98 50.66 21.68 44 60 98

Internet Classes AY 2009-2010

39.948 I

50.83 10.88 96 168

I

For students enrolled in face-to-face sections, results indicate that the mean score on the posttest was 21.68 points higher than mean score of the pretest. For students enrolled in internet sections, results indicate that the mean score on the posttest was 10.88 points higher than mean score of the pretest. While the mean posttest scores for both the face-to-face and internet classes were the same with only a .17 point difference, the face-to-face sections a greater difference between posttest and pretest scores (see table 8).

Table 8 Comparison of Instructional Delivery Modes for Interpersonal Communication

Pretest Mean Score


Increase

I Face-to-Face Classes 28.98 50.66 21.68

Internet Classes 39.95 50.83 10.88

Difference 22.11 .17 10.8

Fa112009. For the face-to-face sections assessed during the fall 2009 semester, the data show, students' mean posttest scores were 5.80 points higher than the mean pretest scores. Results also indicated that the mean score on the posttest for students enrolled in internet sections during fall 2009 was 13.45 points higher than mean score of the pretest (see table 9). The mean score on the posttest for students enrolled in face-to-face interpersonal communication classes was 5.80 points higher than mean score of the pretest.

Table 9 Test Results for fall 2009 Interpersonal Communication Classes. '.' ..... . .. ' .. ......

I Pretest

Mean Score Post test Mean

Score Increase # of Students

Face-to-Face Classes 28.25 33.33 5.80 33

Internet Classes 39.12 52.57 13.45 68

Difference 10.87 19.24 7.65

10

I

Spring 2010. For the face-to-face sections assessed during the spring 2010 semester, the data show students' mean posttest scores were 38.28 points higher than the mean pretest scores. Results also indicate the mean score on the posttest for students enrolled in internet sections during spring 2010 was 5.4 points higher than mean score of the pretest (see table 10). The faceto-face classes scored 32.88 points higher on the posttest than the students enrolled in internet sections classes.

Table 10 Test Results for spring 2010 Interpersonal Communication Classes ... ...

Increase # of Students Pretest Mean Post test Mean Score Score

Face-to-Face Classes 29.72 68 38.28 10

Internet Classes 31.23 36.63 5.4 64

Difference 31.37 32.881.51

One face-to-face section of the interpersonal communication class containing 55 students was not averaged into the spring 2010 assessment scores due to problems with recording data for the posttest. Although the scores were not included in this report, the number of students were included in the report because the pretest scores were collected (see table 11).

Table 11 Test Results for spring 2010 Face-to-face class

Pretest Post test Increase High High Score #of

I Mean Score

Mean Score

Score on Pretest

on Posttest Students

I I

Face-to-face Class from Spring 2010

I

7.02

I I

N/A N/A N/A

I I

N/A

I

55

Assessment 3-Strengths and Weaknesses of the Groups as Indicated by the Data

A strength identified with the data from the interpersonal courses is that the students' mean posttest scores are higher than the mean pretest scores. There was a 16.28 increase in scores for the AY 2009-2010. The data shows that while the increase in test scores improved, the majority of the students are not passing the post test. The low test scores are the probable result of changing textbooks in the internet classes, but continuing to use the same pretest and posttest ·designed for the textbook used in the face-to-face classes. See table 12.

Table 12 . rHt' IDtf Itn erpersona ICIS orlca aa or ommunlca Ion

2009-2010

Pretest Average Posttest Increase Number of students

34.47 50.75 16.28 266 2008-2009 36.68 54.32 17.64 123 2007-2008 42.82 67.80 24.97 224

From the analysis of students' answers on each question of the pretest/posttest from the interpersonal communication class, it was discovered that the percentage of students answering the questions correctly dropped on the posttest. Several students from the internet classes

11

criticized the posttest stating it was not relevant to their textbook. One student stated "I did horrible on the post test! I couldn't connect a lot of the questions with what we read in the text. I feel like I learned a lot from this book and I'm disappointed in my score and myself. I guess I just wanted to tell you that ..." During the summer 2010 semester, the instructor modified the posttest and students average score improved to 75.78. See table 13.

Table for 13 Test Results for summer 2010 Internet class

I

Pretest Mean Score


Increase High Score on Pretest


#of Students

Summer 2010 section W1 I 59.04 75.78 16.74 92 96 36

Assessment 3 - Effectiveness of Modification Recommended in 2008-09 Assessment Report

Recommendations from the previous report about methods of collecting data was not successful. Also, archiVing the data was problematic. A successful recommendation involved an analysis of each question. This analysis led to the discovery of the majority of students submitting incorrect answers on several questions, which indicates a need to revise the test. A third recommendation from the previous report involves the differences in the scores from the face-to-face and internet classes and how these courses collect data. Faculty continue to use different methods to collect data. A fourth recommendation that was implemented involved the requirement to evaluate the posttest as part of the students' grade in order for the students to apply themselves more seriously. Given the rise in the mean scores in the posttest from the previous two years, this recommendation seems to have had a positive effect. Another modifications for data collection based on the 2008-2009 data is the requirement for every student to complete both the pretest and posttest instrument. This modification was not effectively implemented during the 2009-2010 year and the department's concern continues to persist. Another recommendation concerning analyses of pretests and posttests is that the individual instructors should complete sub-analysis of the individual content areas of the assessment instrument. This recommendation was implemented.

Assessment 3 - Recommended Modifications for the AY201 0-2011

Because inputting individual scores into spread sheets is time-consuming task for faculty. It is recommended to seek help spread sheets so that faculty can spend more time analyzing data. Also, the department should develop a system for archiving the data. The recommendation to analyze each question should continue to identify patterns. Also, the department should continue dialogue about methods to collect data. Another modifications for data collection based on the 2008-2009 data is the requirement for every student to complete both the pretest and posttest instrument. Another recommendation concerning analyses of pretests and posttests is that the instructors should discuss the data collected from this academic year.

BUSINESS AND PROFESSINAL SPEAKING (COMM 2213)

Program Objectives and Outcomes Measured [Entry-Level assessment]: 1) Recognize the dominant communication, rhetorical, and mass communication theories; 3) Be able to exhibit communication competence through effective oral and written

presentations; 4) Be able to apply critical and creative thinking to problem solVing; 5) AcqUire a basic understanding of mediated communication through the use of new

communication technologies;

12

1

6) Gain an awareness of, understanding of, and respect of cultural diversity in human interaction;

Assessment 3 PretesUPosttest of introductory communication definitions. concepts, and theories in the Business and Professional Speaking Course [Objective(s) Measured: 1, 3, 4, 5, 6/entry-level].


Assessment 3 - Summary of the Descriptive Data, Quantitative and Qualitative Data:

The PretesUPosttest instrument is comprised of 50 multiple-choice questions. It assessed information regarding topic selection, topic focus, thesis development, and outline and argument development. The sample for AY 2009-2010 consisted of 337 students, which included 151 students from the fall 2009 semester and 186 during spring 2010. Because this class fulfills one of two options for general education requirements, students were from any major. Data collected from the business and professional speaking course show the students' mean posttest scores of 67.55 for all sections during each of the fall 2009 and spring 2010 semesters are higher than the mean pretest scores of 56.88. The mean score for the posttest increased 10.67 points. See table 14.

Table 14 Test Results for Business and Professional Speaking Classes AY 2009-2010

Pretest Average Increase in # of Students Average Posttest

in Class Fall 2009

Mean Scores 56.01 67 13 11.12 151

1 .Sections Spring 2010 55.75 67.96 12.21 186 Sections Averages for 56.88 10.67 337 AY 2009

67.55

1--::2:..:.0....:...10~ J__---L- --...L ----J

A second set of samples used for this year's assessment includes the comparison between two honor sections of the business and professional speaking class taught during the fall 2009 semester (honors and Presidential Leadership Class-PLC), and the other sections required as one of the general education options in the communication category. The honors class consisted of 15 students and the PLC class consisted of 26 students. The data collected from theses sections show the students' mean posttest scores of 66.05 for these sections are higher than the mean pretest scores of 54.44. The mean score for the posttest increased 11.61 points which is 0.18 percent higher than the other general education sections. See table 15.

Table 15 Results for Honors Business & Professional Speaking and Presidential Leadership Class (PLC) fall 2009

I Pretest Average Posttest

AveraQe Increase in Mean Scores

# of Students

Honors 48.38 70 21.6 15 PLC 60.5 62.1 1.64 26 AveraQes 54.44 66.05 11.61 41

Table 16 shows the mean pretest and posttest scores for the general education sections taught during the fall 2009 semester. The mean posttest score of 68.22 for the four sections is higher than the mean pretest score of 57.58. This is an increase of 10.64.

13

I

Table 16 Results for fall 2009 General Education COMM 2213 Sections Fall 2009 Sections

Pretest Average Posttest AveraQe

Increase in Mean Scores

# of Students in Class

Section 1 57.24 66.12 8.88 26 Section 2 56.86 70.46 13.6 30 Section 3 56.88 65.72 8.84 27 Section 4 59.34 70.58 11.24 27 Mean Scores 57.58 68.22 10.64 110

Table 17 shows the mean pretest and posttest scores for the general education sections taught during the spring 2010 semester. The mean posttest score of 67.96 for the eight sections is higher than the mean pretest score of 55.75. This is an increase of 12.21.

Table 17 R It forspnnQ 2010 G eneraI Ed r COMM 2213 S eClonsresu s ucalon

Pretest Average Spring 2010 Sections Section 1 Section 2 Section 3 Section 4 Section 5 Section 6 Section 7 Section 8 Mean Scores

58.26 51.68 53.3 60.7 50.76 59.12 54.82 57.38 55.75

Posttest Average


# of Students in Class

74 15.74 16 67.04 15.36 27 64 10.7 27 71.36 10.66 28 68.12 17.36 26 64.74 5.62 25 69.38 14.56 17 65 7.62 20 67.96 12.21 186

Assessment 3 - Strengths and Weaknesses of the Groups as Indicated by the Data From assessment of the business and professional speaking courses during the 2009-2010 academic year, it was found that students' mean score on the posttest was higher than the pretest. A weakness for this year is that the number of students participating in the assessment decreased from 373 students to 337. During the fall 2009 semester, assessment data from the night classes were not submitted for analysis. An improvement during this academic year is that results were consistent in that every section had an increase in scores. A weakness for this year is that the posttest mean scores were lower than the scores from the previous two years. See table 19.

Table 18 Historical Data for Business & Professional Speaking

Pretest Mean Scores

Posttest Mean Scores

Increase in Scores Number of Students

2009-2010 56.88 67.55 10.67 337 2008-2009 53.68 65.8 12.1 373 2007-2208 57.44 71.9 14.46 154 2006-2007 52.64 63.48 10.84 229

Assessment 3 - Effectiveness of Modifications Recommended in 2008-2009 Assessment Report A concern of the department about collecting quantitative data continues to persist in that not every student enrolled is completing both the pretest and posttest instrument. For example, the department was not effective in collecting and assessing the data to compare individual student's pretest with his or her posttest scores. The department was only able to calculate mean scores for sections and then compare sections of the business and professional speaking classes.

14

Another recommendation from 2008-2009 report is that the honors general education courses should be compared to regular general education classes and this was accomplished. Table 18 shows a comparison between the general education sections taught during the fall 2009 and spring 2010 semesters. The mean scores for the honor classes increased 11.61 points whereas the general education sections of the Business and Professional Speaking class increased 11 .43 points. The honor sections increase was 0.18 percent higher than the other sections. See table 18.

Table 19 Comparison for Honors, Presidential Leadership Class, General Education Sections of COMM 2213 AY 2009-2010

Pretest Average Posttest Average


# of Students

Averages Honor and PLC classes Fall 2009

54.44 66.05 11.61 41

Averages for Gen Ed B& P

I AY2009, 2010

56.66 68.09

I

11.43 296

Difference

I 0.18

Assessment 3 - Recommended Modifications for AY 2010-2011 The department's primary concern is that public speaking is an activity class involving subjective components, not just something that can be tested by a multiple choice instrument. It is recommended that the course not be assessed based on a test that is perhaps insufficient to really gauge the improvement of our students. Because there is little difference between the sections and not a significant increase in the scores, the department should investigate the validity of the test. Another concern of the department is that if the pretest and posttest are used again as the primary assessment tool, each faculty member should be responsible for collecting and analyzing the data and that a systematic method for archiving data should be developed.

RESEARCH IN COMMUNICATION (COMM 4950)

Program Objectives and Outcomes Measured by Research in Communication course

1) Recognize the dominant communication, rhetorical, and mass communication theories; 4) Be able to apply critical and creative thinking to problem solVing; 5) Acquire a basic understanding of mediated communication through the use of new


interaction;

Assessment 4: Performance in Research [Objective(s) Measured: 1, 4, 5. 6 [Mid-Level and Exit Levell.


Assessment 4 - Summary of the Descriptive. Quantitative and Qualitative Data and Sample The research in communication course fulfills the requirements for the communication program goals and objectives. It is performance-based by requiring students to conduct original research

15

projects. Research projects performed in this course are conducted by students majoring in Organizational/Strategic Communication and Media Studies. The sample of students for this assessment included all students enrolled in the fall 2009. These students were both majors and minors in communication.

Assessment 4 - Summary of the Descriptive and Qualitative Data: The data gathered for research performance projects measured student performance on research, scholarship, organization, content, creativity and electronic, oral, and/or written presentation(s) at state, regional and/or national levels. Performance results for AY 2009-2010 are as follows:

State Competition in Research Twenty students participated in the Research Day Oklahoma Consortium of Regional Universities at the University of Central Oklahoma in November, 2009. Each of the students individually and/or in groups created and participated in computerized poster session from their respective projects in research. These students' projects were chosen based on the significance of the research, design and execution of study, and the design relative to the topic. Competition included all regional universities in Oklahoma. In the liberal arts category and communication discipline, 20 of the 36 students from Oklahoma regional universities were students from Southeastern's research in communication class. They presented 14 of the 20 posters in this category One student from the research class presented his research in the Theater & Dance Discipline and was 1 of 3 students statewide participating in this category..

Assessment 4 - Strength(s) and Weakness(e)s of the Groups as Indicated by the Data Analysis of Pretest/Posttest

This data demonstrates students are making significant progress toward understanding the language associated with conducting qualitative and quantitative research.

Assessment 4 - Effectiveness of Modification Recommended in 2008-2009 Assessment Report

Recommendation to follow external competition guidelines was effectively implemented.

Assessment 4 - Recommended Modification A recommendation is that all departmental faculty review textbooks for this course.

External Competitions

Program Objectives and Outcomes Measured by External Competitions


presentations; 4) Be able to apply critical and creative thinking to problem solving; 5) Acquire a basic understanding of mediated communication through the use of new




16

Assessment 5: External Competitions [Objectives Measured: 1,2,3,4,5,6 and 7/Exit Level (Shown Above)].

2009-2010 Forensics Team Season


Forum In e ores,t K'JngwoodC II dR U' 't. th F o ege an Ice n1versltY Clinton Harders Parliamentary Debate W-L: 2-4 Clinton Harders Impromptu Prelim Krister Maxey Parliamentary Debate W-L: 2-4 Krister Maxey Dramatic Interpretation Prelim Krister Maxey Impromptu Prelim

M. Elizabeth Dillow Impromptu Program of Oral Interpretation M. Elizabeth Dillow

Clinton Harders Parliamentary Debate Impromptu Parliamentary Debate Impromptu

Clinton Harders Krister Maxey Krister Maxev

S J . t C IITexas ntercoIIe~ late Forenslcs League, an aCln 0 o ege N0 rth Prelim Prelim W-L: 1-4 Prelim W-L: 1-4 Prelim

CI . C ·tCh'rlStmas aSSIC, ameron U'nlversllY M. Elizabeth Dillow Parliamentary Debate W-L:3-3; 2"0 JV Speaker M. Elizabeth Dillow Impromptu 61n Place M. Elizabeth Dillow Program of Oral Interpretation Prelim Clinton D. Harders Parliamentary Debate W-L: 3-3 Clinton D. Harders Impromptu Prelim

M. Elizabeth Dillow M. Elizabeth Dillow M. Elizabeth Dillow Lindsav Grav Clinton Harders Clinton Harders

Oklahoma ae Tournamen,t U . 't ahoma and Okl h C't U . 'tSt t nlverSlty 0 f Okl a oma ltV nlVerSllY 2"0 Place 61n Place 5

Impromptu Poetrv Program of Oral Interpretation Poetry Communication Analvsis Impromptu

U1 Place Prelim 51n Place Prelim

appa Deta om prehensive ournamentPi K I National C T M. Elizabeth Dillow Impromptu Excellence Award M. Elizabeth Dillow Poetry Prelim M. Elizabeth Dillow Program of Oral Interpretation Prelim Clinton D. Harders Communication Analysis Prelim Clinton D. Harders Impromptu Excellence Award

MEDIA STUDIES: JOURNALISM

Assessment 5 - Number of Students Assessed: 20

Assessment 5 - Summary of the Descriptive, Quantitative and Qualitative Data

Collegiate and professional journalism awards earned by the student newspaper, The Southeastern, in the two-semester span are noted. This is an excellent program

17

assessment tool because most journalism courses are tied directly to The Southeastern newspaper. A significant portion of students' grades are based on contributions to student publications in the following courses:

• Comm 2113, Writing for Mass Comm I (stories and copy editing) • Comm 2223, Writing for Mass Comm II (stories and copy editing) • Comm 2243, Publication Production and Design (page and web page design) • Comm 3223, Editing for Print Media (copy editing, headlines and page design) • Comm 3283, Photojournalism (photos, photo pages, illustrations, projects)

Peer review of the Journalism Program comes in the form of awards from the Oklahoma Collegiate Press Association and the Society of Professional Journalists. In fall 2009 The Southeastern won two awards in the prestigious Oklahoma SPJ competition, second only to one other four-year university in its category. The Southeastern publications staff won six awards at the spring 2009 Oklahoma Collegiate Press Association conference.

Society of Professional Journalists, Oklahoma Chapter - fall 2009

1. The Southeastern earned Third Place from the SPJ for Arts Writing. 2. The Southeastern earned Third Place from the SPJ for Spot News Reporting.

Oklahoma Collegiate Press Association - spring 2010 Awards earned by The Southeastern staff

1. Column Writing, Honorable Mention 2. Overall Newspaper, Honorable Mention 3. Front Page Design, Honorable Mention 4. Cartoons, Honorable Mention 5. Interior Page Design, Honorable Mention 6. Column Writing, Honorable Mention

It should be noted that that the publications staff elected to not participate in the OCPA Yearbook competition for 2009 due to a re-vamping of the yearbook format and a change in design, which will make it a better product in 2010-2011.

Assessment 5- Effectiveness of Modifications Recommended in 08-09 Assessment Report

Very effective, as students enrolled in Writing for Mass Comm I (COMM 2213) and Writing for Mass Comm " (COMM 2223) classes increased their participation in providing content for the weekly student newspaper. There was considerable group discussion involving assignments, including exchange of ideas and critiquing of assignments. Conversion from Quark to InDesign CS4 software for newspaper publication, which was undertaken the previous year, continued to produce improved quality of page layout and design, as well as providing skills which will be helpful to graduates as they seek careers in media. Photojournalism (COMM 3283) students continued to contribute visual content to the student newspaper. Addition of Adobe InDesign CS4 software to all computers in the Communication lab/classroom enhanced the Production and Design (COMM 2243) class, as well as Photojournalism (COMM 3283). The 2010 digital yearbook got a new look with an in-depth pictorial history over the school year, focusing on candid photos with a de-emphasis on staged photos. The number of students proficient in using the Apple version of InDesign software more than tripled from the previous year.

Assessment 5- Recommended Modifications for AY 2010-2011

The student publications staff, adviser and department chair will work together to formulate plans to increase the student newspaper's digital presentation. The staff is moving toward a change in the newspaper's format, from broadsheet to tab size, giving it a more contemporary look. At the same time, plans are being formulated to launch an online version of the student newspaper,

18

which can be updated daily. The digital newspaper will also interface with such relevant social networking sites as Facebook and Twitter. These changes will help prepare students for the challenges they face in a fast-changing media world.

COMPARATIVE GROUP DATA vs. THE NATIONAL NORMED DATA (WHEN APPLICABLE):

Although no nationally standardized exam exists for communication, our program participates in a number of state, regional, and national competitions.

DEPARTMENT ASSESSMENT OF IETV AND/OR WEB-BASED INSTRUCTION:

Courses are assessed from data gathered by faculty. IETV data does not exist at this time because no IETV communication courses are taught. See tables 1-17 for comparisons of internet and face-to-face classes. In addition to the department collecting data on student learning from the internet sections of communication classes, evaluations of classes are collected by Distance learning Council. This information is made available to faculty.

FACULTY INVOLVEMENT IN ASSESSMENT PROCESS

Assessment is a departmental wide function. Faculty have input in the process, modifications, and in contribute to the department's program outcome assessment report. Every faCUlty member submitted a report that included data requested for the assessment process. They submitted information pertaining to the areas they teach that fit within the communication assessment plan as well as submit an analysis of the data from their particular option or emphasis to a faculty member responsible for writing a particular section of the report. This person corresponds to the faculty about the analysis. Faculty make suggestion to each about recommendations before the information is entered into the report. Dr. Brese, Mr Gaskin, Dr. Mangrum, and Dr. McCraw collected and analyzed data about their specific courses. Departmental secretary collected pretesUposttest information from adjunct faculty. The information was given to Dr. Mangrum who structured the report into a cohesive document that represents the communication program as a unified voice. The report was then submitted Mr. Mclain who revised the document. Additionally, all faculty members participated in the development of the communication assessment plan and received a copy of it for their files. Specifically, regarding this outcomes assessment report, faculty members receive a copy of this document for their files as well.

Communication faculty include the following:

1. Dr. lacinda Brese, Organizational and Strategic Communication; Director of Forensics

2. Mr. Ray Gaskin - Media Studies, Journalism; Adviser for The Southeastern student newspaper and The Savage Storm yearbook

3. Dr. Faye Mangrum - Organizational and Strategic Communication

4. Dr. Shannon McCraw- Organizational and Strategic Communication

5. Mr. Dell Mclain -Department Chair

C?u~~J'f~6ell Mcl , ITA.

Department Chair Dean, School of Arts and Sciences Art, Communication, and Theatre

19

Appendix A: Internship Documents Refer to page 5 in the Report

I. SOSU Internship Agreement Form

This form is to be filled out with copies sent to the student, instructor, and the supervisor.

I. Student Information Name _ Address _ Phone Number where you can be reached _

II. Supervisor Information

Name--------------------Address _ Business Number _

I". Instructor Information

Name Dr. Lacinda Brese Address Southeastern Oklahoma State University

Department of Art. Communication & Theatre Durant. OK 74701

Contact Information Phone: 580.745.2556 Fax: 580.745.7475 E-mail: [email protected]

Please give the job title of the internship.

List the general duties and activities that are part of the internship: Credits that are earned _ Hours to be completed _ Time of internship: (Date) to (Date) _ Please sign this agreement if the above conditions are satisfactory.

1. Student's Signature Date

2. Supervisor's Signature Date

20

3. Instructor's Signature Date

SOSU Internship Confirmation

This form is to be filled out and turned in as soon as arrangements for your internship are completed.

Student's name: _

Student's address: _

Students phone number: _

Company name: _

Company address: _

Company phone number: _

Supervisor's name: _

Date internship began: _

Approximate work schedule: _

Additional comments:

(1)

(2) Weekly SOSU Internship Report

Week of _

21

Student' name _

Supervisor's name _

Company _

Hours Worked:

Sunday Monday Tuesday Wednesday _

Thursday Friday Saturday _

Please list any comments and/or observations that you have regarding the work performance of the student.

4. Student's Signature Date

Supervisor's Signature Date

SOSU Internship Midterm (60 Hours) Report

Student's Name _

Student's Signature _

Supervisor's Name _

Supervisor's Signature, _

Company _

22

Summarize your thoughts regarding your internship to date. Include duties you have performed, facts and procedures you have learned, and observations you have made.

a)

b)

c)

d) SOSU Intern's Ending Report

1. Were you provided with learning opportunities and was an effort made to train you? Explain.

2. Has the experience of the internship left you feeling satisfied? Explain.

3. Do you think this company should be used regularly as a part of this internship program? Explain.

4. Give a summary of your thoughts and observations that you have made during this internship.

Supervisor's Evaluation Form

Student's Name'---- _

Company Evaluator _

Evaluator's Signature _

. d' t e ow . t your company, PIease In Ica e on the sca e I b I how your In ern pe rformed WI'th

CHARACTERISTICS SUPERIOR

4

GOOD

3

AVERAGE

2

BELOW AVERAGE

1

DO NOT KNOW

NA

Resourcefulness

Maturity

Interest in Job

Ability to Learn I

23

Ability to Communicate

Ability to Organize

Ability to Work with Others

Ability to Work Independently

Ability to Work Under Deadline Pressure

Ability to Contribute to The Company

I I

Understanding of Company Procedures

I

Acceptance and Constructive I Use of Criticism

Promise of Success in The Profession

1. What the most significant strengths of your intern?

2. What improvements would you suggest for your intern?

3. If your company had an opening for a person with the background of this intern, would you hire him/her?

4. What letter grade do you feel that the intern should receive?

A__ B__ C__ 0__ F__

5. Would you be interested in having another internship in the future?

6. Do you have any suggestions that could improve this internship program?

24

Appendix B: Rubrics for Recording Data for AY201 0-2011 Refer to page 87 in the Report

Rubric for Core Courses Needed for 2010-2011 Assessment Report

Receive d Yes/NO

Name and Section of Class

Semester Faculty Pretest Average

Posttest Average

#of Students Passing

#of Students Failing

COMM Fall 2010

1013.1 1

COMM Spring

1013.1 2011

COMM Summer 1

1013.1 2011

COMM Spring

3113.1 2011

COMM Spring 1

Theory 2011

3113.W1

Research Summer

1

in 2011

COMM

I

1

I

4953 1

I

25

Rubric for Entry-level/General Education Data for 2010-2011 Assessment Report

Receive d Yes/NO

Name and Section of Class

Semester Faculty Pretest Averag e

Posttes t Averag e

#of Students Passing

# of Students Failing

1233.1 Fall 2010

1233.W1 Fall 2010

I

1233.W2 Fall 2010

2213.1 Fall 2010

2213.2 Fall 2010

2213.3 Fall 2010

2213.4 Fall 2010

2213.5 Fall 2010

2213.6 Fall 2010

2213.7 Fall 2010

2213.8 Fall 2010

2213.9 Fall 2010

2213.10 Fall 2010

2213.11 Fall 2010

2213.33 Fall 2010

2213.44 Fall 2010

2213.45 Fall 2010

26

09-10 - Southeastern Oklahoma State University

Documents

Transcript of 09-10 - Southeastern Oklahoma State University