OTC Online Course Review Hints

Fall 2011

Continuous quality improvement

Goal

1. Strive for an objective tone by avoiding first person (“I”).

2. Identify strengths in review criteria and at the conclusion of the review by providing examples from course site or quotations from the instructor.

3. Differentiate between what needs to be changed and suggestions or ideas that you may recommend to help meet the guidelines.

4. Provide resource links/direct to online@otc.edu.

5. To help align scores with my comments, complete the comment box first and then match score to what is noted (0=not evident, 1=partially evident, 2=mostly evident, 3-fully evident).

Hints from Reviewer for Reviewers

6. Divide the development review into two times. For the first session, examine criteria A1-D6. Then, allow more time for the second session, where you focus on the content and application to learning objectives.

7. Group collaboration/discussion may be present in a specific groups area, an entire-class discussion that is broken into separate prompts for students to choose one prompt, Web 2.0 group opportunities, etc. The point is that it provides a structure for students to communicate beyond an entire class forum. It could even be Wimba (or Collaborate) sessions, where students participate in smaller groups.

8. Utilize course reports (Control Panel-> Evaluation->Course Reports).

9. Save file with course ID and then instructor name (e.g., SSM-101-W19-11FA George Washingon).

10.Ask questions.

Think about the why and the how

OTC Online Course Development Review

Fall 2011

There are a lot of extra buttons included, such as lectures. Could these buttons and this material be combined in the lessons area? It seems a bit confusing to have to go to all of the different places.

Buttons are not in recommended order. Other buttons are included as needed.

Formatting is not consistent throughout the course. In the announcements, syllabus and lessons area the font is different sizes. Also in the syllabus there is visible HTML. The folders are black when they should be the same blue throughout the course.

There is consistency, but no banner and there does not seem to be a great deal of visual design or multimedia. ______ is a great field to employ visual images and this could enhance student learning and attentiveness.

There is no course banner. The Announcements page could be made more visually appealing by personalizing it with a banner and changing the menu buttons—do this by accessing “Customization” then “Style.”Green font is used consistently throughout the course to visually unify elements.

Adding a course banner can help individualize the course so students don’t confuse this course with other Blackboard courses in which they may be enrolled.

The banner and other aspects present a unified visual design. Good job on changing the color of the menu buttons—it personalizes the course and is more visually appealing than the default color.

Welcome announcement provides good instructions. Dates from previous semester need to be deleted.

Helpful hint about Adobe Reader. Students are directed to Syllabus and Faculty buttons. There is no mention of what to do (or where to go) to start working on the course. The brief introduction is mentioned, but more direction is needed. A bulleted list may help as items are presented.

Photo and contact info are present and personal story adds presence. Students expect to see turnaround time here, especially as courses become more consistent, so consider adding this information here as well.

There is a short faculty bio, but no photo or turnaround information. Photo helps to personalize instructor for the students and students need to know when to expect grades and communication. This also helps to minimize questions.

Bio and photo are present, the bio part is really well written! The approximate turnaround time is not included here.

Syllabus is current template.

Syllabus is not embedded and the file attachment is a .docx, not a PDF or .rtf. Information is intuitive.There is a printable course calendar, but the alignment with course objectives is not included—it is a .docx, as well

Including a title with the lesson helps students identify objectives and where they are in the course.

Please remove the “week” references from the lessons for an easier transition between an eight week and a sixteen week course.

Units are based on course objectives—the objectives are listed on the unit folders and given for individual lessons.Folders are labeled with titles, i.e. Unit 1-Introduction to _____.

An “Introduce yourself” forum is included, however it is not linked nor information provided in any Lesson folder.

Yes. Bulleted lists are used and are very eye catching!

In Lessons, provide an internal link to discussion forums (Build Content -> Course Link).

Links open in new windows. Course links largely promote efficient navigation.Assignment links should be identified by the type of assignment, i.e. Discussion. Quizzes are identified.

Instructions are easy to read and understand. They are also complete. For example, in the first Lesson, _____ explains how to “find” the discussion board via the link and how to post a thread.

A checklist is suggested because students will know at a glance when assignments are due.

There are print lectures; it seems that these could be more interesting with the addition of perhaps audio explanations or more integration of the video components. This is a lot of text for students to read in addition to the textbook.

Excellent content--lots of valuable resources and activities in the lessons (e.g. “_______” mini-lecture audio file, reflective journaling activity, ________ video, etc.)

Everything for the chapter seems to be Instructor Resources provided by the publisher. Very text intensive and boring.

Some lessons include some lecture notes (format needs to be updated). For video links, provide an introduction and hints.

Students have some choices in discussion board questions which allows for personalization, but it seems that even more content could be more active and individualized. How about some labs that analyze something within the student’s own environment

The group project is active and provides individualization, as do a few discussion questions. There needs to be more content of this type however. How about some __________ or ___________ that are typical and some analysis? Maybe students could look for ads on youtube to discuss? Everyone sees _____________ every day so this is a great area to allow students to see that and analyze what is effective and what is not, etc.

Active and individualized learning is included. As students complete assignments, they are often getting hands-on experience in _________, etc.,

Instructional content includes role playing, article reviews, short answer, essays, and other learning experiences.

Think about incorporating one active learning or individualized learning activities to each lesson, including some of the following examples:

•Ask students to observe and write about the lesson’s _________ in their environments (from TV or from conversations)

•Ask students to write about what which _________they have the most trouble with--this could be done through a discussion forum so that students could read each other’s ideas and experiences and respond to each other.

•Ask students to implement some of the __________ in short writings or audio clips.

•Ask students to reflect on their test results from the previous (lesson) and write about what they learned, what they struggled with, or what strategy they need to work on.Ask students to find and share resources on the course material for that (lesson).

Great use of multimedia! All content is embedded. Instructor has utilized YouTube, mp3, Prezi, and pdf files.

YouTube videos are embedded. PowerPoints need to be changed to SlideShare or other more accessible content and included directly within lessons. In Lesson 4, the questionnaire could be easily loaded into a test and still named as a questionnaire to avoid a file attachment.

This course relies on PDF and PPT. Explore Learning Modules for PDFs or another format. Explore other options mentioned for PPT. ______ quiz should be in the form of a quiz rather than a file attachment.

Youtube videos are nice additions, what about some audio readings of other selected ______, perhaps even by the _____?

Try looking for websites that provide interactive exercises or activities that relate to the content

Tight connection between learning objectives and content.

In online courses, community and connections between students and instructor are essential.

Lessons A and B discussions could be combined for more meaningful opening discussions. Because of the short duration of discussion forums and the number of discussion forums, there is not the opportunity for in-depth discussion. Would increasing weight of discussion for course grade influence participation?

This course taught the students how to write blogs and there was great student to student interaction as well as student to instructor interaction. Would love to see this much interaction in all online courses!

Discussions are assigned for each lesson. Lessons discussions account for 15% of course grade (160 out of 1100 points). A higher point value may encourage more involved participation. Clear guidelines in each forum as far as replies (required?) are needed.

Students might need more direction on what constitutes a suitable source for the discussion boards.

Instructions are easy to understand. Good use of bullets and step-by-step instructions in Seminar instructions.

Questions are well-written and promote higher order thinking. This is a good example:“Also, what are the problems associated with doing ______? How do you combat these problems? What are two (2) outcomes of doing ________?”

Good prompts (e.g. “Take a stand on the issue. Do you think ________ is a useful tool or do you think it infringes on students' rights” and “While it's all right to take a stand based on your opinion, back up your opinion by showing how you analyzed information in the essay to form your opinion.”)

Current events are included in activities. Example – Assignment 4.4.

Again, specific examples, cases, _____c examples, and others could be used to keep the course current, relevant, and applicable. Including discussions of ________, such as _____can be discussed. In addition to the ______ activities, students can discuss specific examples from their viewing.

Not present. Perhaps students could be divided into two teams to conduct two separate investigations, documentation assignments, or other. Each would have a specific role or assignment. Another option is to use groups for feedback on a major course assignment project.

Strengthen correlation between group discussion and objectives by referring to them, possibly beside each lesson’s discussions.

For both times that this area was used the instructions were detailed. Providing detailed instructions is an area where this instructor really excels!

“Read the links and comment” is a bit vague. More detailed guidelines would be helpful.

The time limits for quizzes are excessively short. A general rule of thumb for multiple choice is 2 minutes per question. There were quizzes with 15 multiple choice and 4 or 5 short answers expected to be completed in 15 minutes. This would be very difficult for the best students and impossible for anyone with reading difficulties.

Assignments other than the final project are primarily “regurgitation” and definition. These are easy for students to copy and are not very authentic. How do we know that the student has learned anything? They could easily just copy answers out of the textbook without much retention.

Students are moved through the research process with many steps, including topic selection and rough drafts.

The main assignment beyond writing the words and their definitions are 30 minute tests that range from 11-15 multiple choice or word answers. There is not really a way to be certain that students aren’t Googling definitions of words or using their book to take the exam. This can be the challenge of knowledge-based or recall-based exams.

Authentic assessments might ask students to write sentences using the appropriate words in new contexts. Here is a link to more information about authentic assessment: http://jfmueller.faculty.noctrl.edu/toolbox/whatisit.htm

This course spends a lot of time looking at plagiarism and what counts as plagiarism, this is great and will carry on from this course to many other courses!

No rubrics were visible. Rubrics are important in letting students know how they will be evaluated.

This seems to be incorporated with labs, although quizzes and exams would probably be more valid with other types of questions rather than primarily multiple choice.

Could quiz review be enabled so students could learn from what they missed?

Lessons build upon one another – example lesson 3 is finding sources and lesson 4 is providing a bibliography on those sources.

Provide additional assignment types available in this course. Although the quizzes build on each other as the student must score 70% on them before they are able to move on to the next topic.

There appears to be no hidden columns pertaining to this specific class, but there are hidden columns. In addition, there are visible columns that do not contain grades—the duplicate other columns that do, i.e. there are two columns labeled “_____.” These extra columns are visible with the Edit Mode off, so students would have been able to see them, as well. If the Weighted Total column is not used, it needs to be hidden.

Some of the quizzes are hidden. For assistance, request help from Bb Support at online@otc.edu. Discussion forums linked to Grade Center allows for grading of discussion in same area discussion is scored.

OTC Online and Testing Services are provided. Suggest adding LRC, Writing Center, and TLC.

Sans serif font is used and buttons are one word.

When you enter pictures or videos be sure to provide the alt tag for disability purposes.

Providing transcripts is a “tricky” business—unless they are readily available from the source, transcribing the spoken word would take hours. (Instructors may email online@otc.edu for assistance from OTC Online via a work study student.)

Mini-lectures come with transcripts as do audio lessons.

Not present. Some considerations are to search for YouTube videos with closed captioning or to request some assistance with transcripts via online@otc.edu.

Class is well organized and consistency is high. The use of multimedia is exemplary.

Basic course organization is good, could be great with added multimedia and cleaning up of broken links.

Online, detailed feedback is essential for the growth of students and their master of the material. Providing rubrics and comments directly on papers will benefit them._________ can back up ___ enthusiasm with just a few changes to the course.

Thorough and well crafted course.

Lessons are visually appealing and integrate visuals through images as well as video.

Instructor has designed a course that contains an array of material to enlighten and challenge the students. The variety of content, including movies, films, interactive sites, many links to articles, truly energizes the course.

OTC Online Course Delivery Review

Fall 2011

Announcements are very helpful and continuous, guiding students, for example:“I added a third group--Group 3 has two members. They are --- and ---. Hopefully now, there will not be any further additions to the class, and the groups will remain as they are now. Thank you for your help, ---.”

Only two announcements present. More announcements, updates, reminders, hints, etc. can help guide students through course and establish instructor presence.

Announcements are thorough and frequent. They give students clear guidance and present high expectations.

The drop date in the syllabus is for November—it needed to be updated for summer semester.

Dates included in calendar and course site. Note spring break and other relevant dates.

Current readings and notes are present. Campus resources such as library, student help desk, OTC Online need to be added under Resources.

Most of the links in the resources section open in the same window, just be sure to check the box for “open in new window.” This helps to prevent technical issues for students and makes it easier to get back to where they were in Blackboard.

The course is rich in notes, readings, and relevant external links, including course resources. Not all links under Resources open in new windows. Some of the reading assignment links do not open in new windows. Students would benefit from having OTC Online, Student Help Desk, TLC, and Disability Support Center added to Resources button. Since students are required to cite sources in some of their work, an applicable style guide would be helpful in Resources.

Excellent use of multimedia with audio mini-lectures and videos.

Embedded content and one video are evident. Greater use of multimedia could enhance course.

Prompts are current.Blogs give the students variety!

Discussion topics appear to be current. Example – students must find an article or video online – “Then summarize, in four or five sentences, the main points or argument of the online article or video you found “

Some variance in discussion questions would offer some connections between _____and how it connects to student understanding today.

Twitter and Discussion forums are utilized for this course. Prompts are personalized and current for this course. Instructor promotes student success and academic integrity by being having a frequent presence in Twitter and Discussion forums.

Look for opportunities to updates some discussion prompts from one semester to the next with update and relevant questions.

More current topics and issues could be incorporated into assessment activities.

There was very little instructor presence on the discussion board. This presence is very important to students and also serves as an example.

For example: “It is hard for me to understand ________)” This would be a great place to follow up with an example, or to discuss a general principle from the text, such as the importance of . . .”

While students should be given time to interact and comment on the posts of their peers, instructor input is important to them! While replying to each student is not necessary, they benefit from having their input acknowledged. The instructor also is the facilitator, guiding students in the conversation(s) and/or adding insights that students might not have.

Even if comments are provided in the grading area comment, participation by the instructor in the discussions is important.

If instructors do not want to “intrude” or impose their views on the posts of the students as the discussions are taking place, giving a comprehensive, wrap-up post will suffice in some instances.

Almost all posts were on the last day of the week. Try posting several times throughout the week as a good example and to guide student discussions.

Ideally, discussions should be a dialogue, a conversation that extends beyond the first thought. Also, the goal is to get the students to develop the skills to assess and revise their own work and not always be dependent on an instructor for feedback. Discussions can be a place where students can work through several steps in the thinking process as opposed to being given the answer.

Instructor has an active role in discussions. Facilitating the discussion by participating early in discussion would be a plus for students.

Instructor presence is ongoing, and remarks are constructive.

Yes, in Questions forum. Other participations are lacking.

Without more posts, _____ cannot aptly display subject-knowledge matter or reinforce objectives.

Good tone throughout. Example from announcements – “Hello!  Just wanted to let everybody know that unit 5 is up and it is our last unit for the semester!  Good luck and lets finish strong.  As always if you need anything let me know.

Tone was excellent and positive:“I look forward to getting to know all of you and hope that you will find the class beneficial in your future endeavors.”

Tone in Announcements is professional and approachable, but lack of participation does not set a tone in discussions.

Approachable tone. Example, “You are an insightful group and you communicate clearly and effectively.  I'm very impressed.  I hope to see this continue as we take steps further and further into areas of the course that can be uncomfortable to examine.  “

For the most part, yes. Some grammar and mechanics errors in announcements.

Good participation on the ice-breaker discussion forum. This approach should be incorporated into other learning activities that relate to the course objectives.

Yes. _________ gives the other students an opportunity to answer the questions posed first, but then will answer within 36 hours if no one else answers.

Actively engaged! Instructor’s introduction post sets the tone!

Communication is ongoing thanks in part to the students without his involvement. The instructor is reading all posts, but does need to jump in and guide or help the discussions.

Feedback needs to be a bit more detailed and constructive. Explain why the student might have misunderstood the problem, maybe where they could look for further study on this issue, etc. This seems especially important on quizzes and exams with short answers. Students need to know why they didn’t get full credit.

Instructor is actively participating in some discussions, in others on a limited or non-existent basis. Discussions 4, 5, and 7 no participation. Discussions 2, 8 and 13 is limited to one post or reply.

The number of discussion forums for the course and the short window for each discussion does not allow for much interaction. In fact, there are many discussion posts without replies.

Multiple discussions allow students to choose discussion question. Questions could be changed up some semesters.

Instructor offers hints for getting started.

Instructor makes infrequent interjections in discussion. More instructor facilitation is needed to guide progress for group collaboration.

In addition to getting started hints, instructor posts announcements.

No use of the online medium is apparent in feedback.

Example from GradeCenter – “Good summary and observations in your post about education in these ______.There was some good discussion in your thread-- remember part of your task as a leader (and a responder, too) is to stay involved in your thread and reply to others. Review the scoring criteria at the end of the assignment. Also, work on submitting posts on time, so that others have plenty of opportunity to learn from and interact with your material.”

Feedback is essential; while it might not seem necessary to comment on the work of students who do well, they will benefit from knowing what/how they did well. Students who do not do well need to know how to improve in the future.

Feedback is often positive and emphasizes student strengths. Example “Your Discussion Board posts are robust and your scores on the quizzes are consistently high.  Your efforts are really paying off”

_____ did a great job here. In reading the feedback to the students, she is direct yet supportive. ____ points out what needs to be fixed, but also highlights something they did well.

Evident from student comment “And thanks for planning to have our grades back to us so quickly! “ Instructor also posts updates on grading in announcements.

Instructor provides guidance to external sites that can help with specific student errors.

The feedback is insightful--it clearly identifies issues with the writing. The development needed in this area is to give students specific examples and explanations so that they can learn the patterns in their writing. In this way, feedback is more than diagnosing a piece of writing; it is about helping students figure out the patterns in their writing along with the methods to improve. Even if it is a sentence that says, “Here you are doing, but for your next paper, try …..”

Feedback is present followed by opportunities for remediation on written work.

Good use of multimedia and instructor is active on the discussion board.

Feedback was excellent and there is plenty of material presented. Focus on increasing presence, especially on discussion board.

Tone and knowledge of material are real strengths. Add more direct participation and feedback throughout the course to enhance student learning.

Feedback is one of the most important elements of a class—although saying “Great job” or “Great input” might seem enough to let students know that they are doing well, a few words of how and why the good job is accomplished would give the students a greater sense of accomplishment. Commenting on one or two specifics from assignments or discussions also allow students to know that the instructor paid attention!

Course makes excellent use of multimedia and instructor provides outstanding feedback to students. Suggest incorporating authentic assessment or group work in future iterations.

Instructor is a strong presence in this course and models critical thinking for students. Students are challenged to problem solve and make personal application in this course. Instructor sets a positive and professional tone in the welcome and announcements as well as in interaction with students.