CSP 6123

Jimmy Warren and Aaron Jordan

What we will discuss today Overview of Assessment and Technology

Impact of Technology on Student Life and Learning

Institutional Examples of Assessment and Technology

Overview of Web-Based Data Collection

Advantages and Disadvantages of Web-Based Data Collection

Assessment and Technology Conclusion

Reading Assignment

Overview Technology has made a significant impact on higher

education. The Economist Intelligent Unit, a social media consortium, reports several important facts about the emergence of technology in higher education

63% of public and private institution administrators consider technology to be the main factor in attracting prospective students.

More than two-thirds of reporting institutions utilize online classrooms*

*Taken from the Economist Intelligence Unit, 2008

Overview More important facts from the Economist Intelligence

Unit Many administrators consider technology to have a large

impact on student life

The use of technology is allowing colleges to “globalize” their educational efforts by offering global campuses and open access to international students

Sixty percent of college classrooms utilize more than one medium of teaching

The current generation of students is the most technologically savvy and equipped group of students in the history of higher education in America *

*Taken from the Economist Intelligence Unit, 2008

How to assess technologies’ impact First, define the technology

What specific technologies are in use? Computers, specific software, iPods, etc.

Where is the technology being used? Classroom use, out of class, asynchronously…

Who is using the technology? Individuals, groups, entire classrooms, instructor specific use…

How is the information being used? Note sharing, group discussion, utilizing real time information…*

* North Carolina State University Office of University Planning and Analysis, 2006

How to assess technologies’ impact Define student learning outcomes*

What is the main goal of the course or project?

What is considered a full understanding of the course or project?

What skills will students have at the end of the course of project?

What will be used to determine students have the skills?* North Carolina State University Office of University Planning and Analysis, 2006

How to assess technologies’ impact Assessment tools

Course performances – tests, quizzes, presentations, or projects

Locally developed tests

Surveys completed by students, staff, and faculty pertaining to specific programs

Focus groups utilizing students, staff, and faculty pertaining to specific programs

Tracking students’ course-taking , and advisement patterns*

*North Carolina State University Office of University Planning and Analysis, 2006

How to assess technologies’ impact Utilize information from database resources

Enrollment data

Retention rates

Budget data

Graduation rates

Classroom experiences

Out of classroom experiences** Taken from the North Carolina State University Office of University Planning and Analysis, 2006

How to assess technologies’ impact Garnering feedback on technological impact can come

from all parts of the institution Prospective , withdrawn and current students

Faculty

Staff

Employers

Alumni

Community groups

Trustees

Executive administration** Taken from North Carolina State University Office of University Planning and Analysis, 2006

How to assess technologies’ impact Does technology have an impact?

The impact depends on several factors – instructor, classroom characteristics, the learner, and the technology used

More literature is being produced on the impact of technology on education

Literature also exists to guide professionals assessing technological impacts

Research suggests online learning promotes critical thinking

Technology promotes student engagement, collaboration and authentic learning environments

The main impact on student learning however is the pedagogy*

*Taken from North Carolina State University Office of University Planning and Analysis, 2006

Institutional Examples To maintain the increased use in technology on campus many

institutions have begun to utilize technology into their assessment practices. Old Dominion University

The institution combines technology and assessment into one department –Office of Student Affairs IT & Assessment.

Functions of the department include data tracking, reporting, analysis, database development, student affairs effectiveness, and educational effectiveness.*

*Taken from Old Dominion Office of Institutional Research and Assessment, 2006

Georgia Tech Office of Assessment Within the offices statement of goals they use relevant methods to assess

technology-driven initiatives The office also utilizes web-based data collection methods to assess non-

traditional programs and distance education courses**** Taken from Georgia Institute of Technology Office of Assessment, 2008

Institutional Examples A few more examples of technology and assessment

integration.

Towson University

Utilizes technology to develop and evaluate the effectiveness of distance and hybrid courses*

*Taken from Townson University Office of Assessment, 2010

North Carolina State University

The office of University Planning and Analysis created an entire online resource concerning student learning outcomes, technology effectiveness, commonly used technology tools, and secondary resources to assist in the assessment process** **Taken from North Carolina State University Office of University Planning and Analysis, 2006).

Web-Based Data Collection Overview Aside from assessment being used to measure

technologies’ effect on student life and learning, the emergence of technology allows professionals in the field to collect assessment data through the internet.

Web-based data collection is an intriguing portion of assessment. Not only has it become a common method for collecting data, but it has stirred a debate over its place in higher education.

A primary key to remember with any assessment method is to utilize what works best for the institution and the study at hand.* *Taken from Wortman and Upcraft, 2001

Web-Based Data Collection Advantages

Lower costs Postage and supply costs are greatly reduced Time and costs spent on creation, editing and implementation are

reduced

Web- based data collection allows for qualitative and quantitative data gathering

With the rise of technology, web-based collection is user-friendly Assessments can be completed at the convenience of the respondent

May lead to greater return rate Data collection time may be reduced Data can be easily managed

Allows data to be stored and managed easily – no need for filing cabinets and a specific geographic location

Methods can accommodate change*

*Taken from Wortman and Upcraft, 2001

Web-Based Data Collection Disadvantages

Not all respondents have web access

Distrust in anonymity

Not all respondents have equal computer literacy skills

Respondents may not understand the software in use

Technical problems may occur

Computer difficulties may discourage a respondent from finishing a survey

Respondents computer may have compatibility issues*

*Taken from Wortman and Upcraft, 2001

Web-Based Data Collection Advice for the profession

Include a channel of communication for the respondent

Design user friendly methods

Collaborate and work closely with qualified personnel

Always test the survey before it is sent

Understand that no collection method is perfect**Taken from Wortman and Upcraft, 2001

Reading Assignment and Discussion Read the following article.

What do you think about the course format?

Do you see programs like Learn Anytime becoming widely adopted?

Conclusion Currently ,higher education is undergoing a change in

delivery and format

College students utilize technology throughout their college lives

Faculty and staff utilize technology to increase educational and institutional effectiveness

Institutions are beginning to integrate technology into assessment and student affairs departments

The web may also be used as a data collection tool

No matter the type of assessment used, challenges occur

ReferencesEconomist Intelligence Unit. 2008. The future of higher education: How

technology will shape learning. Retrieved October 9, 2010 from http://www.nmc.org/pdf/Future-of-Higher-Ed-(NMC).pdf.

Georgia Tech Office of Assessment. (2008). RetrievedOctober 9, 2010 from http://www.assessment.gatech.edu/.

North Carolina State University Office of University Planning and Analysis. (2006). Retrieved October 10, 2010 fromhttp://www2.acs.ncsu.edu/UPA/.

Old Dominion Office of Institutional Research and Assessment. (2006). Retrieved October 9, 2010 fromhttp://www.odu.edu/ao/ira/.

Townson University Office of Assessment. (2010).Retrieved October 10, 2010 from http://www.towson.edu/assessment/.

Wortman, T. I., & Upcraft, M. L. (2001). Web-based data collection. In Schuh, J.H., Upcraft, M.L., & Associates (Eds.), Assessment practice in student affairs: An applications manual (101 – 117). San Francisco: Jossey- Bass.