Ready for Online Assessments? Help is Here.
description
Transcript of Ready for Online Assessments? Help is Here.
Ready for Online Assessments? Help is Here.Call-in number: 571-392-7703 | PIN: 635 397 980 649
October 2013
Poll #1
A. SETDA/Assess4ed Promotion
B. Connected Educators Month Promotion
C. Twitter/Facebook
D. A Colleague
How did you hear about this event?
About SETDA
• Ten-year old DC-based, national, non-profit member association
• Serve, support, and represent U.S. state and territorial directors (SEA leadership) for educational technology
• Forum for:– Research and best practices – Inter-state collaboration– Professional development– Public-private partnerships– State-federal relations
SETDA Resources for Online Assessment Readiness
• assess4ed.net• Technology Requirements for Large-scale
Computer-based and Online Assessment• Technology Readiness for College and Career
Ready Teaching, Learning and Assessment• SEPC• Implementing Online Assessments – Pathways to
Success
For access to SETDA’s resources to go -
http://www.setda.org/web/guest/assessment
Agenda
• Welcome– Geoff Fletcher, Deputy Executive Director, SETDA
• Implementing Online Assessment Website Overview District Sharing– Mike Nelson, Director of Curriculum and assessment of Coeur d"Alene
School District 271, ID– Patches Hill, Technology Systems Manager of Indian River School
District, DE– Scott Smith, Chief Technology Officer of Mooresville Graded School
District, NC
• Q&A, Geoff Fletcher, SETDA
Poll #2
A. Technology contact
B. Assessment contact
C. I am driving the whole show
D. I am doing nothing to help prepare – yet
E. I will implement it when it happens
Chose the statement that best describes your role in helping your state/district prepare for online assessments.
Presenters
Scott Smith
Chief Technology Officer
Mooresville Graded School District, NC
Patches Hill Technology Systems ManagerIndian River School District, DE
Mike Nelson Director of Curriculum and Assessment Coeur d’Alene School District 271, ID
Effective Use Online
Platforms for Formative & Summative
Assessment
Michael S. NelsonDirector of Curriculum and Assessment
Coeur d’Alene School District (Idaho)[email protected]
Coeur d’Alene School District 271http://www.cdaschools.org
Public school district of 10,300 students in Idaho 530 teachers in 17 schools (7 of last 14 State teachers of the year)
11 elementary; 3 middle schools; 2 traditional high schools 6 magnet or charter schools
Part of the Smarter Balanced Assessment Consortium Highest performing large district in Idaho 90GB of bandwidth coming into our district
17 mobile laptop/iPad labs available if needed Blended learning for several courses
District Background
Professional Devel. Approach
We empower faculty through:•Job embedded staff collaboration
– Weekly on Mondays– 1 day per month district-driven
•“Top 40” Professional Devel. Team– Stiggins’ Assessment Training Institute
•Building Leadership opportunities
Our Assessments are used for…
• Diagnostic to determine the student’s pre-knowledge and
identify strengths & weaknesses
• Formative and Interim to provide the student with feedback on their
progress during a course
• Summative to estimate performance at the end of a course
and ‘grade’ the student’s work
Smarter Balanced Questioning Types
Our Assessment Quandary Discovery
Education PaperScannable
OMR Schoolnet SkywardDelivery Options
Clicker Delivery XX
Online Delivery XX XX XX
Paper Delivery XX XX XX XX
Question TypesSelected Response
XX XX XX XX
Constructed Response
XX XX XX XX
Technology Enhanced
XX
Performance Tasks
XX
End of Exam
Item Analysis XX XX
Reporting Features
XX XX XX
Import to Grades XX
Costs
Ease of Use XX XX
Administration Ease of Use
XX
Annual Maintain and/or Upgrades
XX XX
Other Considerations
Ties to Standards XX
Connects to Rubrics
XX
We Chose Skyward…
After evaluating options, we found that our student management system did the best in:
•test factual knowledge effectively
•include web-based resources, media objects
•track student attempts
•provide detailed, specific feedback
•mark automatically, quickly, consistently
•provide question banks, randomized ordering
•produce statistical information for test items
•Export to gradebook and statistical packages
Areas of Note• construction of good objective tests still depends on
instructors
• building up question pools is time-consuming
• specialized mathematics and scientific notation still cumbersome to incorporate
• provision of secured test environment
• review or grading essays and project-based assessment/performance tasks
• detect plagiarism in essays
• robust infrastructure is required to avoid failure during exams
Students Taking
Assessments
Getting to the Exam
Click on “Open Student Access” on the upper left corner of the screen.
Student “assignment” will be listed at the top of the screen along with their current grades and academic calendar.
Click on “Take Assignment”
Ending the Exam
When choosing “Save and Complete Exam” a summary screen will appear asking you to review your work and ask whether you want to complete the assessment.
Click “Yes” to end.
Essay Questions
• Read the prompt carefully.• Create a well-crafted essay that fully answers
the prompt.
• You can expand the response box by dragging down on the bottom-right corner of the window.
Selected Response Stems
• These are NOT traditional multiple choice.• Each question may have multiple correct
answers.• Read the question carefully AS WELL AS
EACH answer choice.• Place an X in each of the boxes that correctly
answer the question correctly.• There may be an image, document or other
information available for answering your question.
• Look for the button
X
X
X
Post-Assessment Analysis
Post-test, we have the ability to analyze each assessment by question stem or by teacher distribution.•Report-based or chart form
Final Thoughts
Organizational Support Structure
Phase-In • Work with provider• Pilot work at different levels• Constantly communicate
Provide Supports in a variety of methods
• How to handouts• PowerPoint presentations• Short instructional videos• Top 40 Team• Technology Leaders• Hand holding may be
necessary
Provide Timely Results and Successes
• Empower faculty to manipulate and use results
• All assessment should inform instruction
• Triangulate results with other works
Thank You.
Michael S. NelsonDirector of Curriculum and Assessment
Coeur d’Alene School District (Idaho)[email protected]
Delaware, IRSD and Online Assessments
Summary of levels A multi-step process at multiple levels
◦ State infrastructure implementation◦ State assessment implementation – summative standardization ◦ Personalized learning and formative standardization
History and Background Delaware has only three counties that encompass 19 school districts (16 traditional districts and 3 county-wide vocational/technical districts).
The move to adopt and fully implement Common Core State Standards (CCSS)-aligned assessments in 2014 builds on Delaware’s nearly twenty-year history of planning toward better information sharing across the state.
Faced with standardized testing contract expiration after the 2009-2010 school year, the state formed a taskforce with two priorities for the next iteration of tests: immediate scoring, and the ability to track data over time. These priorities led to the group’s ultimate recommendation for a shift to online assessment.
Delaware wanted to emphasize testing flexibility in how the test is administered and within the assessment. The Delaware Comprehensive Assessment System (DCAS) provides multiple opportunities for students to demonstrate proficiency and provides academic achievement including a measure of fall-to-spring and year-to-year individual student growth.
Delaware has a law requiring all state-mandated and high-stakes assessments to be administered digitally.
Key Success Factors Centralized statewide network that provides Internet connectivity to each school.
Statewide pupil accounting system that tracks accommodations used by every district already in place.
Annual school technology survey to assist in determining readiness.
Stakeholders involved starting very early in the field testing process and focused on technology capacity and hardware issues, particularly the logistics of setting up schools.
What districts did Prior to field testing in 2010, districts and schools were using technology for instruction, but administering paper-and-pencil assessments.
Assessment no longer meant entire buildings would have to shut down for several days while all students inside completed the exams because of the move to broad assessment windows.
Technology Readiness – Including network, bandwidth and computers readiness.
Teacher/Staff Assessment Readiness – Including training of district technology coordinators, as well as training of school and district test administrators.
Student Readiness – All students viewed a presentation that introduced the DCAS field test and completed both a computer skills assessment and a training test.
Scheduling – Districts developed careful schedules to complete in time allowed.
What districts did Displaced some instruction because there aren’t enough laptops with particular features to meet all demand scenarios.
Establishing Wi-Fi in every school to assure secure, reliable, redundant connections. Currently, all but two buildings are wireless at 100% coverage of facility.
PLCs provided a structure for decentralized decision making about the timing and configurations for online assessment that work best.
Despite the considerable flexibility offered by broad testing windows, most schools continue to follow the more traditional approach of holding shorter testing windows comprised of a few days during which nearly all students are tested.
Four schools at IRSD (2 high schools, 1 middle school and 1 elementary school) are participating in the spring 2013 SMARTER Balanced pilot test.
Key Factors for Future Systems Adoption at IRSD
As we move to more personalized learning it will prove key to build systems that follow open standards.
Joined BRINC(State) and IMS GLC (International).
IMS Global and its members have helped us significantly in understanding the total interoperability of our data and how they should integrate with other state and local systems.
Only choosing systems that follow QTI and APIP at district level to keep impact of assessment database longevity and interoperability from being a concern. Also focusing on Common Cartridge (CC) and Learning Tools Interoperability (LTI).
IMS Standards Prove Key @ IRSD
QTI - Question and Test Interoperability is a technical specificationthat describes formats & services for exchange of digital assessment items and tests among assessment system software components.
◦ XHTML based◦ Permits inclusion of other mark-ups, e.g., MathML, ChemML◦ Supports alternative accessibility representations◦ Future version will merge with HTML5 & digital book formats
APIP Accessible Portable Item Protocol is a technical specificationthat describes formats & services for exchange of accessible digital assessment items and tests among assessment system software components.
◦ APIP v1.0 uses a precisely defined subset of QTI v2.1 ◦ Expands the accessibility content feature in QTI◦ Uses accessibility definitions from IMS AccessForAll specifications (Metadata and Personal
Needs and Preferences - PNP)
How does APIP relate to QTI in terms of origin?
QTI has been implemented and evolved over 10+ years of experience of leading assessment, publisher and learning platform providers worldwide.
APIP arose out of seed funding from the U.S. Department of Education to a project led by a consortium of U.S. states.
Conformance Profiles
Additional LinksDistrict Website
http://www.irsd.net/
Materials Used
http://assessmentstudies.setda.org/resources/
IMS Overview
http://www.youtube.com/watch?v=corV3-WsIro
Open Infrastructure
http://www.imsglobal.org/IMSSupportsK12.pdf
Mooresville Graded School District Scott S. Mooresville Graded School District Scott S. SmithSmithDigital ConversionDigital Conversion Ed.DEd.D
Mooresville Graded School District:Mooresville Graded School District:DemographicsDemographics
Number of Schools
8
Certified Staff ~ 400
Non Certified Staff ~ 300
Total StudentsTotal StudentsK-12K-12 59005900
Asian 2%
Multi Racial 3%
Hispanic 8%
African American
17%
Caucasian 70%
Students receiving Free/Reduced Lunch has increased in MGSD...
Why a Digital Why a Digital Conversion?Conversion?• Close the digital divideClose the digital divide
• Relevant instructionRelevant instruction
• 21st century readiness21st century readiness
• Real world experienceReal world experience
• Instructional practiceInstructional practice
• Improved academic Improved academic achievementachievement
• Moral imperativeMoral imperative
IngredientsIngredients
• Building the cultureBuilding the culture
• 21st century tools and 21st century tools and online resourcesonline resources
• Data rich & intensive – Data rich & intensive – with immediate with immediate feedback and resultsfeedback and results
• Capacity buildingCapacity building
• All in!!!All in!!!
The Tech SpecsThe Tech Specs
• Over 5500 MacBook Airs for every teacher and student Over 5500 MacBook Airs for every teacher and student grades 3-12grades 3-12
• Online content available virtually 24 / 7Online content available virtually 24 / 7
• Ubiquitous wirelessUbiquitous wireless
• 802.11n Access Point in every classroom (at 802.11n Access Point in every classroom (at minimum)minimum)
• High availability connectivityHigh availability connectivity
• 500 MB Pipe for the district500 MB Pipe for the district
• GB Fiber between locationsGB Fiber between locations
• Cultural shift / personalized instructionCultural shift / personalized instruction
• Support for online assessmentsSupport for online assessments
Online AssessmentsOnline Assessments
• Daily online assessments in Angel (Learning Daily online assessments in Angel (Learning Management System)Management System)
• Implementation of Lock Down Browser in conjunction Implementation of Lock Down Browser in conjunction with Angelwith Angel
• Quarterly Assessments with ThinkGateQuarterly Assessments with ThinkGate
• End of Grade and end of Course Exams onlineEnd of Grade and end of Course Exams online
Meeting Bandwidth NeedsMeeting Bandwidth Needs
• Prior to Digital Conversion Prior to Digital Conversion
• 40MB Connection40MB Connection
• Year 1 (2008 – 2009)of Digital Conversion (~2200 Year 1 (2008 – 2009)of Digital Conversion (~2200 laptops)laptops)
• 100 MB Connection 100 MB Connection
• Year 2 (2009 – 2010) of Digital Conversion (~4000 Year 2 (2009 – 2010) of Digital Conversion (~4000 laptops)laptops)
• 250 MB Connection 250 MB Connection
• Year 4 (2011-12)of Digital Conversion (~5000 laptops)Year 4 (2011-12)of Digital Conversion (~5000 laptops)
• 500 MB Connection500 MB Connection
• Year 6 (2013-14) of Digital Conversion (~5500 laptops)Year 6 (2013-14) of Digital Conversion (~5500 laptops)
• 500 MB Connection (500 MB Connection (watchingwatching closelyclosely))
Online AssessmentsOnline Assessments
• Average week – 274 MB of a 500 MB pipe (Sept 16 – Average week – 274 MB of a 500 MB pipe (Sept 16 – Sept 20, 2013)Sept 20, 2013)
June 3, 2013 – First day of Online June 3, 2013 – First day of Online ExamsExams
• Average 173 MB of a 500 MB pipeAverage 173 MB of a 500 MB pipe
• Honor School of ExcellenceHonor School of Excellence• High GrowthHigh Growth• Met 100% AMO TargetsMet 100% AMO Targets
Mooresville High SchoolMooresville High SchoolGrades 9 - 12Grades 9 - 12
Composite increase of 23 Composite increase of 23 percentage points over 6 percentage points over 6
yearsyears
Overall Composite for Students Testing ProficientOverall Composite for Students Testing Proficienton State End-of-Course Examson State End-of-Course Exams
Mooresville Graded School DistrictMooresville Graded School DistrictGetting students across the line!Getting students across the line!
4-Year Cohort Graduation Rate4-Year Cohort Graduation Rate
MGSD is 3rd MGSD is 3rd in the state in the state out of 115 out of 115
districts for districts for its 2012 its 2012
graduation graduation rate!rate!
Mooresville Graded School DistrictMooresville Graded School DistrictDistrict AchievementDistrict Achievement
16%16%Growth!Growth!
16%16%Growth!Growth!
Mooresville Graded School DistrictMooresville Graded School DistrictDistrict AchievementDistrict Achievement
DistrictDistrictPercent Proficient Percent Proficient
using EOG Reading, using EOG Reading, Math, Science; EOCMath, Science; EOC
Per Pupil Per Pupil ExpenditureExpenditure
(state ranking)(state ranking)
District District Enrollment Enrollment NumbersNumbers
11 Camden CountyCamden County 90%90% 7989 (73)7989 (73) 18911891
22Mooresville Mooresville
Graded School Graded School DistrictDistrict
89%89% 7463 (100)7463 (100) 56225622
Polk CountyPolk County 89%89% 9604 (22)9604 (22) 23632363
33Chapel Hill Chapel Hill CarrboroCarrboro 88%88% 8979 (39)8979 (39) 11,50411,504
MGSD Ranked MGSD Ranked 2nd in the 2nd in the
State State 2011-122011-12
Formula: Number of Proficient Test Scores on Reading Grades 3-8, Math Grades 3-8, and EOCs divided by the total number of tests Formula: Number of Proficient Test Scores on Reading Grades 3-8, Math Grades 3-8, and EOCs divided by the total number of tests taken in school year 2011-2012.taken in school year 2011-2012.
Questions?
Webinar Archive & Resources
http://www.setda.org/web/guest/connectededucators
http://assessmentstudies.setda.org
More information: Geoff Fletcher, [email protected]
Claim your CEM Digital Badge
• Go to http://badges.connectededucators.org• Click the “Have claim code?” button• Enter the code: rere124• Click “Claim”
NOTE: Participants will have to create a Mozilla account and backpack to claim a badge if they do not have an account already.
CEM Activities for State Leadership
http://connectededucators.org/cem-for-states/
http://connectededucators.org/events/#/audience=249
Webinar Resources
Webinar Archive & Resources
http://www.setda.org/web/guest/connectededucators
http://assessmentstudies.setda.org
More information: Geoff Fletcher, [email protected]