Assessment 2.0 for Canadian Accredited Independent Schools Heads
-
Upload
jonathan-martin -
Category
Education
-
view
7 -
download
1
description
Transcript of Assessment 2.0 for Canadian Accredited Independent Schools Heads
Video
Video
Assessment 2.0: New & Next-Gen Measurements of
Learning
Jonathan MartinHead of School
St. Gregory College PrepTucson, AZ
www.21k12blog.net
How do we assess and measure student learning in ways that advance, rather than stifle, student creativity and innovation?
“How do we measure what we value rather than value what we measure?”
How do we ensure we don’t constrain our teachers?
What about Students?
Let’s commit ourselves to experimentation and
innovation, and to maintaining focus on strong
learning outcomes.
Let’s use assessment and measurement to
lead and manage educational innovation.
Less Actually, More Uncertainty
Measurement isn’t going away: it isn’t likely to diminish in importance in the coming years. Funders, Foundations, Parents/Consumers, Boards
Commission on Accreditation Criterion 13
The standards require a school to provide evidence of a thoughtful process, respectful
of its mission, for the collection and use in school decision-making of data (both
external and internal) about student learning
Broaden the Gauge beyond mandated testing, SAT scores & the College List.
Balance the Portfolio
Broadening Internal Assessments of Learning
Demonstrations of Learning: “What you do, not what you know, is the ultimate test of education.” ~PFB
Tweet
Conduct a fluent conversation in a foreign language about of piece of writing in that language.
Declaim with passion and from memory a passage that is meaningful, of one’s own or from the culture’s literature or history.
Invent a machine or program a robot capable of performing a difficult physical task.
Character and 21st c. skills report cards
www.21k12blog.net
St. Gregory’s “egg”
• Character– Integrity
• Leadership– Positive role model– Facilitating Collaboration
• Scholarship– Inquisitiveness– Analytic thinking– Synthetic thinking– Critical thinking
• Innovation – Adaptability – Initiative – Experimentation www.21k12blog.net
Digital Portfolios• High Tech High Digital Portfolios• New Tech Network Digital Portfolios
New and Next Gen. External Measurements of learning
The questions
• Does the assessment measure what we value?• Does it constrain creativity, narrow curriculum,
or restrict teacher latitude?• Are our students meaningfully engaged in the
experience of the assessment? • Does it provide data we can use for
accountability, communications, and continuous improvement of learning?
• Is it affordable?
In “Crime Reduction” students act as a consultant to Mayor Pat Stone who is running for reelection.
As Mayor Stone’s consultant, students are asked to weigh evidence to analyze the strengths and weaknesses of two policy initiatives.
Crime Statistics
Research Brief
Critical thinkingAnalytic reasoning
Problem solvingWritten communication
Three MetricsInternal growth at a school
Comparison to other participating schoolsCollege readiness
CWRA/CLA median percentile, College Freshman Normed
http://www.youtube.com/watch?v=Y4qgDXUrc4w
CWRA: the good and the bad
• Great Goals: A test worth teaching to; measuring what we value.• Does Not Constrain or Narrow Teaching or Diminish creativity &
innovation. • Authentic assessment: Positive testing experience for students. • Student motivation can be an issue. • Results: Good for communications and accountability, not
detailed enough for continuous improvement or student differentiation.
• Pricy
Note
Experiment with developing internally designed, administered, and assessed performance task assessments.
Some schools and districts are finding this enormously valuable for teachers, such as Virginia Beach.
CLA in the Classroom provides workshops on this.
Measures of Academic Progress™
(MAP)
• Traditional/Conventional Standardized Testing Updated
• Computer Adaptive Assessment• More frequent (3x) than once a year• More rapid feedback to teachers• More detailed analysis of student proficiency
Sample Math Question
• Notice the calculator.
• To use it, click the numbers with your mouse.
Sample Math Question
• Notice the calculator.
• To use it, click the numbers with your mouse.
Sample Reading Question
• Read the passage and select the best answer
• Click the <Go on> button
Sample Reading Question
• Read the passage and select the best answer
• Click the <Go on> button
• Measures what we value? Yes and no. More basic skills than higher order thinking skills.
• It can narrow teaching and curriculum, perhaps diminish creativity and innovation, by its emphases.
• Test does conform itself to each student’s individual proficiency level.
• Student experience of testing, 3x a year, certainly wears on kids. • Results are very comprehensive and detailed, and can be very
useful for continuous improvement and differentiation, but big learning curve.
• Moderately priced.
STG 2009 STG 2010 All Schools (Averaged)
8690
75
I have opportunities to be creative in the classroom
Percentage Strongly Agreeing
All Schools St. Gregory 09 St. Gregory 10
7282
92
We regularly discuss questions with no clear answers
What Excites & Engages Me?
Lecture Discussion and Debate
Group Projects Projects Involving Technology
28
6560
55
All HSSSE students
• Measures what we value. • Does Not Narrow or Constrain• Not painful for students, but does fatigue in
subsequent administrations. • Results useful for accountability and communications,
but limited in value for continuous improvement. • Highly affordable.• Middle School Survey (MSSSE) coming soon.
student, teacher and parent surveys collect data and generate reports focused on school climate, classroom conditions, and student engagement
42
Care
Control
Clarify
Challenge
Captivate
Confer
Consolidate
Control• Students in this class treat the teacher with
respect• My classmates behave the way the teacher wants
them to• Our class stays busy and doesn’t waste time
Challenge• My teacher asks students to explain more about
the answers they give.• My teacher doesn’t let people give up when the
work gets hard.• In this class, we learn to correct our mistakes.• My teacher wants us to use our thinking skills, not
just memorize things.
Tripod Seven Cs Sample Survey Items
Student Engagement Targets(Both Individual and Collective)
1. Trust (vs. Mistrust)
2. Cooperation (vs. Resistance)
3. Ambitiousness (vs. Ambivalence)
4. Diligence (vs. Discouragement or Disengagement)
5. Satisfaction and Efficacy (vs. Disappointment and Failure)
The Seven C’sWhat Teachers Do (What Students Experience)
1. Caring about students (Encouragement and Support)
2. Captivating students (Learning seems Interesting and Relevant
3. Conferring with students (Students Sense their Ideas are Respected)
4. Controlling behavior (Culture of Cooperation and Peer Support)
5. Clarifying lessons (Success Seems Feasible)
6. Challenging students (Press for Effort, Perseverance and Rigor)
7. Consolidating knowledge (Ideas get Connected and Integrated)
Teacher Professional Learning (PLCs) Content Knowledge
Pedagogic Skill Relationship-Building Skills
The Tripod
STUDENT ACHIEVEMENT OUTCOMES
Agreement with selected statements by students in 25th and 75th percentile classrooms from the MET secondary school sample.
( From among 2985 classrooms, each with at least 5 students responding)
25th Percentile
75th Percentile
1. CARE: My teacher in this class makes me feel that s/he really cares about me
40 73
2. CONTROL: Our class stays busy and doesn’t waste time.
36 69
3. CLARIFY: My teacher explains difficult things clearly. 50 794. CHALLENGE: My teacher wants me to explain my answers – why I think what I think.
59 83
5. CAPTIVATE: My teacher makes learning enjoyable. 33 726. CONFER: My teacher wants us to share our thoughts. 47 797. CONSOLIDATE: My teacher takes the time to summarize what we learn each day.
38 67
Beyond test scores: Healthy dispositions and experiencing a high quality of life in school
Happiness in classGood behavior in classHealth vs. unhealthy responses to social pressures (e.g., holding back, hiding effort, help avoidance, etc.) Self-consciousness about being perceived as smart or not
by peersAcademic engagement/effort Satisfaction with performance and progress
46
What Works in the Classroom? Ask the Students
Teachers whose students described them as skillful at maintaining classroom order, at focusing their
instruction and at helping their charges learn from their mistakes are often the same teachers whose students learn the most in the course of a year, as measured by
gains on standardized test scores, according to a progress report on the research.
Example: Happiness Index
1. This class is a happy place for me to be.2. This class feels like a happy family.3. I feel respected in this class.4. Being in this class makes me feel angry.
The next slide shows the relationship to the Seven C’s for this measure of happiness.
48
1 2 3 4 5-0.80
-0.60
-0.40
-0.20
0.00
0.20
0.40
0.60
0.80
Happiness in Class(1260 secondary school classrooms in an urban school
district.)
GPA_DGPA_CGPA_BGPA_A
Classroom Quintiles in the Seven C's Distribution
S. D
. Uni
ts
Tripod Thoughts
• Measures What We Value• Does not narrow. • Not painful for students, but a large task to
administer surveys. • Seems it can provide valuable data for both
communications and continuous improvement.
• Pricing varies.
Information Literacy: “To be information literate, a person must be able to recognize when information is needed and have the ability to locate, evaluate and use effectively the needed information …” (ACRL, 1989)
Digital Fluency = information literacy + digital environments
CommunicateCommunicate
CreateCreate
IntegrateIntegrate
ManageManage
EvaluateEvaluate
AccessAccess
DefineDefine
Digital Fluency:
Skillful use of information via
technology
Digital Fluency:
Skillful use of information via
technology
Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
Featureso Interactive simulations – not multiple choiceo Vendor neutralo 14 real-world, problem-solving scenarios
The ETS iSkills™ Assessment
Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS).
• Task Length: 4-minute duration
• Task Type Integrate: Interpret and represent information in an ICT context, including synthesizing, summarizing, comparing, and contrasting information from multiple sources.
• Within the task, the test taker is asked to…– Summarize information from three different types of sources– Compare the information to reach a conclusion
Measuring How We Integrate
In this task, examinees summarize information from a variety of sources and then draw conclusions from their summary.
Examinees are presented with an information need and three different types of information sources.
After filling in the table, examinees must interpret the summary to rank the three sources correctly.
58
Institutional Skill Area Report
59Confidential and Proprietary. Copyright ©2006 Educational Testing Service.
Aggregate Task Performance Feedback Report
• Measures what we value? Open question. • Narrows teaching: No• Experience for students: Challenging, but the online dynamic
may be engaging for some. • Detailed data seems very useful for communications and
continuous improvement. • Price moderate.
PISA-Based Testing for Schools
Have you ever wanted to compare your school’s effectiveness with the world’s best national educational systems?
Your school’s results are statistically significantly above
Your school’s average score is not significantly different
Your school’s results are statistically significantly belowCompared to Averages
Country or Economy
Reading literacy
Mathematical literacy
Scientific literacy
Shanghai-China
Korea
Finland
Canada
United States
United Kingdom
Germany
Turkey
OECD Average
Reading: 512 Mathematics: 513 Science: 507
EXAMPLE: International comparisons of your school’s performance: PISA 2009 Countries and Economies
Section II. Your School’s Results in an international context
School ABCD
AdvantagePISA Index of socio-economic background
Disadvantage
School performance and schools’ socio-economic background
Student performance and students’ socio-economic background within schools
Private school Public school in rural area Public school in urban area
700
EXAMPLE: Your school’s performance in the context of PISA results for Canada (with socio-economic background)
Section II. Your school’s results in an international context
School ABCDReading: 512
Index calculated
from student questionnaires
-2 -1 0 1 2200
493
Score
Stu
den
t p
erf
orm
an
ce
493
Your School
300
BLOOD DONATION NOTICE
The instruments for taking the blood are sterile and single-use (syringe, tubes, bags).
There is no risk in giving your blood.
Blood donation is essential.
There is no product that can fully substitute for human blood. Blood donation is thus irreplaceable and essential to save lives.
In France, each year, 500,000 patients benefit from a blood transfusion.
Blood donation:
It is the best-known kind of donation, and takes from 45 minutes to 1 hour.
A 450-ml bag is taken as well as some small samples on which tests and checks will be done.
- A man can give his blood five times a year, a woman three times. - Donors can be from 18 to 65 years old.
An 8-week interval is compulsory between each donation.
Question 1An eighteen-year-old woman who has given her blood twice in the last twelve months wants to give blood again. According to “Blood Donation Notice”, on what condition will she be allowed to give blood again?
.................................................................................................................................
.................................................................................................................................
SCORING
Full Credit
Identifies that enough time must have elapsed since her last donation. Depends whether it has been 8 weeks since her last donation or not.
She can if it has been long enough, otherwise she can’t.
Level 2 item – 81.2% of students across OECD can perform tasks at least at this level
Will review option for schools to electronically explore their confidential results online
Proposed Structure of School Reports (Preliminary)
I. Understanding your school’s results from the PISA-Based Test for Schools assessment
II. Your school’s results in an international context
III. Additional insights from international PISA results
AnnexesA. Summary of PISA-Based Test for Schools assessment questions (items,
units and response-types)B. Summary Description of PISA Assessment Frameworks
Contents of print-ready reports will include:
PISA-Based Test for Schools Pilot:
is...• An assessment tool to support
international benchmarking efforts of schools and school networks
• Meant as a tool to assess competencies, skills and application of knowledge of 15-year-old students - comparable to international PISA results
• Based on the same assessment frameworks as the main PISA assessment
• To be available by choice and ‘on-demand’
• A tool that links schools and practitioners to comparative and international policy discussions
is not...• A mandated standardised assessment • Meant as a substitute or alternative to
the main PISA assessment• Meant to inform day-to-day instruction
of teachers• Meant to align 100% with country-
specific standards or curriculum (i.e. “Common Core” although considerable alignment is expected
• Meant as a formative evaluation tool
PISA-Based Test for Schools Pilot and Post-Pilot
Pilot Post-Pilot On-Demand
Timeline July 2011 – September 2012 September 2012 - ongoing
Assessment Window May 2012 On-demand – may be taken anytime during academic year
Test administration, scoring, and reporting
services
International service provider based on Call for Tenders process
by OECD
Local service providers accredited by OECD
School recruitment and communication services
Supported by U.S.-based team and local partners in Canada and the
U.K.
Demand-driven from schools and networks
Number of participating schools
Approx. 120 – 150 in three countries (approx. 15 - 25 in
Canada)
Demand-driven – no annual minimum required
Number of participating students
Approx. 10,000 (75-100 per school)
Demand-driven – no annual minimum required
Testing costs Approx. 80 Eur/Student. TBC by local vendor(s)
Find out more about the PISA-Based Test for Schools:[email protected]
It is important to distinguish clearly between the PISA-Based Test for Schools pilot and the main PISA 2012 assessment also taking place in the following months. The two assessments are separate and independent exercises with different goals.
The next cycle of the main PISA assessment will occur in 2012. Results for participating countries and economies are expected to be published in December 2013. Schools cannot participate in both the main 2012 PISA assessment and the pilot round of PISA-Based Test for Schools. Participating in the national sample of the main PISA 2012 study should be the priority.
PISA Based Testing for Schools
The good and the bad:
• Measures what we value: application of knowledge.
• Don‘t think it constrains. • Great general data for
communications and accountability, but maybe not enough detailed reporting.
• Student experience: unsure. • May be expensive.