PSAT/NMSQT ® Connections: Using PSAT/NMSQT Tools to Inform and Improve Teaching and Learning...

PSAT/NMSQT® Connections:Using PSAT/NMSQT Tools to Inform and Improve Teaching and Learning

Belinda Chung, College Board Consultant

Today’s agendaJanuary 15, 2011

• PSAT/NMSQT Overview

• PSAT/NMSQT Score Report Plus

• AP Potential

• Sample Questions Work

• SOAS Report Overview

• SOAS Group Work

• Group Insights/Recommendations & Action Plan

• Evaluation

The College Board's mission is to connect students to college success and opportunity. We are a not-for-profit membership organization committed to excellence and equity in education.

Mission

PSAT/NMSQT Overview

• Performance compared to the state and nation

• Question-by-question analysis

• Early wake-up call

• 3.57 million students (2010)

• 55.5% of test takers are pre-juniors (2010)

• Utilize tools to help you when Understanding by Design (backwards planning)

PSAT/NMSQT OverviewContent and Question Types

What is the PSAT/NMSQT?

• A comprehensive academic reasoning test, which focuses on skills that all students need to succeed in high school and college.

• These reasoning skills focus on the three main academic areas, but their attainment affects all areas of learning, as they are transferable process skills that are needed to manage and apply knowledge, regardless of content orientation.

• These skills are relevant to all students, and are needed by all students, thus the strong focus on inclusion when testing with the PSAT/NMSQT.

PSAT/NMSQT Overview

• The PSAT/NMSQT is a school-based test

• Comprised of five sections:• 2 Critical Reading• 2 Mathematics• 1 Writing Skills

• Critical Reading, Mathematics and Writing Skills scores are on a 20-80 scale


Critical Reading Two 25-minute sections

48 questions total : - 13 sentence completion - 35 passage based reading

50 mins

Mathematics Two 25-minute sections

38 questions total: - 28 multiple choice - 10 student produced response

50 mins

Writing Skills One 30-minute section

39 questions total: - 20 improving sentences - 14 identifying sentence errors - 5 improving paragraphs

30 mins


• Performance compared to the state and nation• Question-by-question analysis• Early wake-up call• 3.57 million students (2010)• 55.5% of test takers are pre-juniors (2010)

• Utilize tools to help you when Understanding by Design (backwards planning)

Average Test ScoresFor Juniors

Average Test ScoresFor Sophomores

Critical ReadingOverview

Questions focus on:

Student’s ability to read critically

Student’s ability to think logically

Student’s ability to analyze

Student’s ability to evaluate

PSAT/NMSQT OverviewCritical Reading

5 skills• Determining the Meaning of Words

• Author’s Craft

• Reasoning and Inference

• Organization and Ideas

• Understanding Literary Elements

Critical Reading Two 25-minute sections

48 questions total - 13 sentence completion - 35 passage based reading

50 mins

PSAT/NMSQT OverviewCritical Reading

5. Because their behavior was ------- , Frank and Susan served as ------- models for the children under their excellent care.

(A) incorruptible . . pernicious

(B) lamentable . . flawed

(C) commendable . . exemplary

(D) erratic . . unimpeachable

(E) reputable . . imperfect

PSAT/NMSQT OverviewSentence Completion

5. Because their behavior was ------- , Frank and Susan served as ------- models for the children under their excellent care.

(A) incorruptible . . pernicious

(B) lamentable . . flawed

(C) commendable . . exemplary

(D) erratic . . unimpeachable

(E) reputable . . imperfect

PSAT/NMSQT OverviewSentence Completion

Difficulty Level = EasySkill = Determining the Meaning of Words

Determining the Meaning of Words

Use vocabulary skills, context, roots, prefixes, and suffixes to determine the meaning of words.

PSAT/NMSQT OverviewCritical Reading Sample Skill

Passage:Line1 Advertisers are interested in finding out what exactly makes pop-up ads on the Internet irritating to viewers. Why are these ads seen as intrusive? One 2002 marketing study determined that when Internet users are focused,

Line 5 they perceive interruptions as more severe than when they are not focused. What seems to get the attention of viewers (rather than merely annoy them) is to expose them to pop- up ads only at breaks in content—for example, when they are switching between pages. Another marketing strategy

Line 10 is to increase the relevance of the ad. If a consumer is browsing an automobile Web site, a pop-up ad for a car will seem less intrusive than will a travel ad.

PSAT/NMSQT OverviewPassage Based Reading

9. The primary function of the sentence in lines 3-6 (“One . . . not focused”) is to

(A) paraphrase the question posed in line 3

(B) present a relevant research finding

(C) hypothesize about a solution to a problem

(D) shift the focus to a different medium

(E) address a concern of Internet users


9. The primary function of the sentence in lines 3-6 (“One . . . not focused”) is to

(A) paraphrase the question posed in line 3

(B) present a relevant research finding

(C) hypothesize about a solution to a problem

(D) shift the focus to a different medium

(E) address a concern of Internet users


Difficulty Level = MediumSkill = Organization and Ideas

Organization & Ideas

Understand the organization of a reading passage, and identify the main and supporting ideas.

PSAT/NMSQT OverviewCritical Reading Sample Skill

MathematicsOverview

The primary aim of the math section is to assess how well students understand and reason within a math framework:

Can they apply what they already know to new situations, i.e. transfer their learning and knowledge?

Can they reason and problem-solve within a high school math context?

PSAT/NMSQT OverviewMathematics

4 Content Skills

• Number and Operations

• Algebra and Functions

• Geometry & Measurement

• Data, Statistics, & Probability

Mathematics Two 25-minute sections

38 questions total - 28 multiple choice - 10 student produced response

50 mins

5 Process Skills

• Problem Solving

• Representation

• Reasoning

• Connections

• Communication


Question 30 (Student–Produced Response)

Each year from January 1 through January 8, the number of daily customers c at a music store is estimated by c =1200 + 2x, where x is the day of the month. How many more customers are estimated to be at the music store on January 5 than on January 1?


Question 30 (Student–Produced Response)

Each year from January 1 through January 8, the number of daily customers c at a music store is estimated by c =1200 + 2x, where x is the day of the month. How many more customers are estimated to be at the music store on January 5 than on January 1?The correct answer is 30. For January 1, the day of the month, x, is equal to 1, and so the estimated number of customers on January 1 is c = 1200 +21 = 1202. For January 5, the day of the month, x, is equal to 5, and so the estimated number of customers on January 5 is c =1200 +25 = 1232. Therefore, there are estimated to be 1232–1202 = 30 more customers at the music store on January 5 than on January 1.


Level of Difficulty = (M)Skill(s) = Algebra and Functions Problem Solving

PSAT/NMSQT Overview Student-Produced Response

Algebra and Functions

Solve problems using algebraic expressions and symbols to represent relationships, patterns and functions of different types.

Problem Solving

Solve abstract and practical problems, applying and adapting a variety of strategies. Monitor progress and evaluate answers in terms of questions asked.

Writing SkillsOverview

Writing Skills questions test students’ ability to edit written work and find the most effective means to communicate a thought.

PSAT/NMSQT OverviewWriting Skills

5 Skills• Manage Word Choice and Grammatical Relationships

between Words

• Manage Grammatical Structures Used to Modify or Compare

• Manage Phrases and Clauses in a Sentence

• Recognize Correctly Formed Sentences

• Manage Order and Relationships of Sentences and Paragraphs

Writing Skills One 30-minute section

39 questions total: - 20 improving sentences - 14 identifying sentence errors - 5 improving paragraphs

30 mins

PSAT/NMSQT OverviewWriting Skills

Understanding the difference between criticism

and insult enhances one’s ability to engage in

constructive argument. No error

A

E

B C D

PSAT/NMSQT OverviewIdentifying Sentence Errors

Understanding the difference between criticism

and insult enhances one’s ability to engage in

constructive argument. No error

A

E

B C D


Level of Difficulty = (E)Skill = Correctly formed sentences

Correctly Formed SentencesRecognize correct sentence structure.


Student Feedback: Paper Score Report Plus

& My College QuickStart

Score Report Plus

• Scores and percentiles

• Personalized feedback on skills

• Student answers

• Next Steps

My College QuickStart™

Signing in

Students use the access code on their PSAT/NMSQT® paper score report to sign in at www.collegeboard.com/quickstart.

My Online Score Report

Look Beyond Scores

• Performance on each skill measured

• State and national percentiles

• Projected SAT® score ranges

My Online Score Report

Your Answers

• Questions and answer explanations

• Students can filter questions

My SAT Study Plan

Prepare for the SAT®

• Personalized skills to improve

• SAT practice questions

• An official SAT practice test

My SAT Study Plan

Review and Practice

• Review PSAT/NMSQT®

questions missed

• Answer SAT ® practice questions of the same type

My SAT Study Plan

My Skills Feedback

• Based on PSAT/NMSQT results

• Hundreds of practice questions, organized by skill

My SAT Study Plan

Warm Up for the SAT® Essay

•Strategies for success

•Official scoring guide

•Practice essay questions

•Sample essays for the each score 1-6

Express SAT® Sign-Up

My Personality

Connect to thePossibilities

• Major a student chose

• Related majors

• Related careers

My Major & Career Matches

Research Majors• Description of

majors• Courses

students should take in high school to prepare

• Courses they will take in college

• Jobs students can expect achieving different degree levels

My Major & Career Matches

Start the College Search

• Starter list of colleges

• Criteria to customize search

• Ability to save searches

My College Matches

Get the Facts

• Admissions

• Cost and financial aid

• Majors offered

• Sports, housing and more

My College Matches

• www.collegeboard.com/quickstartresources

My College QuickStart Tips

Arrange for a computer lab session to introduce students

to My College QuickStart

Arrange for a computer lab session to introduce students

to My College QuickStart

Use free lesson plans to guide students through structured activities

Use free lesson plans to guide students through structured activities

TIP

TIP

TIP

Check out the demo for educators http://professionals.collegeboard.com/k-12/prepare/quickstart

Check out the demo for educators http://professionals.collegeboard.com/k-12/prepare/quickstart

AP Potential

What is AP Potential?

• A web-based tool

• Free for schools that administer the PSAT/NMSQT ®

• Provided by the maker of the AP ® Exams

• Designed to ensure that no student is overlooked

• Based on proven research

AP Potential

•Identifies “diamond-in-the-rough” students

•Promotes equity

Helps identify students who initially might have been overlooked for AP courses

•Is a useful tool for principals, teachers, and counselors to

Expand AP programs

Increase enrollment in current AP courses

The PSAT/NMSQT and AP Connection

Research shows a strong relationship between PSAT/NMSQT and AP

•Studies conducted in 1998 and 2006

•Analyzed performance of more than a million students

•Showed strong correlations between PSAT/NMSQT® scores and AP® Exam results

•Download the research reports at www.collegeboard.com/appotential

Traditional Methods of Identifying Students for AP Courses

•Teacher recommendations

•Self-nomination

•History of courses completed

•Student grades

•Student discussion

Access to AP is Important

•AP access is linked to success in college. Rigorous high school curriculum is key

•Participation in AP is linked to college completion. It is a stronger indicator than socioeconomic status and GPA.

Signing in to AP Potential™ :www.collegeboard.com/appotential

Signing in to AP Potentialwww.collegeboard.com/appotential

AP PotentialExpectancy Tables

Expectancy TablesSee data from the correlation study showing the percentage of students at any given PSAT/NMSQT score range.

Step 1 of 4

• Select the PSAT/NMSQT ® administration year for which you’d like to generate a roster.

AP PotentialSelect PSAT/NMSQT® Administration Year

Step 2 of 4

• Select the AP subject areas your school offers.

AP PotentialSelect Subjects

Step 3 of 4

• Choose a grade level and define your pool.

AP PotentialSelect Pool

AP PotentialStudent Roster

Step 4 of 4• Read the

instructions for using the roster and see your list of students at the bottom of the screen.

• Student names

• Correlations and

Student scores

• Ethnicity

• Gender

AP PotentialSubject Detail

• All courses selected noted

• Status of meeting the criteria

• Probability for scoring a 3 noted

AP PotentialStudent Detail

AP Potential Tips

Export data into an Excel spreadsheet to make it easy to read and use.

Export data into an Excel spreadsheet to make it easy to read and use.

Create letters to parents. AP Potential™ provides sample letters in English and Spanish.

Create letters to parents. AP Potential™ provides sample letters in English and Spanish.

TIP

TIP

•After considering potential AP students using your school’s standard process, check AP Potential for additional students that might have been overlooked.

•Use AP Potential to make a case for professional development - more teachers will need to be trained to handle an increased number of AP courses.

•Review AP Potential for 9th and 10th graders in preparation for 11th or 12th grade AP courses and ensure that they are taking the correct preparatory courses

AP Potential Tips

Activity A: Getting to know the Test

Audience

• Administrators

• Department heads

• Curriculum specialists

• Teachers

• Other staff interested in students’ college preparatory skills

Note: the more inclusive the testing within each grade, the more meaningful and applicable the results.

SOAS Introduction

Instructions: Part I, Solo Activity (20 minutes)

• Review the PSAT/NMSQT skills list in your binder in order to become familiar with the skills for your academic area.

• You will have 20 minutes to take sample PSAT/NMSQT • Mark your answers in the exam booklet• Immediately after answering a question, refer to the List of

Academic Skills list and, in the test book next to the question number, identify the skill group(s) that you feel is needed to answer the question.

Activity A

Instructions: Group Activity (20 minutes)

• Gather in groups by subject matter (no more than 7 per group). Each group needs a Leader and a Recorder.

• You will have 20 minutes to review at least 5 questions from the test-taking activity, to compare notes on the skills needed to the answer questions:

• English teachers, do at least 3 Critical Reading questions and 2 Writing Skills questions

• Math teachers, do at least 3 multiple choice questions and 2 SPR questions

• Take 15 -20 minutes to answer the following questions:

Activity A

Discussion Topics:

1. How would the content covered in these questions be familiar to your 10th grade students? 11th grade students?

2. How are these test questions similar to question types you would present in class?

3. Was there agreement in the group on the skills coding? If no, what kinds of issues caused disagreement?

4. How do these skills compare to the skills you would expect your students to have? (10th? 11th?)

5. How did the “group’s” coding match the test developer’s coding?

Activity A

The Summary of Answers and Skills (SOAS)

Logging In

Repeat Users

www.collegeboard.com/results

Logging In

New Users

If it’s your first time, you’ll need your school’s SOAS access code

SOAS Resources

Supporting Materials

Your Reports

Page(s)

• Title Page

• Performance Overview 1

• Skills Analysis 2, 6-7, 11

• Question Analysis 3-4, 8-9, 12-13

• Comparable Group Analysis 5, 10, 14

Report Orientation

Table of Contents

• Number of students in report

• Mean scores and

• score distribution

• College Readiness Benchmarks

Performance Overview

Page 1

Pages: 2, 6-7, 11

Skills Analysis

• Skills

• Triangles show a comparison to the state and nation

• Each test question is linked to a skill

Pages: 3-4, 8-9, 12-13

Question Analysis

• Student responses and answer patterns

• Comparisons to the state and nation

Tips:

Question Analysis

Look for questions your

Students answered incorrectly that the students in the

state and nation

answered correctly

Look for commonlyWrong answers

Pages: 5, 10, 14

Comparable Group Analysis

What is a comparable group?

• A useful statistical model

• A statistically created group (virtual group)

• Mirrors your group’s performance profile

• This creates an expected performance indicator for your group on each question.

• Provides more “actionable” feedback than state or national averages on questions/skills.

SOAS Comparable Group

Pages: 5, 10, 14

Comparable Group Analysis

Darker blue =

significantly below the comparable group.

Lighter blue = significantly above the comparable group

Resources

Resources Test Questions and Answer Explanations

How does it all add up?

1. Identify

a question for review

2. Find the skill measured by this question

3. View the question details

Other Relevant Resources:Before, During, and After SOAS review

Understanding PSAT/NMSQT Scores

• PSAT score related FAQ’s

• Information on percentiles and mean scores

• Score conversions

PSAT/NMSQT Summary Reports

PSAT/NMSQT Summary Reports

Summary Reports are provided to all schools that tested at least 50 juniors or sophomores.

State level summary reports are also available (College Bound Juniors/Sophomores reports) www.collegeboard.com/reporting

Reports contain:

• number of test-takers

• ethnic background

• participation in Student Search Service

• self-reported GPA

• performance by score range

• college major

Activities

Instructions

• Work within your own school group/academic area, using your school’s SOAS report.

• Identify the skill that your students had the lowest performance on.

• Using the Test Question and Answer Explanations, look at each question aligned to that skill.

• Things to look for:

Skills furthest below the state/national average

Skills that are furthest to the left (i.e. those that are the weakest for the group).

Skills that came to teachers’ attention after review of Question Analysis.

Activity B Review Skills Analysis

Questions for Consideration

Within your group, answer the following questions:

1. Is there a pattern in the skills you identified as problematic?

2. Are there actions or strategies that might help address the issues?

Activity B Review Skills Analysis

Instructions


• Refer to the Test Questions and Answer Explanations to make note of problem questions and how they might be addressed.


Easier questions (difficulty level 1-3) where less than a 2/3 of the students answered correctly.

A larger proportion of your students answered incorrectly or omitted an answer in comparison to the state/nation

Commonly wrong answers

Review any questions that generated teacher interest when you took the test in Activity A.

Activity C Review Question Analysis

Activity C Review Question Analysis



1. Are there any patterns in the questions you identified as problematic?


3. Summarize your findings for sharing.

Instructions


• Using the Test Question and Answer Explanations, look at each question.


Questions that fall into the dark blue area, where your students performed below expectation.

Questions that fall into the light blue area, where your students performed higher than expectation.

Activity D Comparable Group Analysis



1. Is there a pattern in the skills aligned with the test questions your students performed below expectation on?

2. Is there a pattern in the skills aligned with the test questions your students performed expectation on?


Activity D Comparable Group Analysis

For Group Discussion

To Help Students and Parents

1. How does the school educate parents about the wealth of information on Score Report Plus?

2. What plan does the school have in place to make sure students receive and understand their scores?

For Group Discussion

Using Your SOAS Data

•Identify the skills associated with the questions students answered incorrectly.

•Make inferences about what led to errors.

•Identify where in the curriculum the skills are (or are not) taught.

•Work with departments and classrooms to review common concerns.

•What recommendations can you make to your school for increased student success?

Closing Thoughts

Using Assessments for School Improvement

• Identify skills to help guide instruction and curriculum planning

• Connect more students with challenging high school courses and college attendance

• Increase Advanced Placement Program courses in your school system; more professional development for teachers

• Close the achievement gap in your community

The Excellence and Equity ChallengeThe Excellence and Equity Challenge

“Without preparation, opportunity is an empty promise.”

Alan Page, former NFL star, current Minnesota State Supreme Court Justice

Austin American Statesman, 10.25.02

Looking Ahead:

2011 PSAT/NMSQTWednesday, October 12th

Saturday, October 15th

PSAT/NMSQT ® Connections: Using PSAT/NMSQT Tools to Inform and Improve Teaching and Learning...

Documents

Transcript of PSAT/NMSQT ® Connections: Using PSAT/NMSQT Tools to Inform and Improve Teaching and Learning...