A publication of the Magnet Schools Assistance Program Technical Assistance Center

Volume 2 • Issue 4 • october 2012

Managing with DataLeaders of successful magnet programs make data use a priority.

They nurture a culture that respects evidence, encourages data collection, and uses findings to improve program components, processes, and outcomes. These leaders find data to be a powerful tool that helps all stakeholders create a program that focuses on measurable accomplishments. When used properly, data can help to improve teacher quality, enhance curriculum development, narrow achievement gaps, engage parents, and inform stakeholders. However, it can be daunting to sustain data-driven management practices over time.

Managing with data is a systematic process of collecting and analyzing a wide range of information to guide operational decisions that foster school and student achievement.

Successful magnet programs provide staff development around data use, have access to systems that support data application, and collect high-quality data. The information must be analyzed in meaningful ways, shared with the right people, and used to increase magnet school efficiencies.

This issue of the Magnet CoMpass offers principles, strategies, and tips for implementing programwide data use to effectively manage magnet programs. Inside you will find information on collecting a wide variety of data and managing the data-driven decision making process.

Make Sure Data Are Reliable and ValidData-based decisions are only as strong as the quality of data analyzed. Accordingly, data must be reliable and valid. While these data concepts are simple to understand, maintaining data integrity can be challenging.

The following definitions and tips are starting points for ensuring reliable and valid data. Your district’s research and evaluation department can help you learn to use existing data sources and offer guidance on establishing additional sound data collection and analysis protocols.

Reliable data are collected in a consistent •

•

manner over time. Set standards for data collection to ensure all staff members collect the same type of data in the same manner. Create standardized data collection rubrics to make the data collection process easy for staff to follow.

Valid data measure what they claim to measure. Consider external factors that could possibly discount the trustworthiness of data. Determine whether the indicators you use are valid for all groups from which you are collecting data.

inside THis issUeLearning to Value Data . . . . . . . . . 3

Using Diverse Data . . . . . . . . . . . 5

Leading Effective Data Use . . . . . . 6

Join Our Online Community . . . . . 8

REGULAR FEATURES

The Needle Point . . . . . . . . . . . . . 2

Mapping the Way . . . . . . . . . . . . . 4

Magnet Moments . . . . . . . . . . . . . 7

Promoting diversity, academic excellence and equity through magnet schools

A Message From the Magnet Schools Assistance Program

Left to right: Anna Hinton, Director of Parental Options and Information; Rosie Kelley, MSAP Team Lead, Education Program Specialist; Brittany Beth, MSAP Program Officer, Management and Program Analyst; and Tyrone Harris, MSAP Program Officer, Management and Program Analyst

The ability to collect and analyze data is extremely important to gauging effectiveness and optimizing performance of programs, processes, and resources. Here at the U.S. Department of Education, we do our best

to support the education community, including teachers and administrators, on how to understand and appropriately use data. We also strive to present relevant and accessible information that increases demand for educational attainment and improves educational performance, while maintaining student privacy. Current data efforts include these initiatives:

Update Freedom of Information Act •

•

•

•

(FOIA) processes and procedures to promote timely and complete responses to all requests; Launch interfaces that give the public easy access to data, including information on the performance of states, districts, and schools; grant awards; and promising practices;Develop a centralized college completion website that provides information on state completion efforts and higher education strategic plans; and

Implement and improve National Center for Education Statistics public data tools that support real-time creation of tables and estimation of statistical models via the

Web and improve transparent public access to statistics.

Data become powerful

when you analyze and

process them to inform

decisions.

We want to inform decision making by facilitating the development of interoperable state data systems that capture student data from early learning through the workforce. While the Department has provided funding for the development of such systems, the long-term value comes from state and local commitments to fund and use the systems. This will only happen if states and districts fully understand the benefits of data and how they can be used above and beyond compliance requirements to improve education and productivity.

Like other educators, magnet leaders should see data use as integral to their program management decision making. Data can inform everything you do, from improving your marketing and recruitment to enhancing your magnet theme and continuously building capacity in your staff.

This newsletter provides information that will help you understand the importance of data and think about using data to improve your Magnet Schools Assistance Program project through transparency and accountability. Alone, data are just numbers and facts. Data become powerful when you analyze and process them to inform decisions that lead to program and student success.

This quarterly publication is produced by the Magnet Schools Assistance Program Technical Assistance Center (MSAP Center), a technical assistance resource for MSAP grantees and the general magnet schools community. The MSAP Center provides grantees and magnet schools with technical support by offering tools, information, and strategies to assist in planning, implementing, and sustaining programs. The ultimate goal of the MSAP Center is to help magnet schools provide communities with educational opportunities that promote diversity, academic excellence, and equity.

www .msapcenter .com

DIRECTOR

EDITOR-IN-ChIEf

Manya Walton

CONTRIbUTINg WRITERs

Elizabeth FordNancy Balow

gRaPhIC DEsIgN

Yoshiye Anjos

EDITORIaL sERVICEs

Synergy Enterprises, Inc.

The views expressed in the newsletter do not necessarily reflect the position or policy

of the U.S. Department of Education (ED) and no official endorsement by ED should be inferred.

This newsletter was produced in whole or in part with funds from the U.S. Department of Education under

contract number: ED-OII-10-C-0079.

october 2012 • page 2

Learning to Value DataBuilding common understandings among staff about program goals, practices, and data use can help you foster informed decision making in everyday applications. These principles will help instill the importance of effective data use in your magnet staff and keep data at the forefront of your program management processes.

october 2012 • page 3

Understand the work is an ongoing process.Effective data use is more a continuous conversation than an end goal. Keep refining and updating

common understandings about what the data say and how data can support teaching and learning. Take a long-term approach to data analysis by examining data from year to year. Drill down into the data through subset analyses, and make data comparisons by using cross-tabulations.

As staff members’ understandings of data increase, so will their abilities to make decisions about using resources and strategies for program success. Your decisions about policies and plans, as well as changes in students, staff, technology, and practice, will help set new priorities and create reasons to collect and analyze new data.

Have data inform everything you do. Once staff understand and start to use data, have data inform all decisions—professional

development, curriculum and theme integration, community involvement, marketing and recruitment. Conduct data checkups with district and school leaders, and examine important trends to hone your magnet program vision throughout the year. Demonstrate to staff how data contribute to everyday program management.

Provide opportunities for professional learning.Administrators, teachers, and other magnet school stakeholders have different understandings about

data use and diverse opinions about effective practices. Role-specific, ongoing professional development in individual and small-group settings can build knowledge and skills around data use and decision making. Additionally, establishing professional learning communities can help expand data conversations and support a data-use culture. The communities can be informal settings where staff members connect with each other to ask questions, share knowledge and experiences, and form new relationships.

Make sure everyone understands the purpose and value of data collection and everyone has specific goals to meet. Universal participation ensures that you consider all data needs that exist within the magnet program, and that everyone begins to feel involved in decision making, which generates buy-in.

Sustainability Through Data Usethe Magnet CoMpass is presenting a series of articles that feature exceptional, innovative researchers. This article features Deidra Honeywell, Ph.D., and Elaine Ranieri, President and Evaluator, respectively, at DKH Consulting Services, Inc. Both women have years of experience working with magnet programs at the school and district levels. They share strategies magnet leaders can use to collect and interpret data to make program management decisions.

toward, it can be hard for magnet administrators to lead change. Magnet leaders should, “develop a written action plan with each school and meet with the school regularly to evaluate progress,” Honeywell said.

Communicating with other district offices that handle data and magnet applications can help. “It is very important that project directors work closely with the district data office,” Honeywell said. Without communication, “data may not be meaningful because the data person is not familiar with the goals and requirements of magnet programs,” Honeywell continued.

Magnet leaders should check all data sent by the district accountability office and by the schools “to make sense of

Honeywell and Ranieri work closely with the districts they evaluate to ensure that magnet leaders collect clear, useful data that can be used to determine progress toward implementing the magnet program.

Magnet leaders should collect data in several areas, including marketing and recruitment, staff professional development,

and curriculum. “Data should measure progress against the goals set out in grant applications” or other founding documents, Honeywell said. According to Ranieri, “School walkthroughs by administration to observe best practices of magnet teaching” are a great data collection tool.

Having templates for data collection is also important to “ensure the clarity and consistency of data,” Ranieri said. “Provide continuing professional development on data use and the templates to all staff,” Ranieri suggested, so that everyone knows what data are being collected and why.

Data collection should be continuous. “Leading indicators are important,” Honeywell explained. “You can’t just look at the test data that come at the end of the year. You need to collect benchmarks throughout the year to measure progress and make projections about year-end achievement.” Similarly, “Don’t wait until the lottery takes place to look at applicant pool data,” Honeywell suggested. Looking at applicant demographics as the applications are received will enable magnet leaders to modify the marketing plan in real time if targeted racial groups are not well represented.

Communication is key to effective data use. “I can’t emphasize enough the importance of communication with the schools by the project director,” Ranieri said. “School-level staff must know the purposes and goals of the magnet program.” If staff do not know the goals they are working

the numbers,” Ranieri said. Honeywell also suggested that magnet leaders “work with the district office that handles the magnet lottery to receive regular updates, perhaps weekly, on applicant demographics to continuously inform the marketing plan during the application period.”

To ensure magnet program sustainability, leaders should “write a sustainability plan, monitor it regularly, and evaluate it annually,” Ranieri said. “The key elements to sustaining a magnet program are desegregation goals, application demographics, professional development, and themed curricula,” Honeywell explained. “Data should be collected in these elements to ensure progress toward sustainability,” she said.

Elaine RanieriEvaluator

Deidra HoneywellPresident

october 2012 • page 4

Collect benchmarks throughout

the year to measure progress and

make projections about year-end

achievement.Deidra Honeywell

october 2012 • page 5

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

Using Diverse DataHaving data from a variety of sources and perspectives can support better decisions about magnet program management. Some of these suggestions for data collection and use will be more familiar than others; all can help you think about data use.

Topic area Questions to ask Types of data

student achievement are all of our students achieving aYP, and are there patterns among students who are not?

are students succeeding in our school and beyond?

aYP scores

graduation rates

student portfolios

student surveys and interviews

Marketing and recruitment What schools do our students come from, and where do we lose them to?

What barriers exist for other groups to attend the school?

feeder patterns

Retention rates

Neighborhood characteristics

applicant numbers and demographics

family and community engagement

how does the community feel about our magnet program?

how can we engage families further?

feedback from community organizations

family interviews

hours of parent involvement

Profiles of potential partners

Curriculum and theme integration are we adding new or improving existing theme-based curriculum each year?

Do we offer the courses we set out to in our founding documents?

Curriculum maps and standards

Enrollment in theme-related courses

Class observations

Teacher and student evaluations

Program management Do all staff receive relevant training and have access to required resources?

Do all staff understand the purpose and goals of our magnet program?

staff certifications

hours and types of training by individual and by school

staff surveys and interviews

staff observations

Leading Effective Data UseMagnet leaders who engage in data-driven decision making must ensure that data are also effectively used at each magnet school. Here are steps for defining and leading data use at the school level.

october 2012 • page 6

Start with the school mission statement.Work with school leaders to write or refine clear goals and measurable objectives for the school’s

mission statement. Set up a review process to ensure the support of the entire school community. Examples of measurable objectives would be, “The magnet school will provide theme-based arts instruction in all core content areas for grade 3.” “Adequate yearly progress (AYP) scores for students in all racial and ethnic groups will increase by 2 percent.”

To model this process, write a broader mission statement, with associated goals and objectives, for your entire magnet program. For this, you will collect data to demonstrate success in larger magnet goals, such as increasing school choice and lessening districtwide segregation.

Set benchmarks for progress.Establish benchmarks for each objective so school leaders can monitor progress toward meeting goals. Create benchmarks that show, for example,

the amount you want AYP scores to increase each year, or the number of students in each racial and ethnic group that you want to add to your student body. Data from sources such as standardized assessments and student surveys, as well as conversations with parents before, during, and after school, can provide a holistic examination of how well the program is doing.

Make evidence-based decisions and set priorities.Work with school leadership teams to interpret the data and to identify areas of success and areas for improvement, as well as reasons for each. If

you find, for example, that the theme is not present in the core content areas of all the grade 3 classrooms, put a team together to consider why it is not present, what existing approaches can be expanded, and how to develop additional creative ways to integrate the theme into the core curriculum.

Likewise, if the school has not attracted as diverse a student body as intended, review your marketing and community outreach data to see how you might do a better job of reaching out to diverse family groups. Work alongside school leaders so they understand the data and become comfortable with using the findings in program management decisions. Then set new benchmarks for a new round of data collection.

Collaborating on Data Usethe Magnet CoMpass is presenting a series of articles spotlighting personnel from magnet school districts and programs that have effectively implemented innovative ideas or research-based practices. This article features Anna Kucaj, Project Director at Metropolitan Nashville Public Schools (MNPS) in Tennessee. She discusses using data to make decisions about managing the magnet program in her district.

At MNPS, Kucaj collects a large amount of data that can inform programmatic decisions. Collecting data continuously in the areas of professional development, curriculum, and recruitment supports ongoing decision making.

For curriculum and theme integration, “We monitor how much thematic instruction

our kids are getting throughout the year so if we’re low in a grade level or a content area, we can address that. We constantly

Anna KucajProject Director

october 2012 • page 7

benchmark and make sure we’re on track to make our year-end goals,” Kucaj explained.

“By tracking professional development hours by school and also by teacher we’re able to see where we’re having the most impact,” Kucaj said. In addition, within each school, “the instructional designers do formal and informal surveys to figure out the professional needs in their building.” After collecting those data, “we address it in our professional development plans and in our budgeting process.”

“In terms of our recruitment data, we’ve started identifying our feeder patterns.” Kucaj and her team can pinpoint “schools we get a steady stream of out-of-zone students from, and schools we lose in-zone students to.” Once they identify these schools, they work with the principals to either “create a more concrete feeder pattern or to examine why we are losing students and where they are going. We have conversations with kids, parents, and schools, too,” she said.

Looking at the data, “informs all of the conversations we have and all of the decisions we make,” Kucaj said. “We try to embed

it in everything we do. That way we will always be able to point back to something and say, ‘This is why we’re doing this.’”

Another key to effective data use is collaborating across the district, according to Kucaj. She works closely with the student assignment, communications, purchasing, and business offices in her district. “What we have tried to do is show that this is a collaborative effort and that we’re all trying to move this project forward together. It’s absolutely necessary that everybody be on board.”

Data inform all of the

conversations we have and all

of the decisions we make.

Anna Kucaj

Working with other departments is also useful because those departments can provide data you did not know were available. “It’s very labor-intensive to collect all these data yourself, so instead of trying to reinvent the wheel, talk to the other groups in the district. Figure out what’s already being collected and how you can build on that most effectively. If you still have a gap, then go out and build a system to collect something more.”

Magnet leaders must work with “principals, leadership teams, and teachers” to ensure that effective data use trickles down to the school level. Kucaj works closely with principals, guiding them through the process of “driving their marketing and instructional decision making based on data.”

Kucaj summarized, “At the end of the day, everybody in the schools and the district office is here for the students, so keeping that at the forefront of our decision making is important. Effective data use is all about building capacity to better serve students.”

More Informationfor more about data-driven management, visit http://msapcenter .com/resource .aspx .

BibliographyAmerican Institutes for Research. (2003). Using NRS Data for

Program Management and Improvement. Washington, DC: U.S. Department of Education, Office of Vocational and Adult Education, Division of Adult Education and Literacy. Retrieved July 16, 2012, from http://www.nrsweb.org/docs/Data_Use_Guidebook.pdf

Corrigan, M., Grove, D., and Vincent, P. (2011). Multi-Dimensional Education: A Common Sense Approach to Data-Driven Thinking. Thousand Oaks, California: Corwin Press.

Goldring, E., and Berends, M. (2009). Leading With Data. Thousand Oaks, California: Corwin Press.

Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., & Wayman, J. (2009). Using student achievement data to support instructional decision making (NCEE 2009-4067). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/PracticeGuide.aspx?sid=12

Wayman, J. C., Jimerson, J. B., & Cho, V. (2012). Organizational Considerations in Establishing the Data-Informed District. School Effectiveness and School Improvement: An International Journal of Research, Policy and Practice, 23(2), 159-178. Retrieved July 19, 2012, from http://edadmin.edb.utexas.edu/datause/papers/Wayman%20Jimerson%20Cho%20Org%20Considerations%20SESI%202012.pdf

Join Our Online Community of PracticeAttention members of the 2010 MSAP cohort!

Join the MSAP Center’s Magnet Connector to share experiences and solve problems with your magnet colleagues. The Magnet Connector is located on the private workspace at http://msapcenter.com. Each discussion topic is moderated by an expert who can help you find answers to questions and suggest useful resources.

Discussions have already started around curriculum and theme integration, program sustainability, and community partnerships. Good questions—and practical solutions—can come from any and every community member. We want you to get involved!

To join the Magnet Connector, ask your grant project director to get a login account for you from the MSAP Center. After you receive your username and password, log in, complete your profile, and start posting.

We look forward to hearing about what is happening in your magnet schools!

october 2012 • page 8

For technical assistance, contact the MSAP Center.

Call toll-free: 1-866-997-6727

E-mail: msapcenter@seiservices.com

Visit: www.msapcenter.com