Using Assessment Data Changes the Way Some Schools Do Business

by Sue Johnson and Kathy Bradford

Quality Assurance Review (QAR) Process Provides Valuable Lessons in Helping ALL Students Access the General Curriculum

The Quality Assurance Review (QAR) really seems to work. Schools that have used the QAR process report they have improved the performance of students with disabilities at three levels: classroom, school, and district.

QAR is an action research project in which educators have worked collaboratively to conduct school-based research to improve teaching and learning (see Figure 1). Schools involved in the QAR action research project have achieved improved student results by using research-based practices to guide the cycle of collecting and analyzing meaningful assessment data and then using the information to improve the achievement of students with disabilities.

This FOCUS on Results document will introduce the QAR process, which has been piloted in six Michigan schools since 2002 (see box on left). Although each school uses data in unique ways to fit its needs, all report that the QAR process has changed the way they use data and plan instruction (see Figure 1). As a result, these schools have learned some valuable lessons about how to help
teachers teach so children can learn.

Applying the Lessons Learned from QAR

Lesson One: Data analysis should be brought to the student level.

One major finding after the first year of this pilot was that, while the QAR was driving systems reform, this systems reform did not necessarily improve student performance. Even though the staff and parents at the schools were willing to learn new methods and work in new ways within the system, the students’ achievement didn’t improve.

As a result, QAR teams had to change the way they thought about data. Pilot schools began implementing the eight components of the QAR process at the student level (see Figure 2). They did this by using multiple sources of student data to measure a student’s present level of educational performance, which in turn determined the goal(s) and short-term objectives for a student’s individualized education program (IEP). Then the teams repeated the same assessment analysis to measure the student’s progress and make new decisions about each student’s instruction and accommodations.

Lesson Two: Data analysis should be done on comparable data.

It’s important to remember that data are information, not just scores. Too often in the need to make data-driven decisions, schools gather many different assessment scores that measure student achievement at a moment in time. Individually, these scores provide limited information about an individual student. Consequently, using this limited information often leads to subjective decisions about how to teach a student so the student will learn.

QAR teams overcame this barrier by using student data that could be compared; this comparable data guided instructional decisions about the student.

Unfortunately, it is unlikely to find three comparable sources of data results in each of the core content areas for any student. Therefore, QAR teams needed to develop an alternative—a process that would work for all teachers in all schools. Using lessons from existing research and the QAR process implemented in the six pilot schools, teams developed a data-driven decision-making model that improved student learning and focused classroom instruction (see Figure 3).

Lesson Three: Data analysis must focus on identifying universal skills

Too often educators think of “skills” as things like “multiplication” or “decoding,” or “using a noun and a verb to make a complete sentence.” These are academic skills.

Knowledge gained from this action research and other published literature suggested that students first needed to master certain basic universal skills before they could achieve academic, social, or daily living skills. By analyzing a student’s 12 incorrectly-answered assessment questions, pilot teams identified the basic universal skills the student must be taught to apply in order to learn content material.

Teachers came to recognize that curriculum content is the “what” that students must learn; it provides the context in which students apply their specific universal skills. These are the skills that build bridges for the critical thinking patterns the student needs in order to progress in the general curriculum (see Figure 4).

Project participants—including students, parents, and teachers—reported that students with disabilities found it difficult to transfer skills from one setting to another. Using this as a hypothesis for the QAR student case study (1) research, it was determined that students experience their disability throughout different educational and learning settings. Therefore, a student’s unique need to learn specific universal skills occurs throughout the general curriculum (each core content area) and all school, work, and home-related settings.

For example, historically, an IEP goal for a student with disabilities might have read, “Jimmy will learn to read at sixth-grade level” or “Jane will read three stories, two times a week.” In comparison, using the QAR process to identify and measure the universal skills a student needs to learn, an IEP team might write a quality annual goal that reads, “John will improve 80-100 percent in learning to apply, sequence, prioritize, and organize (2) using the third-grade-level expectations for English Language Arts from the Michigan Curriculum Framework. Progress will be measured each marking period based upon an analysis of three types of assessments: teacher made, textbook chapter tests, and curriculum-based.”

Focusing on the universal skills a student needs to learn transcends all learning environments. This focus also provides continuity for the student to learn and apply these skills in all settings and with four core content areas. The results received from the pilot schools indicate that identifying and measuring each student’s universal skills yields significant improvement for students with
disabilities.

Lesson Four: Collaboration is key to raising student achievement.

Using data effectively requires consistent collaboration. This collaboration needs to happen at two levels: 1) between the general education and special education providers and 2) between the school and home.

First, special education service providers explored new roles, becoming advisors and collaborators with classroom teachers. Special education providers in the QAR pilot schools facilitated collaboration with the student’s general education teachers to collect the multiple sources of student assessment data for analysis. Then together they reviewed assessments and analyzed the data to determine which specific universal skills the student needed to learn or improve. In addition, special education providers used the results to become “accommodation experts” to help classroom teachers identify precisely how to teach and observe these universal skills using the content in each core content area.

As the discussion between special education teachers and general education teachers became more collaborative and focused on the universal skills that the student needed to learn, classroom teachers were more likely to say, “I can do that!”

Second, pilot schools found new ways to collaborate with families. Collabora-tion between school and home has been, and remains, an important practice for most educators. However, using parent input as part of the data set to plan and measure the student’s continuous improvement is new! The QAR process empowers parents to be key partners on the IEP team, helping prepare the IEP and evaluate the results of the data-driven decisions.

In the pilot schools, parents/guardians were invited to look at and respond to the universal skills identified for improvement, based on an analysis of their child’s assessment data in school. They were invited to talk about how their child learns at home. For example, special education teachers asked questions like, “What seems to help your child do a job independently?” and “How does your child solve problems?” Parents provided additional information that became part of the data set used to develop a quality IEP and then applied this information in the classroom. At the conclusion of their child’s IEP, the parents/guardians were surveyed to evaluate whether they believed the school team valued and used information about their child’s learning at home when preparing and implementing the IEP. Survey results showed that parents/guardians indeed felt as if they were working members of their child’s IEP team.

Lesson Five: Using data to improve student achievement requires a new use of time and additional staff support.

Using assessment data to plan a quality IEP, monitor progress, and apply the results to continually improve student performance is not doing business as usual—it is groundbreaking school reform. Consequently, school-based teams need time to implement new practices and personnel development in order to build success.

Pilot schools provided data-driven personnel development to help teachers identify and compare existing assessment data that was available to them. Teachers also learned how to reduce academic skills to the basic universal skills students needed to acquire in order to progress in the general education curriculum. When they applied these lessons to their instruction in the classroom, teachers started seeing improved student performance reflected in better test scores. Once teachers started seeing results, they asked teams to expand the process to help more students—including those without an IEP.

Student assessment results were used to design future personnel development for the pilot teams, based on teacher response. In this way, personnel development responded to the needs of the staff to improve student performance. Then teams used a post test of the same student assessments to see if teachers had successfully implemented what they learned in the classroom.

In addition to personnel development, school-based teams needed time to implement new practices and develop support within the system. Gathering and analyzing comparable data can be frustrating because ideal testing vehicles do not exist. At times, the pilot schools found the process time consuming and difficult. In the beginning, some schools found the process required from 1.5 to 3 hours for each student. However, each time the process was completed, it became easier. School-based teams reported the time was cut to 30 to 45 minutes for each student.

By the second year, all the general education staff members were aware of the process. Each teacher had a prepared list of her/his assessments and was introduced to the universal skills. All involved teachers were working together to measure a student’s universal skills within the content area. As the process became more systemic, it became more streamlined, showing actual results—improving student performance.

When teachers, principals, and parents began seeing results, they knew the time had been well spent. Seeing students learning and reaching their goals generated excitement, which brought new teachers on board. Plus, preparing and implementing IEPs using QAR became a systemic process; school staff began to understand the process and were more willing to participate. School leaders even began to save time by identifying student assessment data they didn’t need. They eliminated some assessments and found new tools that provided more useful student information.

Firing Up Your Team to Use Data-driven Instruction

New tools and knowledge coming out of the QAR pilot schools can make the learning curve shorter for schools that are just starting to use data-driven instruction. However, school teams will still need administrators who will cheer them along, encourage honesty and risk-taking, and provide personnel development that focuses on improving student achievement.

Seeing students succeed should inspire staff and parents. At first, though, education leaders using data-driven processes need a powerful set of messages to share with their staff. When parents and staff ask, “Why should we do this?” here are a few reasons to offer:

• It works for kids, and QAR pilot schools have the results to prove it.
• It works for all teachers, creating a common language (e.g., universal skills, Michigan Curriculum Framework, assessment data.).
• It saves everyone time. You will waste fewer hours on methods that don’t work, and you’ll toss out assessments that don’t help you make decisions for students.
• It makes reporting easier. It became very easy for pilot schools to complete the questions asked by Education YES! and No Child Left Behind (NCLB). Plus, schools felt more confident about their answers, because they were based on real data.
• Schools will be more successful in making Adequate Yearly Progress (AYP). This is what new accountability measures are all about—helping ALL students succeed, no matter where they start.
• Using student-level data will transform your work. The QAR process has shown that when teachers are given methods that work, they’ll never go back to the old way of doing things.