Not content with the letter-grade system that the state uses to assess the quality of Arizona schools, Kyrene School District administrators have developed an assessment tool for tracking student performance in a way that enables them to address it more effectively.
The tool was unveiled at last week’s board meeting by Susie Ostmeyer, the district’s chief information and accountability officer, as the state Education Department released its letter grades for the 2017-18 school year.
Of Kyrene’s 19 elementary schools, 11 received an “A” and seven got a “B.” Among the middle schools, Altadena and Pueblo earned an A, and Centennial and Aprende received a B. Kyrene Middle and Akimel got a C. Among the high schools in Ahwatukee, Mountain Pointe High School received a B, while Desert Vista and Horizon Honors got an A.
In the state’s evaluation, an A rating means the school is “excellent,” a B means it is “highly performing” and a C means it is performing above minimal expectations.
Specifically, the A schools show distinguished performance on the statewide assessment in terms of having significant student growth and overall performance that is significantly higher than state averages. High Schools with A ratings also have high four-year graduation rates.
The B schools are those that have some combination of the A schools’ factors, while a C is awarded to schools that have “adequate performance but needs improvement on some indicators, including proficiency, growth or graduation rate,” according to state guidelines.
Going deeper into the numbers used to calculate those grades and by incorporating some data that the state education board refused to consider last year, Ostmeyer showed the board a broader and deeper look at Kyrene schools’ performance.
Ostmeyer’s analysis showed:
60 percent of Kyrene students passed the state’s math assessment, surpassing the statewide average of 41 percent and the charter school average of 49 percent.
59 percent of Kyrene students passed the state’s English Language Arts assessment, as opposed to 41 percent statewide and 50 percent among charter students.
The 74 percent passing rate for science exceeded the 60 percent passing rate for charter school students and the 51 percent statewide average.
Nine Kyrene schools scored in the top 10 percent of all Arizona schools for English Language Arts and math.
Over a three-year period, 17 schools improved their math results and 12 improved their results in English Language Arts testing.
All 25 schools have more than 60 percent of their students showing growth in English Language Arts, and 21 schools have more than 60 percent of their students showing growth in math.
The district also analyzed students’ performance in different categories so it could identify issues by race, whether students had special needs and grade.
Ostmeyer also noted that teachers and principals had already become familiar with much of the analysis.
“While it’s our first time to look at this publicly, it’s not the first time for our educators,” she told the board, saying teachers and principals began looking at the data “as soon as the results started to roll in in June.”
“Our principals have already presented 90-day action plans, which is incumbent on really having a healthy review of the information learned through the assessments and working with teachers,” Ostmeyer said.
The principals’ plans essentially outlined how they intend to improve student performance, particularly among subgroups where it may be lagging.
Overall, the state assessments measure students’ passage from one of four quadrants to a higher one. The four quadrants are highly proficient, proficient, partially proficient or approaching proficiency and minimally proficient or falls below expectations.
Ostmeyer noted, however, that much of the performance data for schools “doesn’t do a very good job of telling us the change that we’ve had with our children who have grown with us. It gives us the overall organization performance, which is a good thing.
She said the district’s assessment tools will enable teachers and administrators to measure student growth within those quadrants
“We want all of our students to experience that level of success, and so we definitely want to keep our expectations and our goals at the same level,” she said. “However, we also want to measure whether or not students are growing while they’re with us.”
She said the measurements that Kyrene has developed “allow us with a certain degree of accuracy to say how much growth did they receive within whatever quadrant they’re in. And based on that trajectory, how long is it going to take them to get to the proficiency level.”
Ostmeyer noted the district already created intervention teams to work with families of students who are performing below proficiency level and that growth within the four quadrants measures their success.
“Our goal is to obviously eliminate the gap that you see on these screens,” she told the board. “Not by decreasing the achievement of the students on the top of the graph, but by raising the achievement of the students who aren’t there yet.”
Meanwhile, Audrey Beardsley, professor at Arizona State University’s Mary Lou Fulton Teachers College, cautioned parents against putting too much emphasis on AzMerit test scores and school letter grades.
Proficiency on AzMerit tests is one of five quantifiable factors that affect Arizona’s school letter grades.
“There is a rule of thumb on this – the farther away from the classroom they are, the worse (these assessments) get for (grading) classroom level achievement,” she said.
Beardsley said that parents choosing schools should look at a host of variables beyond letter grades, including disciplinary data, attendance records, school safety and per pupil expenditures.
Beardsley, who specializes in educational policy, measurement and research methods, said one issue is the significant relationship between test scores and a school’s demographics.
She said poorer schools tend to do worse on standardized tests than more affluent schools and that researchers can predict test scores with a high degree of accuracy just by looking at a school’s demographics.
“There is a correlation between test scores and student demographics and risk variables,” she said. “The correlations are very strong to the point that we as statisticians can use those risk variables and predict 80 percent of the time how students will perform without them even the test taking place.”
Beardsley served on a technical advisory committee for Arizona State Board of Education that was tasked with interpreting data and advising the board on A-F letter grades and other board policies.
Members of the committee, which began meeting in October 2017, were appointed to three-year terms, but the committee was sun-setted earlier this year after less than six months.
Beardsley said that some committee members suggested that the board should take a more holistic approach to grading schools that looked at factors beyond test scores, including social services, safety and programs for specialized populations offered on campus.
However, the “state wanted to stick with the easy answer test score approach,” she said.
Kyrene board member John King noted that one of the problems with the state’s approach is that it uses one test to base a significant part of the grade on.
“This is a single point in time assessment,” King said. “If the kid’s having a bad day, it’s not a true representation of what the growth level is or where they are at. I’ve never been a big fan of this kind of thing. I like the way we do multiple tests and things like that.”
Beardsley said the risk in relying on test scores is that they are often skewed, and it is difficult to get all students to take the tests seriously – meaning negative performance on an AzMerit test is not always correlated with a lack of understanding of a subject.
She said current federal education policy under The Every Student Succeeds Act puts the onus of accountability on administrators, schools and teachers – not on students.
That leaves teachers and schools scrambling to motivate students to prepare for a test that does not affect their grades and they may or may not care about, she said.
She acknowledged that occasionally schools perform well on the test despite demographics that would indicate otherwise.
However, she said it is important to look critically at what these schools did to promote higher scores in order to determine whether the scores were a one-time outlier or a genuine trend.
“We see outliers and want to celebrate them, but we also want to investigate how does this occur and do these outliers persist?” she said.
Beardsley pointed to several common conditions that result in temporary bumps in standardized test scores, including a turnover in administrators and/or teachers at a given school.
“There could be a new principal that shakes things up and has extreme censuses (based on test scores),” she said. “As soon as those statements are made, that is when it becomes high stakes testing.”
She said that can lead to skewed test scores as teachers feel pressured to improve test scores and do things like teach from the previous year’s test.
“The more seriousness with which we approach the test, the more likely it is for artificial inflation,” she said. “That ensures (teachers) will do things, thinking they are doing them wisely but also things that may not be good for students, to keep their jobs.”
-AFN Staff Writer Wayne Schutsky contributed to this report.