Ensuring That All Groups of Students Matter In School Ratings
School rating criteria that are based on how schools are doing for all groups of students — including low-income students, students of color, students with disabilities, and English learners — can send a powerful signal that the achievement of all students matters, and that schools have a responsibility to serve all students well.
In recent years, however, many states put in place school rating criteria that don’t look at schools’ performance for individual student groups at all. As a result, schools are able to — and do — get high ratings despite low outcomes for one or more groups of students e.g., low-income students or students with disabilities). Instead of sending the signal that all students matter, these states’ rating criteria do the exact opposite.
Passage of the Every Student Succeeds Act (ESSA), which requires states to base school ratings on how schools are doing for all groups of students, offers an important opportunity to turn this trend around. But as states work to revise their accountability systems, they will face lots of pressure to give high ratings to as many schools as possible, and to count the results of individual student groups as little as they can. It will be up to advocates to push states to ensure that new rating criteria meaningfully reflect how well schools are serving each group of students and do not hide some students’ performance behind whole-school averages.
What does ESSA require?
ESSA contains two key requirements related to including performance of individual student groups in school ratings:
- School ratings must be based on how schools are performing for each group of students on each of the indicators that are part of the rating criteria.
- These ratings must differentiate any school that is consistently underperforming for any group of students on the indicators that are part of the rating criteria.
The law also requires any school that is consistently underperforming for any student group to take action to improve.
Making sure that all groups of students matter in school rating criteria: Key things to look for in your state’s system
While there are many ways that states could go about designing their school rating criteria, there are a few things that are especially important for making sure that these criteria reflect results for all groups of students.
Here are some key things to look for — and watch out for — in your state’s rating criteria.
1. All indicators must be disaggregated by individual student group.
Under ESSA, all indicators that go into a rating have to be disaggregated by student group. This doesn’t just mean that these results must be publicly reported; it means that the performance of each group of students on each indicator must matter in how schools are rated.
Under ESSA, all indicators that go into a rating have to be disaggregated by student group. This doesn’t just mean that these results must be publicly reported; it means that the performance of each group of students on each indicator must matter in how schools are rated.
Right now, many states don’t count results for individual groups of students in their ratings at all. Some of these states count results for a “supergroup” of students instead, typically combining some or all of what they often describe as higher need groups. Others count results for the lowest performing 25 percent of students in each school. Neither approach is a substitute for looking at results for individual groups of students. Looking only at outcomes for the lowest performing 25 percent of students, for example, enables schools to ignore inequities in access to advanced coursework experienced by many high-achieving African American students. And a supergroup that combines multiple groups of students ignores meaningful distinctions between the needs of — and civil rights protections afforded to — each group. It also allows one group’s results to mask those of another.
Other states only include results for individual student groups on some indicators. For example, a state might measure percentages of students achieving on grade level by student group, but it may only look at schoolwide averages for graduation rates and rates of participation in advanced courses. This is unacceptable. If something is important enough to count in a school rating for students overall, it’s important enough to count for each student group.
2. Rating criteria have to set the same performance expectations for all groups of students.
Rating criteria must make clear that all schools are required to reach the same performance expectations for all groups of students. In other words, the state cannot call a school “excellent” because its graduation rates for low-income students are in the top 10 percent of low-income graduation rates in the state. Instead, if the state chooses to define “excellent” as “in the top 10 percent of the state,” it must set that threshold based on graduation rates for the all-student group and then apply that same threshold to schools’ results for every group of students (see Figure 1).
3. Results for individual groups of students must affect a school’s rating.
Reporting results for every group of students is not enough; these results have to affect the rating the school receives. If they don’t, schools and districts will be able to continue ignoring inequities in opportunity and achievement and still get an A or B rating.
In recent years, many states justified their decision to assign ratings to schools based only on schoolwide averages by saying that they report results for individual groups of students, including whether schools met their goals for each group (see Figure 2).
Not surprisingly, however, everyone — educators, the public, and even most parents — focused all of their attention on the rating. And because schools were able to get A’s (or “excellent” ratings, or 5 stars) despite not meeting goals for individual student groups, these students’ outcomes simply did not matter (for anybody other than their parents).
So what should states do instead?
There are a number of ways that states could go about making sure that rating criteria are based on how schools are serving all groups of students. They include the following:
- States could calculate school ratings in a way that gives results for each group of students substantial weight. For example, a state could give equal weight to every student group (including the all-student group) when calculating school ratings.1 This means that if a school’s student population comprises four groups of students — Black, White, low-income, and students with disabilities — results for each of these groups would count for one-fifth of the school’s total score (making up four-fifths of the total), and the all-student group would count for the last fifth. This approach would send a clear signal that results of each student group matter.While this approach represents a big shift from what states are doing now, it’s actually fairly straightforward to implement. For example, many states currently assign ratings to schools based on an index that combines results across multiple indicators such as achievement, graduation rates, and student growth, for example) for the all-student group. To weigh all groups equally, these states could apply the same calculations to the results for each group of students in each school, and then average those scores together.
- To further increase transparency, and to clearly identify schools that are consistently underperforming across multiple indicators, states could go a step further. In addition to giving schools an overall rating, states could assign ratings to schools based on their scores for each group of students. In other words, in addition to receiving, say, three stars based on a total score that weighs all groups of students equally, the school in our example below might receive five stars based on its results for White students, but only one star for its results for students with disabilities (see Figure 3).
- Another approach states could take is to ensure that regardless of how the rating is calculated, if a school is consistently underperforming for any group of students, that rating is lowered by one or more levels. For example, consider a school whose schoolwide averages qualify for a four-star rating, but it is consistently underperforming for low-income students. Instead of four stars, this school would receive three.
Things to watch out for: If your state does one of the following, there’s a good chance it is not meeting the law’s requirement to include results for individual student groups in school ratings.
- The state allows schools to get high ratings A or B grades, “excellent” or “good” labels) even if they are consistently underperforming for one or more student groups. States will usually publish a list of schools identified for a certain type of federally required support and improvement. But lists aren’t enough: If a school appears on such a list, it should not receive a high rating.
- The state creates an “extra high-performing” school rating like an A+ grade). It doesn’t allow schools to get an A+ if they are consistently underperforming for a group of students, but it does allow these schools to get A’s.
- The state says that it will use results for individual groups of students to inform the improvement process, but not as part of school ratings. While the state absolutely should consider results of individual groups of students in the improvement process, these results must also be part of the rating. That’s because if schools can get high ratings even when they are consistently underperforming for a group of students, they will have little incentive to tackle that underperformance.
4. The state must define “consistently underperforming” for a group of students in a meaningful way.
ESSA requires school ratings to reflect whether a school is consistently underperforming for one or more student groups. The law also requires states to identify schools that are consistently underperforming for a group of students for targeted support and improvement. The definition of consistently underperforming, however, is left up to the state.
If your state takes the step of rating schools based on their results for each student group (as shown in Figure 3), then the state should use these ratings to identify schools that are consistently underperforming. For example, the state could say that any school that receives a rating of “needs improvement” (or a C grade, or 3 out of 5 stars) or lower for any group for two years in a row would be considered consistently underperforming for that group and would need to work with its district to improve.
If your state does not take this step, then it could base the definition of consistently underperforming on whether a school is meeting its improvement goals (see Setting Goals for Accountability fact sheet). For example, a school that misses its achievement or graduation rate goals for any group for two years in a row could be considered consistently underperforming for that group. This school should receive a lower rating and should be required to work with its district to improve.
Things to watch out for: While there is more than one possible way to meaningfully define consistently underperforming, there are some definitions that states should avoid.
- States should not define consistently underperforming based on the size of within-school achievement gaps. A school that is not doing well for its Latino students should have to take action regardless of how it’s doing for its White students. What matters most is the school’s results for each group, not how those results compare with other groups.
- States should not define consistently underperforming as “performing as badly as the bottom 5 percent of schools are for all students.” Doing so would mean that the only schools that would need to take action are those that are doing as badly for a group of students as the absolute lowest performing schools in the state are for all kids. Schools that are only slightly higher performing for all groups of students — say, at the 6th percentile — would get a free pass.
- States should not in any way cap the number of schools that can be identified as consistently underperforming for a group. In other words, states should not define consistently underperforming as the 10 percent or 15 percent (or 20 percent or 30 percent) of schools with the lowest achievement for any group. Doing so sends the message that results of individual groups of students only matter in those 10 percent or 15 percent of schools.
Clearly, advocates have a lot to watch out for. There are many moving parts in rating systems, and how they work together can be complicated to understand. Moreover, there is no one best way to design a rating system. But there is a worst way, and that is designing a system without the clear message that every group matters — that schools can’t be good unless they are good for every group of students they serve. So, throughout the process, the question for advocates must be: How clear is that signal?
