The New John King Math: 40% = 100%

This new school year will be the first time that teachers in NYC will be evaluated using the evaluation system imposed by State Education Commissioner John King. Under the new system every teacher will receive a score ranging from 0 to 100 and this number will determine if the teacher is deemed Highly Effective, Effective, Developing or Ineffective.

One of the most shocking parts of the new system is that teachers who are measured ineffective on the test-based component of the evaluation must be labeled ineffective overall, no matter what they receive from their principal based on the observation of their teaching (see the King decision, p. 37 and p. 38).

The test-based component, 40% of teachers evaluations, outweighs the observation component, which is 60%.

Here’s how this fuzzy math works out:

Commissioner Imposed Cut Scores for New York City 

Because of the mandate that 40% test based outcomes will outweigh the 60% observation outcomes, some teachers with high scores will be found ineffective, while teachers with lower scores will be effective:

Although these two teacher’s scores below are only slightly different in the first two components and their overall score is identical, one is ineffective, and the other is effective:

It gets worse.

It is quite numerically possible to get an ineffective overall score if only scoring ineffective in one of the testing categories, even rated effective by your principal:

The language in the King decision mandating that 40% = 100% comes from the original 2012 agreement orchestrated by Governor Cuomo between our state union and the state education department. For the rest of the state, the bands were set up numerically to ensure this result (with ineffective teachers only being given 0-2 points for the state and local testing portions of their evaluations).  This, however, created other problems, well-documented by the EdWize blog.

Summary:

• Neither score band (NYC or the original state bands) can keep highly inequitable situations from occurring. Teachers will be found to be ineffective when the categories into which they fall clearly indicate they are not.
• All of these problems emanate from the desire to make relative measures of student learning dominate evaluation while setting the ineffective range so wide that it spans from 0-64 points.

Alternative Race to the Top approved plan—Massachusetts

While MORE opposes the use of test scores in teacher evaluations under any circumstances, it is interesting to compare the headlong rush into mathematical craziness in our state with a more deliberate approach next door. The educator evaluation plan of Massachusetts was part of an approved Race to the Top application and it also received an ESEA waiver. It is substantially different in many regards, and does not have any of the above problems.

Educators in Massachusetts are not thrilled about their new plan: check out the Massachusetts Teachers Association website for more details about their position on the evaluation system. The evaluation law and its implementation manual are both online as well.  However, there ar some major improvements over New York’s system.

First, Massachusetts plan does not assign numbers, only categories to performance.

Second, it inspires professional practice. If you are a distinguished or effective teacher, and your student scores are fine, you create your own goals and a plan for self-improvement. Student learning growth falls into one of three categories, thus avoiding the need for numerical assignment. Test scores are a lens, they do not determine the evaluation.  They are a means by which teachers set goals and they determine their growth plan.

Third, scores will not be used until 2017, and then only after extensive piloting. The following quote is from the evaluation manual.

The Department recommends that districts develop plans for piloting the new assessments they are anticipating using as district-determined measures. Both small- and large-scale piloting offer an opportunity to assess the reliability and validity of measures and work out the logistical details of administering the assessments. 

Fourth, “impact on student learning” is based on trends over time, not one testing event. Again the following is from the manual.

Districts should also be aware that in Massachusetts, an educator’s impact on student learning is determined neither on a single year of data nor on a single measure of student learning. It must be based on a trend over time of at least two years, and it should reflect a pattern in the results on at least two different assessments

Finally, the plan is being carefully phased in according to the timeline below.

TIMELINE of Massachusetts Plan 

• Plan first submitted: September 2013.
• Pilot measures: 2013-14.
• Revise measures: September 2014.
• Begin administering district-determined measures to establish first year of trend data for at least 50 percent of educators: 2014-15.
• Administer district-determined measures to establish first year of trend data for remaining educators: 2015-16.
• Report ratings of impact on student learning for all educators: 2017.