As New York City schools grapple with the requirements of a new evaluation system, school staff told WNYC that they remain skeptical of the portion of the evaluation that holds teachers accountable for their students’ progress.

“This gets put in our lap and we know we have to deal with it, so we make the best of it,” said Christina DiZebba, an eighth grade humanities teacher at I.S. 289 in Manhattan.

"As we are making the best of it," she said, "we realize how some of it is completely ridiculous.”

The new evaluation system, now called Advance, is being rolled out fast, just months after the State Education Department announced the details for the New York City school system. It requires that principals base 40 percent of a teacher’s evaluation on student learning, with this portion divvied up evenly between state measures and local measures. Both the city's Department of Education and teachers union have called this new system fair, in part because the evaluation puts heavy emphasis on multiple classroom observations as well.

But, in interviews with WNYC, several teachers from across the system questioned whether or not the measures of student progress would reflect their actual work with students in the classroom. 

The Prominent Use of State Tests

State tests play a major role in the student learning portion of the evaluation, as outlined by a 2010 New York law. The state uses a complex formula to calculate a growth score, so that teachers are held accountable for their students' progress as opposed to performance. (The state created an animated video to explain the calculation.)

The use of test scores is controversial among educators, and a coalition within the teachers union has been vocal against it. Other school staff members support the idea, at least in theory.

"I think that it makes sense to have some component of the individual teacher’s rating be connected directly to how their students perform," said Brian O'Connell, principal of Scholars' Academy in The Rockaways, Queens. "The mathematics behind it, the statistics behind it and how to do it in a way that’s valid and fair to the teacher -- that’s something that I don’t have statistical capacity to articulate. But I think the concept makes sense.”

Now that the implementation phase is here, state tests are playing an even larger role at many schools where staff opted to have the other half of the student learning portion of the evaluation -- the local part -- also based on state tests. In order to use these tests as a local measure, schools had to calculate growth in another way -- such as looking at the growth only of students who perform in the bottom third.

Jason Zanitsch, a drama teacher at the High School for Public Service in Brooklyn, said he does not view test scores as an accurate measure of teacher practice and expressed concern over how teachers at lower-performing schools, which may have higher populations of special education students or English language learners, would fare with the new system.

He also questioned the evaluation system's effectiveness at high-performing schools like his, saying it could backfire.

"Because we’re a highly functioning school, and we’re using these school-wide measures, teachers who aren’t terribly effective are going to get high marks and are going to get carried along by the work of the whole school," he said.

New Tests For Some Teachers

There are not state tests for every subject and grade-level, however. Instead, there are now brand new assessments for these classes, such as social studies in middle school. New York City developed these tests -- in conjunction with city teachers, the D.O.E. points out -- for the new evaluation system.

DiZebba, for one, was not pleased that students at her school will be taking more tests “solely for the purpose of evaluating teachers,” she said. She has even taken on a new role of testing coordinator this year to manage the new assessments which will be given in the fall and spring so that schools can measure a student's growth in that subject.

(Christina DiZebba teaches humanities at I.S. 289.)

Schools could also choose a New York City test that they already use to assess students, such as the “Running Records” test to determine students’ reading levels. Or they could choose from an approved a list of outside tests, known as third-party assessments, like Advanced Placement exams.

A Dilemma For Teachers in Non-Core Subjects

Still, some classes don’t have any subject-specific tests at all.

Teachers in the arts, physical education and foreign languages, for example, will be rated according to tests in subjects that they do not teach, at least until assessments for these classes are developed.

Educators and D.O.E. officials have said that, while it is an imperfect system, aligning teachers in non-core subjects to tests in math, English, science or social studies could encourage more collaboration among school staff. But some teachers said that this method is unfair or, at the least, an inaccurate measure of their own teaching abilities.

Julius Adams, a music teacher at P.S. 98 The Douglaston School in Queens, said he does indeed collaborate with other teachers to incorporate subjects like English Language Arts or history in his music lessons.

“But I don’t teach the core curriculum," he said. "How does it make sense that what I teach, which has its own set of city standards, is impacted by what someone teaches in ELA and math?”

A computer teacher in Manhattan, who asked to remain anonymous, said he felt that 40 percent of his evaluation was "out of his control." He likened his situation to police officers rated according to whether or not crime rates drop in the neighborhoods they patrol but then rated according to crime rates in neighborhoods they never set foot in.

Both the D.O.E. and the teachers union have acknowledged that the situation for teachers in non-core subjects is less than ideal. 

"To me, it’s the main problem that has to be fixed when we’re doing our negotiations for a contract because we can change it in a collective bargaining agreement," said Michael Mulgrew, the president of the United Federation of Teachers. But he still called the new evaluation system fair and a good plan for teachers.

“We did fight for a system that was no longer just about one single person’s judgment on you -- which is the principal’s," said Mulgrew. "We wanted some objective measures.”

A D.O.E. spokesman, Devon Puglia, hailed the evaluation plan as "transformative" and inclusive of components that the city fought for.

“While we will face implementation challenges in our first year - it’s a huge departure from the pass-fail system used for the last eight decades – Advance will yield invaluable development tools for teachers and improved outcomes for students,” Puglia said in a written statement.

“When it comes to shared growth scores, there are benefits to teachers and students since it can encourage more collaboration in schools. But ultimately, this was a component of the state-mandated plan, and we’re working within its parameters,”  he added.

The Ineffective Rule

Although student test scores factor into less than half of the teacher's performance review, the state gives it extra weight. If teachers are rated “ineffective” on this 40 percent, they must be rated "ineffective overall."

In the city's pilot of the evaluation system, less than one percent of teachers received a rating of "ineffective" on the student growth portion of the evaluation while also receiving a rating of "highly effective" or "effective," according to the portion of the evaluation based on classroom observations.

All teachers who receive an ineffective rating will be able to appeal. The low rating also kicks into effect new supports outlined in the new teacher evaluation system, including "a teacher improvement plan."