note: after a lot of comment, the final evaluation regulations were much improved, but some of my comments remain relevant. 
My main points: It's an expensive, unfunded mandate; there are costs in learning time and equity; benefits are questionable at best.

State Senator Patricia Jehlen

2nd Middlesex District
Medford, Somerville, Winchester and Woburn
Chair, Joint Committee on Elder Affairs
Vice Chair,Joint Committee on Education

To the Board of Elementary and Secondary Education:

Thank you for this opportunity to comment on the draft regulations for evaluation of educators.  The report of the advisory committee makes clear how difficult it will be to balance often conflicting needs.  It also stresses how important adequate resources are if the project is to be successful.  I urge the Board to carefully forecast the initial and ongoing costs to both the state and communities before making final decisions.  I understand there is $10 million in RTTP money for this process: perhaps a realistic budget could be presented, along with a proposal for how much the program will cost in the future.

35.03: Standards and Indicators of Effective Teaching lists many characteristics and indicators of good teaching.  If evaluation is based on these characteristics, supplemented where appropriate with more specific standards such as nurturing creativity in the arts and scientific inquiry in the sciences, this list could be helpful.  In order to make final comments, one would need to see the actual rubrics the Department will recommend, so there may need to be another comment period later, especially if the Department recommends weights for different characteristics and measures.

When parents and teachers at the Healey School in Somerville, which is undergoing an intensive restructuring process, were asked what they would most look for in a classroom, they chose as core values Excellence, Openness, Creativity, and Joy.   I hope that all of these will continue to be valued in rating teachers and schools.

35.06 Evaluation Cycle
35.07: Evidence Used in Evaluation.
35.09: Student Performance Measures

Unfortunately, the document in these sections becomes extremely narrow and prescriptive, requiring yet more tests –  two sets of “measures” for every student, every subject, every grade -- and stating that the outcomes of tests are significant components of evaluation.

While the goal of standardizing the evaluation process may be worthwhile, the Board should question the costs of these regulations, especially those requiring additional testing.

The Cost of Unfunded Mandates

The Board should request an estimate of the cost to local districts of implementing the new evaluation process.  In particular, the regulations mandate
(a) At least two state or district-wide measures of student learning gains shall be employed at each school, grade and subject in determining impact on student learning.
(b) MCAS Student Growth Percentile shall be used as one of the measures where it is available, and
(c) Additional measures comparable across schools, grades and subject matter district-wide as determined by the superintendent and approved by the Department may be used in conjunction with MCAS scores to meet this requirement, or when MCAS growth scores are not available.

Developing or purchasing “measures” for every grade and subject may be very costly.  Art, music, library, phys ed, as well as other subjects at every level will require new tests.  With budgets tight, the simplest solution may appear to be purchasing commercial tests, which may not align with state standards or school needs.  If the tests don’t change every year to sample different skills and information, the temptation to teach straight to the test will be great, especially with teacher evaluation pegged to it.

The proposal is that every educator will be subject to exhaustive requirements, at least every other year, no matter how long they have been teaching, or how inappropriate the measures are.  Indeed, one reason to make the measures every year is that they are so unreliable that results for the same teacher will vary greatly every year! 

There will still be other personnel, key to students’ education, whose contribution will be difficult to measure.  For example, many schools have a career ladder, with skilled teachers serving as teacher coaches.  If a class succeeds, who deserves credit: the teacher, the coach, the principal, the director of curriculum, the band leader who inspires students to arrive an hour early at school, the food service director who provides good breakfasts, the counselor who helped students cope with a classmate’s suicide, the sped personal aide who kept a student from being disruptive?  If the class isn’t successful, how is blame apportioned?

Principals and other administrators are already stretched thin with numerous other mandates in addition to their traditional duties.  The Board should ask for an estimate of the time required for a principal with perhaps 35 teachers to evaluate, most every year (assuming someone else evaluates specialists, sped teachers, lunch supervisors, etc.): pre-conference, review of test scores at the beginning and end of the year, observations, formative discussions.  This is in addition to all the other important duties of being an educational leader outlined in the document, as well as maintaining order.

School districts already face the challenges of trying to do more in response to other mandates, with fewer resources.  In my district, all four communities are below 2001 levels in total state aid, and two receive less Chapter 70 money.

The Mass. Business Alliance for Education reported in December that the foundation budget has not kept up with the actual cost of running schools, and that “from 2000 to 2007, spending on books fell by more than half and spending on teacher training by almost a quarter.” Yet teacher training must be part of the  improvement in educator quality envisioned by these regulations.

The MBAE also found that “over the 17 years since the Education Reform Act passed, there has been virtually no equalization in spending or state aid between rich districts and poor.” I hope the Board will consider this an important issue to address.

The Cost in Learning Time

The Board should request an estimate of how much class time is currently spent on testing, and compare that with how much was spent before NCLB, and with how much will be required under this regulation.

Many people believe than an important educational reform is increasing learning time.  Yet this proposal would decrease learning time and devote it to testing.  For example, if an art teacher has forty periods a year per class, and spends two of them on pre and post tests, the students have lost 5% of their learning time.  If there are two sets of tests, that’s 10%.  This is true for every subject, every grade.

Reading will apparently be tested for both classroom teachers and reading specialists.  Which will get the credit for gains, and which will be blamed for insufficient gains?  Will teachers reject cross-disciplinary teaching, if, for example, a social studies teacher learns that teaching students to write research papers will help students’ ELA scores but not their social studies scores?

There’s an old saying: “A pig don’t get fatter the more you weigh it."

The Cost in Equity

The Board should consider whether teachers will voluntarily seek to teach difficult classes, when their students’ achievement may depend on factors they do not control.

Secretary Reville’s recent article in Education Week, “Why Attention Will Turn to Non-School Factors,” clearly recognizes that the effect of teachers and schools on student achievement is “weaker and less reliable” than socio-economic status, poverty and mobility.

Many people believe that closing the achievement gap will require encouraging the most talented teachers to serve the most challenging students.  Yet, if evaluation and job security depend in large measure on student scores or even on “growth scores,”  why would any teacher who has several job offers choose to work in a school with poor, immigrant, or mobile students?

There are many examples of schools gaming the system already, improving test scores by somehow changing the student body.

The Questionable Benefits

Scholars generally agree that standardized, high-stakes tests are neither valid nor reliable for judging teacher quality. 

For example, “one study found that across five large urban districts, among teachers who were ranked in the top 20% of effectiveness in the first year, fewer than a third were in that top group the next year, and another third moved all the way down to the bottom 40%.”  (quoted in epi.org/publications/entry/bp278)

I know others have called this information to your attention, so I won’t belabor the point, but I hope you will review, for example, the testimony of leading testing experts to the New York Board of Regents, the report of the National Research Council, and cautionary reports on VAM from RAND Educational Testing Service and the National Academy of Education.

Another simple example of problems with over-reliance on tests: some districts, such as Somerville, have up to 25% turnover in students during the school year, making it difficult to compare beginning and ending scores.  If income students are excluded from scoring, teachers may be less likely to focus attention on helping them.

Students’ scores and growth scores are attributable to many factors, including personal experiences and family background, neighborhoods, their entire school experience.

Massachusetts already has schools and teachers that are among the best in the world.  There are certainly some teachers who need either improvement or removal.  Like me, you probably know some principals and superintendents who have been able to improve or remove ineffective teachers.  Is the massive effort proposed in these regulations the best use of our resources?

I hope the Board will consider reforms that have proven successful in other states and countries to:
-- attract talented people to the education profession
-- support beginning and experienced teachers and principals through mentoring and other means
-- evaluate schools and teachers by teams of visiting professionals

Right now, little about education reform would attract new teachers.  It is possible that these regulations could be amended to offer support, especially for new teachers; half of them leave within five years.  Evaluation based on flawed testing criteria could discourage potentially excellent educators from entering the field.

There are many possible improvements to schools.  With limited resources, we must choose among them and budget those resources. 

True reform will cost serious money.  Issuing costly regulations without new state aid will not help students and may harm them.

Recently, the Board, following the Commissioner’s recommendation, postponed requiring the history MCAS for graduation, because there is no funding for remediation.  This was an example of the Board acting responsibly in postponing a goal it values, which I urge you to repeat.  Accountability should be a two-way street.  We cannot require schools and teachers to achieve results for which we do not give them tools.

Thank you for your consideration of these and other comments.

Best wishes,