ORDER

DAVID C. BURY, District Judge.

Teacher and Principal Evaluations (TPE)

On August 28, 2014, at the request of the Mendoza Plaintiffs, the Special Master filed a Report and Recommendation (R&R) related to TUSD's Teacher and Principal Evaluations (TPE). He recommended that the Court direct TUSD to submit Action Plans for dealing with the procedures for evaluating teachers and principals. (Doc. 1659.) Correspondingly, the Special Master had reported in his 2014 Annual Report, that the development of teacher and principal evaluation procedures was delayed. (Doc. 1641.)

TUSD took the position that it had developed such procedures and would begin training to implement them in SY 2013-2014, and the USP did not require the development of an Action Plan for this task, which would trigger USP review and comment by the Plaintiffs and Special Master. The Court found: "The teacher and principal evaluation procedures at issue in this R&R are new and will involve significant changes contemplated to implement the USP, such as basing evaluations on effective efforts to include, engage, and support students from diverse racial, ethnic, cultural, and linguistic backgrounds using culturally responsive pedagogy." (Order (Doc. 1760) at 3.) The Court found the express and unambiguous terms of the USP § I.D.1,1 therefore, required the Special Master and the Plaintiffs to have an opportunity to review and provide input regarding Teacher Evaluation Procedures and Principal Evaluation Procedures. The Court ordered TUSD to prepare an Action Plan and afford the Special Master and the Plaintiffs an opportunity to review and provide input prior to final scheduled approval by the Board of the teacher and principal evaluation procedures. Id. at 3-4, 6.2

A year to the day after the first R&R related to teacher and principal evaluations, on August 28, 2015, the Special Master filed a second R&R, again at the urging of the Mendoza Plaintiffs. The section of the USP at issue is § IV.H, which provides, with emphasis added as follows:

Mendoza Plaintiffs' concerns that prompted their request for the R&R. The Special Master filed a revised R&R addressing only those two issues: 1) whether TUSD agrees to reassess the effectiveness of the teacher evaluation process, if the Mendoza Plaintiffs agree to accept 10 percent, as proposed by TUSD, instead of 17 percent, as proposed by the Special Master, for weighting student surveys in teacher evaluations; and 2) whether teacher and student surveys should be weighted, aggregately, as proposed by TUSD at 10 percent in principal evaluations or at 17 percent as proposed by the Mendoza Plaintiffs and Special Master. (R&R (Doc. 1845)).

As noted by the Special Master: "[T]he two most important in-school influences on student outcomes are teacher and principal effectiveness. Moreover, these are interrelated. Any district's ability to foster improvement of teacher and principal performance depends on having good knowledge of the level of effective practice. Thus, teacher and principal evaluation plans are critically important to achieving the goals of the USP." (R&R (Doc. 1845) at 3.) This explains the context for the Mendoza Plaintiff's request for the R&R.

In the context of TUSD's objections to the Special Master's recommended 17 percent, TUSD refuses to "consider the weights assigned to surveys of teachers and students despite analysis of its own staff concluding that the impact of the [10%] weights identified in both plans will be negligible." Id. at 3, see also (TUSD Objection (Doc. 1853) at 1-3, 4-6.)

In its Objection to the R&R, TUSD informs the Court that it "agrees to review the efficacy of the weight assigned to student surveys as they inform teacher evaluation and does not address that issue further." (TUSD Objection (Doc. 1853) at 2.) This agreement resolves the first issue addressed in the R&R, but TUSD has agreed to do no more than it is required to do under the law, the USP, and the Court's January 6, 2012, Order. Cf., Fisher v. TUSD, 652 F.3d 1131, 1141-42 (9th Cir. 2011) (citing this Court's findings that TUSD failed to make the most basic inquiries necessary to assess the ongoing effectiveness of student assignment plans and failed to assess program effectiveness to address racial imbalances as impossible to reconcile with a finding that TUSD acted in good faith to cure vestiges of segregation to extent practicable); see Order (Doc. 1350) at 7-8, § III: Ongoing Oversight (requiring Special Master's oversite to include annual reports to include evaluation of the effectiveness of programs and provisions established in the USP and recommendations for further review or revisions to these programs and provisions); USP § X: Accountability and Transparency (requiring evidence-based accountability and incorporating the oversight role of the special master as described in the January 6, 2012, Order). Like all the USP provisions, TUSD must review them on an on-going basis for efficacy and, therefore, TUSD must review both the teacher and principal evaluation procedures and instruments especially given the arbitrary nature of the weights set here.

According to TUSD: "The parties are before this Court, incurring thousands of taxpayer dollars for the Special Master, TUSD and Plaintiffs to litigate a 7% weight allocation in principal evaluation instruments, when this decision should be left with TUSD." (TUSD Objection (Doc. 1853) at 2.) TUSD asks the Court to review this issue de novo, Fed. R. Civ. P. 53, and uphold the District's evaluation instruments if it "`conforms to the consent decree entered into by the parties and . . . is compatible with the Constitution.'" (TUSD Objection (Doc. 1853) at 4 (quoting United States v. South Bend Community School Corp., 511 F.Supp. 1352, 1360 (N.D. Ind. 1981)). Here, the Mendoza Plaintiffs and the Special Master assert the Teacher Principal Evaluation (TPE) procedures do not conform to the USP.

TUSD argues that if the Court sets the weight for student and teacher surveys for principal evaluations, it will be acting as a `super school board' by micro-managing a programmatic decision which should rightfully be left to the District. Id. at 4 (citing Anderson v. Canton Mun. Separate School Dist., 232 F.3d 450, 454 (5th Cir. 2000); Richmond Welfare Rights Org. v. Snodgrass, 525 F.2d 197, 207 (9th Cir. 1975); Swann v. Charlotte-Mecklenburg Bd. of Ed., 402 U.S. 1, 12 (1971) (citing Brown v. Bd. of Ed., Brown II, 349 U.S. 249, 299 (1955) ("School authorities have the primary responsibility for elucidating, assessing, and solving these problems; courts [] have to consider whether the action of school authorities constitutes good faith implementation of the governing constitutional principles.")) TUSD asserts that "the USP expressly affords TUSD discretion in this instance": to determine what is "adequate" weight to afford the teacher and student surveys in principal evaluations. Id. at 5.

The Court does not agree. The concept of adequacy, i.e., "being good enough," is not wholly discretionary. It is used in the context of principal evaluations, required pursuant to the USP, which provides that TUSD adopt a principal evaluation instrument using a rubric which gives weight to three things: 1) the efforts by principals to create school conditions, processes, and practices that support learning for racially, ethnically, culturally and linguistically diverse students; 2) the use of classroom and school-level data to improve student outcomes, target interventions, and perform self-monitoring; and 3) aggregated responses from student and teacher surveys which protect the anonymity of survey respondents. Of course, TUSD may consider other things such as state law requirements and other appropriate measures. (USP (Doc. 1713) § IV.H.) TUSD proposes the following weighting rubric: 1) Principal Performance (57%); Student Academic Progress (33%); SAI (the Arizona Department of Education Teacher Survey on Leadership (2%); TUSD Teacher Survey (4%), and TUSD Student Survey (4%). The rubric provides for 100 total points.

The Special Master explains that the State guidelines constrain the number of points that can be assigned to teacher and student surveys to 17% and that the parties agreed to attribute 10 points to student surveys in teacher evaluations. Plaintiffs Mendoza and the Special Master believe 10% is too low, but as noted above they agreed to it for this year contingent on TUSD's review of its efficacy to evaluate principal performance required pursuant to the USP. TUSD admits the 10-points are "negligible" in assessing a teacher's overall score. This is because the results of the Danielson observations, Principal Performance) are weighted the most heavily and at 57% will have the greatest impact on a teacher's overall score. Academic Growth represents 33% of the total model so that it can impact a teacher's overall score, but not necessarily determine the outcome. Attributing 10 points to the teacher survey will be negligible in assessing a principal's overall performance score. (Teacher Evaluation Scaling 2015 Model at 4.)

The same weights are proposed for principal evaluations, but here TUSD combines student surveys (4%) and teacher surveys (6%) to reach 10 percent. This is to account for the differing interactions between students, teachers, and principals. TUSD does not explain why teachers should not weigh in on a principal's performance the same as a student weighs in on a teacher's performance at 10 percent, with student surveys regarding a principal's performance being an additional weight.

Instead, TUSD challenges the Special Master's proposal that student surveys be given 7 points in addition to the 10 points for teacher surveys as a recommendation pulled out of "thin air" and unsupported and lacking any research basis. (TUSD Objection (Doc. 1853) at 6-7.) TUSD presents an affidavit from Dr. Leah Lavigne, Ph.D. which reflects that there is a lack of research suggesting any specific weight, higher or lower, would be an adequate measure of principal performance. Id., Ex. 1: Decl. ¶ 6.)

The Mendoza Plaintiffs object to the introduction of this new evidence and argument in what is essentially a reply brief. (Mendoza Plaintiffs' Motion to Strike (Doc. 1855) at 1-2 (describing procedural posture of TUSD's Objection to the Special Master's R&R which does not allow any response without further leave of the Court); cf. Lentini v. Cal. Ctr. For the Arts, 370 F.3d 837, 843 n. 6 (9th Cir. 2004) (well established rule that courts will not consider arguments raised for the first time in a reply); Gadda v. State Bar of Cal., 511 F.3d 933, 937 n.2 (9th Cir. 2007) (same). The Court finds that either weight, 10 or 17 percent, is not research based. (TUSD Objection (Doc. 1853), Ex. 1: Lavigne Decl. ¶ 6.)

The evidence reflects that "these [10%] weights were deliberately determined through the ELI meet and confer agreement that resulted from an approximate three month process of collaboration and negotiation with TUSD regarding the evaluation of principals." (TUSD Objection (Doc. 1853), Ex. 2: Daniel Erickson Decl. ¶ 3.) The ELI (Education Leaders, Inc.) is the collective bargaining voice for school principals, assistant principals, psychologists, and research project managers. As such, the ELI is concerned with the impact of these evaluations on a principal's livelihood, including eligibility for multi-year contracts and other employment benefits. Id. ¶ 10. While compensation for principals is a legitimate concern for ELI and, correspondingly, administrative-union relations is a concern for TUSD, employee compensation is not a factor for assessing the adequacy of the weight for teacher surveys in the context of the USP. Here, the weight afforded the survey data must be adequate to meet the student achievement and integration goals of the USP, i.e., adequate to improve teacher and principal effectiveness.

The Court, therefore, turns to TUSD's assertions that teacher and student surveys will not serve the purpose of improving the effectiveness of principals. The Special Master makes a strong argument when he asks: How, [without the surveys], will TUSD complete the "School Behaviors" portion of the Principal Performance evaluation, which requires the evaluator to determine what teachers and students believe and experience? But, TUSD responds that any increase in weight given to teacher/student surveys will necessarily reduce the 57% weight it gives to the Principal Performance Measure, which is an objective measure of principal performance as compared to evaluating principals based on subjective surveys. The Court has looked at the Principal Performance measures. It includes criteria which are equally subjective, such as two found in the very first rubric: whether the principal promotes the development of an inclusive school climate characterized by culturally responsive strategies or whether staff members do not see the principal as lead learner in the school. (R&R (Doc. 1845-4), Administrator Evaluation Rubric at CM/ECF page 3.) Depending on the type of evidence the evaluator chooses to rely on—the Principal Performance Measure will be more or less objective than subjective.

The Court understands an "objective test" to be one where the subjective judgment by the grader is eliminated, Webster's New Collegiate Dictionary (1973), i.e., relating to, or based on externally verifiable phenomena, Black's Law Dictionary 1103 (8th ed. 1999). In comparison, a subjective test is based on an individual's perceptions, feelings, or intentions, as opposed to externally verifiable phenomena. Black's Law Dictionary 1465 (8th ed. 1999). So, if the evaluator relies on his or her personal perception of a principal's attitude it will be subjective versus reliance on verifiable data such as counting the number of teacher workshops or student activities promoting cultural-diversity. Based on the record before the Court, TUSD does not prevail on its assertion that the Principal Performance measure is more objective and, therefore, more reliable than the Survey measures. The issue appears to be who should be making the subjective evaluations: teachers or the principal evaluators.4

The issue of whether principals should be responsible for preparing teacher evaluations is no longer in dispute, but the ELI complains that it is an inherent contradiction between the Special Master's determination that principals are not well-suited to perform teacher evaluations because principals have a desire to be "liked" by their teachers and his position that teachers be allowed to weigh-in regarding principal evaluations. The ELI argues that for the Special Master to be consistent, he should recommend that teachers should not evaluate principals.

The two evaluations are not analogous. The better analogy is the role of the principal evaluators, who directly supervise the principals, and their having responsibility for conducting the Principal Performance evaluations in the same way principals are responsible for the Teacher Performance evaluations. In the same way that principals want to be liked by the teachers they supervise, which can cause principals to be overly generous in their teacher evaluations, TUSD's Elementary and Secondary School Directors, who supervise principals, may be compromised in the role of evaluator to critique principals. This argument only suggests the importance of the student and teacher surveys, which are anonymous, and provide a double-check to the Principal Performance measurement completed by the Directors. Students and Teacher surveys are not interchangeable given the differing nature of the relationships they hold with principals.

In conclusion, the Court finds that given the importance of evaluating the efforts by principals to create school conditions, processes, and practices that support learning for racially, ethnically, culturally and linguistically diverse students, there must be adequate weight given to the opinions of students and teachers regarding these conditions, processes, and practices. The Court cannot find that 10 percent, a weight which is admittedly negligible in the overall evaluation of a principal's performance, is adequate. The Court must, however, consider TUSD's question: "When the USP specifies no specific percentage and there is agreement to the 10% weight afforded student surveys in teacher evaluations, how in the world can the 10% weight assigned in the principal evaluation instrument to student/teacher surveys amount to a `clear' violation of the USP?" (Objection (Doc. 1853) at 5.) The answer is that it is an arbitrary weight. The evidence reflects that it was arrived at in large part because the ELI would not agree to more. (R&R (Doc. 1845-10) at Ex. G: TUSD's Alignment Response at 2.) Additionally, the State Teacher Survey (SAI), which accounts for 2 of the 10 points, is generic and its questions are not aimed at identifying efforts by principals to create school conditions, processes, and practices that support learning for racially, ethnically, culturally and linguistically diverse students. (R&R (Doc. 1845-9) at CM/ECF 314-316.)

The Court accepts the 10 percent for the limited purpose of this year, pending further review and evaluation for efficacy related to accomplishing the goals of the USP. Because of the findings outlined above, the Court will not adopt TUSD's proposal to aggregate the weight of the teacher and student surveys. By TUSD's own suggestion: if 10 points are given to student surveys in respect to teacher evaluations, the same should apply to principal evaluations. For now, teacher surveys shall be 10% of the principal evaluation, with student surveys weighing at least half that, i.e., an additional 5% of the principal evaluation.

Accordingly,

IT IS ORDERED that the R&R (Doc. 1845) is adopted in part by the Court as follows: Teacher surveys shall be 10% of the principal evaluation, with student surveys weighing at least an additional 5% of the principal evaluation.

IT IS FURTHER ORDERED that the Clerk of the Court shall correct the case docket to reflect that the R&R (Doc. 1836) is not pending because it was replaced by the R&R (Doc. 1845).

IT IS FURTHER ORDERED that TUSD shall undertake further review and evaluation of the adequacy of these weights to effectively measure teacher and principal performance in the context of USP.

IT IS FURTHER ORDERED denying TUSD's request to strike portions of the Special Master's R&R, and therefore, the Motion to Strike (Doc. 1855) filed by the Plaintiffs Mendoza is DENIED AS MOOT.