STATE OF FLORIDA

DIVISION OF ADMINISTRATIVE HEARINGS

MARYANNE S. SWEENEY, )

)

Petitioner, )

)

vs. ) CASE NO. 86-0023

)

RALPH D. TURLINGTON, )

Commissioner of Education, and ) the STATE BOARD OF EDUCATION, )

)

Respondents. )

)

RECOMMENDED ORDER

Pursuant to notice, an administrative hearing was held before Diane D. Tremor, Hearing Officer with the Division of Administrative Hearings, on April

15 and 16, 1986, in Clearwater, Florida. The issue for determination in this proceeding is whether petitioner is entitled to the designation of Associate Master Teacher retroactive to the 1984-85 school year.

APPEARANCES

For Petitioner: Ronald G. Meyer

Meyer, Brooks and Cooper, P. A. 911 East Park Avenue

Post Office Box 1547 Tallahassee, Florida 32301

For Respondents: Barbara Staros Harmon

Deputy General Counsel State Board of Education Knott Building Tallahassee, Florida 32301

INTRODUCTION

In support of her position of entitlement to endorsement as an Associate Master Teacher, petitioner testified in her own behalf and also presented the testimony of Dr. Betty Fry, John Nicely, Bill Williamson and Dorothy Cheatham. Petitioner's Exhibits 1 through 15 were received into evidence.

The respondent presented the testimony of Dr. Donovan Peterson, accepted as an expert in the area of teacher evaluation; Dr. Betty Fry, accepted as an expert in the development of instructional programs with emphasis in training and utilizing evaluation systems; Anne Wilson; Nancy Zambito; Dr. Robert S. Soar, accepted as an expert in the area of developing measures of classroom behavior and relating those measures to student outcome; and, by deposition, Dr. Harold E. Mitzel, accepted as an expert in the area of teacher evaluation.

Respondent's Exhibits A through E, G through K, and M through P were received into evidence.

Subsequent to the hearing, the parties filed proposed findings of fact and proposed conclusions of law. To the extent that the parties' proposed factual findings are not included in this Recommended Order, they are rejected for the reasons set forth in the Appendix.

FINDINGS OF FACT

Upon consideration of the oral and documentary evidence adduced at the hearing, as well as those facts stipulated to by the parties, the following facts are found:

1. The petitioner has been employed with the Pinellas County school system since 1969. She is currently an instructor and department chairperson at Clearwater High School. In the 1984-85 school year, petitioner taught Introduction to Data Processing and COBOL Programming, Levels 2, 3 and 4, to sophomores, juniors and seniors.

   
   

2. Along with twelve to fifteen other teachers at Clearwater High School, petitioner applied for the 1984-85 designation as Associate Master Teacher. The Master Teacher Program requires candidates to receive superior scores on both a subject area examination and a performance evaluation. Both components require scores at or above the seventy-fifth (75th) percentile on norms for candidates tested or evaluated. Petitioner received a score in the 86th percentile on her subject area examination.

   
   

3. For the performance evaluation, each candidate must have two observations by approved observers who utilize an approved performance observation instrument known as the FPMS (Florida Performance Measurement System. Summative Observation Instrument. To become an approved observer, one must undergo a period of training involving approximately three days. The training programs entail extensive instruction in the research base of the FPMS, how to use the Summative Observation Instrument and in coding procedures.

   During the training session, the observers are given practice sessions with video taped lessons and then feedback on their coding accuracy. The observers are trained as data collectors and are not taught to score the observation instrument. This scoring function is performed at the University of South Florida's Teacher Evaluation and Assessment Center (TEAC). The three observers in this case had all successfully completed the training session and were approved observers for the State Master Teacher Program.

   
   

4. The three persons who observed the petitioner had a part in observing

   12 Master Teacher Program candidates for the 1984-85 school year. Of those 12 candidates, 30 percent passed the performance evaluation component. This was slightly higher than the state norm of a 25 percent passage rate.

   
   

5. Petitioner's initial two performance observations were conducted on February 20 and 21, 1985, in her Introduction to Data Processing course which she had taught for five years. In preparation for her observations, petitioner created a detailed lesson plan which she believed contained the requirements necessary to receive a passing score on the FPMS. She was not personally trained to code or score the Summative Observation Instrument. Petitioner utilized the same lesson plan for both observations, though a different group of students were being taught on each day.

   
   

6. Assistant Principal John Nicely, an approved observer, utilized the FPMS Summative Observation Instrument to conduct the petitioner's performance

   evaluation on February 20, 1985. This was the first or second evaluation he had conducted after receiving his training as an observer for the Master Teacher Program. Mr. Nicely felt that his training had been adequate and that she was competent to mark the positive and negative indicators required on the Summative Observation Instrument. He felt that he marked the instrument in accordance with what he had observed. His personal opinion of the class which he observed was that petitioner was well prepared, presented a very good lesson, and that student learning was taking place during that class.

   
   

7. Bill Williamson, the then Principal of Clearwater High School, and an approved observer, conducted the performance evaluation of petitioner on February 21, 1985. He felt that he had received adequate and comprehensive training and felt confident in his ability to record the behaviors he observed on the Summative Observation Instrument. Mr. Williamson's personal opinion after observing petitioner's class on February 21, 1985, was that she had done a nice job and had given a very fine lesson.

   
   

8. The observation instruments completed by Mr. Nicely and Mr. Williamson were sent to TEAC for scoring, and the average of the two scores was formed with the scores of approximately 20,000 candidates for the 1984-85 Master Teacher Program. As the result of the two initial evaluations, petitioner received a score of 18.92 percent on the performance component of the Master Teacher Program. Upon her request, petitioner's scores were hand verified, which did not alter the 18.92 percentile score.

   
   

9. The rules governing the Master Teacher Program provide a procedure for candidates who are dissatisfied with their initial performance scores to request reevaluation by a third observer. Petitioner made such a request and was reevaluated by Assistant Principal Dorothy Cheatham on August 29, 1985. Ms. Cheatham was an approved observer of the FPMS, having attended the three-day training session, as well as a review session. Believing that her training was adequate, she felt confident to complete the Summative Observation Instrument. Ms. Cheatham had conducted four Master Teacher Program observations prior to observing petitioner in August. She felt that she had observed an effective lesson in petitioner's class.

   
   

10. Following the reevaluation performed by Ms. Cheatham, the three observation scores were averaged and petitioner's revised score improved to the

    32.44 percentile. Pursuant to Rule 6A-4.46(5)(b), Florida Administrative Code, a candidate's Principal may recommend to the Department of Education that the candidate receive up to ten additional evaluation points. Mr. Williamson recommended that petitioner receive all ten points. This recommendation was approved, resulting in a final percentile score on the performance evaluation component of 53.17, still below the required 75th percentile.

    
    

11. Each of the three approved observers of petitioner's classroom performance were surprised and could not understand or explain how their completion of the observation forms could have resulted in such a low score for petitioner's performance evaluation. Each believed that they had correctly completed the Summative Observation Instrument, yet each felt that the low score did not accurately reflect the level of teaching they had observed. The three observers were trained to utilize the "intervening behavior" or "intervening episode" rule, which means that a teaching behavior that continues is recorded a second time only if there is an intervening behavior or episode. Petitioner's observers utilized this rule in evaluating her performance on February 20 and 21, and August 29, 1985. It does appear from the testimony that at least some of the petitioner's behaviors with regard to her use of audio visual equipment

    throughout her lesson may have been undercoded. However, there is insufficient evidence to establish how many such instances occurred or how their omission may have affected her overall score or ranking.

    
    

12. The cover page of the Summative Observation Instrument contains a section entitled "Instruction Format/Frame Factors." Petitioner's three observers did not recall specific indepth instruction during the training sessions as to the proper manner of completing this portion of the form. They circled different methods of teaching for petitioner's observed lessons. Mr. Nicely indicated a combination of lecture, discussion, recitation and interaction. Mr. Williamson indicated a combination of all methods used in the classroom. Ms. Cheatham indicated a combination of lecture and independent study or work. While the form of instruction a teacher utilizes during a performance evaluation can result in a point differential in the norming of a candidate's score, the total effect is small. The score adjustment was not determined to be great enough to justify norming separate norm groups based upon instructional format.

    
    

13. The FPMS was developed to be used for teachers of all levels and in all subjects, with two norm groups, elementary and post-elementary. It is a low inference observation instrument, designed to collect objective data and to be scored against a norm group. As opposed to a high inference or rating instrument, the FPMS does not require the observer to make a judgment or opinion about the teacher's performance At the time the data is collected and recorded, no scores are entered on the instrument. Indeed, observers are not taught to score the instrument; they are trained as data collectors. The evaluator's purpose is to code what the teacher does and not what the evaluator thinks of the teacher. The observer records data concerning the teacher's behavior in the classroom by simply describing the numbers of times they see a listed behavior. The data recorded is then compared with all other teachers who are being scored on the same instrument. With low inference instruments, the only judgment made is after the scoring or recording of data is completed. At that time, the norm table is determined and the teachers are placed in rank order. Low inference evaluations based upon structured observations produce more useful data than high inference or rating evaluations in relating teacher performance with student achievement. Indeed it has been determined that the use of high inference instruments in evaluating teacher performance bears little or no correlation with student behavior and student achievement.

    
    

14. The Pinellas County School Board utilizes a high inference rating system to perform the annual evaluations for instructional personnel. The main purpose of this evaluation, while it contains some of the same items as the FPMS, is feedback and improvement of the teachers for employment purposes. Unlike the FPMS, whose purpose is to recognize superior ability, the Pinellas County evaluation system does not compare teachers when ratings are given. In contrast, Master Teacher Program candidates are compared to each other statewide to determine whose teaching performance is superior. For the 1984-85 school year, approximately 47 percent of the teachers at Clearwater High School received the highest rating of "exceeds expectations" in the instructional area. Another 49 percent received the next highest rating designation of "meets expectations." Petitioner's annual performance evaluations during her years of employment with the Pinellas County School Board have been consistently above average. For the 1984-85 school year, petitioner was evaluated by Dorothy Cheatham and received an overall rating of "exceeds expectations" in the area of instructional performance.

15. A three-member team of outside evaluation experts reviewed the FPMS for compliance with the criteria specified in Rule 6A-4.46, Florida Administrative Code. While the team had some recommendations for improvement and suggested certain areas of caution in future use, it enthusiastically endorsed the use of the FPMS for the merit program. The team found the observer training program to be effective and impressive. Testing results have indicated a good distribution of awards among the various disciplines and instruction modes and indicate that the FPMS was appropriately normed.

    
    

    CONCLUSIONS OF LAW

    
    

16. The petitioner contends in this proceeding that her three performance observers were not properly trained in the use of the Florida Performance Measurement System and did not properly complete the Summative Observation Instrument, and, therefore, her "normed" evaluation score is not reliable. In addition, petitioner urges that since more than 90 days had elapsed between the filing of her application (a date never established in the record of this proceeding) for designation as Associate Master Teacher and the date of notification of approval or denial of that application, she is entitled to the designation, along with its financial award, as a matter of law.

    
    

17. As with any applicant for a license or certification, the petitioner in this case bears the burden of demonstrating that she has met the statutory and regulatory criteria for endorsement as an Associate Master Teacher. This concept is also set forth in Rule 6A-4.46(10)(b), Florida Administrative Code. Here, the petitioner has attempted to demonstrate that her high annual evaluations indicate her superior teaching ability, that she prepared and followed her lesson plans on the days of her classroom performance observations in a manner that would demonstrate her superior teaching ability and that, because of inadequate training of her observers and/or because the FPMS is not a reliable evaluation instrument, she is entitled to the designation.

    
    

18. The evidence demonstrates that the ratings received on the Pinellas County School Board's annual evaluation instrument are in no way comparable to the scores received under the FPMS. The former is a method of evaluation without measurement. It does not attempt to differentiate between the quality of performance by different teachers. Its purpose is to provide feedback to teachers and to identify areas needing improvement or change for employment purposes. The latter FPMS is a system of measurement followed by comparison with norms. Only then does evaluation and identification of superiority occur, which is the purpose and intent of the Master Teacher Program. Section 231.533, Florida Statutes; Rule 6A-4.46(1), Florida Administrative Code. The results of the 1984-85 annual performance evaluations at Clearwater High School are evidence that that type of evaluation could not be utilized to achieve the intent of the Master Teacher Program of rewarding teachers of superior ability, defined as those in the 75th and above percentile of those evaluated. At petitioner's school, 47 percent of the teachers received the highest rating, while 49 percent received the next highest rating. Thus, 96 percent of the teachers in that school were evaluated as either meeting or exceeding the expectations of the Pinellas County School Board. No comparisons between teachers were made and the Pinellas evaluation system is incapable of discriminating at the 75th percentile as is required to implement the statute allowing awards to be given for superior performance.

    
    

19. Petitioner has not been trained to either complete or score the FPMS Summative Observation Instrument. Thus, her attempt to devise and intent to follow a lesson plan which would assure her of a superior performance evaluation

    result does not lead to the conclusion that she is entitled to a passing score. The approved observers were not trained to predict the outcome of their observations, and it is unlikely that the teachers observed could plan and follow a lesson which would predict and assure a superior rating.

    
    

20. Petitioner's contention that her three observers were not properly trained in the administration of the FPMS is likewise not substantiated in the record of this proceeding. Instead, the evidence demonstrates a thorough training program with extensive training materials for both the observers and the training of observer trainers. Mr. Nicely, Mr. Williamson and Ms. Cheatham each felt that they had received adequate training in the FPMS and were confident and comfortable when completing the Summative Observation Instrument while observing petitioner's classroom performance. They each were of the opinion that they had accurately recorded what they had observed during the class period. The fact that they each marked a different method of instruction utilized by petitioner could well be explained by the fact that they each observed a class with different students. In any event, the evidence establishes that the instructional format coded by the observer does not affect the candidate's score to any great degree. There is nothing further in the record to indicate that if the instruction format had been coded in any other manner, petitioner would have received a passing score. Likewise, even if there were some undercoding regarding the petitioner's use of audiovisual equipment, there is nothing in the record to indicate what the extent of that miscoding may have been. The observers correctly understood the concept of "intervening behaviors," and it cannot be determined from this record that that concept was misapplied to the extent that petitioner would have received a passing score.

    
    

21. Petitioner relies upon her observers' subjective opinion regarding her teaching performance to support her claim that they were either untrained or miscoded the Summative Instrument. Yet, the elimination of personal judgment on the part of the evaluator is the prime purpose of the low inference instrument chosen to be utilized for the Master Teacher Program. The instrument is designed so it does not require the observer to make a judgment or opinion about the candidate's teaching performance.

    
    

22. The petitioner failed to prove that the FPMS is not a reliable evaluation system of recognizing superior teaching ability. Rule 6A-4.46, Florida Administrative Code, implements the Master Teacher Program and prescribes the manner of evaluating performance. That rule is not being challenged in this proceeding, and it therefore is presumed valid. The evidence demonstrates that the method of training observers and the FPMS meet the criteria set forth in Rule 6A-4.46 and are in keeping with the goals of the Florida Master Teacher Program. Petitioner presented no evidence to demonstrate that the norming of the scores or the distribution of the awards did not comply with the rule or statute. Likewise, petitioner has failed to show that the rule was inappropriately or unfairly applied to her.

    
    

23. Finally, petitioner's contention that she is entitled to a retroactive designation as an Associate Master Teacher, based upon Rule 6A-4.46(10)(a), Florida Administrative Code, and Section 120.60, Florida Statutes, is without merit. The cited rule requires the Commissioner of Education to inform the candidate that his or her application has been granted or denied within 90 days of receipt of the completed application. The cited statute contains a similar time requirement for license applicants. However, the application procedure for certification as an Associate Master Teacher encompasses more than the filing of an application. It also encompasses the submission of verification of a superior performance evaluation, along with other information. Therefore, the

    90-day clock did not begin to run until petitioner's submission of her performance score, which, by her request, could not have occurred until after her third observation by Ms. Cheatham in late August. She was timely notified that she had failed to score in the 75th percentile after that observation.

    Petitioner's reliance on the recent decision in Lanier v. Turlington, 11 FLW 1066 (Fla. 1st DCA, 1986) is totally misplaced. The facts reported in that decision indicate that certification as an Associate Master Teacher was denied solely on the basis of an incomplete initial application form; to wit: the failure to include the applicant's social security number on the application form. Notice of that deficiency was untimely provided to the applicant. The Court held that the master teacher certification process was a licensing process governed by the time requirements set forth in Section 120.60(2), Florida Statutes. However, the Court specifically held that the failure to timely notify the applicant results only in the inability to deny certification based on the omission in the application, and the applicant was permitted to pursue certification as if her application had been complete. The Court noted the prescriptions of Section 120.60(2) that failure to comply with the time limitations results in the application being "deemed approved . . . subject to the satisfactory completion" of required examinations and other prerequisites for certification.

    
    

24. Here, petitioner was never denied the right to take an exam or to be eligible for the FPMS performance evaluation. That evaluation process was not complete until after her third observation conducted in late August, 1985. Petitioner was timely notified of the results and, having failed to achieve a score in the 75th percentile, she could not complete the application process. She therefore is not entitled to certification or designation on the grounds that she was not notified of a grant or denial of her application within 90 days.

    
    

25. In summary, it is concluded that petitioner has failed to demonstrate that Rule 6A-4.46, Florida Administrative Code, was not fairly and properly applied to her in the performance evaluation component of the Master Teacher Program, and has not otherwise proven that she is entitled to a retroactive award of the designation for the 1984-85 school year.

RECOMMENDATION

Based upon the findings of fact and conclusions of law recited herein, it is RECOMMENDED that the petitioner's amended petition seeking the granting of the Associate Master Teacher Endorsement and the $3,000.00 merit incentive award be DENIED.

Respectfully submitted and entered this 28th day of July, 1986, in Tallahassee, Florida.

DIANE D. TREMOR

Hearing Officer

Division of Administrative Hearings The Oakland Building

2009 Apalachee Parkway

Tallahassee, Florida 32301

(904) 488-9675

Filed with the Clerk of the Division of Administrative Hearings this 28th day of July, 1986.

APPENDIX TO RECOMMENDED ORDER IN CASE NO. 86-0023

The proposed findings of fact submitted by the parties have been carefully considered and are accepted and/or incorporated in this Recommended Order, except as noted below:

Petitioner

1. The date of application is rejected as unsupported by any evidence.

7. Last sentence rejected as contrary to the greater weight of the evidence.

15. The words "gave the same lesson" rejected as unsupported by competent, substantial evidence.

1. Rejected as unsupported by competent, substantial evidence, and that portion relating to Williamson's understanding of the coding rules is rejected as contrary to the evidence.

2. Partially rejected in that it was not conclusively demonstrated that undercoding occurred.

   20 and 21. Rejected as irrelevant and immaterial to the issues in dispute.

   23 - 26. Rejected as unsupported by competent, substantial evidence.

   
   

   Respondents

   
   

3. Rejected as immaterial to any issue in dispute.

22 and 26. Rejected as constituting a legal conclusion as opposed to a factual finding.

27. This proposed finding is supported by the evidence, but is not particularly relevant to any issue in dispute.

COPIES FURNISHED:

Ronald G. Meyer, Esquire

Meyer, Brooks and Cooper, P. A. 911 East Park Avenue

Post Office Box 1547 Tallahassee, Florida 32301

Barbara Staros Harmon Deputy General Counsel State Board of Education Knott Building Tallahassee, Florida 32301

Ralph D. Turlington Commissioner of Education The Capitol

Tallahassee, Florida 32301

Judith Brechner General Counsel

Department of Education Knott Building Tallahassee, Florida 32301

Docket for Case No: 86-000023

Orders for Case No: 86-000023