STATE OF FLORIDA

DIVISION OF ADMINISTRATIVE HEARINGS

JUNE M. GARLAND, )

)

Petitioner, )

)

vs. ) CASE NO. 87-5135

) DEPARTMENT OF EDUCATION, )

)

Respondent. )

)

RECOMMENDED ORDER

Pursuant to notice, this cause was heard by Linda M. Rigot, the assigned Hearing Officer of the Division of Administrative Hearings, on April 1, 1988, in Miami, Florida.

Petitioner June M. Garland was represented by Bruce Lamchick, Esquire, Miami, Florida; and Respondent Department of Education was represented by Charles S. Ruberg, Esquire, Tallahassee, Florida.

Petitioner testified on her own behalf and presented the testimony of Jodi De La Torre and Joanne Eliot. Respondent presented the testimony of Dr. Arthur

1. Gilford and Dr. Thomas H. Fisher. Additionally, Petitioner's Exhibits numbered 1 and 2 and Respondent's Exhibits numbered 1 and 2 were admitted in evidence. By agreement of the parties at the final hearing, the evidence in this cause was supplemented by an affidavit of Dr. Arthur M. Gilford filed on April 11, 1988 and by Petitioner's Position as to Question Number 72 filed on April 25, 1988.

   
   

   Petitioner challenges five questions and her score on the Communication Disorders Examination of March/April 1986, administered in conjunction with the 1985-86 State Master Teacher Program. Accordingly, the issue for determination herein is whether Petitioner's answers should be deemed the best correct answers so as to require recalculation of the examination results.

   
   

   Both parties requested and were granted leave to file post-hearing proposed findings of fact in the form of proposed recommended orders. Those proposed recommended orders were required to be filed with the Division of Administrative Hearings no later than May 16, 1988. Petitioner's Proposed Finding [sic] of Facts was filed on May 23, 1988. Respondent failed to submit proposed findings of fact. By telephonic motion hearing on May 24, 1988, Respondent's are tenus motion to strike Petitioner's proposed findings on the basis that they were not timely filed was granted.

   
   

   FINDINGS OF FACT

   
   

   1. Petitioner is a speech-language pathologist employed by the Dade County Public schools. In April, 1986, she took the Communication Disorders Examination as part of her candidacy for the designation of associate master teacher pursuant to the 1985-86 State Master Teacher Program. She received a

      raw score of 90, which placed her in the 74th percentile on the examination. She challenges the best correct answers to questions numbered 72, 81, 95, and

      117 on that examination. She also believes that she should receive credit for her answer to question numbered 33, which question was deleted from the examination prior to the examination being graded.

      
      

   2. The examination was not a test for beginning teachers or a minimum competency test. Rather, the purpose of the examination was to recognize the superior ability of experienced teachers. Excelling on the examination would require experience in a public school setting. The examination was a state-wide examination, reflecting state-wide policy issues and state guidelines. Under no circumstances was one school district's manuals or policies allowed to control the selection of the best answer to each of the multiple-choice questions.

      
      

   3. The Master Teacher Communication Disorders Examination was administered for the first time in March/April of 1986. A task force of professionals in communication disorders was selected to prepare a test blueprint defining the various topics to be covered by the examination and the relative weights to be given each. The writing team, although highly experienced in examination preparation, received training on how to write examination questions in order that uniform standards be utilized by each member of that committee. Questions were written by various members of the committee and reviewed by all other members of the committee. The questions were further reviewed by staff members of the Institute of Instructional Research and Practice at the University of South Florida, the unit designated by the legislature to assist Respondent in the creation of examinations pursuant to the Master Teacher's Program. Additionally, some questions were taken from the Georgia Beginning Teacher Competency Examination which has been extensively validated.

      
      

   4. After the questions were approved by all members of the writing team and had been edited to conform to national examination guidelines, they were submitted to a validation team which independently reviewed each of the test questions in the test pool. The persons involved in both the development and validation of the test questions were practitioners in the field. After the questions in the test pool were further refined by the validation team, they were submitted to "keyers", more practitioners in the field to ascertain if those persons would all agree on the best correct answer. Those questions which made it through the editing, validating, and keying processes without disagreement as to the best correct answer remained in the test pool.

      
      

   5. Pilot tests were then administered in a further effort to validate each item on the examination. The end product was an examination composed of questions and answers with subtle distinctions in order to identify the advanced or top-level teacher for merit pay. Since it was a merit pay test, it was impossible to field test the examination prior to its first administration since the people who would likely be candidates to take the test would be the only persons with whom the examination could be field tested. Accordingly, the first administration of the test became the field test, and the processees that would normally be done in a field test took place in the first administration.

      
      

   6. The test was administered and then scored but no individual candidate's data was processed at that time. Each test item was first statistically analyzed on the basis of pyschometric standards in order to determine the examination's reliability and validity. Any test items with unusual statistics were removed from the processing. Thereafter, the candidates' raw scores were obtained.

   7. Petitioner challenges herein the deletion of question numbered 33 from the examination. Question 33 was deleted prior to the actual scoring of the examination. The decision to delete question number 33 from the examination was made prior to the first administration of the test since disagreement had developed during the development and validation phase as to the best correct answer for that question. Question number 33 was simply not scored. The deletion of question number 33 from the examination without scoring that question was proper and in accordance with professional testing guidelines.

      
      

   8. Petitioner failed to select the best correct answer to questions numbered 72, 81, 95 and 117 on the examination. Although Petitioner defends her answers to two of those questions by relying upon Dade County policies, guidlines for any particular county were properly irrelevant in the determination of the best correct answer to those questions on this state-wide examination.

      
      

   9. The Communication Disorders Examination of March/April, 1986, was a valid and reliable test. It was properly administered, and it was graded correctly.

      
      

   10. It cannot be determined that one more correct answer by Petitioner would have raised her score so that she would have been in the 75th percentile on the examination. The legislature mandated that only persons receiving scores in the top 25 percent would receive merit pay, not that anyone with a score of 75 or better would pass the examination. Once the scores were computed for each candidate, a percentile ranking was then established in order to determine those persons with scores in the top quartile. Petitioner did not rank within the top quartile. No evidence was offered to show that Petitioner's percentile ranking would be effected favorably by her receiving credit for any of her incorrect answers. It is as likely that Petitioner's percentile ranking would have been affected unfavorably or not at all.

       
       

       CONCLUSIONS OF LAW

       
       

   11. The Division of Administrative Hearings has jurisdiction over the subject matter hereof and the parties hereto. Section 120.57(1), Florida Statutes.

       
       

   12. Petitioner argues that question number 33 should not have been deleted from the examination since that adversely affected her score. Petitioner cites no authority for the proposition that a candidate has a right to have certain questions on an examination graded and included in that candidate's score. The evidence is clear that it was appropriate to delete question number 33 from the scoring of the examination since the question was faulty, and retaining it could not be justified pursuant to acceptable statistical guidelines.

       
       

   13. Petitioner next argues that she should have been given credit for her answers to four other questions since she believes her answers to have been good answers. The evidence is uncontroverted that two of her answers were selected pursuant to Dade County Public Schools' guidelines and not state-wide guidelines. While it could be argued that Petitioner did give good answers to those four items in dispute, Petitioner has failed to prove that she gave the best answer to those questions or that the committee's choice of best answer was wrong.

       
       

   14. Lastly, Petitioner's argument that one more correct answer would have raised her score so that she would have attained the 75th percentile is

       erroneous. This examination had an absolute cutoff so that only the top 25 percent of those taking the examination would be eligible for merit pay. If one of Petitioner's answers were deemed correct, then the entire examination would need to be re-scored to determine which of the other candidates answered that question in accordance with Petitioner's answer. Upon re-scoring, some scores would be higher, some scores would be lower, and the percentile ranking would necessarily be recalculated. Since the legislature required that the awards under the Master Teacher's Program will go only to persons in the top 25 percent of teachers, a ranking of candidates must occur. If the answer to a question is changed, the ranking of everyone taking that exam will necessarily change. It could be that Petitioner's percentile ranking would be lowered, and persons who received the award would have to return it. Since it cannot be determined from the evidence in this cause that giving Petitioner credit for any of her answers in dispute would affect her percentile rank favorably or unfavorably, Petitioner has failed in her burden of proof.

       
       

   15. The Communication Disorders Examination was a valid and reliable test; it was properly administered; and it was properly graded. Petitioner has failed to prove that her answers to questions numbered 72, 81, 95, and 117 were the best correct answers or that she should have been given credit for her answers to those questions. Finally, Petitioner has failed to prove that she ranked within the top 25 percent of those taking the March/April, 1986, examination.

       
       

   16. The parties have stipulated to security measures surrounding the questions and answers on this examination. The exhibits in this cause, which contain the examination and other confidential information, together with the transcript of this proceeding, which also contains several questions and answers from the examination, are being transmitted to the Department of Education with this Recommended Order so that their confidentiality can be maintained. The affidavit of Dr. Arthur M. Gilford filed on April 11, 1988; Petitioner's Position as to Question Number 72 filed on April 25, 1988; and Petitioner's Proposed Finding [sic] of Facts filed on May 23, 1988, also contain information as to the test content. Accordingly, those documents, which are being retained at the Division of Administrative Hearings as part of the record in this cause, are hereby sealed and will not be available for review by anyone without an order authorizing such review, examination, or copying.

RECOMMENDATION

Based upon the foregoing Findings of Fact and Conclusions of Law, it is,

RECOMMENDED that a Final Order be entered denying Petitioner's appeal of her raw score and percentile ranking on the March/April, 1986, Communication Disorders Examination.

DONE and RECOMMENDED this 13th day of June, 1988, at Tallahassee, Florida.

LINDA M. RIGOT, Hearing Officer Division of Administrative Hearings The Oakland Building

2900 Apalachee Parkway

Tallahassee, Florida 32399-1550

(904) 488-9675

COPIES FURNISHED:

Betty Castor, Commissioner Department of Education The Capitol

Tallahassee, Florida 32399

Bruce Lamchick, Esquire 10725 Southwest 104th Street Killian Parkway

Miami, Florida 33176

Charles S. Ruberg, Esquire Department of Education Knott Building Tallahassee, Florida 32399

Sydney H. MacKenzie, Esquire Department of Education Knott Building

Tallahassee, Florida 32399

Filed with the Clerk of the Division of Administrative Hearings this 13th day of June, 1988.

Docket for Case No: 87-005135

Orders for Case No: 87-005135