BRUCE K. BARR, )
vs. ) CASE NO. 82-2326
Respondent. )
This matter came on for hearing in Jacksonville, Florida, before the Division of Administrative Hearings, by its duly designated Hearing Officer, Robert T. Benton, II, on January 20, 1983. Petitioner appeared on his own behalf. Respondent was represented by counsel:
For Petitioner: M. Catherine Lannon, Esquire
Assistant Attorney General Department of Legal Affairs The Capitol, Room 1601 Tallahassee, Florida 32301
After petitioner received a failing grade on the clinical portion of his examination for a dental license, he instituted formal administrative proceedings. The Division of Administrative Hearings received a copy of the transcript of these proceedings on January 28, 1983. Respondent's proposed recommended order was filed on February 14, 1983. Time for filing petitioner's proposed recommended order was extended, by agreement of the parties, and petitioner's proposed recommended order was filed on March 29, 1983.
The issue is whether the examination was unfairly graded, and if so, whether petitioner would have passed, if it had been graded fairly.
Petitioner Bruce K. Barr almost passed the clinical portion of the December 1981 examination administered to applicants for Florida dental licenses. On a scale of 0 to 5, he scored 2.992, rounded to 2.99, the merest fraction below the lowest possible passing grade, 3.0.
In the course of the clinical portion of his examination, Dr. Barr performed various dental procedures, each of which was evaluated by three and, in one instance, five examiners. An average score was computed for each procedure and these scores were used to calculate another weighted average,
which was the final score awarded Dr. Barr on the clinical portion of the examination.
The following chart, based on joint exhibits B and E, reflects the scores Dr. Barr received from each examiner for each clinical procedure, reflects the average score calculated for each procedure, and indicates the weight given each procedure in calculating the weighted average of 2.99.
Procedure Scores Grade for Weight
Procedure
Endodontics
Posterior 4,3,3 3.33 7.5 percent
Cavity Prep. Final | 4,4,3 | 3.67 | 13.3 | percent |
Restoration | 3,3,2 | 2.67 | 6.6 | percent |
Anterior 4,3,3 3.33 7.5 percent Amalga
Denture
Occlusal R & Pressure and | T | 2,2,2 | 2.00 | 10.0 | percent |
Articulation |
| 4,3,3 | 3.33 | 10.0 | percent |
Peridontal
Cavity Prep. Final | 3,1,1 | 1.67 | 10.0 | percent |
Restoration | 3,2,2 | 2.33 | 5.0 | percent |
Evaluation 5,4,4,3,2 3.60 20.0 percent Cast Gold
Pin Amalgam
Prep. Final | 3,3,3 | 3.00 | 6.6 | percent |
Restoration | 3,3,2 | 2.67 | 3.3 | percent |
The weight to be given each procedure is specified by Rule 21G-213(3), Florida Administrative Code.
Dr. Barr contends that the 2 he received from one of the examiners who evaluated his periodontal work, the 2 he received from one of the examiners who evaluated the final amalgam restoration he performed, one of the two 3s he received from examiners who evaluated the denture pressure and articulation procedures he performed, and the 2 he received from one of the examiners who evaluated his pin amalgam final restoration were improper for various reasons.
All of the clinical examiners were licensed as dentists in Florida and none had practiced less than five years. After they had been selected as examiners, they gathered for an all-day standardization session to "fine tune [the criteria] and come to a consensus about how they [we]re going to grade." (T. 62) At this session, the examiners applied the "criteria in a full mock examination. . ." (T. 62) Department heads from the dental school of the University of Florida participated in the standardization exercises (T. 81). The grade of 3.0 was chosen to represent "minimally acceptable."
In no case did one examiner know what grade another examiner had given. In an effort to ensure uniformity in grading, two additional examiners were asked to evaluate a procedure, whenever any two of the first set of three scores were separated by three or more points. When additional examiners were assigned to a procedure, they were not told how many other evaluations had been performed, although circumstances were sometimes such that they could deduce that they were not among the first three examiners to evaluate.
DOCUMENTATION
The examiners were furnished a form for each evaluation of each of the procedures. More than 17,000 evaluations took place in connection with the December 1981 Examination. On the forms were listed the criteria to be applied and "canned comments" pertaining to each procedure. The "comments" section on the periodontal evaluation form, for example, read as follows: "0-No Comment; 1-Stain; 2-Supra-gingival Calculus; 3-Root Roughness; 4-Sub-gingival Calculus; 5-Tissue Management." The numbers were to permit coding so that the form comments could be read by a machine and do not correlate to any particular score. Examiners were asked to indicate on the form a grade for each procedure they evaluated and, for each procedure which they gave a failing grade (2.0 or lower), they were asked to assign a reason. Whether they made comments on procedures to which they gave grades of 3.0 or better was left to their discretion.
PERIODONTAL EVALUATION
Because of the three point spread between the 2 and the 5 he received from two of the three examiners who originally evaluated respondent's periodontal work, two additional examiners were asked to make evaluations. All five scores were then averaged, in keeping with the procedure applied in every such case. Examiner No. 36 assigned a grade of 5 and indicated, "No Comment." Examiners Nos. 37 and 71 each assigned a grade of 4 and indicated, "Sub-gingival calculus." Examiner No. 72 awarded the procedure a 3, noting root roughness and sub-gingival calculus. Examiner No. 5 assigned a grade of 2, noting sub- gingival calculus and "Tissue Management." The person on whose teeth petitioner performed the periodontal procedures had moderate roof roughness and "pockets," extensive calculus above and below the gum line, and extensively stained teeth. Respondent's Exhibit No. 3. It was a difficult assignment, and the examiners were so advised. Tissue mismanagement, if any, was not such as to justify a failing grade. Petitioner has had extensive training and experience in periodontics, which is his specialty.
AMALGAM RESTORATION
Examiners Nos. 5 and 72 each assigned a grade of 3 to petitioner's "final amalgam restoration," indicating problems with "functional anatomy" and "proximal contour." Examiner No. 36 gave this procedure a grade of 2, noting the same problems as the other examiners had indicated, and, in addition, "light contact" and a problem with "margin." Light contact refers to the resistance dental floss met when inserted between the filling and the adjacent tooth; and insufficient resistance could be characterized as a problem with "proximal contour."
As for the pin amalgam, final restoration, all three examiners noted problems with functional anatomy. Examiner No. 37, who gave this procedure a grade of 2 wrote out "innocclusion" on the form. The other examiners assigned a
grade of 3 but examiner No. 71 noted a problem with "proximal contour" and examiner No. 36 noted a problem with "margin."
DENTURE PRESSURE AND ARTICULATION
All three examiners who evaluated petitioner's work on dentures commented on "[e]xtension" which relates to the fit. That was the only comment of Examiner No. 72 who gave petitioner a grade of 4 on this procedure. Examiner No. 71 who awarded petitioner a grade of 3 on this, noted a problem with "surface detail" in addition. Even at the time of hearing, there was some detail on the model made by petitioner in performing the required procedure for the examination. Examiner No. 36, who also awarded petitioner a grade of 3 for this procedure, indicated still other problems: "Pressure Areas" and "Distribution."
TEST DESIGN
The clinical portion of the examination prescribed for licensure as a dentist proceeds on the assumption that different clinicians' evaluations of an applicant's work will vary, even after the examiners have discussed criteria for each procedure and taken other steps toward standardization. Because disagreement is anticipated, three examiners evaluate each procedure independently of one another. Whenever there was more than a two point difference between grades awarded for the same procedure, two additional examiners were called in, in an effort to enhance the reliability of the grade for that procedure. In petitioner's case, there was a three point spread between two evaluations of his periodontal work and a two point spread between different examiners' evaluations of his cast gold cavity preparation, but no more than a single point disparity on any other procedure. The test design contemplates differences of this magnitude.
All these safeguards notwithstanding, the test also assumes that there will be errors with respect even to average scores on given procedures. It depends for its reliability on the probability that such errors will not all be in the same direction. Florida's clinical examination employs more examiners and more procedures than any other state's, and compensating errors should make it among the most reliable of examinations of its kind.
Proposed findings of fact and proposed recommended orders have been considered and, in many instances, adopted in substance. Otherwise they have been deemed immaterial or unsupported by the weight of the evidence.
CONCLUSIONS OF LAW
After respondent "set forth in writing the grounds or basis for [proposed] denial of a license," Rule 28-6.08(2), Florida Administrative Code; see Section 120.60(2), Florida Statutes (1981), petitioner had "the burden of establishing entitlement." Rule 28-6.08(3), Florida Administrative Code. See, e.g., Zemour, Inc. v. State Division of Beverage, 347 So.2d 1102 (Fla. 1st DCA 1977)
FAILURE DOCUMENTED
Petitioner contends that examiners checking precoded comments on a form does not comport with the requirements of Rule 21G-2.17(3), Florida Administrative Code, which provides:
Failure of an applicant in any clinical procedure shall be documented on the grading sheet by the examiner. Documentation shall be accomplished through the use of "comments" contained on the grade sheet. The "comments" section may contain any technical terms that define, illustrate or otherwise explain the criteria utilized in grading a particular procedure. For the purpose of experience and
brevity in grading, appropriate "comments" may be noted by recording "comments" on an optical scan field contained on the grade sheet. When there exists a discrepancy on the grade sheet between the hand written grade and the grade penciled in on the optical scan field and the discrepancy cannot be resolved by an examination of the grade sheet, the hand written grade shall be accepted as the
intended grade.
At the time petitioner made application for licensure, Rule 21G-2.17(3), Florida Administrative Code, provided simply that, "failure of a candidate in any clinical procedure shall be documented on a grading form in writing by the examiner." The rule assumed its present wording effective December 5, 1982.
The amendment of Rule 21G-2.17(3), Florida Administrative Code, in no way deprived petitioner of any substantive right. There is no impediment to giving the current version full effect even though it was adopted after petitioner's application was filed and after the examination took place. See State Department of Environmental Regulation v. Oyster Bay Estates, Inc., 384 So.2d 891 (Fla. 1st DCA 1980). Nor is there anything in Rule 21G-2.17(3), Florida Administrative Code, as it was worded when petitioner applied for licensure that precludes the use of forms with precoded comments, especially in view of the fact that the agency's practice reflected an interpretation of the rule authorizing the use of these forms. See e.g., City of Miami Beach v. Miller, 122 So.2d 578 (Fla. 3rd DCA 1960). Faced with evaluating thousands of procedures, the use of something like the forms petitioner has used is eminently reasonable. Even if there had been a deviation from some rule, petitioner has not shown how he was harmed, moreover.
FAIR TEST
Petitioner did not show that he was graded unfairly, with respect to any particular procedure. The same examiner who gave him a "2" on his periodontal work graded numerous other applicants' work, as well. Unremoved subgingival calculus, noted by four of the five examiners, can be an appropriate basis for a grade of 2. The fact that one examiner awarded a grade of 5 and the other a grade of 2 does not prove that either was wrong. Petitioner's average grade for that procedure was 3.6, his second highest average grade on the eleven procedures, which, even on petitioner's thesis that, because periodontics is his specialty, he had to do well on that aspect of the examination, indicates the test's reliability. His contentions with respect to the other procedures are speculative and unsupported by evidence. There was no proof that there was anything amiss in the grading of any particular procedure that worked to the detriment of petitioner, much less anything that was not compensated for by an unjustifiably favorable grade on other procedures.
Petitioner contends that respondent should have treated his score as a 3.0, since it was so close. Instead of rounding to the second or third decimal place, respondent might as well round to the first, petitioner argues. The result would be to lower the passing grade to 2.95, as a practical matter, and petitioner's grade would be a passing one. Petitioner made no evidentiary showing, however, that would support his contention that this approach would be preferable to the approach respondent has taken; and the rule plainly provides that 3.0 is the grade for a "[m]inimal acceptable dental procedure." Rule 21G- 2.13(1), Florida Administrative Code.
That respondent deny petitioner's application for licensure with leave to reapply.
DONE and ENTERED this 20th day of April, 1983, in Tallahassee, Florida.
ROBERT T. BENTON, II
Filed with the Clerk of the Division of Administrative Hearings this 20th day of April, 1983.
M. Catherine Lannon, Assistant Attorney General Department of Legal Affairs The Capitol, Room 1601 Tallahassee, Florida 32301
Bruce K. Barr, D.D.S
Fred M. Roche, Secretary
