Findings Of Fact Dr. Farah is a candidate for licensure by the Board of Dentistry, having taken the dental clinical examination in June 1985. The examination covers ten domains of dental knowledge and practice; each is separately graded, and then weighted according to an algorithm. Rule 21G-2.13(3), Florida Administrative Code. A weighted grade of 3.0 is required to pass the clinical dental examination. Rule 21G-2.13(2)(c), Florida Administrative Code. Dr. Farah received a grade of 2.96. The June 1985 examination was Dr. Farah's second attempt to pass the clinical examination. The grading scale for each procedure is established in Rule 21G- 2.13(1), Florida Administrative Code, as follows: complete failure unacceptable dental procedure below minimal acceptable dental procedure 3- minimal acceptable dental procedure better than minimally acceptable dental procedure outstanding dental procedure An examiner is required to record a comment in support of any grade below 5. Examiners for the dental examination are experienced licensed Florida dentists. Rule 21G-2.20(4), Florida Administrative Code. They are trained by the completion of 8 to 10 hours of standardization exercises. During the standardization exercises the examiners receive examination grading criteria, grade identical procedures, discuss any grade variance and attempt to eliminate any discrepancies in interpretations of the grading criteria in order to bring the examiners to a consensus on grading. In the periodontal portion of the examination there are five criteria which are accorded equal importance in grading. These are: (a) presence of stain on the assigned teeth, (b) presence of supra-gingival calculus on assigned teeth, (c) presence of sub-gingival calculus on assigned teeth, (d) root roughness on the assigned teeth, (e) improper management of tissue such as gums which may have been lacerated during the procedure. Rule 21G-2.13(4)(b), Florida Administrative Code. The grading is holistic and each examiner assigns a grade based on the examiner's evaluation of the overall procedure. Three examiner's grades are averaged to obtain a final grade score for the individual procedure. Rule 21G- 2.17(1), Florida Administrative Code. The score for that procedure is then weighted and added with the other weighted scores to obtain the overall grade on the clinical examination. As a standardization technique in grading the periodontal exercise, an examiner marks off for root roughness when use of an explorer on treated teeth reveals a tactile roughness but the examiner is unable to visually confirm the presence of sub-gingival calculus. Use of an explorer reveals the presence of root roughness or calculus below the gum level (i.e., calculus which is sub- gingival). Dr. Farah was assigned teeth number 2, 3, 4, 12, 13, l4 and 15 on her periodontal patient. A prior candidate (Candidate 20057) had treated the same patient in her periodontal exercise, and had been assigned some of the same teeth as Dr. Farah, viz., teeth 2, 3, 4, 5, 6, 7 and 8. Candidate 20057 received individual holistic grades of 4, 4 and 5, which average to a grade of 4.33 for the periodontal procedure. Dr. Farah received grades of 1, 2 and 3, which average to a grade of 2.00 for the procedure. (Petitioner's Exhibit 8) Examiner #006 graded both Dr. Farah and Candidate 20057 on their periodontal treatment. That examiner gave Candidate 20057 a holistic grade of 4 (better than minimally acceptable), noting a deduction for "root roughness," but there is no indication on the grade sheet of the tooth or teeth on which roughness was found. Examiner #015 also gave Candidate 20057 a grade of 4, and noted "root roughness" on the mesial side of tooth number 7, which was not one of the teeth later treated by Dr. Farah. The third examiner gave Candidate 20057 a grade of 5 with no comments. (All comments are found on Respondent's Exhibit 3.) After Dr. Farah's treatment of the patient, which occurred two days after the treatment provided by Candidate 20057, Examiner #006 gave Dr. Farah a grade of 3, and recorded that he found sub-gingival calculus on the mesial side of tooth number 3. Calculus is a mineral deposit on teeth which does not form in 48 hours; Examiner #006 missed the calculus on tooth 3 when grading Candidate 20057 (perhaps because it was obscured by the inflammation and bleeding of the gums which the patient testified about at the hearing) or the calculus was on a tooth other than tooth 3, and the wrong tooth was noted by Examiner #006 on Dr. Farah's grade report. Examiner #005 gave Dr. Farah a grade of 2, finding root roughness and sub-gingival calculus on the distal side of tooth number 12, a tooth not treated by Candidate 20057. Examiner #048 gave Petitioner a grade of 1, commenting on "several" instances of sub-gingival calculus on teeth treated by Dr. Farah, as well as the presence of root roughness. (All comments are found on Petitioner's Exhibit 4.) Examiner #006 gave Dr. Farah the highest of her grades on the periodontal procedure, which was that it was minimally acceptable. The other examiners determined that Dr. Farah's treatment left sub-gingival calculus, and was below minimally acceptable standards (the grade of 2) or was unacceptable (the grade of 1). At the hearing Dr. Farah agreed that if calculus remained the appropriate grade would be 2 or lower. There is no reason to adjust the grades assigned on the periodontal exercise. Dr. Farah also prepared a cast class II restoration onlay wax up on a posterior tooth on a stone mannequin of a lower jaw. She received grades of 5, 3 and 2, which average to 3.33. Examiner #080 assigned a grade of 2, wrote on the grading form "undercuts," and also noted that the procedure had a marginal surface finish. Examiner #133 assigned a grade of 3, and noted "poor outline form" but added no comment concerning an undercut. The third examiner, #048, made no deductions and assigned a grade of 5. An "undercut" is an improper preparation of a tooth surface which is to support a crown. During the preparation of the assigned tooth, the center portion of the tooth was reduced to create a trapezoidal shape, similar to an equilateral triangle, the top of which has been cut by a plane parallel to its floor. A wax model of the crown is then prepared. If the side walls of the trapezoid, when the prepared surface is viewed from the top, do not slope downward and slightly outward, when the wax cast is removed, the wax deforms, and the crown made from it will not seat correctly on the tooth. This may cause the crown to fail, and is a serious error. When a curved dental explorer is placed against the base of the tooth and against the surface of the tooth vertically, one may observe whether there is an angular displacement outward from the vertical at the top, indicating an undercut. On Dr. Farah's preparation this test reveals an undercut. The testimony of Dr. Farah's expert, Dr. Robert Murrell, was that a "surveyor" is the proper instrument to use to evaluate a tooth preparation surface for an undercut. Dr. Murrell did so and determined there was no undercut on the Petitioner's work. There are two difficulties in determining whether there is an undercut using the surveyor. The surveyor's rod is fixed in a vertical position and cannot reflect whether it is actually up against the base of the tooth or not, and viewing the rod from the top down does not give visual confirmation whether the top edge is wider than the bottom; neither can one visually inspect the vertical alignment from the side because the remaining portion of the tooth would prevent one from viewing the alignment from the side position. Secondly, as the expert for the Department, Dr. Theodor Simkin, testified, the surveyor is not a proper instrument for determining undercuts on a mannequin, but is meant to be used on castings and other bridge or denture work done outside the patient's mouth. Logic supports Dr. Simkin's assessment, because a surveyor simply cannot be inserted into a patient's mouth. Dr. Simkin's testimony is also more persuasive because he has been, for several years, an experienced dental examiner and examination grading consultant. Dr. Murrell, while certainly a well-qualified dentist, has never been trained to grade the Florida clinical dental examination. Laying aside the question whether the surveyor or the explorer is the better instrument for assessing whether there is an undercut on a tooth, the other method for determining an undercut explained by Dr. Simkin is persuasive. If no undercut is present, when the stone mannequin of the mouth on which Dr. Farah worked is viewed from directly above, it should be possible to view all four bottom corners of the preparation surface at the same time; if there is an undercut, the undercut bottom corner will be hidden when all of the other corners are viewed. Visual examination confirms the presence of an undercut in the front right corner of Dr. Farah's preparation.
Recommendation It is recommended that the petition for regrading of the failing score assigned to Dr. Farah on the June 1985 clinical dental examination be DENIED. DONE AND ORDERED this 27th day of March 1986 in Tallahassee, Leon County, Florida. WILLIAM R. DORSEY, JR. Hearing Officer Division of Administrative Hearings The Oakland Building 2009 Apalachee Parkway Tallahassee, Florida 32301 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 27th day of March 1986. APPENDIX TO RECOMMENDED ORDER CASE NO. 86-0235 The following constitute my specific rulings pursuant to Section 120.59(2), Florida Statutes (1985) on the proposed findings of fact submitted by the parties. Rulings on Proposed Findings of Fact Submitted by Petitioner Findings of Fact (onlay) Rejected for the reasons stated in Findings of Fact 14 and 16. Accepted in Finding of Fact 15. Rejected for the reasons stated in Finding of Fact 15. Findings of Fact (periodontal)1 Generally accepted in Findings of Fact 1 and 8, except for the final sentence, which is rejected as argument. Rejected for the reasons stated in Findings of Fact 10 and 11. In addition, the question of whether the performance of Candidate 20057 was properly graded does not arise in this proceeding. If Candidate 20057 received high grades although three of the seven teeth treated had to be retreated 48 ours later by Dr. Farah, this does not address the central question in this case: Did the treatment provided by Dr. Farah meet minimum standards? [page 7] Rejected because there is no competent substantial evidence that Dr. Simkin was Examiner #015, but if he was, the proposal is argument, not a finding of fact. Rulings on Findings of Fact Submitted by Respondent Accepted in Findings of Fact 4, 5 and 6. Accepted in Finding of Fact 5. Accepted in Finding of Fact 4. Accepted in Finding of Fact 3. Accepted in Findings of Fact 3 and 7. Accepted in Finding of Fact 8, 9 and 10. Accepted in Finding of Fact 11. Accepted in Finding of Fact 12. Accepted, but clarified in Findings of Fact 13 and 14. Accepted in Finding of Fact 15. COPIES FURNISHED: Mr. Fred Varn Executive Director Board of Dentistry 130 North Monroe Street Tallahassee, Florida 32301 Mr. Fred Roche Secretary Department of Professional Regulation 130 North Monroe Street Tallahassee, Florida 32301 Salvatore A. Carpino, Esquire General Counsel Department of Professional Regulation 130 North Monroe Street Tallahassee, Florida 32301 H. Reynolds Sampson, Esquire Department of Professional Regulation 130 North Monroe Street Tallahassee, Florida 32301 Dr. Mina Farah 21-32 Crescent Street #D-7 Astoria, NY 11105
Findings Of Fact At all times relevant, Respondent, John W. Delk, held a license to practice dentistry, number DN 0005106, issued by the State of Florida, Department of Professional Regulation's Board of Dentistry. James Whisman was a patient at the Delk Dental Center from July 14, 1981 until May 22, 1984. Dr. John W. Delk was the dentist of record for James Whisman. On July 14, 1981, Dr. John W. Delk prepared teeth #6, 7, and 8 for crown and bridgework which included the placement of a post in tooth #8. On July 29, 1981, James Whisman returned to the Delk Dental Center to have the crowns on #6, 7, and 8 seated. On July 29, 1981, Don Berman seated permanent crowns on teeth #6, 7, and 8 for James Whisman using a permanent cement. Don Berman was a technician (dental assistant) for the Delk Dental Center and was not a licensed dentist or dental hygienist. He did not have an expanded duties certificate. On August 11, 1981, Respondent diagnosed a need, and had Berman prepare a treatment plan, for future dental work for James Whisman. During the establishment of Mr. Whisman's August 11, 1981 treatment plan, there was no documentation or oral advisement that an abnormality, such as a retained root tip or abscess, existed at tooth #10. Later, the bridge work on teeth #6, 7, 8 became loose, and Whisman called for an appointment with the Delk Dental Center. On September 21, 1981, the crowns on teeth #6, 7, and 8 were re-cemented with a permanent cement, zinc phosphate, by technician Don Berman. Dr. John W. Delk did not supervise Don Berman when he used the permanent cement to seat the crowns on teeth #6, 7, and 8 for a second time. From February 8, 1982 through February 25, 1982, Dr. James Costello provided dental services to James Whisman, specifically preparing teeth #9-15 and teeth #1-5 for crowns and bridgework and seating the crowns and bridgework. Dr. Costello did not advise James Whisman that an abnormality, specifically a retained root tip or abscess, was present at tooth #10. The patient chart for James Whisman failed to document that tooth #10 had a retained root tip and abscess present. The failure to chart a retained root tip and abscess at tooth #10 is critical to diagnosis and treatment. Fourteen months later, James Whisman returned to the Delk Dental Center for continued dental work. Respondent on March 6, 1984, diagnosed the need, and had Berman prepare a treatment plan, for fixed bridge-splints on teeth #19-22 and #27- 30. On March 7, 1984, Dr. John W. Delk prepared teeth #19- 22 and #27-30 for crown and bridgework. On March 9, 1984, Don Berman cemented temporary crowns on teeth #27-30 without supervision from Dr. John W. Delk. On March 12, 1984, Don Berman re-cemented temporary crowns on teeth #27-30, using a permanent cement called Durelon, without supervision from Dr. John W. Delk. On April 19, 1984, Don Berman used a permanent cement, Durelon, to seat the crown and bridgework on teeth #19-22 and #27-30 without supervision from Dr. John W. Delk. On May 15, 1984, James Whisman returned to the Delk Dental Center complaining of loose teeth in the area of #7 and 8. On May 22, 1984, James Whisman returned to the Delk Dental Center continuing in his complaint that teeth #7 and 8 were loose. On May 22, 1984, Don Berman did an oral inspection of teeth #7 and 8 and with the aid of a dental instrument removed said teeth. Teeth #7 and 8 fractured off inside the crowns. On May 22, 1984, based on what Don Berman had reported to him, Respondent diagnosed the need, and had Berman prepare a treatment plan, for Mr. Whisman which encompassed an estimate for two crowns, one root canal filling and two pin and core build-ups for a total fee of $708.00. On May 22, 1984, Dr. John W. Delk did not examine Mr. Whisman nor did Dr. Delk supervise the actions of Don Berman. Don Berman re-cemented the crowns for teeth #7 and 8 using a permanent cement, Durelon, with no supervision from Dr. John W. Delk. James Whisman suffered from areas of sensitivity around the bridgework, poor dental work and an unnoticed abscess and retained root tip. James Whisman discontinued the dental work with the Delk Dental Center and sought a second opinion from Dr. Albert P. Hodges on June 7, 1984. After the services performed by Dr. Delk and/or his employees, an examination of James Whisman's teeth revealed dental work that fell below the standard of care as recognized by the prevailing peer community. The standard of care for crown and bridgework recognized by the prevailing dental peer community is as follows: No open or shy margins around the crowns; no active decay present; proper retention in multiple-unit splints; proper dowel lengths in crowns that are needed to support multiple unit bridges; proper occlusal contact and recognition; and treatment of any pathological condition prior to crown and bridge placement. Specifically, tooth #7 had margins that were open and shy, active decay was present and there was a distinct lack of retention to support the two-unit splint. The dental treatment provided on tooth #7 fell below the minimum acceptable standards of care as recognized by the prevailing peer community. Specifically, tooth #8 had margins that were open, active decay was present and the dowel length was totally inadequate for useful retentive support. The dental treatment provided on tooth #8 fell below the minimum acceptable standards of care as recognized by the prevailing peer community. Specifically with tooth #10, Dr. Delk failed to diagnose and treat a retained root tip and a pathological condition which was visible and discoverable. The retained root tip and abscess were clearly visible radiographically as early as the July 14, 1981 visit to Dr. Delk's facility. James Whisman was not advised during the course of his treatment that a retained root tip existed and that the pathological condition should be treated prior to the placement of a crown over tooth #10. James Whisman's records, made at Dr. Delk's facility, failed to reflect the existence of the retained root tip and abscess at tooth #10. Failing to chart or notify the patient of the existence of a retained root tip and the accompanying cyst falls below the standard of care as recognized in the prevailing dental community. The dental treatment provided on tooth #10 fell below the minimum acceptable standards of care as recognized by the prevailing peer community. Specifically, tooth #20 had margins that were open and shy, it was sensitive to probing, and it was out of occlusion because it had no contact with the opposing tooth when the mouth was in the closed position. The dental treatment provided on tooth #20 fell below the minimum acceptable standards of care as recognized by the prevailing peer community. Specifically, tooth #21 had margins that were open and shy. The dental treatment provided on tooth #21 fell below the minimum acceptable standards of care as recognized by the prevailing peer community. Specifically, tooth #22 had margins that were shy. The dental treatment provided on tooth #22 fell below the minimum, acceptable standards of care as recognized by the prevailing peer community. Specifically, tooth #27 had margins that were shy and the crown was over-contoured causing potential gum irritation and food impaction. The dental treatment provided on tooth #27 fell below the minimum acceptable standards of care as recognized by the prevailing peer community. Specifically, tooth #29 had margins that were open and shy. The dental treatment provided on tooth #29 fell below the minimum acceptable standards of care as recognized by the prevailing peer community. Cementing crowns with permanent cement is an irremediable procedure. Cementing temporary crowns with permanent cement is justified on a short-term basis but only if the procedure is done by a licensed dentist. To be within acceptable dental standards, a dentist must do a physical oral examination of a patient before developing a treatment plan. Failure to do a physical oral examination in the development of a treatment plan falls below the minimum standards as recognized in the prevailing peer community. An assistant with an expanded duties certificate may use temporary cement only to seat temporary crowns provided a licensed dentist provides direct supervision.
Recommendation Based upon the foregoing Findings Of Fact and Conclusions Of Law, it is recommended that the Florida Board of Dentistry enter a final order: (1) holding the Respondent guilty on both counts of the Administrative Complaint; (2) fining Respondent $1000 for each count, said amount to be paid within 30 days from the signing of the final order or Respondent's license automatically to be suspended until the fine is paid; (3) suspending Respondent's license to practice dentistry for 6 months for each count of the Administrative Complaint, to run consecutively; (4) placing Respondent on probation for 12 months subsequent to the expiration of the suspension period; and (5) conditioning reinstatement of Respondent's license to practice dentistry on successful completion of 100 hours of university credit course work in crown and bridge restorations by the end of the probation period and on an appearance by Respondent before the Board to provide evidence of compliance with the final order. RECOMMENDED this 3rd day of March, 1987 in Tallahassee, Florida. J. LAWRENCE JOHNSTON Hearing Officer Division of Administrative Hearings 2009 Apalachee Parkway Tallahassee, Florida 32399 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 3rd day of March, 1987. COPIES FURNISHED: Nancy M. Snurkowski, Esq. Department of Professional Regulation 130 North Monroe Street Tallahassee, F1 32399-0750 Michael T. Hand, Esq. 230 East Marks Street Orlando, F1 32803 John W. Delk, D.D.S. 2918 North Pine Hills Drive Orlando, F1 32808 Pat Guilford Executive Director Board of Dentistry Department of Professional Regulation 130 North Monroe Street Tallahassee, F1 32399-0750 Van Poole, Secretary Department of Professional Regulation 130 North Monroe Street Tallahassee, F1 32399-0750 Wings T. Benton, Esq. General Counsel Department of Professional Regulation 130 North Monroe Street Tallahassee, F1 32399-0750 APPENDIX To comply with Section 120.59(2), Florida Statutes (1985), the following rulings are made on Petitioner's proposed findings of fact (Respondent did not submit proposed findings of fact): 1.-3. Accepted and incorporated. 4. Accepted but unnecessary. 5.-8. Accepted and incorporated. Accepted but unnecessary. Rejected as not proved. The evidence suggested that Respondent examined the patient and instructed Berman how to prepare the plan. 11.-15. Accepted and incorporated. 16. Rejected as not proved. Dr. Costello testified he seated the crowns and bridgework, and the office notes do not reflect that Berman was involved at all. Whisman's memory probably was in error on this point. 17.-20. Accepted and incorporated. 21. Rejected. See 10 above. 22.-29. Accepted and incorporated except the correct date in 24 is March 12, 1984, and the correct teeth in 25 are #19-22, not #19-20. 30. Rejected. See 10 above. 31.-56. Accepted and incorporated. Accepted but cumulative. Accepted and incorporated. Rejected as conclusion of law. 60.-61. Accepted but unnecessary. Accepted and incorporated. Accepted but cumulative. 64.-65. Accepted and incorporated.
The Issue Is Petitioner entitled to receive a passing score on the June 2001 dental licensure examination?
Findings Of Fact Upon consideration of the oral and documentary evidence adduced at the hearing, the following relevant findings of fact are made: The Department is the agency of the State of Florida responsible for administering the dental licensure examination. Petitioner was an unsuccessful candidate for the June 2001 dental licensure examination in that she failed the clinical portion of the June 2001 dental examination. Originally Petitioner received a score 1.89 on the Clinical portion, but on re-grade received a score of 2.10. However, a score of 3.00 was required to pass the Clinical portion. The June 2001 dental licensure examination consists of two parts: (1) the Clinical portion; and (2) the Laws and Rules portion. The Clinical portion consists of nine procedures. Petitioner challenges five of the nine procedures, which are: (1) Periodontal procedure; (2) Class IV Composite Restoration; (3) Class II Composite Restoration; (4) preparation for a three- unit fixed partial denture; and (5) Class II Amalgam Restoration on a model. The Department selects three examiners to independently grade each candidate’s performance, and the average of the three scores from each examiner produces the overall grade for that procedure. The average grade for each procedure is then weighted in accordance with Rule 64B5-2.013, Florida Administrative Code, which produces an overall score for the entire Clinical portion of the examination. This procedure provides for a more reliable indication of the candidate’s competency. Each examiner must be a licensed dentist for a minimum of five years without having any complaints or disciplinary actions against the examiner’s license. The examiners are not allowed to have any contact with the candidates they are grading. Each examiner must attend, and successfully complete, a standardization session, which trains each examiner to use the same internal grading criteria. In this standardization session, the examiners are thoroughly taught specific grading criteria, which instruct the examiners on how to evaluate the work of the candidates. 8. Examiners numbers 005, 316, 346, 360, 361, and 375, who graded Petitioner’s examination, successfully completed the standardization session. The Department’s post-exam check found these examiners' grading to be reliable. Petitioner received a score of 1.66 on the Class IV Composite Restoration. Petitioner contested this score contending that she was downgraded on this procedure because she mistakenly stained that procedure. The Class IV Composite Restoration consists of the restoration of a chipped tooth. The grading is based on the candidate’s ability to restore the tooth as it appeared before restoration. The goal is to restore the tooth to its proper contact and to restore the contact between the teeth. The fact that Petitioner stained the Class IV Composite Restoration did not result in the examiners downgrading the Petitioner’s procedure. Examiner 005 gave Petitioner a score of 2.00, which was based on the contact being open and not having a flushed fit (marginal error). Examiner 316 gave Petitioner a score of 2.00, which was based on Petitioner’s problems with the functional anatomy, the proximal contour, and with the margin. Examiner 346 gave Petitioner a score of 1.00, which was based on Petitioner’s problems with functional anatomy, proximal contour, and mutilation of opposing or adjacent teeth. Petitioner received a score of 0.00 on the Class II Composite Restoration. Petitioner contested this score contending that she was downgraded twice for the same mistake. A Class II Composite Restoration is a procedure that involves the candidate’s ability to fill an opening inside the tooth with composite, which is a tooth-colored filling. The Candidates were instructed, for security reasons, to place dye in the composite and that failure to place dye in the composite would result in a failing grade. Petitioner failed to place dye in the composite. In addition to his comment concerning no dye in the composite, Examiner 005 also commented that Petitioner’s occlusion was very high, which would result in the premature failure of the restoration. Examiner 005 gave Petitioner a score of 0.00. Examiner 316 also gave Petitioner a score of 0.00, which was based on the absence of dye in the composite and the occlusion being high, which would result in the premature failure of the restoration. Examiner 346 also gave Petitioner a score of 0.00, which was based on the absence of dye in the composite. Petitioner contested the score she received on the Preparation for a 3-unit Fixed Partial Denture procedure claiming that the examiners’ comments regarding insufficient and excessive reduction were conflicting comments. The Preparation for a 3-unit Fixed Partial Denture procedure is a procedure that involves the candidate’s ability to replace a missing tooth with a fixed partial denture or fixed bridge. Petitioner received a score of 2.00 on this procedure. A tooth has five surfaces (front, back, top, inside and outside). Therefore, one surface of the tooth may have insufficient reduction, while another surface of the tooth may have excessive reduction. It is not unusual for examiners to see and comment on different errors. Examiner 316 gave Petitioner a score of 2.00 on this procedure because there was a problem with the outline form, insufficient reduction on the preparation and errors on the marginal finish. Examiner 005 gave Petitioner a score of 2.00 on this procedure because there was a problem with the outline form and there was both insufficient reduction and excessive reduction on the preparation. Examiner 346 gave Petitioner a score of 2.00 on this procedure because there was excessive reduction on the preparation, marginal finish, and mutilation of opposing or adjacent teeth. Petitioner contested the score of 0.66 that she received on the Class II Amalgam Restoration on a model procedure. This procedure is similar to Class II Composite, which involves the candidate’s ability to restore a cavity in the tooth so that the finished product restores proper form and function to the tooth. The difference is that amalgam rather than composite is used for the restoration. The restored tooth should closely resemble its original size and shape. Examiner 316 gave Petitioner a score of 1.00 on this procedure because there was a gingival overhang on the distal lingual aspect of the restoration, which could cause tooth decay and gingivitis. Examiner 346 also gave Petitioner a score of 1.00 because of problems with functional anatomy, proximal contour, margin, and gingival overhang. Examiner 005 gave Petitioner a score of 0.00 because of problems with proximal contour and gingival overhang. Petitioner contested the score of 1.66 that she received on the Periodontal procedure alleging that she was graded unfairly because she could not remove all of the calculus on this procedure, and that one examiner gave her a score of 3.00. The Periodontal procedure involves the candidate’s ability to completely remove any stains, calculus deposits or any foreign debris from the surface of the tooth. Patient selection is very important for the periodontal procedure. It is the candidate’s responsibility to select a suitable patient as clearly outlined in the Candidate’s Information Booklet, which is mailed to the candidate prior to the examination. Petitioner chose a difficult patient, considered to have heavy calculus deposits and severe periodontal disease. Petitioner admitted that she did not remove all of the calculus deposits on her patient. Petitioner failed to present sufficient evidence to show that it was impossible to remove all of the calculus on the patient she had chosen. Examiner 360 gave Petitioner a score of 3.00, but commented that sub-gingival calculus remained on the tooth, and there was root roughness. Examiner 375 gave Petitioner a score of 2.00 because sub-gingival calculus remained on the tooth and there was root roughness. Examiner 361 gave Petitioner a score of 0.00. The basis for this score was that there were heavy deposits of calculus and root roughness on teeth number 19, 29, and 30, and that the procedure was of little value to the patient. The Department provides a re-grade process for all candidates who timely request a hearing. The purpose of the re- grade is to determine if any of the grades rendered were inconsistent. The Department selects the top three examiners who had the highest reliability from that examination to participate in the re-grade. On re-grade, Petitioner’s overall grade increased slightly from 1.89 to 2.10 but not enough for Petitioner to receive a passing grade. The Department’s post-standardization statistics of the examiners’ performance indicated that Petitioner’s examiners graded reliably. The post-standardization statistics indicate the examiner’s performance on grading of models during standardization. In addition, the Department calculates post- examination statistics for the examiners who graded the Petitioner’s challenged procedures. They are: Examiner Accuracy Index & Rating 361 94.2 – Very Good 360 95.1 – Excellent 375 96.0 – Excellent 005 94.3 – Very Good 316 97.0 – Excellent 346 97.2 – Excellent All examiners’ reliability was significantly above the minimum acceptable accuracy index of 85.00.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is accordingly Recommended that the Board of Dentistry enter a Final Order dismissing Petitioner’s challenge to the grades she received on the Clinical portion of the June 2001 dental licensure examination and denying Petitioner licensure as a dentist in the State of Florida due to her failure to receive a passing grade on the June 2001 dental licensure examination. DONE AND ENTERED this 31st day of January, 2002, in Tallahassee, Leon County, Florida. WILLIAM R. CAVE Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 31st day of January, 2002. COPIES FURNISHED: Cecilia Diaz 8810 Memorial Highway Tampa, Florida 33615 Cherry A. Shaw, Esquire Department of Health Office of the General Counsel BIN A02 4052 Bald Cypress Way Tallahassee, Florida 32399-1703 Theodore M. Henderson, Agency Clerk Department of Health 4052 Bald Cypress Way BIN A02 Tallahassee, Florida 32399-1701 William H. Buckhalt, Executive Director Board of Dentistry Department of Health 4052 Bald Cypress Way BIN C06 Tallahassee, Florida 32399-1701 William W. Large, General Counsel Department of Health 4052 Bald Cypress Way BIN A02 Tallahassee, Florida 32399-1701
Findings Of Fact The Parties stipulated to the fact that Dr. Moore was a dentist, licensed by the State Board, holding license number 1464, issued by the State Board. The Parties further stipulated that the facts alleged in the Board's complaint were accurate as of the date of filing. Dr. Ackel testified concerning professional practices. It is a part of the professional services of a dentist to fill out forms necessary for insurance coverage, although they may charge a fee for the time required to do so. The time required to fill out such forms, which are in the main prepared by clerical personnel in the dentist's office, varies from fifteen to forty-five minutes, to include the dentist's time taken to review the entries. The failure to prepare the forms results in nonpayment or delayed payment of insurance claims to the patient. Dr. Moore had delayed over a year the preparation and submission of the forms on the patients involved in this complaint. Dr. Ackel said this was the first such complaint that the Broward County Dental Association has had in his eight-year association with the Association's board which investigates patient complaints. Dr. Moore, having been cautioned about his rights in this case, took the stand and testified that he had had multiple personal problems beginning in 1973. These problems included within a two-year period a personal bankruptcy, a son who flunked out of medical school at the halfway point and subsequently was critically ill with ulcers, another son who suffered a mental depression which resulted in his hospitalization, a reduction in his office staff, and a separation from his wife who also worked in his office. While Dr. Moore acknowledged his ultimate responsibility for the failure to process the insurance forms involved, he did request the Board to consider the foregoing facts in mitigation. Dr. Moore's office is currently a one-man office with one receptionist who has been with the Doctor for twenty-two (22) years. There has been an increase recently in dental insurance claims; and Dr. Moore, who is an older dentist who had a good professional reputation in the community until these incidents, has apparently not adjusted his office administration to keep pace with the changes. This, together with his various personal problems, prevented him from attending to these important matters. Dr. Ackel stated that Dr. Moore had been suspended from the County Association for ninety (90) days as a result of its investigation and findings; however, that this suspension did not cause Dr. Moore to submit the forms. Dr. Moore apologized to all the parties concerned, indicated that he was acting immediately to hire additional personnel in his office, and that all the insurance forms in his office would be filled out and submitted immediately. The Hearing Officer notes, however, that the statements of Dr. Moore's patients indicate he had made similar assurances to his patients.
Recommendation The Dental Board's interest in this case is apparently twofold: To rectify the existing situation and enable Dr. Moore's patients to obtain reimbursement, and To prevent any further failures of this type by Dr. Moore. The Hearing Officer would recommend the following Board action based upon the Findings of Fact and Conclusions of Law: Dr. Moore's license be suspended for three to six months, said suspension or a portion thereof to be held in abeyance or suspended upon Dr. Moore's doing the following: Immediately filing the insurance forms involved here, with copies to the Board, and Permitting and reimbursing, if necessary, a representative or designee of the Dental Board with a reputation for effective office management within the profession to inspect Dr. Moore's office and make a written report to Dr. Moore and the Board suggested ways of improving his office management to prevent a recurrence of this type of failure. DONE and ORDERED this 4th day of September, 1976 in Tallahassee, Florida. STEPHEN F. DEAN, Hearing Officer Division of Administrative Hearings Room 530, Carlton Building Tallahassee, Florida 32304 (904) 488-9675 COPIES FURNISHED: L. Haldane Taylor, Esquire 605 Florida Theatre Building 128 East Forsyth Street Jacksonville, Florida 32202 Castles W. Moore, D.D.S. 852 N. E. 20th Avenue Fort Lauderdale, Florida 33304
The Issue The issue is whether the score that Respondent assigned to the Patient Amalgam Preparation and Periodontal sections of the clinical part of Petitioner's June 2003 Florida Dental License Examination taken was arbitrary or capricious.
Findings Of Fact Petitioner took the June 2003 Florida Dental License Examination. A passing score for the clinical part of the examination is 3.0. Petitioner received a score of 2.94, so he failed the clinical part of the examination. Petitioner has challenged the grades of 2.0 that he received on the Patient Amalgam Preparation and Periodontal sections of the clinical part of the examination. In both sections, the score of 2.0 is derived from averaging the 3s that Petitioner received from two evaluators and the 0 that he received from one evaluator. For both procedures, Petitioner challenges only the scores of 0, and he needs two additional points to pass the clinical part of the examination. The administration of the clinical part of the dental examination requires Respondent to recruit and train numerous examiners and monitors, all of whom are experienced, licensed dentists. The training process includes standardization exercises designed to ensure that all examiners are applying the same scoring criteria. The evaluation of specific procedures are double-blind, with scoring sheets that identify candidates by test numbers, so examiners do not know the identity of the candidate whose procedures they are scoring. The two sections that are the subject of this case require the candidate to demonstrate certain skills on a live patient. While working with the patient, the candidate is supervised by a monitor. When the candidate has completed the required dental work to his satisfaction, he so advises the monitor, who sends the patient to the dental examiners. For each section that is the subject of this case, three dental examiners examine the patient and score the procedure. These examiners do not communicate with each other, and each performs his or her examinations and scores the procedure in isolation from the other examiners. Communication between examiners and candidates is exclusively through monitor notes. For each section that is the subject of this case, the maximum possible score that a candidate may receive is a 5. Passing grades are 3, 4, or 5. Nonpassing grades are 0, 1, or A score of 3 indicates minimal competence. The Periodontal section of the clinical part of the dental examination required Petitioner to debride five teeth. Removing calculus from teeth, especially below the gums, is an important procedure because the build-up of tartar and plaque may cause pockets to form between the tooth and gum. Eventually, the gum tissue may deteriorate, ultimately resulting in the loss of the tooth. Prior to the examination, written materials explain to the candidates and examiners that the debridement is to remove all supragingival and subgingival foreign deposits. For the Periodontal procedure, Examiners 207 and 296 each gave Petitioner a 3, and Examiner 394 gave him a 0. The scoring sheets provide a space for preprinted notes relevant to the procedure. All three examiners noted root roughness. However, Examiner 394 detected "heavy" subgingival calculus on four teeth and documented his findings, as required to do when scoring a 0. Petitioner contends that two examiners and he correctly detected no calculus, and Examiner 394 incorrectly detected calculus. As an explanation, Petitioner showed that Examiner 394 knows Petitioner in an employment setting, and their relationship may have been tense at times. However, Petitioner never proved that Examiner 394 associated Petitioner's candidate number with Petitioner. Thus, personal bias does not explain Examiner 394's score. On the other hand, Examiners 296 and 207 are extremely experienced dental examiners. Examiner 296 has served nine years in this capacity, and Examiner 207 has served ten years, conducting 15-20 dental examinations during this period of time. By contrast, Examiner 394 has been licensed in Florida only since 1995 and has been serving as a dental examiner for only three years. However, the most likely explanation for this scoring discrepancy is that Examiner 394 explored more deeply the subgingival area than did Examiners 207 and 296 or Petitioner. Examiner 394 testified with certainty that he found the calculus at 5-6 mm beneath the gums. This is likely deeper than the others penetrated, but not unreasonably deep. For the Periodontal procedure, an examiner who found calculus on four teeth would be entitled to award the candidate 0 points. Examiners may deduct two points per tooth that has been incompletely cleaned, although the lowest score is 0. Examiner 394's score of 0 is therefore legitimate and at least as reliable as the other scores of 3. The Amalgam Preparation section of the clinical part of the examination required Petitioner to remove caries from one tooth and prepare the tooth for restoration. These procedures are of obvious importance to dental health. Poor preparation of the tooth surface will probably result in the premature failure of the restoration. A restoration following incomplete removal of caries will probably result in ongoing disease, possibly resulting in the loss of the tooth. Written materials, as well as Respondent's rules, which are discussed below, require a 0 if caries remain, after the candidate has presented the patient as ready for restoration. Other criteria apply to the Amalgam Preparation procedure, but this criterion is the only one of importance in this case. Examiners 207 and 417 each assigned Petitioner a 3 for this procedure, but Examiner 420 assigned him a 0. Examiners 207 and 417 noted some problems with the preparation of the tooth, but neither detected any caries. Examiner 420 detected caries and documented his finding, as required to do when scoring a 0. As noted above, Examiner 207 is a highly experienced evaluator, but the other two evaluators are experienced dentists. Examiner 417 graduated from dental school in 1979, and Examiner 420 has been licensed in Florida since 1981. The instructions to examiners emphasize that they are to detect caries "exclusively" tactilely, not visually. Tactile detection of the stickiness characteristic of caries is more reliable than visual detection. For example, caries assumes the color of dentin as the decay approaches the dentin. Examiner 420 testified definitively that he detected caries tactilely, not visually, in Petitioner's patient. This testimony is credited. It is difficult to reconcile Examiner 420's finding of caries with the contrary finding by the highly experienced Examiner 207. It does not seem especially likely that an experienced dentist would miss decay, especially in the artificial setting of a dental examination, in which everyone's attention is focused on one tooth. Examiner 207's finding of no caries is corroborated by the same finding of Examiner 417. However, Examiner 417's finding is given little weight. She readily suggested that she must have missed the caries. What at first appeared to be no more than a gracious gesture by a witness willing to aid Respondent's case took on different meaning when Examiner 417 testified, in DOAH Case No. 03-3955, first that she had detected visually and then retreated to testifying that she did not know if she had detected caries visually or tactilely--a significant concession because examiners were instructed explicitly not to rely on visual findings of caries. Returning, then, to the conflict between the findings of Examiner 420 and Examiner 207, substantially unaided by the corroborating findings of Examiner 417, either an experienced, credible dentist has found caries where none exists, or an experienced credible dentist has missed caries. The specificity of Examiner 420's testimony makes it more likely, as logic would suggest, that he did not imagine the existence of caries, and Examiner 207 somehow missed the caries. It is thus slightly more likely than not that Petitioner failed to remove the caries prior to presenting the patient. More importantly, though, for reasons stated in the Conclusions of Law, Examiner 420, in finding caries, adhered strictly to Respondent's rules and policies for evaluating candidates' work, and his finding was not arbitrary or capricious.
Recommendation It is RECOMMENDED that the Department of Health enter a final order dismissing Petitioner's challenge to the scoring of the clinical part of the June 2003 Florida Dental License Examination. DONE AND ENTERED this 27th day of February, 2004, in Tallahassee, Leon County, Florida. S ROBERT E. MEALE Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 27th day of February, 2004. COPIES FURNISHED: R. S. Power, Agency Clerk Department of Health 4052 Bald Cypress Way, BIN A02 Tallahassee, Florida 32399-1701 William W. Large, General Counsel Department of Health 4052 Bald Cypress Way, Bin A02 Tallahassee, Florida 32399-1701 William H. Buckhalt, Executive Director Board of Dentistry Department of Health 4052 Bald Cypress Way, Bin C06 Tallahassee, Florida 32399-1701 James Randolph Quick Driftwood Plaza 2151 South U.S. Highway One Jupiter, Florida 33477 Cassandra Pasley Senior Attorney Department of Health Office of the General Counsel 4052 Bald Cypress Way, Bin A02 Tallahassee, Florida 32399-1703
The Issue The issue for determination is whether Petitioner successfully completed the December 1999 dental licensure examination.
Findings Of Fact Petitioner has been given a temporary permit to practice dentistry due to his being a resident in training. In December 1999, Petitioner took the dental licensure Examination. He successfully completed the Laws and Rules part of the Examination having received a score of 78.00, where a minimum score of 75.00 was required to pass that part. Petitioner failed to successfully complete the Clinical part of the Examination having received a score of 2.89, where a minimum score of 3.00 was required to pass the Clinical part. As a result, Petitioner failed to successfully complete the overall Examination. On the Periodontal section of the Clinical part, Petitioner received a score of 1.66. He challenges this score. Each candidate is graded by three examiners. Each examiner is a dentist who is licensed in the State of Florida, with a minimum of five active years' experience, and who, among other things, has no complaints or negative actions against his/her license. Before every examination, each examiner is trained in evaluating a procedure to make sure that the procedure is properly performed. The Department of Health (Department) conducts training in which each examiner is trained to grade using the same internal criteria. Such training results in a standardization of grading criteria. In this training process, the examiners are trained by assistant examiner supervisors on the different criteria that are used during the examination. The assistant examiner supervisors are dentists licensed in the State of Florida. To further their training, the examiners after receiving verbal training are shown slides of teeth which do not meet the clinical criteria of the examination. Following the standardization, to make sure that the examiners have been able to internalize the criteria, the examiners, themselves, are given an examination. Included in the examination is a hands-on clinical, where models are used and the examiners check for errors on the models. The examiners are evaluated on how they perform when they grade the models, to make sure that the examiners are grading the candidates the same, using the same criteria, and with reliability. Each examiner grades the examination independently. The examiners do not confer with each other while scoring the examination. The examiners do not have contact with the candidates. As to grading, the average of the three grades from the examiners produces the overall grade for the exercise performed by the candidate. Having three examiners grading provides a more reliable indication of the candidate's competency and true grade. Furthermore, the examination is double-blind graded, which is a grading process in which the candidates have no contact with the examiners. The candidates are located in one clinic and perform the dental procedures on their human patient. The clinic is monitored. When the candidate completes the procedures, a proctor accompanies the human patient to another clinic where the examiners are located, and the examiners grade the procedures performed by the candidates. Monitors are used by the Department at the examination. The role of a monitor is to preserve and secure the integrity of the examination. The monitor, among other things, gives instructions to the candidates, answers questions of the candidates, and acts as a messenger between the candidate and the examiner. Monitors also ensure that candidates do not have contact with the examiners. For the Periodontal section, a candidate, as Petitioner, performs a periodontal exercise on a human patient who is chosen by the candidate. The human patient must also be approved by the Department in accordance with criteria specified by rule.2 The criteria includes a requirement that the human patient must have a minimum of five teeth, each of which must have pockets of a minimum of four mm in depth with sub-gingival calculus. Petitioner chose his human patient. The Department approved Petitioner's human patient. Petitioner's human patient was a periodontally involved patient. Petitioner performed the periodontal exercise on his human patient. Petitioner's exercise was graded by three examiners, i.e., Examiners 131, 346, and 264. All three examiners participated in and successfully completed the standardization training, and it is inferred that they were considered qualified to act as examiners for the Examination. Petitioner's examination was double-blind graded. Each examiner independently graded Petitioner's examination. Examiner 131 found no errors and awarded Petitioner a grade of five (5). Examiner 346 found gross mutilation of the human patient's soft tissue of areas 26, 27, and 28, and awarded Petitioner a grade of zero (0). Examiner 264 also found gross mutilation of the human patient's soft tissue of areas 26, 27, and 28, and awarded Petitioner a grade of zero (0). The criteria for the Periodontal exercise mandates a grade of zero (0) where there is gross mutilation of gingival tissue.3 Consequently, Examiners 346 and 264 had no choice but to award Petitioner a grade of zero (0). After the grading, both graders who found gross mutilation of gingival tissue made written comments, regarding the tissue mutilation, on the Examiner-To-Monitor Instructions form. Examiner 264's comment was "Please have candidate place perio pak, area 26, 27, 28" and was not intended to be instructions to Petitioner but was directed to follow-up work or to attention that the human patient may need afterwards. The Examiner-To-Monitor Instructions form, with the written comments, was provided to the monitor who related the comments to Petitioner. The monitor did not allow Petitioner to view the written comments. The monitor informed Petitioner that further work needed to be done as to the human patient. The monitor indicated on the Examiner-To-Monitor Instructions form that Examiner 264's comment was related to Petitioner by the monitor writing "Candidate complied with" and writing and circling his assigned monitor number. The monitor writing "Candidate complied with" meant only that the monitor informed Petitioner that further work needed to be done, not that the Petitioner correctly performed the procedure. No evidence was presented that Petitioner sutured the human patient or that he placed a perio pak on the affected tissue of areas 26, 27, and 28. The evidence shows that the monitor only related to Petitioner that further work needed to be done without the monitor specifying what needed to be done. Moreover, the evidence shows that the monitor did not indicate that Petitioner had done what was requested of him. A candidate is not informed of his/her performance by the examiner because there is no contact between the examiner and the candidate. Additionally, such notification at the Examination site is not done because it is believed to have the effect of alarming the candidate and raising the candidate's anxiety level. The human patient was not informed that there was mutilation of soft tissue as a result of the periodontal exercise. Before an individual is accepted by the Department as a patient, the individual must complete and sign a "Patient Disclosure Statement and Express Assumption of Risk" form. This form, among other things, relieves the Department of any responsibility for poor work done by a candidate or for notifying the human patient of any poor work done by the candidate and places the responsibility on the human patient to have a licensed dentist check the work done by the candidate. The grading of Petitioner's Periodontal exercise is not arbitrary or capricious or an abuse of discretion. The grading process is not devoid of logic and reason.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that the Department of Health, Board of Dentistry, enter a final order dismissing Rami Ghurani's examination challenge to the periodontal section of the clinical part of the dental licensure examination administered in December 1999. DONE AND ENTERED this 15th day of December, 2000, in Tallahassee, Leon County, Florida. ERROL H. POWELL Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 15th day of December, 2000.
The Issue This is a license discipline case in which the Respondent has been charged in a Corrected Administrative Complaint with a violation of Section 466.028(1)(m), Florida Statutes.
Findings Of Fact At all times material to this proceeding, the Respondent, Dr. Merle N. Jacobs, has been licensed to practice dentistry in the State of Florida. He currently holds license number DN 0005940. During the period from January 22, 1993, through March 27, 1995, T. C. was a patient of the Respondent. During that period of time, the Respondent performed various dental services for T. C., including the making and fitting of a partial denture. The Respondent prepared and kept dental records and medical history records of his care of patient T. C. The Respondent's records of such care are sufficient to comply with all relevant statutory requirements. The Respondent's records of such care do not include any notations specifically identified or captioned as a treatment plan. The records do, however, include marginal notes of the course of treatment the Respondent intended to follow in his care of patient T. C. Those marginal notes describe the treatment the Respondent planned to provide to patient T. C.
Recommendation On the basis of all of the foregoing it is RECOMMENDED that a Final Order be issued in this case dismissing all charges against the Respondent. DONE AND ENTERED this 29th day of May, 1998, in Tallahassee, Leon County, Florida. MICHAEL M. PARRISH Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 Filed with the Clerk of the Division of Administrative Hearings this 29th day of May, 1998.
Findings Of Fact Dr. Roberts and His Background Dr. Steven Roberts is a dentist licensed to practice in the State of New York. He attended the United States Military Academy and received his undergraduate degree in 1970. He graduated from the New York University College of Dentistry in 1978, and practiced dentistry in New York, New York from 1978- 1987. To be licensed in New York, Dr. Roberts passed the national boards and the northeast regional board examination. During the course of his practice in New York, Dr. Roberts never received a complaint or had a claim for malpractice made or filed against him. Clinical Examinations Dr. Roberts took the Florida clinical dental examinations in June of 1986, January of 1987, and June of 1987. His grade on the June of 1987, examination is the subject of this proceeding. Dr. Roberts has successfully passed the written examination and the diagnostic examination required for licensure by Section 466.066(4)(a) and (c), Florida Statutes. Dr. Roberts' score for the June of 1987, clinical dental examination was 1.95; the minimum passing score is 3.00. The procedures tested during the June 1987, Clinical Dental Examination and Dr. Roberts' scores were as follows: The Procedure The Score The Revised Score Periodontal 1.67 Amalgam Cavity Preparation 1.67 Amalgam Cavity Restoration 3.00 Composite Preparation .67 Composite Restoration .33 Posterior Endodontics 2.00 3.66 Cast Preparation 2.67 3.00 Pin Amalgam Preparation 1.00 Pin Amalgam Restoration 1.67 Denture 3.63 Total Score 1.95 2.15 Dr. Roberts made a timely request to review his grade, and filed objections to his grades; a regrading procedure resulted in the regrading of his scores for posterior endodontics and cast restoration as set forth above. Each of the procedures tested in the clinical dental examination is scored by three different examiners. For each procedure examiners record their scores on separate 8 1/2" X 11" sheets. Each sheet has a matrix of circles which are blackened with a pencil so that they can be machine scored. On each sheet the candidate's identification number and the examiner's identification number are recorded along with the number for the procedure involved and the candidate's grade. On the sheet for each procedure the criteria for successful performance of the procedure are printed, along with preprinted comments which the examiners may use to explain the reason for the grade assigned. These comments relate to the criteria being examined. The following grades may be assigned by examiners: Complete failure Unacceptable dental procedure Below minimum acceptable dental procedure 3- Minimum acceptable dental procedure 4- Better than minimal acceptable dental procedure 5- Outstanding dental procedure An examiner is not required to mark a comment if the grade assigned is 5, a comment is marked for any grade below 5. Each procedure is graded in a holistic manner. Grades assigned by each of the three examiners for a procedure are averaged; the averaged scores for each procedure are then weighted and the weighted scores are summed to provide the overall clinical grade. By averaging the scores of three examiners for each procedure, variation from examiner to examiner is minimized. The examiners are experienced Florida dentists selected by the Board of Dentistry. An examiner must have at least five years of experience as a dentist and be an active practitioner. Potential examiners attend a standardization training exercise. This training is required by Section 466.006(4)(d), Florida Statutes. Its purpose is to instruct examiners in examination procedures and the criteria to be applied in grading. Through the training the examiner group as a whole arrives at a consensus opinion about the level of grading, so that candidates' scores on the examination will be valid and reliable. The training attempts to focus on each examiner's subjective, internalized evaluation criteria, so that they can be modified, as necessary, to reflect the consensus of all graders. A standardizer explains grading criteria to the potential examiners, and discusses various divisions among schools of thought and training on the procedures which will be the subject of the examination. The standardizer uses dental exhibits from prior dental exams as examples, and identifies grades and errors on the exhibits so that the graders learn and can adhere to uniform grading standards. The training focuses on three problems which professional literature has identified in evaluation: errors of central tendency, proximity errors, and bias a priori. Errors of central tendency result when graders are uncertain of criteria, hesitate to give extreme judgments, even in appropriate cases, and thus tend to improperly grade near the average. Proximity error is a type of halo effect which is applicable in grading of mannequin exhibits. The examiner grades all of the mannequin exhibits for each candidate at one time. If the first example of the candidate's work is especially good, and deserves a grade of 5, the grader may tend to transfer a generally positive attitude towards the next example of the candidate's work and assign a grade which may not be based solely upon the merits of that second piece of work. The same process can improperly depress the grades on subsequent mannequins if the first example of a candidate's work is poor. Bias a priori is the tendency to grade harshly or leniently based upon the examiner's knowledge of the use that will be made of the grade, rather than only on the quality of the work graded. After an 8 to 12 hour standardization training session, the Department administers an examination to those who have been trained. Those with the highest scores become the examiners, i.e., dentists who will grade candidates' work, while those with the lower scores in the training session become monitors, who supervise the candidates in their work on mannequins or on patients, but who do not actually grade student work. There is, however, no minimum score which a dentist who attends the standardization session must obtain in order to be an examiner rather than a monitor. This results, in part, from the limited pool of dentists who participate in the examination processes as monitors or examiners. For the 1987 clinical dental examination 31 dentists accepted selection by the Board and attended the standardization session, 20 were then selected as examiners and 11 became monitors for the examination. None of the dentists who attended the standardization session were dismissed by Department of Professional Regulation from further service at the examination session. The process by which the Department selected the examiners for the 1987 clinical dental exam was neither arbitrary nor capricious, but comports with Rule 21G- 2.020(4), Florida Administrative Code. The standardization training and examination of dentists to determine who will serve as examiners and monitors does not provide any bright line for distinguishing among potential examiners those who will make the most assiduous effort to apply the grading criteria explained in the training session versus those who retain an innate sense of a passing work based on what the examiner considers acceptable work in his own practice. The effort to convey to examiners the standard of "minimum competency" has imperfect success, but the Department's training is appropriate. Out-of-State Candidates' Scores 11. There is a substantial difference in the failure rates for out-of- state candidates and for in-state candidates on the clinical dental examinations. In the June of 1987, exam 82.5% of the candidates who graduated from the only in-state dental school, the University of Florida, passed the entire examination, while 54.2% of the out-of-state graduates passed, and only 37.8% of candidates from foreign schools were successful. Overall, 86.5% of the candidates passed the written portion of the examination, 93.5% the portion on oral diagnosis, but only 63.3% the clinical portion of the examination. Dr. Roberts has failed to prove that the lower pass rate for out-of- state candidates is the result of any sort of conscious effort on the part of examiners to be more stringent in grading out-of- state candidates. Dr. Kennedy's testimony indicated only that the data bear more analysis, not that they prove improper grading. Procedures Performed on Mannequins The Board of Dentistry tests between 600 and 700 dental candidates per year. It is extremely difficult for the candidates to find patients who have exactly the problem which is to be tested and bring them to the examination to work on. Some portions of the clinical dental examination, therefore, are not performed on patients, but on cast models of human teeth which resemble dentures, and which are known as mannequins. This is expressly authorized by Section 466.006(4)(a), Florida Statutes. The notice to appear which candidates receive approximately 30 days before the examination informs them of the types of mannequins which will be used in the examination. Before that time, however, dental supply companies obtain lists of those eligible to take the examination, and contact the candidates in an attempt to sell them the mannequins. Candidates must bring mannequins with them to the examination and can purchase additional mannequins for practice. Testing with mannequins is also more efficient because with live patients, the student must be graded at the time of the examination, while a model can be retained and graded a day or two later. The decision of the Board to have certain procedures performed on mannequins, so that each candidate would be graded on exactly the same procedure, is reasonable. The Board had also considered having students perform all test procedures on extracted human teeth, but there are not a sufficient number of all natural teeth available, given the number of students who are tested, both for the examination itself and for practice. The Board determined that it would be better to use mannequins for some of the procedures tested in the examination because they are readily available and students can purchase extra copies for practice. For certain procedures, such as endodontics, specific natural teeth (such as first bicuspids) are often extracted and so are generally available; for procedures performed on those teeth, it is possible to have candidates work on human teeth. By contrast, testing procedures performed on teeth such as incisors is not practicable. It is impossible to obtain enough incisors in good condition, without restorations and chips, for use during an examination. The statute governing the dental examination does require that one restoration performed by candidates must be done on a live patient, and for the June 1987, clinical dental examination that procedure was a class 2 amalgam restoration. The Board directed by rule that mannequins be utilized for five test procedures: the pin amalgam preparation and restoration, Rule 21G-2.013(3)(d), Florida Administrative Code; the endodontic procedure, Rule 21G-2.013(3)(e) Florida Administrative Code; the posterior tooth preparation for a cast restoration, Rule 21G-013(3)(f), Florida Administrative Code, the class III acid etch composite preparation and class IV acid etch composite restoration, Rule 21G-2.013(3)(g), Florida Administrative Code. Performing these procedures on mannequins is not exactly the same as performing procedures on human teeth in a patient. In view of the difficulty involved in finding patients whose teeth present virgin lesions, so that each candidate would be tested on exactly the same problem, the difficulty in grading a large number of procedures performed on live patients, and the difficulty in obtaining a large number of human teeth necessary for testing and for practice, the Board's decision to use the mannequins for these procedures is reasonable. The Legislature recognized this in Section 466.006(4)(b), Florida Statutes, which prescribes that the clinical dental examination shall include restorations "performed on mannequins, live patients, or both. At least one restoration shall be on a live patient." The Board was within its authority when it determined the procedures to be performed on mannequins. Violation of Blind Grading The dental examiners who grade the work of candidates grade blindly, i.e., they do not know which candidate's work they are grading. The Clinical Monitor and Examiner Instruction Manual for the June of 1987, examination makes this clear. At page 24 paragraph 3 the Manual states Examiners are requested to disqualify themselves at anytime they are presented with models or patients treated by a dentist who they know personally or with whom they have had professional contact. All examiners are requested to give department staff the name of any examination candidate who is personally known to them to be taking the exam. The department staff will assist the examiners in avoiding any work performed by the candidates they know. Rationale: Allegations have been made about examiners who knew candidates taking the exam even though the examiners only see candidate numbers. Monitors and Examiners are strongly urged to avoid discussion with candidates about the examination. Even conversation about non-examination related matters can be misinterpreted by other candidates as an unfair privileged communication. Despite this admonition, one of the examiners, Dr. Cohen, who knew Dr. Roberts, graded the work of Dr. Roberts. Dr. Cohen met Dr. Roberts the first time Dr. Roberts took the Florida Clinical Dental Examination in June of 1986. Dr. Roberts had with him a bag which would have identified him as a student from New York University, where Dr. Cohen had taught. Dr. Cohen came over to Dr. Roberts, introduced himself, gave Dr. Roberts his card, (exhibit 44) and invited Dr. Cohen to his hotel room where they discussed practicing dentistry in Florida. In 1986 Dr. Cohen was associated with another dentist, Gerald P. Gultz, who had recently moved to Florida from New York. Dr. Gultz had also been a part-time clinical assistant professor of dentistry at New York University College of Dentistry. After Dr. Cohen returned from the June 1986, administration of the clinical dental examination, he had a conversation with Dr. Gultz in which Dr. Cohen asked Gultz if he knew Dr. Roberts, and commented on Dr. Roberts performance on the clinical examination. Dr. Cohen said Dr. Roberts had done terribly, and Dr. Cohen believed that Dr. Roberts would never get his license to practice in Florida. (Tr. 5/26/88 at 73). Dr. Roberts saw Dr. Cohen at the January of 1987, clinical dental examination, but they did not speak. In June of 1987, Dr. Cohen also spoke briefly to the wife of Dr. Gerald Gultz, Lauren Gultz, saying that he would be seeing Dr. Roberts at the June of 1987, clinical dental examination, which was coming up. He told Mrs. Gultz that Dr. Roberts was a poor practitioner, and that he did not think he would pass the examination. At the June 1987, exam, Dr. Roberts' periodontal patient was his uncle, Mr. Finkelstein. Dr. Cohen was one of the examiners who reviewed Mr. Finkelstein to determine whether his condition was appropriate to serve as a patient for Dr. Roberts on the periodontal portion of the examination. Dr. Cohen had a conversation with Mr. Finkelstein in which he told him "tell your dentist to do a good job". Because Mr. Finkelstein had stated that his dentist was a graduate from N. Y. U. Dental School, Mr. Finkelstein was convinced that Dr. Cohen knew exactly who the dental candidate who would work on Dr. Finkelstein was -- Dr. Roberts. After accepting Mr. Finkelstein as an appropriate periodontal patient, Dr. Cohen also served as a grader on the periodontal procedure performed on Mr. Finkelstein. After grading the work which Dr. Roberts had done, Dr. Cohen told Mr. Finkelstein to tell his dentist that Dr. Cohen would see him later in the hotel where they were staying. At the hotel, Dr. Cohen talked to Dr. Roberts about the dental examination, that he himself had to take the examination three times, although he considered himself to be a superior dentist, and that Dr. Cohen could help Dr. Roberts with his grades but that he could never grade Dr. Roberts more that one grade higher than any of the other examiners. Dr. Cohen served as an examiner (i.e. grader) for Dr. Roberts on six of the nine procedures tested. There were: procedure number 1, the periodontal evaluation where he assigned a failing grade of 2; procedure number 4, the class III composite preparation, where he assigned a failing grade of 1; procedure number 5, the class IV composite restoration, where he assigned a failing grade of 1; procedure number 6, the endodontic evaluation, where he assigned a passing grade of 3; procedure number 7, the preparation for a cast restoration, where he assigned a passing grade of 3; and procedure number 8, the pin amalgam preparation, where he assigned a failing grade of 1. This failure of blind grading is a serious irregularity in the evaluation of Dr. Roberts' performance on the 1987 clinical dental examination, given his prior negative comments about Dr. Roberts before the examination. By ignoring those scores, Dr. Roberts would be evaluated only by two examiners, on all the procedures for which Dr. Cohen gave a grade. This would mean that his scores would not be comparable with those of any other candidate, for his grade on each procedure would not be the result of blind grading by three independent examiners. Dr. Roberts' Challenges to Grades Assigned by Other Examiners The full nine procedures evaluated in the 1987 dental clinical examination and Dr. Roberts' grades were: A periodontal exercise performed on a live patient, Mr. Finkelstein, which involved the scaling of five teeth both above and below the gum and stain removal. Dr. Roberts was assigned scores of 1, 2, and 2 by the examiners (one grade of 2 was assigned by Dr. Cohen) An amalgam cavity preparation, performed on a live patient, Elizabeth Cox, which is the preparation of a tooth for filling. When the preparation is completed a proctor escorts the patient to the three examiners who independently grade this part. After grading, the patient returns to the candidate who completes the filling of the tooth (the restoration) which is subsequently graded independently by three examiners. Dr. Roberts was assigned grades of 1, 1, and 3 for the preparation (none of these grades were assigned by Dr. Cohen). A final amalgam restoration, which is the filling of the tooth prepared in the prior procedure. Dr. Roberts received grades of 3, 3, and 3 on this procedure (none of the grades were assigned by Dr. Cohen). A class III composite preparation, which is preformed on a model, not a live patient. This involves removing decay and shaping a tooth to hold a class III filling, i.e., one located on the side surface of an incisor. Dr. Roberts received scores of 1, 0, and 1 (Dr. Cohen assigned one of the grades of 1) A class IV composite restoration, which is performed on a model, not a live patient. This involves restoring a fractured tooth with a composite restoration material. On this procedure Dr. Roberts received scores of 0, 0, and 1 (Dr. Cohen assigned the grade of 1). An endodontic evaluation performed on a posterior tooth, which is performed on a mannequin, and involves the opening of a molar, and identification of the canals in the tooth in preparation for a root canal procedure. Originally Dr. Roberts received grades of 3, 3, and 0 (one of the grades of 3 was assigned by Dr. Cohen). Dr. Roberts work was regraded by three new examiners and the grades of the original examiners were discarded. Dr. Roberts ultimately received a grade of 3.67 on the endodontic portion of the examination A preparation of a posterior tooth for a cast restoration, which is performed on a mannequin. It involves preparing a tooth to receive a crown. Dr. Roberts' original grades were 2, 3, and 3 (Dr. Cohen had assigned a grade of 3 on this procedure). On review, Dr. Roberts' was regraded by three new examiners, and the original grades were discarded. Dr. Roberts received a final grade of 3 on this portion of the examination. A pin amalgam preparation, which is performed on a model, not on a live patient. This involves the preparation of a tooth to hold an amalgam filling by inserting a pin into a portion of the tooth, which serves to anchor the filling. Dr. Roberts was assigned grades of 2, 0, and 1 on this procedure (Dr. Cohen assigned the grade of 1). Pin amalgam final restoration, which is performed on a model. It involves filling a tooth with amalgam filling material. Dr. Roberts was assigned grades of 2, 1, and 2 on this procedure (Dr. Cohen assigned one of the grades of 2). Due to the involvement of Dr. Cohen in so many of the procedures involved here, Dr. Roberts performance on the June of 1987, clinical dental examination was not fairly evaluated. A fair evaluation cannot be provided after the fact by merely dropping Dr. Cohen's grades, because Dr. Roberts' performance would not be subject to the independent evaluation of three examiners. Dr. Roberts relies, to a large extent, on the testimony of Dr. Gultz as the basis for regrading his procedures to a passing grade of 3, or better. The testimony of Dr. Gultz does not, however, show that he has ever participated in the standardization exercises for examiners at Florida clinical dental examinations. Dr. Gultz experience as a clinical professor of dentistry at New York University provides a substantial basis for his evaluation of dental procedures. The difficulty, however, is that as with any qualified examiner, his evaluations will be based on internalized standards which are personal to him. There is no way to know whether Dr. Gultz standards for adequate performance are equivalent to those which the standardization training produces among examiners at the standardization exercise before a clinical dental examination. The standardization process "attempts to bring all examiners to the same level of grading, so that each [examiner] is grading in a valid and reliable manner." Clinical Monitor and Examiner Instruction Manual, June of 1987, at page 42. The Florida dental clinical examination uses a holistic grading method. Each score sheet which an examiner fills out has on it the criteria to be applied in evaluating the candidates performance on that procedure. They all contain a statement which reads: It is the intent of the Board that each of the criteria are to be accorded equal importance in grading. Equal importance does not mean that each criteria has a numerical or point value, but means that any one of the criteria, if missed to a severe enough degree so as to render the completed procedure potentially useless or harmful to the patient in the judgment of the examiner, could result in a failing grade on the procedure. The criteria do not have any assigned numerical or point value, but are to be utilized in making a holistic evaluation of the procedure. Each grading sheet also points out to the examiner certain critical factors which, if present, require a grade of 0 for the procedure. The standardization in grading which the Board diligently attempts to achieve through the standardization training and the standardization testing of examiners done at the close of the training is elusive at best. Nonetheless, in the absence of showing that Dr. Gultz standards of evaluation are equivalent to those of an examiner trained at a standardization session, it is impossible to know whether his standards of evaluation are more rigorous or less rigorous than those reflected by the grades assigned to other candidates by the corps of examiners which evaluated the work of candidates at the June of 1987, clinical dental examination. The same is true with respect to the testimony of Dr. Simkins, the expert for the Board in this proceeding. No useful purpose would be served in attempting to choose between the testimony of Dr. Gultz, on the one hand, and the testimony of Dr. Simkins and of the other examiners who testified by deposition in this proceeding. If this were to be done, all the hearing officer would have determined is whose testimony about the appropriate grade to be assigned for each procedure is more believable. On this record it would be impossible to make a further finding about whether that more believable testimony reflects a scoring standard more stringent, less stringent or the same as that generally applied to all candidates by the corps of examiners in the June of 1987, clinical dental examination.
Recommendation It is recommended that the results of the clinical dental examination which Dr. Roberts took in June of 1987, be found invalid, and that he be permitted to take the next clinical dental examination offered by the Department of Professional Regulation at no cost to him. DONE AND ENTERED in Tallahassee, Leon County, Florida, this 8th day of December, 1989. WILLIAM R. DORSEY, JR. Hearing Officer Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-1550 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 8th day of December, 1989.
Findings Of Fact Arnaldo Odio, D.D.S. received his education and dental training in Cuba, completing dental school there in 1942. Thereafter he was in private practice with his uncle in Havana, Cuba for about 5 years following which he worked at a government tuberculosis hospital until he came to the United States in 1965. At the time of Dr. Odio's arrival in the United States, Florida, as well as most of the other states, did not allow applicants for dental licensure to qualify based upon graduation from a foreign dental school. Upon his arrival in the United States Odio operated a dental laboratory in the Miami area. He applied for admission to various U.S. dental schools but apparently was not accepted. In 1970-71 New York changed its law to permit foreign graduates to qualify for licensure and Odio made application, took the necessary tests, and was admitted to practice in New York. He is currently practicing dentistry in New York. Chapter 72-185 Laws of Florida, 1972, deleted the requirement that graduates of foreign dental schools must, prior to examination, graduate from a United States school and substituted the provision that an applicant graduating from a foreign dental school may qualify for the examination for licensure by passing the National Board of Dental Examiners' examination and exhibiting manual skills on a laboratory model to the satisfaction of the Board. The exhibition of manual skills is called the mannequin examination and consists of work done on extracted human teeth inserted in a model upon which the applicant performs dental work without risk of harm to a patient. Having passed the National Board in 1970 or 1971 Odio applied to take the mannequin examination in Florida. Only after successfully completing the mannequin examination is an applicant deemed qualified to sit for the licensure examination which is comprised of a written portion as well as a clinical portion where work is performed on live patients. There is currently no limit on the number of times an applicant may take the mannequin examination. Dr. Odio took the mannequin examinations in December 1974, March 1975, and December 1975 and was informed after each examination that he had failed to attain the minimum qualifying score. It is these failures Dr. Odio is here contesting. The examiners for the mannequin examination are selected from the eight members of the Board of Dental Examiners. Before instituting these mannequin examinations the Board consulted professionals in the field of examining procedures and followed the recommendations so received. The evening before each examination is given the examiners go through a training session in which they are shown dental work and each assigns a grade which they later defend at a group meeting against the grades assigned by other examiners. This process is repeated several times, after which the grades become more uniform. By this procedure divergences in examining techniques are minimized. Before the examination each applicant is given an instruction sheet in which the procedures to be followed, the work that must be performed, and the grading procedures are explained. Dr. Odio testified that he fully understood the steps he was required to complete, and which dental procedures were expected for each grade assigned. During the course of the examination, at the times taken by Odio, the applicant performed three basic procedures and received seven grades. The range of scores on each step graded is from 0 to 5, with a score of 3 representing a passing grade of 75 percentile. This scoring system was adopted as a result of the recommendations of the examining professionals employed to help set up the examination scoring procedures. Two examiners assign a grade for each procedure completed by an applicant. The mannequin examinations are conducted in a room similar to a teaching dental laboratory with work space for each applicant and the examiners at the front of the room. No time limit is set for each procedure, but the applicant is given the full working day from 8:30 A.M. to 6:00 P.M. with one hour off for lunch to complete the examination. As each of the steps upon which the applicant is graded is completed, he takes his mannequin to an examiner who, after examining the work done, assigns a grade. The applicant then must have the procedure graded by a second examiner. After the examinations are completed the original examination score sheets are forwarded to a grading service where the scores are totaled and the applicant advised of his score. Before the originals are sent the Board makes a xerox copy of each examination sheet in case the originals are subsequently lost or misplaced. This copy is retained by the Board. The examiners' scores for each procedure are averaged to obtain the applicant's grade on that procedure. A failing grade of 2 on one procedure could be offset by a grade of 4 on another procedure to attain a passing grade of 3. Since one score is assigned for the preparation of the tooth for filling and a second score assigned after the tooth has been filled, the filling would preclude reevaluation of the score assigned for preparation. Accordingly retention of the mannequins after the work was graded would serve no useful purpose. Following two of Dr. Odio's mannequin examinations he met with a member of the Board who went over the grade sheet with Dr. Odio to show how the final score was computed. Odio's testimony that these members could not advise specifically why he received a failing score would not be unexpected. The examiners occupy positions similar to judges in a talent show or in a figure skating competition where marks are assigned on observed performance. Certain performances would be expected to be found substandard and others excellent and a third party could not look at a score sheet and say why. On each grade sheet the various factors that are to be considered by the examiner are listed alongside each score box. Occasionally the examiner will place a check mark alongside the factor wherein he considered the applicant's work to be deficient. Petitioner presented a summary of the grades assigned at the mannequin examinations in March and December, 1975. These show that of the 25 examination scores of the 7 procedures graded (March 1975) the two examiners assigned different grades a total of 74 times. These grades varied by more than one point only 10 times. Similarly on the December 1975 mannequin examination, the examiners differed by 1 point or more 78 times, but on only 6 grades did they differ by more than one point. No particular significance to the variance of these scores was established. A quick perusal of Exhibit 2, the test sheets for the December 1975 examination indicates that approximately half of the applicants passed the examination. Once the examination grades are assigned and turned in by the examiner, no changes are made to these scores. The originals of the test scores for the December 1974 and March 1975 mannequin examinations could not be located by Respondent; however, the copies made before the exams were delivered for grading, were produced. The originals of the December 1975 exam sheet except for that of Dr. Odio were produced and a copy of Odio's December 1975 exam was admitted as Exhibit 3. Although Petitioner contends that Respondent erred in failing to maintain these originals for two years as required by the statutes, no prejudice to Petitioner resulting from this error was shown or that this contributed in any manner toward Petitioner's failure to pass these mannequin examinations. No evidence was presented to indicate that different standards were applied in grading Petitioner than were applied to all other applicants. Petitioner was approved to take the December 1976 mannequin exam but did not appear. He had received a stab wound in the hand shortly before the exam and did not have the dexterity needed to complete the exam at that time. Petitioner's proposed findings of fact are generally in agreement with the findings above except that some of Petitioner's proposed findings were not included in the Findings of Fact because they were not deemed material. In the Proposed Findings Petitioner emphasized the discrepancies in the grades assigned by the two examiners and showed that the two examiners grading each step of a mannequin exam were in agreement only some 55 percent of the time and not the 85 percent agreement sought by the board. A review of Dr. Leo Foster's testimony (Exhibit 8) reveals his testimony on p. 12 to be that the grading consistency sought was that the grade given by the two examiners be within one point of each other 85 percent of the time. In the exam in March 1975 where there were 10 differences of more than one point on 175 grades in exam scores given by the two examiners this equals approximately 94 percent agreement. On the December 1975 exam where there were only 6 grades assigned by the two examiners which were more than one point apart the agreement between examiners exceeded 96 percent.
