STATE OF FLORIDA
DIVISION OF ADMINISTRATIVE HEARINGS
DR. PASCUAL ESTEVEZ and )
DR. VICTOR VERJANO, )
)
Petitioners, )
)
vs. ) CASE NOs: 84-0828
) 84-0829
DEPARTMENT OF PROFESSIONAL ) REGULATION, BOARD OF PODIATRY, )
)
Respondent. )
)
RECOMMENDED ORDER
Pursuant to notice, a formal hearing was conducted in this case in Miami, Florida, on June 7, 1985, before Michael M. Parrish, a duly designated Hearing Officer of the Division of Administrative Hearings. At the hearing the parties were represented by counsel as follows:
APPEARANCES
For Petitioners: Manuel Vega, Jr., Esquire
477 Northwest 27th Avenue Miami, Florida 33125
For Respondent: H. Reynolds Sampson, Esquire
Deputy General Counsel
Department of Professional Regulation
130 North Monroe Street Tallahassee, Florida 32301
INTRODUCTION AND ISSUES
The basic issue in this case concerns the validity of the 1983 Board of Podiatry licensure examination. The Petitioners contend that the examination was invalid for several reasons, and that because of such invalidity, they should be licensed as podiatrists even though they both failed the exam. The Respondent contends that the examination was valid, and that even if invalid, the Petitioners are not entitled to licensure unless and until they receive a passing grade on a licensure examination.
Subsequent to the hearing a transcript of the proceedings was filed with the Hearing Officer on July 12, 1985. Pursuant to agreement of counsel, the parties were allowed three weeks from the filing of the transcript within which to file their proposed findings of fact and conclusions of law, which time period was later extended at the request of counsel for the Petitioners. On August 9, 1985, the Petitioners filed a proposed recommended order containing proposed findings of fact and conclusions of law, and the Respondent filed proposed findings of fact and a memorandum of law. The posthearing submissions filed by the parties have been given careful consideration in the preparation of
this Recommended Order. Specific rulings on each proposed finding of fact in the posthearing submissions are set forth in the appendix which is attached to and incorporated into this Recommended Order.
FINDINGS OF FACT
Based on the stipulations of the parties, on the exhibits received in evidence, and on the testimony of the witnesses at the formal hearing, I make the following findings of fact.
At all times relevant to these proceedings the licensure examination administered by the Board of Podiatry has tested the following nine subject matter areas: anatomy, biochemistry, orthopedic podiatry, surgery; clinical podiatry and differential diagnosis, physiology, materia medica and pharmacology, pathology, and dermatology. At all relevant times the licensure examination has consisted of a total of 360 questions: forty questions on each of the nine subject matter areas covered by the examination. At all relevant times an overall average of seventy-five per cent (75 percent) has been required to achieve a passing score for the examination. An additional proviso at all relevant times is that a passing grade will not be given to any person who fails to achieve a minimum grade of fifty per cent (50 percent) in any one of the nine subject areas.
The Petitioners Dr. Pascual Estevez and Dr. Victor Verjano, took the Board of Podiatry licensure examination in each of the following years: 1982, 1983, 1984. Both Petitioners failed all three exams. 1/
Dr. Estevez' scores on the 1982 and 1983 examinations were as follows:
SUBJECT 1982 SCORE 1983 SCORE
anatomy 30.0 percent 42.5 percent
biochemistry 50.0 percent 60.0 percent
orthopedic podiatry 45.0 percent 27.5 percent
surgery 47.5 percent 50.0 percent clinical podiatry and differential
diagnosis 32.5 percent 35.0 percent
physiology 37.5 percent 40.0 percent materia medica and pharmacology 25.0 percent 45.0 percent pathology 40.0 percent 52.5 percent
dermatology 45.0 percent 60.0 percent
OVERALL AVERAGE 39.17 percent 45.8 percent
Dr. Verjano's scores on the 1982 and 1983 examinations were as follows:
SUBJECT | 1982 | SCORE | 1983 | SCORE |
anatomy | 17.5 | percent | 37.5 | percent |
biochemistry | 40.0 | percent | 57.5 | percent |
orthopedic podiatry | 30.0 | percent | 30.0 | percent |
surgery | 27.5 | percent | 47.5 | percent |
clinical podiatry and | differential | |||
diagnosis | 35.0 | percent | 27.5 | percent |
physiology 22.5 percent 37.5 percent materia medica and pharmacology 20.0 percent 35.0 percent pathology 30.0 percent 55.0 percent
dermatology 37.5 percent 57.5 percent
OVERALL AVERAGE 28.89 percent 42.7 percent
The 1983 Board of Podiatry licensure examination was the first podiatry examination that used questions solicited from professors of podiatry at several colleges of podiatry outside the state of Florida. The Board could not obtain questions from professors of podiatry within the state of Florida because there are no colleges of podiatry in this state. The Board preferred to avoid soliciting questions from podiatrists practicing in this state so that there would not be any local knowledge of the examination content readily available within the state.
The decision to obtain a new pool of questions from professors of podiatry was a result of the opinion of the Board of Podiatry that the level of difficulty of previous examinations probably tested less than minimum competency. The Board felt that questions should be of a higher difficulty level than had been used on previous examinations because they were concerned that incompetent people were managing to "sneak through."
When the Office of Examination Services wrote to the college professors to request that they prepare questions to be used on the examination, the letters to the professors included the following information with respect to the level of difficulty the Board preferred:
For our purposes, the content tested by an item should be clearly pertinent to the mainstream practice of podiatry and ideally of a difficulty level such that you would expect most of the upper third of a graduating class to answer correctly while the majority of the lower third would find the item to be quite difficult. As regards to the difficulty level, the Board would prefer that in departing from the ideal you tend to favor higher difficulty levels.
The college professors who prepared questions for the 1983 Board of Podiatry licensure examination did not all follow the guidelines quoted immediately above. Some of them wrote questions which in the opinion of the Board were too easy.
The questions for the 1983 Board of Podiatry licensure examination were derived from three sources: those submitted under contract by selected professional faculty members at colleges of podiatry in California, Iowa, and Pennsylvania (approximately 50 percent of the questions), those prepared by the five professional members of the Podiatry Board (about 25 percent), and items selected by the developer from the item bank (about 25 percent). The latter were specifically selected for high discrimination between passing and failing candidates on a previous exam. This group of 89 test items was used to estimate the relative capability of the 1983 candidates. The remaining test items were selected by the professional members of the Board from the 337 items submitted by consultants and those contributed by the Board members themselves.
Each question that was used on the 1983 Board of Podiatry licensure examination was submitted to the Board for review before being included on the examination. The Board reviewed and considered every question submitted and selected only those they felt were adequate to test the candidates' competency. They rejected questions which were too hard as well as those which were too easy.
Following the administration of the 1983 Board of Podiatry licensure examination, each answer sheet was scored and a complete item analysis was generated for review by the examination development team. All items which were passed by 50 per cent or fewer of the candidates were scrutinized. Additional scrutiny was also given to items which displayed negative discrimination indices, i.e., those items passed by a greater proportion of low scoring examinees than high scoring examinees. A total of 109 items were identified for review by the Board. At its meeting on August 6, 1983, the Board determined that three items had been mis-keyed and that 39 items merited credit for all responses because of various defects. There remained a large number of questions which were of questionable validity because of the low percentage of upper half candidates who answered them correctly. A second review was conducted following submission of objections filed by candidates. Five additional examination items were credited for all responses and two were double keyed. Following the foregoing actions, a final psychometric review was conducted by the Office of Examination Services and 15 additional items were credited for all responses on statistical grounds.
As originally administered and scored, the 1983 Board of Podiatry licensure examination was of a difficulty level that tested for greater than minimum competency and was substantially more difficult than the examination that had been given in 1982. However, with the adjustments described above in paragraph 11 of these findings of fact, the difficulty level of the 1983 examination was substantially the same as the difficulty level of the 1982 examination. As adjusted, the difficulty level of the 1983 examination was such that it tested for minimum competency or perhaps less than minimum competency. As adjusted, the difficulty level of the 1983 Board of Podiatry examination was such that it did not test for greater than minimum competency. The effect of the adjustments described above was to delete from the examination the initial bias of the examination toward the more difficult items. As adjusted, the percentage of candidates who passed the 1983 examination was 50.4 percent, which compares favorably with the 51.1 percent pass rate for 1982.
Following the final Board review which credited all of the items described above in paragraph 11 of these findings of fact, several members of the Board of Podiatry, including its chairman, Dr. Owen P. Macken felt that although the examination was a valid measurement of minimal competency as initially given, once it was "watered down" by the removal of so many items it became an invalid measurement because the Department had given credit for too many questions.
A total of 117 candidates took the 1983 Board of Podiatry licensure examination. As finally scored, fifty-nine of those candidates passed the examination. Expressed as a percentage, 50.4 percent of those who took the 1983 examination received a passing grade. Out of the total of 117 candidates who took the 1983 examination; the score of Dr. Verjano was; at best, 113th from the top, and perhaps as low as 116th from the top. The 1983 score of Dr. Estevez was, at best 110th from the top, and perhaps as low as 112th from the top. Compared from the other end of the scale, Dr. Verjano had perhaps the second
worst grade of all 117 who took the 1983 examination and had no better than the fifth worst grade of all who took the examination. Dr. Estevez had perhaps the sixth worst grade of all 117 who took the 1983 examination and had no better than the eighth worst grade. 2/
As demonstrated by the "anchor questions," the candidates for examination who took the 1983 Board of Podiatry Licensure examination had characteristics very similar to the characteristics of the candidates who took the 1982 examination. In view of this similarity of the two groups of candidates who took the 1983 and the 1982 examinations, the fact that their success rate was very similar indicates that the difficulty level of the two examinations (as finally adjusted) was very similar. Accordingly, a candidate who passed one examination would probably have passed the other examination, and a candidate who made a very poor grade on one examination would probably have made a very poor grade on the other examination.
The Board of Podiatry is composed of two lay members and five professional members. Each of the professional members is a licensed Florida podiatrist currently engaged in the active practice of podiatry in the state of Florida.
The function of the Board differs from that of the Department of Professional Regulation. The Board is charged with determining the content of the examination questions so as to ensure that every podiatrist practicing in the state meets minimum requirements of safe practice and that podiatrists who fall below such minimum competency or who otherwise present a danger to the public health would be prohibited from practicing in the state. The Board also determines the general areas to be tested and the score that shall be necessary evidence of passing the examination. The Department's function through its examination development specialists is to ensure that the test items or questions are functioning as they were intended and to advise the Board as to the worth of the individual items.
Although a national podiatry examination is available, the Board of Podiatry has chosen not to use it. This appears to be due in part to the differences from state to state in the lawful scope of the practice of podiatry, some states limiting the practice to the foot while in Florida the scope of practice extends up to the knee.
CONCLUSIONS OF LAW
Based on the foregoing findings of fact and on the applicable legal principles, I make the following conclusions of law:
The Division of Administrative Hearings has jurisdiction over the subject matter of and the parties to these consolidated cases. See Sec. 120.57(1), Fla. Stat.
Rule 21T-11.02 Florida Administrative Code, contains the following provision regarding passing scores:
The Board establishes an overall average grade of 75 percent as necessary to achieve a passing grade on the examination; however, a passing grade will not be assigned to any person who fails to achieve a minimum of 50 percent in any one (1) subject area.
The Petitioners have not filed a rule challenge petition pursuant to Section 120.56, Florida Statutes, attacking the validity of the foregoing rule provision. Accordingly, the rule must be presumed to be valid and must be applied in these cases.
The burden of showing entitlement to the licenses they seek is on the applicants. See Rule 28-6.08(3), Florida Administrative Code; J.W.C. Company, Inc. v. Department of Transportation, 396 So.2d 778 (Fla. 1st DCA 1981). On the basis of the foregoing findings of fact and for the reasons set forth in the following paragraphs, the Petitioners have failed to meet their burden of proof.
The first matter to which attention should be directed with respect to the proof in this case is that there are numerous conflicts in the testimony of the expert witnesses on the subject of testing. For the most part I have found the testimony of the Respondent's testing expert to be more credible than the opposing expert opinions and have resolved most, if not all, disputes in the expert testimony in favor of the Respondent's version of the case. This is largely because of the fact that the Respondent's testing expert was the witness most familiar with the manner in which the 1983 examination was administered, the manner in which it was graded, the manner in which it was adjusted to correct what appeared to be irregularities, and the manner in which the examination was examined statistically to verify its validity and correlation to the 1982 exam. Further, the opinions of the testing expert called by the Respondent were more consistent with other evidence in the case than were the opinions of the Petitioners' experts.
With regard to another aspect of the Petitioner's proof, I have not based any findings of fact on the testimony of Petitioners' podiatric expert, Dr. Sheldon Marne, not because of any deficiency in his credentials but because he was not asked the right questions. All of Dr. Marne's opinions are directed at the total 1983 examination consisting of 360 questions as originally administered. His opinion that the examination was too difficult as originally administered and originally graded is irrelevant because the grades ultimately given to these Petitioners and to all other candidates were based on their performance on an examination on which they were given full credit-- regardless of the answers they chose--on 59 of the questions. Dr. Marne was not asked to express an opinion as to the difficulty level of the 1983 examination with those
59 questions removed from consideration.
Similarly, neither Dr. Marne nor any other witness called by the Petitioner testified that any specific question on the 1983 examination was more difficult than the appropriate difficulty standard. Assuming for the sake of argument that there may have been some questions which were too difficult, there is still no evidence that either of those Petitioners missed any such questions. In light of the nature of the Petitioners' scores on the 1982 and 1983 examinations, it is reasonable to conclude that the Petitioners guessed at a large number of answers. For all that is shown on the evidence in this case, these Petitioners may have guessed correct answers to all of the difficult questions and guessed incorrectly on all of the easy questions. In other words, even if some of the questions were more difficult than appropriate under the rule and statutory standards, these Petitioners have not shown that any such impropriety in any question had an adverse impact on their substantial interests. In order to show that a question was improper and that they should receive some relief on the basis of that impropriety, the Petitioners would have to show that they missed the answer to that question because of the impropriety in the question.
Assumptions and inferences about the quality of the questions on the 1983 podiatry examination which have been urged by the Petitioners are not warranted when considered in light of the other evidence in the record. The professors who wrote many of the questions on the examination did not all follow the difficulty standard which was set forth in Mr. Healy's instruction letters. Rather, many of them wrote questions which were too easy. Further, not all of the professors' questions were used in the examination because the Board members screened some of those questions out of consideration. And though there was no formal task analysis, the Board members considered task relevancy of each question as part of their decision on whether the question was appropriate for examination. Inasmuch as a majority of the Board members were practicing podiatrists at the time of their review of the questions, this screening process is the functional equivalent of a task analysis because all of the podiatrist members of the Board were familiar with what is involved in the day-to-day practice of podiatry in the state of Florida.
If there was any invalidity to the final scoring of the 1983 podiatry examination it was in being too generous rather than too parsimonious in awarding credit for examination questions during the post-examination statistical review of the examination. These Petitioners cannot complain of any such generosity because it had no adverse impact on their substantial interests
. 3/
Even if these Petitioners had been successful in showing that the 1983 examination was totally invalid, the most that could have been awarded to them by way of relief in these proceedings would have been an opportunity to take another examination without cost. The requested remedy of licensure without further examination would be inappropriate because there is nothing in this record to support a conclusion that either of these Petitioners is minimally competent to practice podiatry without danger to the public. Their very low scores on at least two examinations raise serious questions as to their competency. And while the Petitioners are entitled to a fair examination (which this record indicates they received in both 1982 and 1983), even if they were deprived of such an opportunity the remedy of licensure by default so to speak would be detrimental to the public and would frustrate the very purpose of the legislation which requires an examination prior to licensure to practice podiatry. The purpose of the statutory scheme is to protect the public, and while a candidate who was deprived of an opportunity to demonstrate his competence by fair examination may have some other form of remedy available in some other forum, it would be contrary to the public health, safety, and welfare to license any applicant who has not demonstrated minimum competency to practice the profession.
On the basis of all the foregoing it is recommended that the Board of Podiatry enter a Final Order concluding that the Petitioners have failed the 1983 examination and denying the Petitioners' applications for licensure on the basis of their having failed the 1983 examination.
DONE AND ORDERED this 1st day of November, 1985, at Tallahassee Florida.
MICHAEL M. PARRISH, Hearing Officer Division of Administrative Hearings The Oakland Building
2009 Apalachee Parkway
Tallahassee, Florida 32301
(904) 488-9675
Filed with the Clerk of the Division of Administrative Hearings this 1st day of November, 1985.
ENDNOTES
1/ The record does not contain evidence of the Petitioners' scores on the 1984 examination. Their scores on the 1982 and 1983 examinations are set forth in the findings of fact.
2/ The relative standings of Dr. Estevez and Dr. Verjano are derived from a comparison of the following exhibits: Pet. Ex. 7 and Res. Ex. 3 and 5.
3/ The likelihood that the 1983 examination, as finally adjusted and graded was easier than the 1982 examination is supported, inter alia, by the fact that both Petitioners scored better on the 1983 examination than on the 1982 examination.
APPENDIX TO RECOMMENDED ORDER 84-0828 AND 84-0829
The following constitutes my specific rulings pursuant to Section 120.59(2), Florida Statutes, on all of the proposed findings of fact submitted by the parties to these consolidated proceedings.
Rulings on Petitioners' proposed findings of fact:
Accepted in substance with modification of various details.
Rejected as unnecessary. It is more in the nature of a roll call of the witnesses than a proposed finding of fact.
The first two sentences are accepted with additional findings for context and understanding. The last two sentences are rejected as irrelevant and unnecessary in light of other findings regarding how the test questions were screened.
Accepted, with additional findings for context and understanding.
For the most part rejected due to erroneous details and inclusion of irrelevant details.
Accepted in substance, with clarification and correction of various erroneous details.
Rejected for two reasons: (a) It is a summary of testimony and not a proposed finding of fact and (b) the implication of the summary is misleading because it suggests something the opposite of the witness' actual opinion.
Rejected for several reasons: (a) It is a summary of testimony and not a proposed finding of fact, (b) it involves expert testimony which is based at least in part on assumptions which were not proved at hearing (or the opposite of which was proved at hearing), and (c) there is conflicting expert opinion
testimony, specifically the testimony of Mr. Healy, which I find to be more persuasive and more consistent with other evidence in the record.
Rejected for the same reasons as number 8, above.
Rejected for several reasons: (a) It is a summary of the testimony and not a proposed finding of fact, (b) it is irrelevant (the reasons for its irrelevancy are discussed at further length in the conclusions of law), (c) the testimony upon which the proposed finding is based is not persuasive even where not contradicted [See Nettles v. State, 409 So.2d 85 (Fla. 1st DCA 1982)3 and
there is opposing expert witness testimony which I find to be more persuasive and more consistent with other evidence in the record.
Rulings on Respondent's proposed findings of fact:
Accepted, with additional clarifying details.
Accepted, with minor editorial corrections.
Accepted.
Rejected as inconsistent with the greater weight of the evidence. [A finding on this subject has been made.]
Accepted.
Accepted.
Accepted.
Accepted.
Accepted in substance, with additional findings for context and understanding.
Accepted in substance, with additional findings for context and understanding.
Accepted.
Accepted, with additional findings for context and understanding and correction of minor erroneous details.
Accepted, with the exception of the introductory clause.
Accepted, with additional findings for context and understanding and correction of minor erroneous details.
Accepted in substance in findings which include more extensive details.
Accepted in substance in findings which include more extensive details.
Rejected as irrelevant.
Rejected as irrelevant.
Rejected as being misleading due to being incomplete. [A finding on this subject has been made.]
COPIES FURNISHED:
H. Reynolds Sampson, Esquire Department of Professional
Regulation
130 North Monroe Street Tallahassee, Florida 32301
Manuel Vega Jr., Esquire
477 N.W. 27th Avenue Miami, Florida 33125
Marcelle Flanagan Executive Director Board of Podiatry
Department of Professional Regulation
130 North Monroe Street Tallahassee, Florida 32301
Salvatore A. Carpino, Esquire General Counsel
Department of Professional Regulation
130 North Monroe Street Tallahassee, Florida 32301
Fred Rochep, Secretary Department of professional
Regulation
130 North Monroe Street Tallahassee, Florida 32301
| Issue Date | Proceedings |
|---|---|
| Nov. 01, 1985 | Recommended Order (hearing held , 2013). CASE CLOSED. |
| Issue Date | Document | Summary |
|---|---|---|
| Nov. 01, 1985 | Recommended Order | In exam challenge cases candidates for licensure must prove entitlement to higher grade; proof fails in this case. |
