STATE OF FLORIDA DIVISION OF ADMINISTRATIVE HEARINGS
JOHN EUGENE HARDEN and DOVA CAUTHEN, | ) ) | |
) | ||
Petitioners, | ) | |
) | ||
vs. | ) CASE NOS. | 82-1275 |
) | 82-2924 | |
DEPARTMENT OF PROFESSIONAL | ) | 83-0878 |
REGULATION, ELECTRICAL | ) | |
CONTRACTORS' LICENSING BOARD, | ) | |
) | ||
Respondents. | ) |
)
RECOMMENDED ORDER
Pursuant to notice, a formal hearing was held in these consolidated examination Challenge cases at Miami, Florida, before Michael M. Parrish, a duly designated Hearing Officer of the Division of Administrative Hearings, on the following days: December 17, 18, 19, 20, and 21, 1984; January 9, 10, and II,
February 4, 5, 7, 8, 11, 12, and 14, 1985. The parties also stipulated to the incorporation into the record of these consolidated cases all of the testimony and exhibits received during the hearing in Case No. 84-0309R, which is a rule challenge case involving these same parties. The parties were represented at the hearing by the following counsel: 1/
For Petitioners: Meah Rothman Tell, Esquire
13595 South West 114th Terrace Miami, Florida 33186
For Respondents: W. Douglas Moody, Jr., Esquire
119 North Monroe Street Tallahassee, Florida 32301
At the hearing numerous witnesses were called by both parties and voluminous exhibits were offered into evidence. 2/ The fifteen days of hearing in these cases were reported by court reporters, but no transcript of the proceedings has been prepared 3/ Following the completion of the hearing all parties submitted post-hearing materials to the Hearing Officer in the form of Proposed Recommended Orders containing proposed findings of fact, proposed conclusions of law, and argument in support of their
respective positions. 4/ All of these post-hearing submissions have been given careful Consideration in the formulation of this Recommended Order. Specific rulings on all proposed findings of fact submitted by all parties are contained in the Appendix which is attached to and incorporated in this Recommended Order. The parties waived the requirement in Rule 28-5.402, Florida Administrative Code, providing for rendition of this Recommended Order within 30 days of the hearing. (Even absent the waiver, compliance with the rule would have been impossible due to the fact that the parties requested and were granted over three months within which to submit their proposed recommended orders.)
INTRODUCTION
The basic contentions in these consolidated cases may be summarized as follows: By timely filed petitions and by subsequent intervention granted by the Hearing Officer, the Petitioners, Dova F. Cauthen and John Eugene Harden, challenge certain licensure examinations prepared and administered by the Respondents. The specific examinations challenged in these proceedings are the Electrical Contractors' Licensing Board certification examinations administered on January 7, 1982, and on July 13, 1982. The passing score on both examinations is 75 points. On both examinations both of the Petitioners were assigned grades less than 75 and, therefore, were advised by the Board that they had both failed the examinations on both of the subject occasions.
The subject examinations are challenged on numerous grounds, the most significant of which are discussed hereinafter in the Conclusions of Law portion of this Recommended Order. The examinations as a whole, as well as numerous individual questions within each examination, have been challenged as invalid devices for determining the minimal competency of an individual to practice as a state certified electrical contractor. The Petitioners also seek to have additional points added to their grades by contending that specific questions, or groups of questions, were improper, misleading, ambiguous, or erroneous.
The Petitioners also seek to demonstrate that the examinations are invalid in toto and that, accordingly, the Petitioners should both be licensed without further examination because they have twice been administered invalid examinations.
The Respondents contend that, notwithstanding certain inconsistencies within the examinations, the examinations are nevertheless generally valid for the purposes for which they were constructed, validity being a concept not expressed in absolute terms. Further, Respondents assert that all credit which should be granted as a result of some intrinsic anomaly within an individual question has been granted either during the course of
the statutory review proceedings following the administration of each of the examinations or by stipulation at the hearing.
POST-HEARING MOTIONS
Both the Petitioners and the Respondents have filed several post-hearing motions in these consolidated cases which remain to be ruled upon. The Petitioners' pending motions are: Motion To Amend Petitions filed July 8, 1985; Motion For Expert Witness Fees, Legal Fees And Costs filed January 28, 1986, and Renewed Motion For Leave To Withdraw And For Charging Liens filed February 28, 1986. By letter dated July 2, 1985, Petitioners also requested rulings on two motions related to the appeal taken by Respondents in Case No. AS-401, namely, a Motion To Tax Costs of June 22, 1983, and a Motion For Attorney's Fees of June 29, 1983. The Respondents' pending motion is: Motion For Award of Expert's Fee And For Reduction Of Requested Fees. Rulings on the Petitioners' motions are included in the Conclusions of Law portion of this Recommended Order. The Respondents' motion for payment of expert witness fees is addressed in a separate order issued simultaneously with this Recommended Order.
FINDINGS OF FACT
Based on the stipulations and admissions of the parties, the exhibits received in evidence, and the testimony of the witnesses at the hearing in these consolidated cases, I make the following findings of fact.
Findings based on stipulations and admissions of the parties
The Petitioners, John Eugene Harden and Dova Cauthen, qualified for and were administered the January 1982 Electrical Contractors' Licensing Examination. After their examinations were graded and re-graded, Petitioners were notified that they had not received a "Passing" score of 75 or more.
The Petitioners, John Eugene Harden and Dova Cauthen, qualified for and were administered the July 1982 Electrical Contractors' Licensing Examination. After their examinations were graded and re-graded, Petitioners were notified that they had not received a "Passing" score of 75 or more.
Both Petitioners reside in Dade County, Florida.
The Respondent Department of Professional Regulation (hereafter "DPR" or the "Department") is an agency of the State of Florida.
The Respondent Electrical Contractors' Licensing Board (hereafter referred to as the "Board") is an agency of the State of Florida statutorily responsible, along with DPR, for licensing certified electrical contractors. The office of the Executive Director of the Board is located at 130 North Monroe Street, Tallahassee, Florida.
The business address of Petitioner Harden is Harden Electric, 311 N.E. 8th Street, Homestead, Florida 33030.
The business address of Petitioner Cauthen is 959 N.E. 79th Street, Miami, Florida 23138.
Petitioner Harden specifically requested to review his January and July 1982 Electrical Contractors' Licensing Examinations.
On March 25, 1982, Petitioner Harden went to Tallahassee, Florida, to review his January 1982 Licensing Examination papers, but although he requested an "examination review" he was never provided his own examination booklet or a copy thereof. During this alleged "examination review" Mr. Harden wrote written objections to the January 1982 examination on the forms provided to him based on the master copy of the examination provided to him.
On August 31, 1982, Petitioner Harden went to Tallahassee, Florida, to review his July 1982 Licensing Examination papers, but although he requested an "examination review" he was never provided his own examination booklet or a copy thereof. During this alleged "examination review" Mr. Harden wrote written objections to the July 1982 examination on the forms provided to him based on the master copy of the examination provided to him.
Petitioner Cauthen specifically requested an "examination review" with respect to her January and July 1982 Licensing Examination papers.
On March 17, 1982, Petitioner Cauthen went to Tallahassee, Florida, to review her January 1982 Licensing Examination papers, but although she requested an "examination review" she was never provided her own examination booklet or a copy thereof. During this alleged "examination review" Ms. Cauthen wrote written objections to the January 1982 examination on the forms provided to her based on the master copy of the examination provided to her.
On August 19, 1982, Petitioner Cauthen went to Tallahassee, Florida, to review her July 1982 Licensing
Examination papers, but although she requested an "examination review" she was never provided her own examination booklet or a copy thereof. During this alleged "examination review" Ms.
Cauthen wrote written objections to the July 1982 examination on the forms provided to her based on the master copy of the examination provided to her.
Petitioners Harden and Cauthen sought Board review of their January and July 1982 Electrical Contractors' Licensing Examinations. Petitioners were advised in July 1953 that no copies of their actual booklets exist; Petitioners were advised in October 1982 that their actual booklets were shredded.
With regard to question 71 on the afternoon portion of the July 1982 Licensing Examination, two of the four possible responses, "A" and "C" were credited. Candidates like Petitioner Cauthen who answered "B" received no credit for the question.
DPR destroyed the Petitioners' examination booklets before the end of the two year period immediately following each of the 1982 examinations.
That Section 455.217 of the Florida Statutes requires the Board "by rule" to designate areas of competency to be covered by each licensing examination.
That Section 455.217 states that the Board shall "by rule specify the general areas of competency to be covered by each examination, the relative weight to be assigned in grading each area tested, and the score necessary to achieve a passing grade."
That the amendment of Rule 2IGG-6.01(2) required candidates to commit to memory portions of the electrical code, accounting, law, worker's compensation rules, federal employer's tax guide, AIA General Conditions, business practices, legal and insurance requirements.
In the July 1982 Electrical Contractors' Licensing Examination candidates were given one hour more time in which to take the examination than was given during the January 1982 examination.
That DPR did not and has never provided Petitioners, their attorneys or agents with their own examination booklets for the January and July 1982 Electrical Contractors' Licensing Examinations. DPR has provided Petitioners only with copies of "master" examinations for their review.
That the Respondents produced Notice of Destruction of Examination Booklets and Other Examination Materials dated April 14, 1982, which allegedly evidences destruction of Petitioners' examination booklets for the January 7, 1982, Examination on April 14, 1982.
That the Respondents produced Notice of Destruction of Examination Booklets and Other Examination Materials dated October 15, 1982, which allegedly evidences destruction of Petitioners' examination booklets for the July 13, 1982, Examination on October 15, 1982.
The findings in paragraphs 1 through 23, immediately above, are based directly on the stipulations of the parties in Case No. 84-0309R, the evidence in which has been incorporated into the record in these cases. The findings in the following paragraphs are based on testimony and exhibits, but some of them are also based in whole or in part on stipulations. In the findings which follow there are certain to be at least some repetitious findings in the course of putting matters into context and making additional findings which are related to some of the stipulated findings.
Findings incorporated from Final Order in Case No. 84-0309R
Both of the Petitioners in these consolidated proceedings are individuals who have applied to the Electrical Contractors' Licensing Board for licensure as certified electrical contractors. Both of them have been approved to sit for the licensure examination. Petitioner Harden took the Board's licensure examination on each of the following occasions: July 1981, January 1982, July 1982, and January 1983. The Board has not given him a passing grade on any of those four examinations. Petitioner Cauthen took the Board's licensure examination on two occasions: January 1982 and July 1982. The Board has not given her a passing grade on either of those two examinations.
The Electrical Contractors' Licensing Board certification examination is different from a master electrician examination. However, the two examinations are in many ways similar because of the overlap in the nature of the subject matter to be tested on both examinations.
The passing score or cut score on the local master electrician examination administered by Dade County is 70 percent. Most of the questions on that examination are about the National Electrical Code. Part of the Dade County master electrician examination is closed book.
Prior to 1972, persons wishing to engage in electrical contracting in the state of Florida were required to be licensed by the local governments in the areas in which they sought to operate. Since 1972, persons wishing to engage in electrical contracting in the state of Florida must be licensed by a unit of local government or by the Electrical Contractors' Licensing Board. Persons who are licensed by the Electrical Contractors' Licensing Board are known as certified electrical contractors. A certified electrical contractor can engage in electrical contracting anywhere in the state of Florida without local licensure. Persons who are licensed by one or more local governments (typically a municipality or a county) are known as registered electrical contractors. Registered electrical contractors are licensed to engage in electrical contracting only in the geographic areas encompassed by the boundaries of the local government entities that issued their local licenses.
There is, however, a certain amount of reciprocity from one local
government to another.
Certification as a state certified electrical contractor does not authorize the electrical contractor to work as an electrician, although a certified electrical contractor can pull permits. A person does not have to be a licensed electrician in order to become a state certified electrical contractor, although many electrical contractors are also licensed as journeyman or master electricians. Candidates for the electrical contractor licensure examination have a great variety in the nature and scope of their background and experience. This variety in background and experience is among the reasons which cause testing for minimum competence as an electrical contractor not to be an exact science.
The Electrical Contractors' Licensing Board administered its own certification licensure examination from 1972 through 1980. During that period of time the Board administered the examination a total of twenty-two times. During those twenty-two sessions of Board administered examinations, a total of 824 candidates sat for the examination, of which 392 were successful.
In 1979, the regulation of professions and occupations was reorganized. The Department of Professional Regulation (the Department) was created as an umbrella agency over numerous boards, including the Electrical Contractors' Licensing Board and the Construction Industry Licensing Board. The responsibility for examining applicants, which in the case of the Electrical Contractors' Licensing Board had been handled by the Board members themselves, was taken over by the Office of Examination Services (O.E.S.).
Previous Board examinations were open book with certain reference material allowed. When the Office of Examination Services took over the administration of the January 1981 exams, it sent to the candidates a form letter adapted from the Construction Industry Licensing Board. This form letter instructed the candidates that they would be able to bring into the examination any notes and other materials desired. This was contrary to the past practice of the Board and contrary to the Board's wishes.
The Board was very frustrated and concerned about the circumstances which led up to the January 1981 examination because the O.E.S. had refused to allow the Board to have any input into the examination and the Board thought the O.E.S. had come up with an apprentice level examination. In the Board's opinion, the January 1981 examination prepared by O.E.S. clearly tested at a level less than that of minimum competency. And although O.E.S. did not agree with the Board that the January examination was invalid, O.E.S. did agree that the examination was a lot easier than they had thought it would be.
The Electrical Contractors' Licensing Board did not conduct any formal studies or formal analysis prior to making the 1981 amendments to Rule 21GG-6.01 changing the format of the examination and the cut score. However, the Board's Rules Committee did a great deal of work on the matter and the matter was discussed extensively at Board meetings. In this regard it is important to keep in mind that the Board members (with the exception of the two lay members) were all experienced practicing electrical contractors who were familiar with the requirements of day-to-day electrical contracting.
The Board's decision to change part of the examination to closed book was due to a concern that people with no electrical knowledge could become licensed if the examination was all open book and had a low percentage of technical questions. The Board felt that more than half of the examination should be technical questions. The closed book National Electrical Code questions were for the purpose of testing what a person with experience in the field of electrical contracting should know about everyday matters.
The primary, if not the sole, motivation for the Board's 1981 amendments to Rule 21GG-6.01 was the Board's concern about the quality of the O.E.S. administered examination and the Board's feeling that, due to its estranged relationship with O.E.S., the only vehicle through which it could effectively influence the quality of future examinations was through rulemaking. Those amendments were not motivated by any desire on
the part of the Board to restrict competition in the field of electrical contracting.
During the past few years the job requirements of an electrical contractor have remained substantially the same.
Block and Associates (hereinafter "Block") is a company based in Gainesville, Florida, that writes licensure examinations for contractors in various trades. Block writes such examinations for cities, counties, and states. Block has prepared electrical contracting licensure examinations for the states of Florida, Georgia, and Oklahoma, as well as for St. Johns County, Florida. Block prepares licensure examinations for just about anything that involves electricity, including such things as electrical contracting, electrical journeyman, electrical master, low voltage, elevator, burglar alarm, and fire alarm. Block prepares master electrician examinations for over one hundred governmental entities. The Block electrical master exams and the Block electrical contractor exams use some of the same questions.
Subsequent to the 1982 examinations which form the gravamen of these proceedings, the Board has contracted with Block and Associates to prepare the Board's certification licensure examination. The Board has never said anything to Block about wanting to achieve any particular passing rate or wanting to have any particular percentage of the candidates taking the examination achieve a passing score. The Board has never interfered with Block's autonomy in the preparation of the electrical contractor exam.
All of Block's electrical contractor examinations include business questions. It is important to test for business skills because an electrical contractor who gets into business trouble may start cutting corners and cutting corners is a hazard to the public. Some of Block's electrical contractor examinations also have specific sections on safety.
Block always does a post-administration statistical analysis of the examination questions it uses. The reason for this is to find out if anything is wrong with the questions; in other words, to determine whether the questions are valid. The computer program used by Block to check the validity of its tests was prepared for Block by a professor in the College of Education at the University of Florida. Block uses a computer to conduct a validity analysis of each test it administers.
The Board's purpose in amending Rule 21GG- 6.01(1)(c) to delete the reference to "72A, B, C and D" was to make it clear to candidates for examination that the Board intended to include
in the safety portion of its examinations questions from portions of Volume Seven of the NFPA other than Chapters 72 A, B, C, and D of that volume. Volume Seven of the NFPA contains quite a bit of material in addition to the material included in Chapters 72 A, B, C, and D of that volume. In view of the modifying phrase "shall include but not be limited to," the amendment to delete the reference to Chapters 72 A, B, C, and D was not a necessary prerequisite to the use of examination questions based on other portions of Volume Seven of the NFPA. The Board could ask the same examination questions before and after the amendment.
Several weeks before each examination, the Department of Professional Regulation sent all candidates for examination, including these Petitioners, a notice to appear for the examination. The notice to appear included a list of reference books on which the examination was to be based. That list of reference books contained a specific reference to Volume Seven of the NFPA. All sixteen volumes which comprise the entire NFPA were not listed as reference books.
Neither of these Petitioners demonstrated that he or she was genuinely confused or misdirected by the amendment to Rule 21GG-6.01(1)(c). Neither Petitioner contended he or she had studied all sixteen volumes of the NFPA. Instead, both Petitioners studied from the books on the reference list.
In November of 1983 the Electrical Contractors' Licensing Board again amended Rule 21GG-6.01(3) to change the relative grading weights of the areas of competency to be tested. After November of 1982 the relative grading weights of the areas of competency were as follows: Technical increased from 60 percent to 65 percent; electrical calculations increased from 40 percent to 45 percent; code- related questions remained 20 percent of the examination; general business was decreased from
30 percent to 25 percent of the examination; and safety remained
10 percent of the examination.
The licensing examination given by the Board has historically contained materials testing knowledge of electrical work and also materials testing the business qualifications of the applicants. Since the inception of the Board's licensing exam, the electrical work portions have contained electrical calculations.
Throughout the time the Board has been administering licensure examinations for certification, the percentages of the examination devoted to particular subject areas have been based upon the Board's belief as to the appropriate levels of knowledge needed by a certified electrical contractor. The weighting of the examination was equitable when it was weighted 60 percent
technical, 30 percent business, and 10 percent safety, and it is still equitable now that it is weighted 65 percent technical, 25 percent business, and 10 percent safety. The Board changed to the current percentages at the suggestion of Block and Associates. The Board's examination has had similar ratios for a long time, at least as far back as 1971.
The proportions of the examination devoted to each particular subject matter did not change due to the March 1981 amendments. With the exception of the January 1981 examination put together by O.E.S., electrical calculations had historically been 40 percent of the test. Since the Board felt this percentage should be maintained, and since it was necessary to divide those calculations from the other portion of the technical section (since one would be tested in the closed book portion and the other in the open book portion), the division of the technical part of the exam was set out in the rule.
On an electrical contractor licensure examination it is appropriate to include in the area of "technical" questions, questions dealing with the following subjects: electrical calculations, parts of estimating, and designs and electrical schematics. In the "business" portion of an electrical contractor examination, it is appropriate to include questions dealing with the following: accounting, worker compensation, and social security.
Local licensing examinations had, in 1981, included a closed book portion. Local licensing examinations and state licensing examinations given in other states still include such a closed book portion. The basic Block examination has always included a closed book portion.
Closed book examinations are not all that unusual in occupational and professional licensure testing. The CPA and nursing examinations are closed book. Also, virtually all of the electrical examinations administered by Block and Associates include a closed book portion. The advantage of a closed book examination technique is that it is easier to identify the candidates who have had practical experience in the subject matter being tested. A closed book examination is a better test for field experience and for general knowledge of the subject matter being tested.
Certain questions known as "anchor" questions were given both in the open book examination before the format change and in the closed book section after the format changed. The relative performance by candidates on the same questions in the two formats shows the format change not to have prejudiced the candidates.
Further, an examination analysis of the results of the closed book portions of the Board examinations reveals that, based on candidate performance, the closed book portion of the examinations was easier than the open book portion. This also shows that the format change did not prejudice the candidates.
To the extent that the content domain of Board examinations was established by the March 1981 amendments to Rule 21GG-6.01(3), the content domain was established on the basis of the expert judgment of persons who were experts in the field of electrical Contracting, namely, the members of the Electrical Contractors' Licensing Board. The relative grading weights established in the 1981 amendments to Rule 2100-6.01(3) have at least a possible correlation to electrical contractor competence. In fact, the persuasive expert testimony establishes that those relative grading weights did bear a reasonable relationship to electrical contractor competence. The current slightly different relative grading weights also bear a reasonable relationship to electrical contractor competence even though there has been no recent change in the nature of the responsibilities of electrical contractors. In this regard it must be remembered that testing is not an exact science and no test for minimum competency can ever be expected to be an exact model of actual work experience. The best that can be expected is a reasonable model.
There are various accepted methodologies for the creation of standardized examinations, adherence to which tends to enhance the likelihood that the examination instrument finally produced will be valid and reliable. However, adherence to those accepted procedures is not a necessary prerequisite to creation of a valid and reliable examination. This is particularly true when one is going to be testing small groups of people with varied backgrounds instead of the massive groups of people with similar backgrounds for whom standardized tests are more typically designed. In the final analysis the only practical and reliable measure of the validity of an examination is by statistical analysis of the examination after it has been administered.
Given the nature of the circumstances faced by the Board at the time of the March 1981 amendments to the examination format and cut score, given the nature of the pool of candidates to be examined, given the nature of the changes contemplated by the Board, and given the very nature of the process of testing for minimum competency--which involves perhaps as much art as it does science--there is no study or data which would have been particularly useful to the Board in helping to determine exactly what the effect of their changes would be. Such effects can only be determined or measured with any degree of accuracy after the
administration of an examination that incorporated the changes. Following the administration of such an examination, it is possible to perform a statistical analysis of all questions used on the examination and to eliminate or give credit for any questions which are shown by statistical analysis to be invalid or unreliable. This is precisely the process that is used by Block in the validation of their examinations and is an accepted testing procedure.
It would have served no useful purpose to have conducted a trial run of an examination using licensed certified electrical contractors as a test group for the new examination format. First, it would be virtually impossible to try to put together an accurate cross-section of certified electrical contractors to use as a test group. Second, one would expect them all to pass the examination, so when they did so nothing of value would have been learned. Finally, the administration of such a trial run would risk the possibility of compromising examination question security.
One aspect of accepted methodology for the preparation of standardized examinations is the definition of the content domain of the examination, i.e., a determination of what knowledge is essential to demonstrate that the candidates for licensure are minimally competent. This aspect of examination preparation is often accomplished by performing a formal job analysis, which is, in essence, a study of all of the usual tasks performed by a person engaged in the occupation or profession to be tested, including an evaluation of the relative importance of each of those tests to minimum competence. The content domain can also be defined on the basis of the judgment of a group of experts in the occupation or profession to be tested.
With the exception of the lay members who were added in recent years, all of the members of the Electrical Contractors' Licensing Board are, and have been, persons certified to engage in electrical contracting in the state of Florida and actively engaged in the electrical contracting business. Therefore, at all relevant times all of the professional members of the Board had extensive personal knowledge of what was involved in the practice of electrical contracting, which personal knowledge was as useful in defining content domain as would have been a formal job analysis. (In this regard it is important to note that even with the addition of lay members to the Board in recent years, the experienced professional members have continued to constitute a substantial majority of the Board.)
When examination booklets are being prepared by the Department of Professional Regulation prior to an examination, the booklets are carefully inspected to make sure that all of the
booklets are identical. Following that inspection the booklets are sealed and stored in a secure place in order to insure, among other things, that no changes are made to any of the examination booklets before they are handed out to the candidates.
After an examination is given, the Department of Professional Regulation retrieves all of the examination booklets, including all booklets that were used by all of the candidates, and retains them in a secure place until the excess booklets can be destroyed. The examination supervisor selects the booklets which are to be retained from among the booklets that are not handed out to the candidates at the examination. In the normal course of events all of the examination booklets that were actually handled by the candidates at the examination are destroyed within a very few months of the date of the examination.
With the exception of the examination booklet of one other candidate (which exception is not relevant to the disposition of these proceedings), all of the examination booklets which were handed out to candidates during the examinations taken by these two Petitioners were destroyed approximately 90 days after each of the examinations. When such destruction took place, the Department retained copies of the examination booklets which had not been used by any candidate, which copies were identical to the copies that had been handed out to the candidates during the examination.
As part of the examination instructions, all candidates for examination are advised not to write anything in their examination booklets because all of the booklets used by the candidates will be shredded. They are specifically told to do all of their computations on sheets of work paper that are provided to them at the examination. All candidates are specifically told that the only things they turn in that will be saved are their answer sheets and their sheets of work paper.
The findings in paragraphs 25 through 63, immediately above, are substantially identical to certain of the findings of fact made in the Final Order in Case No. 84-0309R. They are, of course, based on the evidence received during the hearing in that case, which evidence has by stipulation been incorporated into the record in these cases. Most of the findings which follow are based on evidence received during the hearing of these examination challenge cases, bat some of them are also based on the evidence incorporated from the record in Case No. 84-0309R.
The rest of the findings
The examination prequalification process imposed by the Board is designed to ascertain whether an individual meets the professional criteria to attempt the examination and has no specific relationship to whether a candidate will pass or fail the examination.
The January and July 1982 examinations were administered in geographical locations accessible to the Petitioners and although there were certain minor distractions and interruptions which occurred during the administration of both examinations, there were no major irregularities in the administration of either examination. Specifically, none of the irregularities in the administration of either examination was of sufficient magnitude to adversely impact the fundamental fairness of the manner in which the examinations were administered. The proof in this case is insufficient to show that the examination conditions on either occasion were such as to have interfered with the abilities of either of these Petitioners to choose correct answers on the examinations.
The copies of the examination booklets which were furnished to the Petitioners at their examination reviews and thereafter were true and correct copies of the examination booklets which were given to all candidates at the time of the examinations. There is no persuasive or convincing evidence that either of these Petitioners has suffered any harm as a result of not being able to see the very same booklets that they personally used during the examination. The evidence in this regard is purely speculative. And, in any event, in view of the instructions to all candidates that their individual examination booklets would be shredded and that only their answer sheets and work paper sheets would be saved, if the Petitioners wrote useful information in their examination booklets and failed to also write that information on their work paper sheets, they were the authors of their own inconvenience and are not entitled to any relief on the basis of a matter that they could have prevented.
When the Board conducts its post-examination review, it has been the policy of the Board to grant credit to all candidates for questions which are determined by the Board to be "bad items" for one reason or another. The Board has also on occasion granted credit for either of two answers when in the judgment of the Board it appeared that either of the two answers should be considered a correct answer. Both of these types of adjustments to the grading of the exams are reasonable ways to compensate for shortcomings in the examination instrument depending upon the particular circumstances of a given examination question.
The examination scores achieved by the Petitioners when their scores were properly tabulated in the manner in which the scores of other candidates were tabulated are as follows:
January 1982 examination:
Dova Cauthen 68.0
John Harden 52.0
July 1982 examination:
Dova Cauthen 73.0
John Harden 61.0
The foregoing tabulation of scores takes into consideration every reasonable manner in which examination scores could be tabulated. The foregoing tabulation ascribes the highest score possible to each candidate under all scenarios which are consistent with the greater weight of the evidence.
During the course of the hearing, it was stipulated that, on the basis of proof at the hearing, both Petitioners were entitled to have certain additional points added to their grades where the proof demonstrated that the answers chosen by the Petitioners were as good as the keyed answers.
On the basis of the tabulated scores plus the stipulated additions to all of those scores, both of the Petitioners would still be assigned failing grades. Accordingly, it is necessary to address the remainder of the Petitioners' contentions to determine whether they are entitled to have further points added to their scores or are otherwise entitled to licensure regardless of whether their scores can properly be raised to a passing level by the award of additional points.
Findings regarding examination directions
The directions which accompanied the challenged examinations are clear and to the point. They are easy to understand and consist of the type of information one would normally expect to see in the directions for a multiple choice examination. The most persuasive of the Petitioners' testing experts said the directions were good examination directions, and I so find.
The Petitioners contend that, as applied, the directions are misleading and confusing because they instruct the candidates to mark only one choice for each question, but in some instances there was more than one correct answer among the choices. This is a shortcoming not of the directions, but of some of the questions and the answer choices that were given with
them. In those instances in which a question does contain two correct answer choices or in which none of the answer choices are correct under any reasonable interpretation, the remedy adopted by the Respondent, which is to grant credit for additional responses and in some instances to grant credit for all responses, is an appropriate and fair resolution to any problem created by the application of the directions to a problematic question.
Findings regarding "domino" questions
The Petitioners have challenged a number of questions on the grounds that they are unfair or improper because they are "domino" questions. "Domino" questions are questions which require for their solution the application of information gleaned from obtaining the correct answer to one or more prior questions. Because the correct answer to one question depends upon application of the correct answer to one or more prior questions, it is argued that "domino" questions are inherently unfair inasmuch as an incorrect answer to an early question virtually guarantees an incorrect answer to any later "domino" questions.
The "domino" question format is not a preferred testing format in many testing situations. Nevertheless, it appears to be a technique particularly well suited for testing for minimal competency in the field of electrical contracting because it provides a testing model which closely parallels functions electrical contractors are required to perform on a regular basis in the practice of their profession. In the practice of electrical contracting, it is common for a contractor to be required to make a series of calculations, each of which incorporates the answer obtained in one or more prior calculations. Accordingly, the "domino" question format is an appropriate format for testing candidates for licensure as electrical contractors. The "domino" question format is also used on other professional examinations in Florida.
Findings regarding "Walker" questions
The closed book portion of both examinations contained questions taken from an approved reference source entitled Practical Accounting and Cost Keeping for Contractors by Frank R. Walker Company, Eighth Edition. The Walker reference is a collection of materials regarding various topics related to bookkeeping and record management. The accounting information contained therein is fundamental. The Walker reference uses a number of antiquated terms and peculiar phrases. Nevertheless, most of the concepts in the Walker reference are rather straight- forward and uncomplicated. The Walker reference book could not be regarded as a very good textbook on the subject of accounting,
but it is adequate for the purposes for which it was intended. Since the administration of the subject examinations, the Board has become disenchanted with the Walker book and has replaced it with another book.
On a variety of different grounds, the Petitioners have challenged each and every one of the questions taken from the Walker reference. With the one exception noted below, the Petitioners' challenges to the questions taken from the Walker reference are without merit because virtually all of the questions are drawn directly from material in the Walker reference and in virtually all cases the language of the correct answer choice is the same, as or is a close paraphrase of, some explicit statement in the Walker reference. Anyone familiar with the Walker reference should not have any trouble selecting the correct answers to the Walker based questions. Furthermore, even without any familiarity with the Walker reference, a person with a reasonable amount of business experience should have been able to answer many of the questions just on the basis of a common sense application of that business experience. In sum, the Walker derived questions are fair and reasonable questions.
The one exception to the foregoing is question 80 on the January closed book examination. Answer choices "C" and "D" should also be credited on this question. Accordingly, both Petitioners should receive credit for this question.
Findings regarding the January "load" questions
On the January 1982 examination all candidates (including the Petitioners) were given credit for all answers to a series of load questions (questions 13 through 20 of the open book portion) on that examination. For reasons which are not elucidated in the record in these cases, it was apparently determined by the Department or by the Board that those load questions were "too difficult." The Petitioners contend that because credit was given for all answers to all load questions on the January examination, the same should be done with regard to all of the load questions on the July examination. However, the Petitioners have failed to prove on what basis the January load questions were deemed to be "too difficult" and, therefore, there is insufficient evidence upon which to find whether the July load questions are "too difficult" for the same reasons. As found further below, standing alone, the July load questions do not appear to be too difficult.
Findings regarding the July "load" questions
Like the January exam, the July 1982 examination also contained a series of questions referred to as the "load"
questions. The purpose of these questions (some of which were "domino" questions) was to require the candidates to demonstrate an ability to determine various amperages, voltages, and conduit sizes based on the content, character, and size of the buildings and equipment in a hypothetical shopping center described in an exhibit which was included in the examination booklet.
The Petitioners have challenged all of the July load questions that either or both of them missed, asserting, inter alia, that the Board's solution to most of the questions is erroneous. For the reasons set forth below, I find that the Petitioners are not entitled to have any points added to their scores on the basis of their challenges to the load questions other than the one point the Respondents stipulated should be added to Petitioner Harden's score.
Both Petitioners missed question 66 in the load series. Both Petitioners missed this question because they both omitted a crucial and undisputed step in their calculations. Both failed to take into account the continuous load factor. The reason for their incorrect answers had nothing to do with the issues in dispute between the experts as to how other aspects of the question should be calculated.
Petitioner Harden answered question 68 correctly. Petitioner Cauthen chose an incorrect answer for reasons which are not elucidated in the record. Nevertheless, it would appear that her incorrect answer was not due to any defect in the question, because her answer choice is inconsistent with either of the answers advanced by the competing expert witnesses.
Petitioner Harden answered questions 69 and 70 correctly. There is no showing of any causal connection between Petitioner Cauthen's incorrect answers to these questions and any defects in the Board's solution to the questions. To the contrary, it appears most likely that Petitioner Cauthen missed these questions by using information from her incorrect answers to prior questions.
Petitioner Harden answered question 71 correctly. Although the competing experts used different methods to calculate the answer, they both came up with the same answer. Thus, there is nothing more than speculation or conjecture as to why Petitioner Cauthen missed this question.
It is stipulated that Petitioner Harden should be given credit for question 72 because an alternative method of calculating the answer would result in the answer he chose. However calculated, the only arguably correct answers are the one chosen by Petitioner Harden and the one chosen by the Board and
the competing experts. Thus, there is nothing more than speculation or conjecture as to why Petitioner Cauthen chose a different answer.
Both Petitioners missed question 73. The experts for the opposing parties disagreed on how this question should be answered. But however it should be answered, the fact remains that neither of the Petitioners appears to have been familiar with either of the methods of solution proposed by the experts, because both chose answers that were inconsistent with those methods of solution. Thus, even assuming that the Board's solution was incorrect, there is no proof that any such shortcoming caused either of the Petitioners to miss this question. In any event, I am persuaded that the Board's solution was the better of the competing solutions offered at hearing.
Both Petitioners missed questions 74 and 75. They both appear to have missed both of these questions because of choosing incorrect answers to prior questions and not because of anything improper or incorrect in questions 74 and 75.
Both Petitioners missed question 76. Nothing improper or incorrect about the question affected the Petitioners' ability to select the correct answer, because under any interpretation of the correct solution to the question, the correct answer was "none of the above."
Petitioner Harden answered question 77 incorrectly. Nothing improper or incorrect about this question affected his ability to select the correct answer. Although the answer depended in part on the answer to a prior question, the correct answer would be calculated using any of the solution methods proposed by the expert witnesses.
Findings regarding National Electrical Code Questions
Question 18 on the January closed book examination and question 20 on the July closed book examination are challenged as being "tricky" questions. While the selection of the correct answer to these questions requires close attention to detail, both questions are fair and reasonable.
Question 22 on the July closed book examination contains a minor typographical error. Despite the typographical error, 84 per cent of the candidates taking the examination selected the correct answer. Accordingly, I find that the minor typographical error did not have any significant adverse impact on the ability of the candidates to understand and answer the question.
Question 26 on the January closed book examination is a fair and reasonable question. It was answered correctly by 92 per cent of the candidates.
Question 43 on the July closed book examination is a fair and reasonable question. The correct answer is derived directly from a specific exception to a provision of the National Electrical Code. It is reasonable to expect candidates for licensure as electrical contractors to be familiar with exceptions to the Code.
Question 47 on the July closed book examination is a fair and reasonable question if credit is given for both responses "C" and "D." Both Petitioners selected response "A," so they are not entitled to additional credit for this question.
Question 51 on the July closed book examination would have been easier to understand if the word "`metal" had been inserted before the word "raceway" in the stem of the question. But the evidence in this case is insufficient to establish that the omission of the word "metal" confused the Petitioners and caused them to miss this question. Their answers are consistent with their having assumed that the question involved a metal raceway, but also having simply chosen the wrong answer.
Question 53 on the July closed book examination has only one correct answer, "seal or plug." The Petitioners' answer to this question is not a reasonable alternative answer.
Question 78 on the January open book examination and question 25 on the July open book examination are both fair and reasonable questions. The omission of the power factor did not make it unreasonably difficult to answer these questions.
Question 80 on the January open book examination and question 38 on the July open book examination are fair and reasonable questions. The fact that the question stem does not specifically state that the ranges are residential or commercial does not appear to have affected the ability of candidates to answer the question. Further, it is implicit from the available answer choices that the question contemplates residential ranges.
Question 43 on the July open book examination is a fair and reasonable question if credit is given for both response "A" and "D." There is no persuasive competent substantial evidence that response "B" is an acceptable answer. Accordingly, Petitioner Cauthen is entitled to credit for this question, but Petitioner Harden is not.
Question 53 on the July open book examination was miskeyed. Accordingly, Petitioner Harden should be given credit for this question. (Petitioner Cauthens original score has been computed on the basis of her having correctly answered this question.)
Question 58 on the July open book examination has an ambiguous last sentence in the stem. As a result of that ambiguity, credit should be given for both responses "C" and "D." Accordingly, Petitioner Harden should be given credit for this question.
Question 28 on the January open book examination has a word missing from the stem. Nevertheless, 73 per cent of the candidates were able to answer the question correctly. Accordingly, I find that the missing word did not have any significant adverse impact on the ability of the candidates to understand and answer the question.
Question 41 on the January open book examination did not contain a correct answer in any of the answer choices. Accordingly, it is stipulated that Petitioner Harden should receive credit for this question. (Petitioner Cauthen's original score has been computed on the basis of her having correctly answered this question.)
Question 42 on the January open book examination was miskeyed. Accordingly, Petitioner Cauthen should be given credit for this question. (Petitioner Harden's original score has been computed on the basis of his having correctly answered this question.)
Question 44 on the January open book examination was miskeyed. Accordingly, Petitioner Cauthen should be given credit for this question, but Mr. Harden should not.
Question 63 on the January open book examination is a fair and reasonable question. Although Petitioners contend the question is confusing, the fact remains that it was clear enough for 75 per cent of the candidates to select the correct answer.
Question 43 on the January closed book examination contains a minor typographical error. Despite the typographical error, 90 per cent of the candidates taking the examination selected the correct answer. Accordingly, I find that the minor typographical error did not have any significant adverse impact on the ability of the candidates to understand and answer the question.
Question 45 on the January closed book examination had four possible correct answers, depending upon which exception to Section 300-5, National Electrical Code, was applied. This question, unlike others appearing on the examinations, gave no indication in the content of the question stem as to whether the exceptions should be applied. Accordingly, all responses should be treated as correct and both Petitioners should be given credit for this question.
Question 16 on the January closed book examination was miskeyed. Credit should be given for responses "A" and "B." Accordingly, Petitioner Harden should be given credit for this question. (Petitioner Cauthen's original score has been computed on the basis of her having correctly answered this question.)
Question 26 on the January closed book examination is a fair and reasonable question. Although Petitioners' expert described a complex way of answering the question which would lead to a different answer, the fact remains that there is a fairly simple solution which leads to the correct answer which was used by 88 per cent of the candidates to select the correct answer.
Question 69 on the January closed book examination does not have any correct answer choice. Accordingly, both Petitioners should be given credit for this question.
Findings regarding financial analysis questions
A major point of contention in these cases concerns the propriety of a series of questions on both the January and July examinations which can perhaps most accurately be described as financial analysis questions. This group of questions has been challenged on a variety of grounds including, (a) the candidates were not provided with a reference source for the questions, (b) the questions were too difficult for the skill level of the group being examined, (c) in order to obtain the correct answer to some questions the candidates had to guess which way the Board expected them to round their answers, and (d) that the exhibits used in conjunction with the financial analysis questions were incorrect and inadequate.
With regard to the assertion that the financial analysis questions were too difficult for the skill level of the group taking the examinations, Petitioner Cauthen correctly answered 17 of 20 of this category of questions on the January examination and all 20 in this category on the July examination. Statistical analysis of the performance of all candidates on this category of questions reveals that on the January examination an average of 74.8 per cent of all candidates responding answered
the financial analysis questions correctly, and on the July examination 81 per cent of all candidates responding answered the financial analysis questions correctly. These averages are significantly higher than the average percentage of correct responses achieved on the remainder of the questions on the morning portion of the two examinations. Accordingly, I find that the financial analysis questions were not too difficult for the skill level of the group being tested.
Some of the financial analysis questions required the application of knowledge or information that is not specifically covered in any of the reference books listed in the notices to appear for the examinations. Petitioners contend that these questions are therefore unfair because even if they had studied all of the reference books, they would not have learned what they needed to know to answer some of these questions.
The high percentage of correct answers achieved by all candidates on the financial analysis questions indicates that these questions were quite fair. All of the concepts that were involved in the solution of the financial analysis questions were concepts that an experienced businessman would normally be expected to be familiar with. All of the financial analysis questions were at a difficulty level such that a typical high school graduate who had some familiarity with bookkeeping should have been able to answer them. The unfairness contention is also dispelled by the fact that the notices to appear specifically stated that the examinations would include questions regarding financial statements, which put all candidates on notice to be prepared to answer such questions. Both of the Petitioners attended examination preparation courses, which courses included financial statement analysis in the course materials.
The exhibits relating to the financial analysis questions which were included in the examination booklets contained all of the information necessary to calculate the answers to the financial analysis questions. The exhibits were fairly comprehensive and contained the type of basic "nuts and bolts" information a businessman would use for his own internal business purposes.
The answer choices to several of the financial analysis questions were such as to require the candidate to round the calculated answer in order to determine the correct response. Some of the rounding functions were not in strict accordance with classical rounding principles and on some occasions the calculated answer would have to be rounded down, for example, when the classical rule would have mandated rounding up to the nearest whole number. The examination instructions did not tell the candidates what type of rounding principles they should use.
The foregoing notwithstanding, irregularities in rounding do not tend to have an adverse effect on the performance of examination candidates unless the candidates are required to choose between very close answer choices. In the subject examinations there is only one question which arguably presented a rounding irregularity in conjunction with very close answer choices. This was question 32 on the January open book examination, which was missed by Petitioner Harden. However, even though it is possible that the format of the answer choices to question 32 confused Petitioner Harden and caused him to choose the wrong answer by rounding his calculated answer, it is equally possible that Petitioner Harden missed this question for some other reason. The evidence is insufficient to show that the format of question 32 on the January open book examination is what caused Petitioner Harden to miss it when so many others got it right. What the evidence does show (both through his Scores and his candid admission) is that Petitioner Harden has difficulty solving mathematical problems that involve the calculation of or application of percentages.
Question 39 on the January closed book examination should be credited to Petitioner Harden for the same reasons that the identical question was credited to all candidates on the July examination. (Petitioner Cauthen's original score has been computed on the basis of her having correctly answered this question.)
The evidence is insufficient to establish that the order of questions 8, 9, and 10 on the January closed book examination had any adverse effect on Petitioner Harden's ability to select the correct answer to those questions.
Findings regarding safety questions
In light of the language of the rule describing what as meant by "safety" questions, and the information about safety questions in the notices to appear for the examinations, there was nothing improper or unfair about the subject matter of the safety questions that appeared on the subject examinations.
Findings regarding miscellaneous questions
Question 23 on the January closed book examination is a fair and reasonable question. It has only one correct answer choice, which choice was correctly selected by 83 per cent of all candidates.
Question 50 on the January closed book examination is a fair and reasonable question. It has only one correct answer
choice, which choice was correctly selected by 92 per cent of all candidates.
Question 80 on the July closed book examination is a fair and reasonable question. The question contains all the information that is necessary to select the correct answer.
On the basis of the findings in the foregoing paragraphs, the tabulated scores of the Petitioners should be adjusted as follows:
January 1982 Examination
Dova Cauthen: Tabulated score of 68, plus addition of 3.5 points, equals final score of 71.5
John Harden Harden: Tabulated score of 2, plus addition of
4.0 points, equals final score of 56.0 July 1982 examination
Dova Cauthen: Tabulated score of 73, plus addition of 0.5 points, equals final score of 73.5
John Harden: Tabulated score of 61.0, plus addition of 2.0 points, equals final score of 63.0
Findings regarding several other matters
In the normal course of events, the questions on a licensure examination should be job related. The licensure examinations prepared by Block and Associates are designed to test for everyday knowledge. The Board is of the opinion that the questions asked on the subject examinations directly relate to the everyday practice of electrical contracting.
The National Electrical Code is Volume 6 of the Fire Safety Code.
Budgetary considerations of necessity affect the quality and nature of examinations prepared for professional licensure. For obvious reasons it would be unreasonable to spend millions of dollars to develop an examination for just a few people. Without question, the examinations at issue here are not as well constructed as the CPA examinations. Not nearly as much effort was put into the construction of these examinations as is typically put into the construction of a CPA examination. The subject examinations could perhaps even be fairly described as mediocre, at best. But on the whole, and in light of the circumstances under which the examinations were given, these
examinations were reasonable. They bear a reasonable relationship to the task they seek to accomplish and they do a reasonable job of giving each candidate a fair chance to demonstrate minimal competence as an electrical contractor.
Statistical analysis of the results of the January and July 1982 examinations indicates that those examinations, as adjusted to grant additional credit for some questions, are reasonably valid and reliable testing instruments.
At least one person who failed one of the subject examinations has since been a successful candidate on a later examination. Where, as here, it is contended that candidates have been subjected to an invalid examination, the remedy which best addresses the competing interests of the candidates' desire for licensure and the public's need for protection is for the candidate to retake the examination.
CONCLUSIONS OF LAW
Based on the foregoing findings of fact and on the applicable legal principles, I make the following conclusions of law:
The Division of Administrative Hearings has jurisdiction over the parties to and the subject matter of these consolidated proceedings. (See, generally, Sec. 120.57, Fla. Stat.)
Section 489.501, Florida Statutes, reads as follows:
The Legislature finds that electrical contracting is an important service and potentially dangerous if not properly provided and, therefore, deems it necessary in the interest of public health, safety, and welfare to regulate the electrical contractors in this state.
In implementation of the foregoing finding the Legislature has imposed a requirement that those who seek to be licensed as certified electrical contractors shall be required to pass an examination (Sections 455.213(1) and 489.511(1) and (2), Florida Statutes) and has also established certain educational and experience prerequisites to taking the examination (Section 489.521(1) and (2), Florida Statutes).
Section 455.217, Florida Statutes, provides as follows, in pertinent part:
The Division of Examination and Licensure of the Department of Professional Regulation shall provide services for the preparation and administration of all examinations.
The department . . . shall ensure that the examinations adequately and reliably measure an applicant's ability to practice the profession regulated by the department and shall seek the advice of the appropriate board in the preparation and administration of the examinations. After an examination has been administered, the board may reject any question which does not reliably measure the general areas of competency specified in thee rules of the board.
At the time of the subject examinations, Rule 21GG- 6.01(1), (2), (3), and (4) of the Electrical Contractors' Licensing Board read as follows:
The areas of competency to be covered by the certification examination shall be as follows:
Technical, which shall include, but not be limited to, electrical calculations, estimating, designs, and electrical schematics; and which shall be divided into two (2) separate areas, one
containing electrical calculations, the other containing Code-related questions not requiring calculations.
General business, which shall include but not be limited to, problems relating to accounting, law, insurance, workers' compensation and social security;
Safety, which shall include but not be limited to, OSHA regulations, life safety codes, and the Fire Safety Code (NFPA).
The format of the examination shall be as follows:
The portions of the examination containing the electrical calculations and safety questions shall be open book. The applicant is responsible for bringing and may use during these portions the applicable code books, reference materials as approved by the Board, and noiseless mechanical or non- mechanical instruments he wishes to use.
The portions of the examination containing the non-calculation technical questions and all business questions shall be closed-book. The applicant will not be permitted to use the reference materials listed in 2(a) or any other reference materials.
Security measures as set forth by the Department shall be followed during both portions of the examination.
The relative grading weight to be assigned to each area of competency shall be approximately as follows:
Technical - 60 percent
Electrical Calculations-40 percent of Test
Code-related questions-20 percent of Test
General business - 30 percent
Safety - 10 percent
An applicant shall be required to achieve a score of a general average of not
less than seventy-five percent (75 percent) in order to pass the examination and be certified for licensure. There shall not be a practical or clinical examination.
The criteria to be evaluated in the examination, the content of the examination, the grading criteria to be used in considering the adequacy of an applicant's response to the requirements of the examination, as well as the criteria for determining passing grades, are contained in Rules 2100-6.01, 2100-6.03, and 2100-6.04, Florida Administrative Code.
The Petitioners have the burden to show by a preponderance of the evidence that the grades they were given on their examinations were arbitrarily or capriciously given by the examining board. See State ex rel. Olaser v. J. M. Pepper, 155 So.2d 383 (Fla. 1st DCA 1963), in which a candidate for licensure as a dentist sought to compel the Board of Dentistry to issue him a license even though his score was less than the required general average of 75. In discussing the applicable burden of proof the court noted, at page 384:
From the foregoing it appears that the burden assumed by appellant to establish his right [to] the relief prayed was to prove by a preponderance of the evidence that he actually passed the dental examination taken by him. In addition to this burden it was also incumbent upon petitioner to prove by a preponderance of the evidence that appellees
capriciously and arbitrarily failed to give petitioner the grade he earned, but instead gave him a failing grade thereby illegally depriving him of the license to which he is entitled.
Absent a showing that the examining board failed to follow standard procedures for conducting and/or grading the examination, or that the candidate was treated differently from other examination candidates, test results will not generally be disturbed. See Macino v. Dept. of Professional Regulation, Bd. of Architecture, 5 FALR 994A (1983).
The Petitioners' presentation of their case, when viewed in its most favorable light, demonstrates that reasonable men may differ regarding the interpretations to be given certain questions and the adequacy of the responses to the questions posed. This, however, is not sufficient to support a determination that an error has been made on the part of the Respondent in grading the Petitioners' examinations. In State ex rel. I. H. Topp v. Board of Electrical Examiners, 101 So.2d 583 (Fla. 4th DCA, 1958), the court stated:
Examining boards such as the one here present are generally constituted and established for the purpose of protecting the public against incompetents who seek to enter the various vocations and professions. Such boards are not vested with arbitrary hegemony over the rights of the individual, but are charged with the duty to administer their rules and regulations equally and justly as between all persons and groups who come within the bounds of their jurisdiction, So long as these boards conduct their examinations fairly and uniformly in accordance with lawful authority and their own rules and regulations, their judgment as to the proper grading of such examinations will not be disturbed by the courts, unless clearly shown to be arbitrary or devoid of logic and reason.
It is clear from the evidence in the instant case that the respondent Board, in the exercise of its lawful authority, determined that the relator failed to earn a passing grade on its examination. Admittedly there type for which the amount of credit to be given various answers may differ in the minds of reasonable men. That such condition
exists is not alone sufficient cause upon which to bottom an alleged abuse of discretion, particularly when as here the ultimate responsibility for assigning grades to such answers falls on those who have been duly elected or appointed to the board and whose function it is to issue a certificate of competency only after being satisfied as to the applicant's entitlement. Under such circumstance the court will be extremely reluctant to substitute its judgment for that of the duly authorized board; else the board would be compelled through the judicial arm of mandamus to issue its certificates of competency not in its own discretion, but upon that of the court.
The legal principles set forth in Topp, supra, were followed in State ex rel. Lane v. Dade County, 258 So.2d 347 (Fla. 3d DCA 1972), where the court also addressed the question of the nature of an appropriate remedy following the administration of an allegedly improper licensure examination, and concluded:
If appellant is correct that authority to give and grade the examination was illegally delegated, then he might seek an order directing that he be given a lawful examination. But he cannot file a petition for mandamus and expect Dade County to be compelled to issue a Master Plumber's certificate to him because the examination was defective.
In a similar vein, the Court in Alvarez, et al. v. Department of Professional Regulation, 458 So.2d 808 (Fla. 1st DCA 1984), balanced the rights of examination applicants and the interests of the public by ordering that applicants who failed the practical portion of the 1982 state acupuncture exam because of insufficient and misleading exam instructions be permitted to sit for another exam free of charge. At page 811 the Alvarez court explained its conclusion as follows:
Accordingly, those examinees who failed only Part IV, Section 1 of the examination and who were, for that reason alone, denied licensure may, upon making of a proper application, but at no cost to them, retake
the clinical portion, Part IV, Section 1, of the Acupuncture Licensure Examination at the
next regularly scheduled examination. Rule 21-11.13(1), Florida Administrative Code.
Under the circumstances, such action is fair to both sides in that it protects the public but does not penalize examinees for the agency's inadequate and sometimes misleading directions. (emphasis added)
Although the Alvarez court was addressing the remedy in a case involving a practical examination, which is encompassed by Rule 21-11.13(1), Florida Administrative Code, the public protection considerations which underlie both the rule and the Alvarez remedy are equally applicable to examinations such as those at issue here.
What Topp, Lane, and Alvarez, supra, demonstrate is the extreme reluctance of the judiciary to order professional boards to license an exam applicant without a clear and unequivocal showing that the applicant is competent to practice his or her profession notwithstanding the grade received on a licensure exam. Such a philosophy is consistent with the legislative purpose of Chapter 489, Florida Statutes, in which the legislature expressly stated that "electrical contracting is an important service and potentially dangerous if not properly provided and, therefore, deems it necessary in the interest of public health, safety, and welfare to regulate the electrical contractors in this state." Section 489.501, Florida Statutes. Thus, absent a showing that the Board acted arbitrarily and capriciously in administering the electrical contractor's exams and as a result, the Petitioners failed to receive a passing grade, the relief requested by the Petitioners must be denied except as to those individual questions which were proved to be deficient in a manner which clearly impaired the Petitioners' ability to ascertain the correct answer.
Proximate cause--a necessary element
It is axiomatic to the presentation of proof in any disputed matter that the individual alleging a wrong must affirmatively demonstrate a nexus between the wrong alleged and the damage resulting to the individual. This is not a determination that can be inferred by mere proof of an occurrence except in the most unique circumstances where the cause and effect of the incident are apparent on their face as, for example, the application of the doctrine of res ipsa loquitor.
In the instant case, the Petitioners presented myriad scenarios relative to individual examination questions which may have had the effect of misleading or confusing an examination candidate to a point where the candidates' ability to properly
answer a question may have been prejudiced. However, what is generally lacking in the Petitioners' proof is any specific testimony that a given circumstance contributed to their failure in answering any of the challenged questions. It is true that Petitioner Harden made general statements about finding the numerical rounding problems in the accounting questions confusing, but he failed, for example, to explain how or if this confusion resulted in an inability to answer a given accounting question correctly. Under such circumstances, any injury resulting to the Petitioners from an alleged defective question is purely speculative.
To prove only that something is defective is not enough; a party must take the additional step of demonstrating that what was defective in some manner affected that party's substantial interest to the party's detriment. By way of specific example, it is not enough to prove that failure to place the words "assume unity" or their equivalent in the stem of question number 25 on the July open book examination which dealt with motor efficiency may possibly have been misleading without proof that Petitioner Cauthen missed this question because of that omission. Without this additional proof one can only speculate as to why Petitioner Cauthen missed the question. It may very well be that she did not know how to utilize the efficiency formula in the first instance, which would render the alleged problem with this question moot as it relates to her.
The burden of proof upon Petitioners in this cause is two-fold. First, proof must be presented that a question is erroneous or misleading, and then it must be demonstrated that the erroneous or misleading aspect of the question substantially affected the interest of each Petitioner. Merely to show that an examinee missed a question is not an adequate response to this requirement. Accordingly, all questions missed by Petitioners wherein the fundamental requirements of proximate cause were not established by competent evidence must be disregarded.
Relative to the numerous questions on the subject examinations taken from the Walker reference, the Petitioners presented extensive testimony as to whether or not a particular answer was correct even though it appeared verbatim in the Walker reference. Again, notwithstanding the accuracy of the information presented in the Walker text, the Petitioners failed to show that they chose an answer other than the keyed Walker answer because they possessed knowledge that was superior to the information in Walker or otherwise that the testimonial basis for choosing a particular response or eliminating one as presented by expert testimony constituted the manner in which they approached an individual question.
Appropriateness of examination questions
Throughout these proceedings, the Petitioners have argued that certain questions which appeared on the closed book examinations were "inappropriate." The Petitioners presented expert testimony in support of this proposition, while the Respondents presented their own expert who disagreed and testified that the challenged questions have a relationship to the practice of electrical contracting.
It seems apparent that the basic issue to be decided concerning whether a question is appropriately on an exam is whether the licensing agency possesses the requisite authority, either by statute or rule, to cover the particular subject area on the licensing exam. Since in this case the Board possesses the authority by statute, Section 455.217(1), Florida Statutes, to prescribe the areas of competency to be covered on the certification examination, and since the Board has exercised its authority in this regarding by adopting Rule 2100- 6.01, Florida Administrative Code, the fact that experts may differ on whether certain topics should be appropriately made test questions is irrelevant. The policy decision to test for competency in these subject areas is a choice which the legislature has vested in the Board, and unless the Board exceeds its authority, it cannot be successfully challenged on the ground that such a policy is "inappropriate.?? With respect to the scope of a Board's discretion in the exercise of its statutory authority, in Department of Professional Regulation, Board of Medical Examiners
v. Durrani, 455 So.2d 515 (Fla. 1st DCA 1984), the court stated:
As earlier observed, the criteria selected by the Board to constitute an acceptable examination . . . need not be the only possible interpretation or even the best interpretation of statutory language need only be within the range of possibilities.
The particular questions challenged dealt with the National Electrical Code, AlA Conditions of Contract, Business and Accounting, Workmen's Compensation, and other related areas. A corollary of this proposition is the appropriateness of questions appearing on closed book examinations. Again, experts may disagree as to what type of question should appear on a closed book examination but this is another of the discretionary functions vested in the Respondent Board which reviewed both subject examinations in their entirety and unless there appears to be a flagrant abuse of discretion, these decisions should not be disturbed. This is particularly true where, as here, statistical analysis of "anchor" questions demonstrates that the
use of the closed book format did not have an adverse impact on the ability of candidates to answer questions
In a case strikingly similar to the instant proceeding, Palaveda, et al. v. Roche, et al., 7 FALR 95 (1984), the Hearing Officer rejected the Petitioners' argument that the plumbing contractor's exam was unduly difficult and inappropriate to determine if an applicant was qualified to be a plumbing contractor. Specifically, the Petitioners challenged numerous business-related questions as outside the scope of the state's contractor's licensure law which the Petitioners argued was solely to protect the public from dishonest or incompetent contractors. The Hearing Officer found that the legislative purpose of the contractor's law was to ensure that contractors were solvent and that the public was protected from "short-lived" contractors. Accordingly, the Contracting Board was authorized to require an applicant to demonstrate minimum business acumen in the areas of social security, worker's compensation, unemployment compensation, et cetera, and the Petitioners exam challenge on the grounds of "inappropriateness?? was dismissed.
Appropriateness of closed book examinations
With regard to the closed book portion of the examination set forth in subsection (2) of the rule, as noted in the findings of fact, it is quite common for licensure examinations to be entirely closed book or to have portions that are closed book. The fact that a portion of an examination is closed book does not ipso facto have anything to do with the validity or reliability of an examination, except to the extent that a closed book examination is generally a better instrument for determining the extent of a candidate's practical experience. There is nothing arbitrary or capricious about including a closed book portion on an electrical contractor licensing examination.
Appropriateness of relative grading weights
With regard to the relative grading weights established in subsection (3) of the rule, the relative grading weights which were in effect in 1982 and the ones which are in effect now all bear a reasonable relationship to testing for minimum competency in the field of electrical contracting. Although there have been changes in the relative grading weights over the years, the changes have been slight. As noted in the findings of fact, the determination of what the relative grading weights should be is a matter involving the exercise of judgment by experts in the field, in this case the professional members of the Board. The greater weight of the evidence supports the conclusion that their judgment in this regard was reasonable. It
certainly bears at least a possible correlation to electrical contractor competence, and it is not arbitrary or capricious.
Interrogatories as evidence at final hearing
The Petitioners argued during the final hearing that the Respondents' Response to Petitioners' First Set of Interrogatories, dated December 13, 1984, was dispositive of the questions posed therein and that the Respondents were, therefore, precluded from placing into evidence any information adding to or altering in any manner the responses as filed.
The Petitioners' position would be basically correct if the discovery matter in issue were requests for admissions rather than interrogatories A fundamental difference between the two types of discovery is that requests for admissions are dispositive of an issue while other forms of discovery, such as interrogatories and depositions, are not. Compare Rule 1.340, Florida Rules of Civil Procedure and Rule 1.370, Florida Rules of Civil Procedure. For example, answers to interrogatories may be used to the extent permitted by the rules of evidence. Requests for admissions, however, conclusively establish a fact unless the court on motion permits withdrawal or amendment. Once part of the record, interrogatories or depositions may be contradicted, impeached or proven to be inaccurate, as may any other form of admissible evidence.
Thus, the interrogatories at issue form only one piece of evidence to be considered in conjunction with all other admissible evidence to establish the facts of this case, and the Petitioners' assertion that they conclusively establish a set of facts is erroneous.
Conclusions regarding the Petitioners' several post-hearing motions
By their July 8, 1985, post-hearing motion to amend their petitions in these consolidated cases, the Petitioners seek to add a request for costs as part of their request for relief in these cases. By their January 28, 1986, motion the Petitioners seek both legal fees and costs. With an exception not relevant here, there is no statutory provision which authorizes a Hearing Officer in a Section 120.57(1) proceeding of this nature to enter an order awarding legal fees and costs. It is also well-settled law in this state that attorney's fees should not be awarded against another party except where such an award is expressly authorized. Therefore, because there is no statutory provision for the relief sought by these two motions, the Petitioners' July 8, 1985, motion to amend their petition to request costs and
their January 28, 1986, motion seeking legal fees and costs are DENIED.
With regard to the Renewed Motion For Leave To Withdraw And For Charging Lien, it is first noted that by order dated February 13, 1986, counsel for the Petitioners was permitted to withdraw. The renewed motion is occasioned by the fact that at a later date the former counsel for the Petitioners filed additional pleadings on their behalf and is concerned that such filing may constitute a new appearance. Accordingly, the renewed motion for leave to withdraw as counsel is GRANTED, and counsel is again permitted to withdraw. The renewed motion for a lien for attorney's fees is DENIED for the reasons set forth in the order of February 15, 1986.
With regard to the letter of July 2, 1985, requesting specific rulings on the motions of July 22, 1983, and June 29, 1983, regarding costs and attorney's fees on appeal, both of said motions are hereby DENIED. (In this regard it is also noted that similar motions were denied by the appellate court and, in any event, the circumstances of the related appellate litigation do not seem to be such as would warrant an award of costs and attorney's fees.)
Accordingly, based on the foregoing, it is:
RECOMMENDED that Petitioner Cauthen be awarded credit for questions number 45, A.M. (.5), number 69, A.M. (.5), number 80,
A.M. (.5), number 42, A.M. (1) and 44, P.M. (1) and that a grade of 71.5 be assigned to her on the January 1982 examination and that question number 43, P.M. (5) on the July 1982 examination be credited to Petitioner Cauthen and that she be assigned a grade of 73.5 there on.
It is further recommended that Petitioner Harden receive credit for questions number 39, A.M. (1), number 69, A.M. (.5),
number 80, A.M. (.5), number 16, A.M. (.5), number 45, A.M. (.5) and number 41, P.M. (1) for the January 1982 examination and be assigned the grade of 56 thereon and further that questions number 53, P.M. (.5), number 58, P.M. (.5), and number 72, P.M.
(1) be awarded to Petitioner Harden and he be assigned a grade of
63 for the July 1982 examination.
It is recommended that the Petitions in these three cases be dismissed in all other material respects and that the Petitioners' applications for licensure as state certified electrical contractors be DENIED.
DONE AND ORDERED this 11th day of June, 1986, at Tallahassee, Florida.
MICHAEL M. PARRISH
Hearing Officer
Division of Administrative Hearings The Oakland Building
2009 Apalachee Parkway
Tallahassee, Florida 32399-1550
(904) 488-9675
Filed with the Clerk of the Division of Administrative Hearings this 11th day of June, 1986.
ENDNOTES
1/ Mrs. Tell withdrew as counsel for the Petitioners subsequent to the hearing. Prior to the withdrawal she filed a proposed recommended order and several motions on behalf of the Petitioners.
2/ The exhibits in these cases alone fill one large cardboard box and constitute literally hundreds of pages of paper. The exhibits in Case No. 84-0309R, which are incorporated in the record in this case, fill two large cardboard boxes and constitute literally thousands of pieces of paper.
3/ Similarly, the ten days of testimony in Case No. 84-0309R, which are incorporated into the records in this case, were reported by court reporters, but no transcript of those proceedings has been prepared.
4/ The parties filed their proposed recommended orders on May 20, 1985. The Petitioners' submission comprises 150 pages. The Respondents' submission comprises 72 pages.
COPIES FURNISHED:
W. Douglas Moody, Esquire
119 North Monroe Street Tallahassee, Florida 32301
Sarah Logan, Executive Director Electrical Contractors' Licensing Board Department of Professional Regulation
130 North Monroe Street Tallahassee, Florida 32301
Fred Roche, Secretary
Department of Professional Regulation
130 North Monroe Street Tallahassee, Florida 32301
Salvatore A. Carpino, Esquire Department of Professional Regulation
130 North Monroe Street Tallahassee, Florida 32301
Mr. John Eugene Harden Harden Electric
331 N.E. 8th Street Suite 108
Homestead, Florida 33030
Ms. Dova F. Cauthen 959 N.E. 79th Street Miami, Florida 33138
| Issue Date | Proceedings |
|---|---|
| Jun. 11, 1986 | Recommended Order (hearing held , 2013). CASE CLOSED. |
| Issue Date | Document | Summary |
|---|---|---|
| Sep. 25, 1986 | Agency Final Order | |
| Jun. 11, 1986 | Recommended Order | Case dismissed. Application for licensure as state certified electrical contractors denied. Petitioners failed to receive passing score of 75. |
