Findings Of Fact In April 1995, Abraham S. Inlong (Petitioner) took the Electrical Engineer part of the Professional Engineering Examination (Examination). A minimum grade of 70 is required to pass the Examination. The Department of Business and Professional Regulation, Board of Professional Engineers (Respondent) notified Petitioner that he had failed the Examination, having received a grade of 69.10. The Examination is a national examination and is graded by national examiners. Respondent issues licenses to practice professional engineering in the State of Florida and administers the Examination on behalf of the State. Petitioner challenges, the answer selected by the national examiners to Problem 433, Question 6 of the Examination, which is A. Respondent selected D as the answer, which states that A, B, and C are all correct. As part of the instructions for candidates taking the Examination, the candidates were to choose the best answer. The best answer is the correct answer. Respondent's response to Problem 433, Question 6 was regraded by the national examiners. They denied Respondent any additional credit. The best and correct answer to Problem 433, Question 6 is the answer identified by Respondent as the answer by the national examiners, i. e., A. The answer selected by Petitioner is not the best and correct answer. A diagram is part of the challenged problem and question. The diagram is clear and unambiguous. The scope of knowledge required for the challenged problem and question is not beyond the knowledge reasonably expected from a candidate for licensure. The challenged problem and question contain sufficient information for a candidate for licensure to select the best and correct answer. Additional information was unnecessary, including whether the system was balanced or unbalanced. The challenged problem and question are clear and unambiguous. The challenged problem and question are not devoid of logic and reason. The challenged problem and question are valid. Statistics indicate that 60 percent of the candidates for licensure (candidates), who took the Examination, answered Problem 433 correctly and that 48 percent of the candidates answered Problem 433, Question 6 correctly.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that the Department of Business and Professional Regulation, Board of Professional Engineers, enter a final order dismissing Abraham S. Inlong's examination challenge and denying him licensure. DONE AND ENTERED this 1st day of August, 1996, in Tallahassee, Leon County, Florida. ERROL H. POWELL, Hearing Officer Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-1550 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 1st day of August, 1996. APPENDIX TO RECOMMENDED ORDER, CASE NO. 96-0031 The following rulings are made on the parties' proposed findings of fact: Petitioner's Proposed Findings of Fact Partially accepted in findings of fact 1 and 2. Partially accepted in finding of fact 4. Partially accepted in findings of fact 4 and 8. Partially accepted in finding of fact 7. Rejected as being not supported by the greater weight of the evidence, not supported by the more credible evidence, argument, or a conclusion of law. Rejected as being not supported by the greater weight of the evidence, not supported by the more credible evidence, argument, or a conclusion of law. Rejected as being not supported by the greater weight of the evidence, not supported by the more credible evidence, argument, or a conclusion of law. Rejected as being not supported by the greater weight of the evidence, or not supported by the more credible evidence. Rejected as being not supported by the greater weight of the evidence, or not supported by the more credible evidence. Respondent's Proposed Findings of Fact 1. Partially accepted in finding of fact 1. 2. Partially accepted in finding of fact 2. 3. Partially accepted in finding of fact 2. 4. Partially accepted in finding of fact 2. 5. See Preliminary Statement. 6. See Preliminary Statement. 7. Partially accepted in finding of fact 4. 8. Partially accepted in finding of fact 4. 9. Partially accepted in finding of fact 7. 10. Partially accepted in finding of fact 11. 11. Partially accepted in finding of fact 10. 12. Partially accepted in finding of fact 9. 13. Partially accepted in finding of fact 10. 14. Partially accepted in finding of fact 10. Partially accepted in findings of fact 9 - 12. Partially accepted in finding of fact 12. Partially accepted in finding of fact 6. Rejected as being subordinate, argument, or a conclusion of law. Partially accepted in finding of fact 8. Partially accepted in finding of fact 14. Partially accepted in finding of fact 13. Rejected as being subordinate, or unnecessary. Rejected as being subordinate, or unnecessary. Rejected as being subordinate, or unnecessary. NOTE--Where a proposed finding has been partially accepted, the remainder has been rejected as being subordinate, irrelevant, unnecessary, not supported by the greater weight of the evidence, not supported by the more credible evidence, argument, or a conclusion of law. COPIES FURNISHED: Harold M. Braxton, Esquire One Datran Center, Suite 400 9100 South Dadeland Boulevard Miami, Florida 33156-7815 R. Beth Atchison Assistant General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0750 Angel Gonzalez, Executive Director Department of Business and Professional Regulation Board of Professional Engineers 1940 North Monroe Street Tallahassee, Florida 32399-0755 Lynda Goodgame, General Counsel Department of Business and Professional Regulation Northwood Centre 1940 North Monroe Street Tallahassee, Florida 32399-0792
The Issue Whether Respondent, Department of Business and Professional Regulation, Electrical Contractors Licensing Board, appropriately denied Petitioner's, Kevin Harrington, application to take the examination for licensure as an electrical contractor.
Findings Of Fact Based on the oral and documentary evidence presented at the final hearing, the following findings of fact are made: Respondent is the state agency vested with the authority to test applicants seeking certification as electrical contractors. Petitioner is seeking certification (licensure) as an electrical contractor. On December 12, 2001, Respondent received Petitioner's application to take the Electrical Contractors Unlimited examination. On December 24, 2001, Respondent mailed Petitioner a letter requesting additional information. The letter requested that the additional information be received by Respondent no later than January 4, 2002. This date was later extended to January 8, 2002. On January 4 and January 7, 2002, Petitioner, through his attorney, forwarded the requested additional information to Respondent. Petitioner's application was complete on January 10, 2002. Respondent, Electrical Contractors Licensing Board("Board"), met on January 16 and 17, 2002. The Board delegates initial consideration of applications to take certification examinations to an Applications Committee consisting of members of the Board who make recommendations to the full Board on each application. Each application is examined by at least two Applications Committee members; if both recommend "denial of the application," or, if one recommends "approval of the application" and one recommends "denial of the application," the application is reviewed by a third Applications Committee member. Each member of the Applications Committee is provided a worksheet as a part of the application package which lists reasons for denial drawn from Subsection 489.511(2)(a)(3), Florida Statutes. After each application is considered by members of the Applications Committee, the application with the Applications Committee's recommendations, reasons for denial (if applicable), and other comments are given to a staff employee who prepares an approved/denial list which is presented to the full Board for consideration. The foregoing procedure was followed in the instant case. On January 16, 2002, Board members, Roger Lange and Kim DeBerry, who were members of the Applications Committee, considered Petitioner's application; both recommended denial of the application. Because there were two recommendations of denial, the application was considered by a third Applications Committee member, Dawn Johnson; she, too, recommended denial. Petitioner's application and the recommendations, reasons for denial, and comments of the Applications Committee were then given to a staff employee who prepared a summary list of all applicants with the recommendations for approval or denial by the Applications Committee with reasons given for denial for submission to the full Board. The full Electrical Contractors Licensing Board considered Petitioner's application on January 17, 2002, and unanimously denied the application. Petitioner was advised of the denial by letter dated February 8, 2002. Petitioner's Applicant's Affidavit dated November 16, 2001, indicates that he was seeking a license under Subsection 489 .511(2)(a)(3)(a), Florida Statutes. The Applicant's Affidavit specifically states: 489.511(2)(a)(3)(a), F.S. Has, within the six (6) years immediately preceding the filing of the application, at least three (3) years proven "management experience" in the trade or education equivalent thereto, or a combination thereof, but not more than one- half of such experience may be educational. (Please submit at least three (3) years of W- 2 Forms) The occupational skills and responsibilities of an electrical contractor are unique and require experience and understanding which are typically acquired by extensive, direct "on-the-job" training in the electrical contracting trade. Petitioner is an experienced General Contractor's project manager. His credentials as a General Contractor's project manager are impressive and the projects he has supervised are extensive. Petitioner has little or no direct supervisory experience in the electrical contracting trade. Petitioner's construction management experience is as a General Contractor project manager, not an Electrical Contractor project manager or similar position, and, as a consequence, he does not meet the "'management experience' in the trade" statutory requirement. Petitioner has an enviable academic record: an Associate of Science Degree With Honors from Miami-Dade Community College (1990), a Bachelor of Science in Building Construction from University of Florida (1993), 21 hours of graduate studies at Florida International University leading towards a Master of Building Construction degree. Petitioner's academic credentials have little direct application to the electrical contracting trade and, as a consequence, do not meet the "educational equivalent" to management experience statutory requirement.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that a final order be entered denying Petitioner's application to take the Certified Electrical Contractor's Licensure Examination. DONE AND ENTERED this 10th day of January, 2003, in Tallahassee, Leon County, Florida. JEFF B. CLARK Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 10th day of January, 2003. COPIES FURNISHED: Barbara Rockhill Edwards, Esquire Department of Legal Affairs The Capitol, Plaza Level 01 Tallahassee, Florida 32399-1050 Rosemary Hanna Hayes, Esquire Hayes & Associates 3117 Edgewater Drive Orlando, Florida 32804 Hardy L. Roberts, III, General Counsel Department of Business and Professional Regulation Northwood Centre 1940 North Monroe Street Tallahassee, Florida 32399-2202 Anthony B. Spivey, Executive Director Electrical Contractors Licensing Board Department of Business and Professional Regulation Northwood Centre 1940 North Monroe Street Tallahassee, Florida 32399-0792
The Issue Whether Petitioner is entitled to additional credit for his responses to Questions 132 and 294 of the Principles & Practice of Engineering portion of the engineering licensure examination administered on April 23, 1999, by the National Council of Examiners for Engineers and Surveyors (the NCEES).
Findings Of Fact On April 23, 1999, Solorzono sat for the Principles and Practice Engineering Examination in electrical engineering. This national examination is developed, controlled, and administered by the NCEES. The examination candidates receive raw scores, which result in a converted score for the final examination score. A minimum converted score of 70 is required to pass the examination. A raw score of 48 equates to a converted score of 70. Solorzano received a raw score of 45, resulting in a converted score of 67. If a candidate is not satisfied with his examination score, he may request the NCEES to review and rescore his examination answers. Solorzano formally requested the NCEES to rescore his examination. Upon rescoring, the NCEES determined that Solozano's raw score should be decreased to 43. The examination questions at issue in this proceeding are Questions 132 and 294. Solorzano received a raw score of 4 on Question 132 and a raw score of 4 on Question 294. When the NCEES rescored the examination, it did not award any additional points for Question 132 and deducted two points for Question 294. The NCEES develops an item-specific scoring plan (ISSP) for each examination question. Question 132 was scored by the NCEES according to the ISSP for that question. Question 132 contains three subparts, which require the examinee to address five discrete requirements: The problem solution as a three-phase problem, (2) The total MW, MVAR, and MVA of the load without the capacitor bank, (3) The size of the capacitor bank in kVAR to make the power factor equal to 0.9 lagging, (4) The complex power diagrams with and without the capacitor bank (MW same for both diagrams and correct phasor directions for both diagrams), (5) The MVA load with the capacitor bank connected. Solozano correctly identified the problem as a three- phase power problem and satisfied the first requirement. Solozano incorrectly calculated the MW, MVAR, and MVA, the real power, the imaginary power, and complex power for the load on the transformer without the capacitor bank. He failed to apply the correct concepts for "Y" transformer as given in the problem statement and based his solution on the concepts for "Delta" transformer. Solorzano failed to satisfy the second requirement. Even though Solorzano's calculations carried through his error from the second requirement, he showed understanding of correcting the power factor and performed a correct analysis to size the capacitor bank. Solorzano satisfied the third requirement. Solorzano made a significant conceptual error by showing an incorrect vector direction for the calculated Q value. He showed a negative polarity for the Q component when it should have been positive. Solorzano failed to satisfy the fourth requirement. In calculating the real complex power load on the transformer, with the capacitor bank connected, Solorzano used an incorrect concept, simply subtracting the load with the capacitor bank from the transformer's rating. He failed to satisfy the fifth requirement. Having satisfied only two of the five requirements for Question 132, Solorzano is entitled to a raw score of 4 for Question 132. Question 294 requires the examinee to address the following five requirements: Correct truth table for 0-9 with at most one error. Correct truth table for 10-15. Map or table showing correct values for w,0,1 entries. Correct assignment for w,0,1 entries to circuit with at most 1 error and no x,y,z entries. Correct polarity for truth table and circuit for w,0,1 (requires correct circuit values). Solorzano constructed a truth table for 0-9 with one mistake for polarity. He fulfilled the first requirement. Solorzano failed to complete the truth table for 10-15, arguing that the 10-15 segments were not used; therefore, it was not necessary to construct a truth table. The second requirement calls for the construction of a truth table for 10-15. It is necessary for a complete truth table to ensure that the output for segment E is not affected by an input beyond 9. He failed to meet the second requirement. Solorzano made a conceptual error by reversing the most significant bit and least significant bit, resulting in his failure to map a table showing correct values for judging zero and one. He failed to satisfy the third requirement. In his development of the fourth requirement, Solorzano carried through an earlier error. However, he correctly utilized the incorrect information, satisfying the fourth requirement. Requirement five called for the correct circuit values. Because Solorzano had used the incorrect polarity throughout his solution, he failed to meet the fifth requirement. Solorzano satisfied two of the five requirements for Question 294; thus, he is entitled to a raw score of 45. Questions 132 and 294, with their problem statements, provide all the necessary information necessary for an examinee to solve the problems. The questions are properly designed to test an examinee's competence in electrical engineering. Solorzano is entitled to a raw score of 45, equating to a converted score of 67.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that a final order be entered finding that Solorzano is entitled to a converted score of 67 on the electrical engineering examination given on April 23, 1999, and has failed the examination. DONE AND ENTERED this 6th day of June, 2000, in Tallahassee, Leon County, Florida. SUSAN B. KIRKLAND Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 6th day of June, 2000. COPIES FURNISHED: Alejandro Solorzano 6675 Southwest 103 Court Miami, Florida 33173 William H. Hollimon, Esquire Ausley & McMullen 227 South Calhoun Street Tallahassee, Florida 32301 Dennis Barton, Executive Director Florida Board of Professional Engineers 1208 Hays Street Tallahassee, Florida 32301 Natalie A. Lowe, Esquire Vice President for Legal Affairs Florida Engineers Management Corporation 1208 Hays Street Tallahassee, Florida 32301 Barbara D. Auger, General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792
The Issue The ultimate issue in this proceeding is whether proposed Florida Administrative Code Rule 61G15-21 is an invalid exercise of delegated legislative authority.
Findings Of Fact Florida Administrative Code Rule 61G15-21.004, in relevant part, states: The criteria for determining the minimum score necessary for passing the Engineering Fundamentals Examination shall be developed through the collective judgment of qualified experts appointed by NCEES to set the raw score that represents the minimum amount of knowledge necessary to pass the examination. The judges shall use a Modified Angoff Method in determining the minimally acceptable raw score necessary to pass the Fundamentals of Engineering Examination. Using the above mentioned Modified Angoff Method, the judges will indicate the probability that a minimally knowledgeable Fundamentals of Engineering examinee would answer any specific questions correctly. The probability of a correct response is then assigned to each question. Each judge will then make an estimate of the percentage of minimally knowledgeable examinees who would know the answer to each question. The totals each of the judges is added together and divided by the number of judges to determine the overall estimate of the minimum standards necessary. The minimum number of correct answers required to achieve a passing score will take into account the relative difficulty of each examination through scaling and equating each examination to the base examination. The raw score necessary to show competence shall be deemed to be a 70 on a scale of 100. A passing grade on Part Two of the examination is defined as a grade of 70 or better. The grades are determined by a group of knowledgeable professional engineers, who are familiar with engineering practice and with what is required for an applicable engineering practice and with what is required for an applicable engineering task. These professional engineers will establish a minimum passing score on each individual test item (i.e., examination problem). An Item Specific Scoring Plan (ISSP) will be prepared for each examination item based upon the NCEES standard scoring plan outline form. An ISSP will be developed by persons who are familiar with each discipline including the item author, the item scorer, and other NCEES experts. On a scale of 0-10, six (6) will be a minimum passing standard and scores between six (6) and ten (10) will be considered to be passing scores for each examination item. A score of five (5) or lower will be considered an unsatisfactory score for that item and examinee will be considered to have failed that item. To pass, an examinee must average six (6) or greater on his/her choice of eight (8) exam items, that is, the raw score must be forty- eight (48) or greater based on a scale of eighty (80). This raw score is then converted to a base 100 on which, as is noted above, a passing grade will be seventy (70). The proposed changes to Florida Administrative Code Rule 61G15-21.004, in relevant part, state: The passing grade for the Engineering Fundamentals Examination is 70 or better. The criteria for determining the minimum score necessary for passing the Engineering Fundamentals Examination shall be developed through the collective judgment of qualified experts appointed by NCEES to set the raw score that represents the minimum amount of knowledge necessary to pass the examination. The judges shall use a Modified Angoff Method in determining the minimally acceptable raw score necessary to pass the Fundamentals of Engineering Examination. Using the above mentioned Modified Angoff Method, the judges will indicate the probability that a minimally knowledgeable Fundamentals of Engineering examinee would answer any specific questions correctly. The probability of a correct response is then assigned to each question. Each judge will then make an estimate of the percentage of minimally knowledgeable examinees who would know the answer to each question. The totals each of the judges is added together and divided by the number of judges to determine the overall estimate of the minimum standards necessary. The minimum number of correct answers required to achieve a passing score will take into account the relative difficulty of each examination through scaling and equating each examination to the base examination. The raw score necessary to show competence shall be deemed to be a 70 on a scale of 100. The passing grade for the Principles and Practice Examination is 70 or better. A passing grade on Part Two of the examination is defined as a grade of 70 or better. The grades are determined by a group of knowledgeable professional engineers, who are familiar with engineering practice and with what is required for an applicable engineering practice and with what is required for an applicable engineering task. These professional engineers will establish a minimum passing score on each individual test item (i.e., examination problem). An Item Specific Scoring Plan (ISSP) will be prepared for each examination item based upon the NCEES standard scoring plan outline form. An ISSP will be developed by persons who are familiar with each discipline including the item author, the item scorer, and other NCEES experts. On a scale of 0-10, six (6) will be a minimum passing standard and scores between six (6) and ten (10) will be considered to be passing scores for each examination item. A score of five (5) or lower will be considered an unsatisfactory score for that item and examinee will be considered to have failed that item. To pass, an examinee must average six (6) or greater on his/her choice of eight (8) exam items, that is, the raw score must be forty- eight (48) or greater based on a scale of eighty (80). This raw score is then converted to a base 100 on which, as is noted above, a passing grade will be seventy (70). Petitioner resides in Tampa, Florida. On April 11, 2003, Petitioner took a national examination that Petitioner must pass to be licensed by the state as a professional engineer. On July 1, 2003, Petitioner received a letter from the Board advising Petitioner that he had received a failing grade on the examination. On July 2, 2003, Petitioner unsuccessfully requested the raw scores on his examination from a representative of the National Council of Examiners for Engineering and Surveying (NCEES). The NCEES is the national testing entity that conducts examinations and determines scores for the professional engineer examination required by the state. On July 9, 2003, Petitioner submitted a formal request to the Board for all of the raw scores related to Petitioner "and all past P.E. Exams that the Petitioner had taken." A representative of the Board denied Petitioner's request explaining that the raw scores are kept by the NCEES and "it is not their policy to release them." The Board's representative stated that the Board was in the process of adopting new rules "that were in-line with the policies of the NCEES." On July 31, 2003, Petitioner requested the Board to provide Petitioner with any statute or rule that authorized the Board to deny Petitioner's request for raw scores pursuant to Section 119.07(1)(a), Florida Statutes (2003). On the same day, counsel for the Board explained to Petitioner that the Board is not denying the request. The Board is unable to comply with the request because the Board does not have physical possession of the raw scores. Petitioner and counsel for Respondent engaged in subsequent discussions that are not material to this proceeding. On August 6, 2003, Petitioner requested counsel for Respondent to provide Petitioner with copies of the proposed rule changes that the Board intended to consider on August 8, 2003. On August 27, 2003, Petitioner filed a petition with the Board challenging existing Florida Administrative Code Rule 61G15-21.004. The petition alleged that parts of the existing rule are invalid. Petitioner did not file a challenge to the existing rule with DOAH. The Petition for Hearing states that Petitioner is filing the Petition for Hearing pursuant to Subsections 120.56(1) and (3)(b), Florida Statutes (2003). However, the statement of how Petitioner's substantial interests are affected is limited to the proposed changes to the existing rule. During the hearing conducted on January 29, 2004, Petitioner explained that he does not assert that the existing rule is invalid. Rather, Petitioner argues that the Board deviates from the existing rule by not providing examinees with copies of their raw scores and by failing to use raw scores in the determination of whether an applicant achieved a passing grade on the exam. Petitioner further argues that the existing rule benefits Petitioner by purportedly requiring the Board to use raw scores in the determination of passing grades. The elimination of that requirement in the proposed rule arguably will adversely affect Petitioner's substantial interests. The Petition for Hearing requests several forms of relief. The Petition for Hearing seeks an order granting Petitioner access to raw scores, a determination that Petitioner has met the minimum standards required under the existing rule, and an order that the Board grant a license to Petitioner. The Petition for Hearing does not request an order determining that the proposed rule changes constitute an invalid exercise of delegated legislative authority.
The Issue Whether Petitioner is entitled to additional credit for his responses to Questions 23 and 27 of the Contract Administration section of the General Contractor licensure examination administered in July 1996, and, if so, whether the additional credit would give him a passing grade. Whether Petitioner is entitled to additional credit for his responses to Questions 11, 23, and 35 of the Contract Administration section of the General Contractor licensure examination administered in April 1997 and, if so, whether the additional credit would give him a passing grade.
Findings Of Fact Petitioner took the Contract Administration section of the General Contractor’s licensure examination in July 1996 and in April 1997. Between the two exams, Petitioner passed all sections of the examination except the Contract Administration section. Petitioner’s score on the Contract Administration section of the July 1996 examination, as graded by Respondent’s Bureau of Testing, was 65. His score on the Contract Administration section of the April 1997 examination was 67.5. For both examinations, there were 40 questions on the Contract Administration section. A candidate had to achieve a score of 70 to pass that section of the examination. Because each question was equally weighted, a candidate would have to correctly answer 28 questions to earn the passing score. All questions challenged by Petitioner were multiple- choice questions where the candidate was instructed to give the best answer from four possible choices. Prior to the examinations, the candidates were given a list of approved reference materials. The candidates were permitted to refer to those reference materials while taking the examinations. Respondent’s score of 65 on the July 1996 examination was based on the Bureau of Testing’s determination that Petitioner correctly answered 26 of the 40 questions. To earn a passing grade on the Contract Administration section of the July 1996 examination, Petitioner would have to receive credit for correctly answering two additional questions. His score of 67.5 on the April 1997 was based on the determination that he correctly answered 27 of the 40 questions. To earn a passing grade on the Contract Administration section of the April 1997 examination, Petitioner would have to receive credit for correctly answering one additional question. QUESTION 23 OF THE JULY 1996 EXAM The correct answer for Question 23 of the July examination is choice “D.” Of the four possible responses, choice “D” is the best answer to the question. Petitioner’s answer to this question was choice “A.” Petitioner did not receive credit for his response to this question because he did not select the best answer. The answer selected by Petitioner would not be the most accurate and cost-effective because the methodology he selected would not detect errors made by the first person performing the computations. The challenged question is a question that a candidate for licensure should be able to answer. The challenged question is not beyond the scope of knowledge that a candidate for licensure should have. The challenged question is not ambiguous. Petitioner is not entitled to additional credit for his response to Question 23 of the July 1996 exam. QUESTION 27 OF THE JULY 1996 EXAM The correct answer for Question 27 of the July examination is choice “C.” This correct answer is supported by reference materials made available to all candidates. Petitioner’s answer to this question was choice “B.” Petitioner did not receive credit for his response to this question because he did not select the correct answer to the question. The challenged question is a question that a candidate for licensure should be able to answer. The challenged question is not beyond the scope of knowledge that a candidate for licensure should have. The challenged question is not ambiguous. Petitioner is not entitled to additional credit for his response to Question 27 of the July 1996 exam. QUESTION 11 OF THE APRIL 1997 EXAM The correct answer for Question 11 of the April 1997 examination is choice “C.” This correct answer is supported by reference materials made available to all candidates. Petitioner’s answer to this question was choice “D.” Petitioner did not receive credit for his response to this question because he did not select the correct answer to the question. The challenged question is a question that a candidate for licensure should be able to answer. The challenged question is not beyond the scope of knowledge that a candidate for licensure should have. The challenged question is not ambiguous. Petitioner is not entitled to additional credit for his response to Question 11 of the April 1997 exam. QUESTION 23 OF THE APRIL 1997 EXAM The best answer for Question 23 of the April 1997 examination is choice “C.” This correct answer is supported by reference materials made available to all candidates. Petitioner’s answer to this question was choice “A.” While there is some support in the reference material for Petitioner's answer, the greater weight of the evidence established that his choice was not the best answer. Petitioner did not receive credit for his response to this question because he did not select the best answer to the question. The challenged question is a question that a candidate for licensure should be able to answer. The challenged question is not beyond the scope of knowledge that a candidate for licensure should have. The challenged question is not ambiguous. Petitioner is not entitled to additional credit for his response to Question 23 of the April 1997 exam. QUESTION 35 OF THE APRIL 1997 EXAM The correct answer for Question 11 of the April 1997 examination is choice “C.” This correct answer is supported by reference materials made available to all candidates. Petitioner’s answer to this question was choice “D.” Petitioner did not receive credit for his response to this question because he did not select the correct answer to the question. The challenged question is a question that a candidate for licensure should be able to answer. The challenged question is not beyond the scope of knowledge that a candidate for licensure should have. The challenged question is not ambiguous. Petitioner is not entitled to additional credit for his response to Question 11 of the April 1997 exam.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that Respondent enter a Final Order that dismisses the challenges brought by Petitioner to Questions 23 and 27 on the July 1996 exam and to Questions 11, 23, and 35 of the April 1997 exam. DONE AND ENTERED this 3rd day of December, 1997, in Tallahassee, Leon County, Florida. CLAUDE B. ARRINGTON Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (904) 488-9675 SUNCOM 278-9675 Fax Filing (904) 921-6847 Filed with the Clerk of the Division of Administrative Hearings this 3rd day of December, 1997. COPIES FURNISHED: R. Beth Atchison, Esquire Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0750 Mr. Kenneth Marshall 624 Southwest 11th Court Fort Lauderdale, Florida 33315 John Preston Seiler, Esquire 2900 East Oakland Park Boulevard, No. 200 Fort Lauderdale, Florida 33306 Lynda L. Goodgame, General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792 Rodney Hurst, Executive Director Construction Industry Licensing Board 7960 Arlington Expressway, Suite 300 Jacksonville, Florida 32211-7467
Findings Of Fact Based upon all of the evidence, the following findings of fact are determined: On April 19, 1990, petitioner, Don R. Blackburn, was a candidate on the engineering intern portion of the professional engineer examination given in Miami, Florida. The test was administered by the Department of Professional Regulation (DPR) on behalf of respondent, Board of Professional Engineers (Board). On July 25, 1990, the Board issued a written uniform grade notice advising petitioner that he had received a grade of 66 on the examination. A grade of 70 is necessary to pass this part of the examination. By letter dated August 15, 1990, petitioner requested a formal hearing to contest his score. In his letter, Blackburn generally contended that the examination was unfairly administered because certain books were allowed to be used by some but not all candidates, untrained proctors were given the authority to scan review materials and determine which could or could not be used by the candidates, and because of the chaos and confusion that occurred during the examination, he was unable to attain a score that he otherwise would have been able to achieve. Blackburn is an engineer for Lee County and is seeking to pass the engineering intern portion of the examination. A passing grade on that portion is a prerequisite to sitting on the second part of the professional engineer examination. He has taken the examination on a number of occasions and has gradually improved his score to just short of passing. Indeed, on the October 1989 examination, Blackburn scored a 69, or just one point less than the required 70. Prior to the April 1990 examination, the engineering intern portion of the professional engineer examination was an unrestricted open book examination. This meant candidates could use any and all reference and review materials during the examination. Beginning with the April 1990 examination, the Board imposed certain restrictions on the use of review materials. As early as October 9, 1989, the Board's executive director sent a memorandum to all candidates on the October 1989 examination, including Blackburn, concerning the new restrictions. The memorandum stated in part: Please be advised of certain restrictions listed in the Candidate Information Booklet which will not be implemented until the April 1990 examination. These restrictions are found in the "Examination Administration Information" section and are concerning the following two areas: * * * 2. Books or information containing sample questions or engineering problems may also be brought provided they are bound. Again, the new restrictions listed in the Candidate Information Booklet regarding the above two areas WILL NOT be implemented until the April 1990 examination. All candidates on the April 1990 examination were given a Candidate Information Booklet prepared in January 1990 by DPR's Bureau of Examination Services. On pages 13 and 14 of the booklet was found the following information: This is an open book examination. Candidates may use textbooks, handbooks, notes, and reference materials which are bound, copyrighted and printed. The term "bound" refers to material that is bound permanently, hard or paperback stitched or glued, or spiral, plastic or three-ringed bound. The printed material must remain contained (bound) in its cover during the entire examination. No writing tablets, unbound tablets or unbound "loose notes" will be allowed. No books with contents directed toward sample questions or solutions of engineering problems are permitted in the examination room. Examinees are not permitted to exchange reference materials or aids during the examination. (Emphasis in original) What the emphasized language meant is that "review" manuals, which contain problems and solutions, were prohibited from use during the examination while "reference" books were not. However, the booklet did not list the specific names of published materials that would be permitted or excluded. In order to ascertain which books he might use on the next examination, on March 27, 1990, Blackburn telephoned the Board in Tallahassee and spoke with a female employee named "B. J." who advised him that "review publications directed principally towards the solution of engineering problems" would be excluded. When asked if "Lindeburg's Sixth Edition" would be authorized, B. J. told Blackburn she wasn't sure and that it would be left up to the proctors in the room. She did say, however, that a review manual authored by Schaum could be used. The engineering intern examination in April 1990 was administered in two separate rooms at the Radisson Hotel in Miami, Florida. Blackburn was in a "very large" upstairs room with approximately thirty other candidates while a similar number took the examination in a downstairs room. The examination in the upstairs room began at 8:43 a.m. after various instructions were read to the candidates by the examination supervisor, Jeannie Smith, a veteran of twenty years in proctoring and supervising professional examinations. According to Smith, there was "considerable confusion" concerning which books could be used by the candidates, particularly since this was the first examination given with the new restrictions. She also acknowledged that there was "chaos" prior to the beginning of the examination and that this was, "extremely upsetting" to the examinees. However, before the examination began, Smith announced on a microphone the names of certain books which the Board had given her that were either prohibited or could be used by candidates. She further advised that if candidates had any questions they were to come to a bulletin board by the microphone where she had posted Xerox copies of the covers of various books. If a book could be used, it had the word "YES" printed on the cover while a "NO" was printed on those covers of books that could not be used. 1/ It is noted that only one cover sheet with a "YES" was posted, that being the Civil Engineering Reference Manual, Fourth Edition, Michael R. Lindeburg. However, at least three candidates who took the examination that morning, including petitioner, did not see the posted materials nor hear the invitation for candidates to come to the bulletin board. One book in issue that was specifically prohibited was Engineer In Training Review Manual, Sixth Edition, Michael R. Lindeburg, which contained 378 solved problems, and thus fell within the general prohibition of review manuals described on page 14 of the Candidate Information Booklet. However, those candidates who had the Seventh Edition of the same book were allowed to keep and use that manual even though it contained 422 solved problems, or some 44 more solved problems than was contained in the prohibited Sixth Edition. By allowing those students having the Seventh Edition to use the same even though it contained "review" materials, DPR violated the instructions contained in the Candidate Information Booklet and gave an advantage to those candidates not enjoyed by others, including petitioner. In addition, at least one other candidate in the upstairs group was allowed to use a prohibited review manual (Schaum's Outline Series, Theory and Problems of Electric Power Systems) but still that candidate did not attain a passing grade. Petitioner also contended that candidates taking the examination in the downstairs room were allowed to use language dictionaries during the morning part of the examination while those upstairs could not. 2/ Petitioner's contention is grounded upon hearsay evidence and accordingly it is found that no competent proof to support this claim was submitted. However, there was obviously some confusion over this matter because, after receiving complaints of this nature from two candidates, Smith telephoned the Board's offices in Tallahassee during the lunch break to ascertain whether such books could be used. Upon learning that they could not, she advised the upstairs group at the beginning of the afternoon session that dictionaries were not allowed. Blackburn also established that during the examination proctors went from desk to desk examining the materials that each candidate had in his possession. If a candidate had what the proctor perceived to be a book containing solutions to problems, the candidate was told to put the book on the floor. In the alternative, she candidates were told that if they tore the offending pages out of the book, they could continue using the remaining materials. Petitioner has complained that the proctors were not engineers and they were untrained in determining whether a book was acceptable or not. The Board has conceded that engineers do not proctor examinations but asserted that they are intelligent enough to determine whether books fall within the proscribed category. According to Blackburn's proctor at the examination, George Walton, a retired Coast Guard captain and engineering graduate of the Coast Guard Academy, he relied upon the list of approved and disapproved books supplied by the Board prior to the examination in determining whether materials would be excluded or not. Walton also stated that if he examined a book and found it contained solutions, he would disallow the same unless the offending pages were removed. A DPR expert in testing and measurements, Dr. Joseph A. Klock, examined the pass/fail rate for the examination taken by Blackburn and compared that rate to the October 1989 examination rate. Doctor Klock found no significant difference in the two rates and concluded that there was no statistically significant difference in performance of candidates over those time periods despite the confusion which occurred during the April 1990 examination. Blackburn did not present any evidence to show that if he had used the Seventh Edition of the Engineer In Training Review Manual, he would have been able to achieve more points on a particular problem and thus would have had a passing grade. Blackburn's principal complaint was that he had spent many hours preparing for the examination in question, that he was forced to guess which books to bring to the examination, and because of the confusion and chaos that took place at the beginning of the examination as well as his awareness that others were using a review manual with solved problems, it was impossible for him to give his best effort on the examination.
Recommendation Based on the foregoing findings of fact and conclusions of law, it is RECOMMENDED that petitioner's request to receive a passing grade on the April 1990 professional engineer examination be DENIED. However, petitioner should be entitled to retake the next examination at no charge. DONE and ENTERED this 28th day of November, 1990, in Tallahassee, Florida. DONALD R. ALEXANDER Hearing Officer Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, FL 32399-1550 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 28th day of November, 1990.
The Issue Whether Petitioner is entitled to additional credit for his responses to the "Principles and Practice" portion of the electrical engineer examination administered by Respondent in October 1996.
Findings Of Fact Petitioner took the electrical engineer licensing examination administered by Respondent in October 1996. Respondent is the agency of the State of Florida with the duty to regulate the practice of electrical engineering in Florida. Pursuant to Section 471.015, Florida Statutes, an applicant for licensure as an electrical engineer is required to successfully pass both parts of a licensure examination.1 The electrical engineer licensure examination at issue in this proceeding was developed and graded by the National Council of Examiners for Engineering and Surveying (NCEES). Following the initial grading of the "Principles and Practice" section of the exam, Petitioner was awarded a score of 68. A total score of 70 was required to pass that portion of the examination. Petitioner thereafter timely challenged the grading of two questions on the "Principles and Practice" portion of the exam. His challenge was limited to Questions 130 and 132. Petitioner did not specifically challenge Question 131. In response to that challenge, Respondent sent Petitioner’s examination package back to NCEES to have the "Principles and Practice" portion of the examination re-graded. NCEES re-graded all of Petitioner's answers to the "Principles and Practice" portion of the examination, including his responses to Questions 130, 131, and 132. NCEES initially awarded Petitioner a score of 2 points for his answer to Question 130. When the answer was re-graded, Petitioner was not awarded any additional credit for his answer to Question 130. The record in this proceeding established that Petitioner's answer to Question 130 was properly re-graded. Petitioner is not entitled to any additional credit for his response to Question 130. NCEES initially awarded Petitioner a score of 2 points for his answer to Question 132. When the answer was re-graded, Petitioner was awarded a score of 4 points for his answer to Question 132. The record in this proceeding established that Petitioner's answer to Question 132 was properly re-graded. Petitioner is entitled to a score of 4 points for his answer to Question 132. NCEES initially awarded Petitioner a score of 8 points for his answer to Question 131. When the answer was re-graded, Petitioner was awarded a score of 6 points for his answer to Question 131. The record in this proceeding established that Petitioner's answer to Question 131 was properly re-graded. Petitioner is entitled to a score of 6 points for his answer to Question 131. Petitioner is not entitled to a score of 8 for his answer to Question 131. Each of the three questions at issue in this proceeding is a problem that requires multiple steps and computations to solve. If a candidate correctly answers all parts of the question a score of 10 points is awarded. Partial credit can be awarded based on how many of the parts of the question are correctly answered. There is no allegation that the three questions involved in this proceeding are ambiguous or otherwise inappropriate for a licensure examination. The record is not clear when Respondent notified Petitioner of its position following the re-grading of the questions at issue. It is clear that Petitioner was aware of Respondent's position prior to the start of the formal hearing. During the formal hearing and in his post-hearing submittal, Petitioner challenged Respondent's right to re-grade Question 131 since he had not specifically challenged that question. Petitioner has not asserted that he was provided insufficient notice of Respondent's position.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that Respondent enter a Final Order that awards Petitioner a score of 68 on the "Principles and Practice" portion of the October 1996 licensure examination. DONE AND ENTERED this 11th day of December, 1997, in Tallahassee, Leon County, Florida. CLAUDE B. ARRINGTON Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (904) 488-9675 SUNCOM 278-9675 Fax Filing (904) 921-6847 Filed with the Clerk of the Division of Administrative Hearings this 11th day of December, 1997.
Findings Of Fact Prior to September 12, 1991, Verda Dupuy, manager of administrative services for the Newtron Group, contacted the Florida Electrical Contractors Licensing Board to obtain information needed to apply for licensure as a statewide electrical contractor on behalf of Phillip Ray Fife, Petitioner. Fife is employed by TRIAD Electric, a wholly owned subsidiary of Newtron Group. Both of these companies are out-of-state corporations which perform electrical contracting services in the United States. Newtron Group has annual revenues of some $80 million dollars and TRIAD has annual revenues of some $30 million dollars. An application packet was provided pursuant to this request and on September 12, 1991 Fife applied to the Board for licensure as an unlimited electrical contractor. The application packet did not include either a copy of chapter 21GG-5, Florida Administrative Code or Chapter 489, Florida Statutes. On October 31, 1991 the Application Committee met and considered Petitioner's application. James P. Williams, who also testified in these proceedings, was chairman of the Committee who reviewed Petitioner's application and recommended denial to the Board. By letter dated November 21, 1991 David O'Brien, Executive Director of the Electrical Contractors Licensing Board, notified Petitioner that his application had been denied by the Board for the following reasons: 21GG 5.003(1) lacks three years proven responsible management experience and/or; six years of comprehensive, specialized training, education, or experience associated with an electrical or alarm system contracting business. 21GG 5.005 - Failure to submit any of the items required by the rule. Your business entity portion lacks a Florida Corporate Charter (Certificate of Good Standing). Dunn and Bradstreet Credit Report not acceptable. Upon receipt of this information Dupuy, on behalf of Petitioner, contacted O'Brien several times to get specific details of the documentation desired. Since Petitioner was attempting to qualify for the January 27, 1992 examination time became important. After receiving additional clarification from O'Brien, the business entity portion was satisfied. In response to the demand for evidence that Petitioner had management experience a summary job description of Petitioner's responsibilities as a project manager for TRIAD was sent by FAX to O'Brien (Exhibit 8). As a result of these several telephone conversations and a listing of the duties of a project manager for TRIAD, O'Brien notified Ed James, Bureau of Examination Services, that Fife was authorized to sit for the January 27, 1992 examination and an examination packet containing a pass to take the exam was sent to Fife. On January 27, 1992 Fife travelled to Tampa and took the exam. Results of this exam have not been provided to Petitioner. On February 11, 1992 the Board met to again consider Petitioner's application. In addition to the summary of his job responsibilities as project manager the application file showed that Petitioner was licensed as an electrical contractor in the states of Arkansas, California, Nevada and Tennessee. However, the Board concluded these licenses may have been for a master or journeyman electrician rather than as a electrical contractor, and the summary job description submitted was not conclusive of Petitioner's management experience. On March 2, 1992, O'Brien mailed a second notice of denial to Petitioner (Exhibit 9). This letter states the application was denied for the following reasons: 21GG 5.003(1) Applicant must show that he has three years proven experience in the trade as an electrical contractor or alarm contractor or in a responsible management position with an electrical contractor or an alarm contractor. Please refer to definitions: 21GG- 5.001(3). 489.511 Has at least six years of comprehensive specialized training, education, or experience associated with an electrical or alarm system contracting business. Informal exposure to the trade wherein knowledge and skill is obtained. At the hearing petitioner presented a plethora of evidence attesting to his approximately 20 years experience in the electrical contracting business showing he executed contracts on behalf of TRIAD and Newtron Group, that he hired and fired employees, that as project manager he exercise full control over the projects including approving amendments and changes in the contracts, opened local bank accounts on which he paid the employees working on the project, etc. Petitioner's proposed findings 13-32 are adopted as facts presented to demonstrate Petitioner's experience in the field of electrical contracting. At the conclusion of Petitioner's case, James A. Williams, Respondent's expert who considered Petitioner's initial application and testified in these proceedings, acknowledged that licensure as an electrical contractor in another state with similar requirements as Florida would qualify Petitioner to sit for the examination in Florida; and that had he been privy to the testimony presented at this hearing, when he initially considered Petitioner's application, the application would have been approved.
Recommendation It is RECOMMENDED that a Final Order be entered finding Phillip Ray Fife fully qualified by experience and training to sit for the electrical contractors examination. It is further RECOMMENDED that Phillip Ray Fife's examination grades be released; and, if he passed the January 27, 1992 examination, that he be issued a statewide license as an electrical contractor. DONE and ENTERED this 20th day of July, 1992, in Tallahassee, Leon County, Florida. K. N. AYERS Hearing Officer Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-1550 (904)488-9675 Filed with the Clerk of the Division of Administrative Hearings this 20th day of July, 1992. APPENDIX TO RECOMMENDED ORDER Proposed findings submitted by Petitioner are accepted. Those proposed findings not included in the Hearing Officer's findings of fact were deemed unnecessary to the conclusions reached. Proposed findings submitted by Respondent are accepted except as noted below. Those proposed findings neither included in the Hearing Officer's findings nor noted below were deemed unnecessary to the conclusions reached. Rejected. Rejected insofar as Petitioner was fully aware of the reasons his application was rejected. No letter from the Board listed the documentation required by Rule 21GG-5.003(1)(a), Florida Administrative Code. Copies furnished: DAVISSON F DUNLAP ESQ WILLIAM W BLUE ESQ PO BOX 13527 TALLAHASSEE FL 32317 3527 JEFF G PETERS ESQ DEPT OF LEGAL AFFAIRS THE CAPITOL SUITE 1603 TALLAHASSEE FL 32399 1000 DANIEL O'BRIEN/EXECUTIVE DIRECTOR FLORIDA ELECTRICAL CONTRACTORS LICENSING BOARD PO BOX 2 JACKSONVILLE FL 32202 JACK McRAY ESQ GENERAL COUNSEL DEPT OF PROFESSIONAL REGULATION 1940 N MONROE ST TALLAHASSEE FL 32399 0792
Findings Of Fact Upon consideration of the oral and documentary evidence adduced at the hearing, the following relevant findings of fact are made: At all times pertinent to the issue herein, Petitioner, Ronald R. Corum, Examinee Identification No. 200619, was a candidate for licensure by examination as a professional engineer, and the Board of Professional Engineers was and is the state agency in Florida responsible for the licensing of Professional Engineers and the regulation of the practice of professional engineering in the state of Florida. Petitioner sat for the October 1990, Florida Professional Engineer Licensure Examination (Principles and Practice of Engineering). This part of the examination is divided into a morning session and an afternoon session. The morning session requires the examinee to choose four essay questions from a choice of twelve essay questions and produce a numerical solution to each question. The afternoon session is multiple choice and the examinee has to solve four questions from a choice of twelve questions. Each of the questions, both morning and afternoon, are worth ten points (raw score) for a total maximum raw score of 80 points, with a minimum passing raw score of 48 points. Petitioner received a raw score of 47 points. Question 124 was one of the essay question selected by Petitioner to solve in morning session of examination. Question 124 consisted three parts, 124A, B and C which required the examinee to: compute the area of traverse (in acres) a five-sided polygon; compute the net area (in acres) in the land parcel after adding sector area AB and excluding sector area DE; and compute the length of curve DE (in feet). The problems posed by Question 124 are not uncommon in the day to day practice of professional engineering and are not particularly difficult to solve. Petitioner attempted to solve Part A by using the method of coordinates which is an acceptable method of determining the area of a traverse. However, the Petitioner made a fundamental error in applying the method, not a simple mathematical error, in that he did not return to the beginning point of the traverse which resulted in an unrealistic answer. The correct answer to Part A was 16.946 acres. The Petitioner calculated the area to be 126.12 acres. In attempting to solve Part B, the Petitioner misapplied a correct methodology by erroneously expressing the central angle of the area in degrees rather than in radians. A radian is equal to approximately 57 degrees and this resulted in substantial error in Petitioner's calculation. The correct answer was 17.607 acres. Petitioner's answer was 219.63 acres which was not possible in relation to the area the Petitioner had already calculated for the traverse in Part A. This was a very serious error, a fundamental error, not a mathematical error. The maximum raw score for question 124 was ten points. Petitioner received a raw score of two points. On review, Petitioner was again granted only two points out of ten possible points. The examinee's identity is not known to the scorer during the initial scoring or the review. Both question 124 and the scoring plan used in grading question 124 were approved by the National Council of Examiners of Engineers and Surveyors (NCEES). The scoring of question 124 was weighted so that Parts A and B were worth four points each, and Part C was worth two points. Petitioner correctly answered Part C and received two points. The Petitioner did not receive any points for Part A or Part B. The examinee was not aware of this weighting policy at the time of the examination. The scoring plan for question 124 which was used by the NCEES grader was set up in six (6) categories from 0 - 10 in two-point increments as follows: 10 - Exceptionally competent. 8 - More than minimum competence but less than exceptionally competent. 6 - Minimum competence. 4 - More than rudimentary knowledge but insufficient to demonstrate competence. 2 - Rudimentary knowledge. 0 - Nothing presented to indicate significant knowledge of the problem. Petitioner's use of acceptable methodologies in attempting to solve the problems of Parts A and B may indicate at least rudimentary knowledge and possibly more than rudimentary knowledge but insufficient knowledge to demonstrate competence which would have entitled Petitioner to at least two points on Parts A and B each. However, the unreasonableness and the impossibility of his answers and his failure to recognize the unreasonableness and impossibility of his answers coupled with his fundamental error in solving the problems of Parts A and B were such that the Petitioner did not demonstrate significant knowledge of the problems for Parts A and B. Therefore, any credit that would have been given for using acceptable methodologies in attempting to solve the problems would be negated by this lack of significant knowledge of the problems. Because of this lack of significant knowledge of the problems the scorer correctly adjusted Petitioner's score on Part A and Part B each to zero. Unreasonable answers result in credit being deleted, and this policy is uniform among all of the states. However, the examinee is not made aware of this policy at the time of the examination. There was no instruction or guide to indicate to the examinee that if the examinee recognized that any answer was unrealistic that the examinee should so indicate on the answer sheet. Likewise, there was no instruction or guide to indicate that the examinee would be more heavily penalized if the examinee did not indicate on the answer sheet that the answer was unrealistic. An examinee's inability to recognize an unrealistic answer and to so indicate on the answer sheet without specific instruction goes to the examinee's competence as a professional engineer. Therefore, Petitioner has not been treated unfairly by the lack of instruction or guide advising him to indicate his ability to recognize an unrealistic answer on the answer sheet. The NCEES scorer for question 124 attempted to award the same score to all examinees of the October 1990 examination who gave similar unrealistic answers to question 124 as did Petitioner without noting on the answer sheet that the answer was unrealistic. The examinees are not informed of how the scoring plan will be applied in advance of the examination or that the essay question will be scored in two- point increments only. There was no evidence that this information would be of significant benefit to the examinee. In fact, the Petitioner did allocate his time in attempting to solve question 124 similar to the weighting of the scoring plan, spending only a small part of the time on Part C. Part B should have identified the curved areas to be computed as segments, rather than sectors. Petitioner attempted to solve Part B as though it referred to segments, and did not raise this issue in the request for review. Petitioner's use of degrees rather than radians would have been equally erroneous in determining the area of a sector. There was no evidence to show that identifying the curved area as a sector rather than a segment had any effect on Petitioner's attempt to solve the problem. The official solution to Part B contained a typographical error made during the transcription of the grader's handwritten solution. This had no effect on the scoring of Part B. The solution cannot affect the answer given by the examinees, as the solution is only available after the examinee has completed the examination and is challenging the scoring. There is a lack of competent substantial evidence in the record to establish that the scores which Petitioner received on Part A and Part B of question 124 of the October, 1990 Professional Engineering Licensure Examination were incorrect, unfair or invalid, or that the examination, and subsequent review, were administered in an arbitrary or capricious manner.
Recommendation Based upon the foregoing, it is RECOMMENDED: That Respondent enter a Final Order dismissing the Petitioner's challenge to the grading of his response to question 124 on the October 1990 Professional Engineer's Licensure Examination. DONE and ENTERED this 26th day of November, 1991, in Tallahassee, Florida. WILLIAM R. CAVE Hearing Officer Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, FL 32399-1550 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 26th day of November, 1991. APPENDIX TO RECOMMENDED ORDER The following constitutes my specific rulings pursuant to Section 120.59(2), Florida Statutes, on all of the Proposed Findings of Fact submitted by the parties in the case. Rulings on Proposed Finding of Fact Submitted by the Petitioner Each of the following proposed findings of fact are adopted in substance as modified in the Recommended Order. The number in parenthesis is the Finding(s) of Fact which adopts the proposed finding of fact: 1 (2); 2 (3); 3 (3); 4 (5); 5 (5); 6 (7); 7 (9); 8 (9); 9 (13); 10 (13); 11 (12); 12 (9); 13 (5); 14 (6); 15 (9); 16 (11); 17 (7); 18 (1); 20 (16); 21 (16) and 22 (18). Proposed finding of fact 19 is not supported by substantial competent evidence in the record but see finding of fact 11. Rulings on Proposed Findings of Fact Submitted by the Respondent 1. Each of the following proposed findings of fact are adopted in substance as modified in the Recommended Order. The number in parenthesis is the Finding(s) of Fact which adopts the proposed findings of fact: 1 (2); 2 (3); 3 (4); 4 (5); 5 (6); 6 (6); 7 (7); 8 (8); 9 (9); 10 (12); 11 (11); 12 (15); 13 (9, 16); 14 (17); 15 (18). COPIES FURNISHED: Wellington H. Meffert, II, Esquire Department of Professional Regulation 1940 North Monroe Street, Suite 60 Tallahassee, FL 32399-0792 David W. Persky, Esquire Spicola & Larkin 806 Jackson Street Tampa, FL 33602 Angel Gonzalez, Executive Director Board of Professional Engineers 1940 North Monroe Street, Suite 60 Tallahassee, FL 32399-0792 Jack McRay, General Counsel Department of Professional Regulation 1940 North Monroe Street, Suite 60 Tallahassee, FL 32399-0792
Findings Of Fact Based on the stipulations and admissions of the parties, the exhibits received in evidence, and the testimony of the witnesses at the hearing in this case, I make the following findings of fact. Findings based on stipulations and admissions of the parties The Petitioners, John Eugene Harden and Dova Cauthen, qualified for and were administered the January 1982 Electrical Contractors' Licensing Examination. After their examinations were graded and regraded, Petitioners were notified that they had not received a "Passing" score of 75 or more. The Petitioners, John Eugene Harden and Dova Cauthen, qualified for and were administered the July 1982 Electrical Contractors' Licensing Examination. After their examinations were graded and regraded, Petitioners were notified that they had not received a "Passing" score of 75 or more. Both Petitioners reside in Dade County, Florida. The Respondent Department of Professional Regulation (hereafter DPR or the "Department") is an agency of the State of Florida. The Respondent Electrical Contractors' Licensing Board (hereafter referred to as the "Board") is an agency of the State of Florida statutorily responsible, along with DPR, for licensing certified electrical contractors. The Office of the Executive Director of the Board is located at 130 North Monroe Street, Tallahassee, Florida. The business address of Petitioner Harden is Harden Electric, 311 N.E. 8th Street, Homestead, Florida 33030. The business address of Petitioner Cauthen is 959 N.E. 79th Street, Miami, Florida 33138. 8. On March 20, 1981, Rules 21GG-6.01 (2), 21GG-6.01 (3), 21GG-6.01 (4), of the Rules of the Electrical Contractors' Licensing Board, were amended. Rule 21GG-6.01 (2) altered the format of the licensing examination so that the formerly entirely open book examination became a part open book, part closed book examination. Rule 21GG-6.01 (4) raised the passing grade on the licensing examination from 70 to 75. Rule 21GG-6.01 (3) altered the technical format. On December 3, 1981, Susan Tully, counsel to the Electrical Contractors' Licensing Board, caused a change to be made in Rule 21GG-6.01 (1) (c) of the Rules of the Electrical Contractors' Licensing Board. This change was made without formal notice or informal notice to the public and without a vote of the Electrical Contractors' Licensing Board, although the change was discussed at a Board meeting. None of the procedures set forth in Section 120.54 of the Florida Statutes were followed in amending this Rule. Liz Cloud was the Bureau Chief of the Division of Elections, Bureau of Administrative Code, Department of State. The address of the Bureau is Room 1802, The Capitol Building, Tallahassee, Florida. The change in Rule 21GG-6.01 (1)(c) eliminated parts 72 A, B, C and D and indicated that the entire Fire Safety Code (and not just the aforementioned parts) would be a subject of the Electrical Contractors' Licensing Examinations. In fact, the reason for the alleged "technical change" was that the Board intended and tested examinees in the January and July 1982 Licensing Examinations on materials in parts of the Fire Safety Code in addition to those contained in 72 A, B, C and D of the Fire Safety Code. Petitioner Harden specifically requested to review his January and July 1982 Electrical Contractors' Licensing Examinations. On March 25, 1982, Petitioner Harden went to Tallahassee, Florida to review his January 1982 Licensing Examination papers, but although he requested an "examination review" he was never provided with his own examination booklet or a copy thereof. During this alleged "examination review" Mr. Harden wrote written objections to the January 1982 examination on the forms provided to him based on the master copy of the examination provided to him. On August 31, 1982, Petitioner Harden went to Tallahassee, Florida to review his July 1982 Licensing Examination papers, but although he requested an "examination review" he was never provided with his own examination booklet or a copy thereof. During this alleged "examination review" Mr. Harden wrote written objections to the July 1982 examination on the forms provided to him based on the master copy of the examination provided to him. Petitioner Cauthen specifically requested an "examination review" with respect to her January and July 1982 Licensing Examination papers. On March 17, 1982, Petitioner Cauthen went to Tallahassee, Florida to review her January 1982 Licensing Exami-nation papers, but although she requested an "examination review" she was never provided with her own examination booklet or a copy thereof. During this alleged "examination review" Ms. Cauthen wrote written objections to the January 1982 examination on the forms provided to her based on the master copy of the examination provided to her. On August, 19, 1982 Petitioner Cauthen went to Tallahassee, Florida to review her July 1982 Licensing Examination papers, but although she requested an "examination review" she was never provided with her own examination booklet or a copy thereof. During this alleged "examination review" Ms. Cauthen wrote written objections to the July 1982 examination on the forms provided to her based on the master copy of the examination provided to her. Petitioners Harden and Cauthen sought Board review of their January and July 1982 Electrical Contractors' Licensing Examinations. Petitioners were advised in July 1983 that no copies of their actual booklets exist; Petitioners were advised in October 1983 that their actual booklets were shredded. At the November 8, 1982, meeting of the Board, Ms. Ida Cameron representing DPR, presented to the Board a package of information (assembled in package form) for the Board to consider during the examination review. With regard to Question Number 71 on the afternoon portion of the July 1982 Licensing Examination, two of the four possible responses, "A" and "C" were credited. Candidates like Petitioner Cauthen who answered "B" received no credit for the question. Petitioners Harden and Cauthen have requested Chapter 120.57 hearings with respect to their January and July 1982 licensing examinations and the review, grading and agency action with respect to same. Although request for production was specifically made for said packages of information with respect to the January 1982 and July 1982 licensing examinations in February, 1983, no packages have been produced to date. DPR destroyed the Petitioners' examination booklets before the end of the two year period immediately following each of the 1982 examinations. That Section 455.217 of the Florida Statutes requires the Board "by rule" to designate areas of competency to be covered by each licensing examination. That Section 455.217 states that the Board shall "by rule specify the general areas of competency to be covered by each examination, the relative weight to be assigned in grading each area tested, and the score necessary to achieve a passing grade." That the amendment of Rule 21GG-6.01 (2) required candidates to commit to memory portions of the electrical code, accounting, law, worker's compensation rules, federal employer's tax guide, A1A General Conditions, business practices; legal and insurance requirements. In the July 1982 Electrical Contractors' Licensing Examination candidates were given one hour more time in which to take the examination than was given during the January 1982 examination. Rule 21GG-6.01 (4) was amended in conjunction with Rules 21GG-6.01 (1)(a), (2) and (3) at a board hearing in March, 1981. Rule 21GG-6.01 (4) raised the passing score on the January 1982 Electrical Contractors' Examination and the July 1982 Electrical Contractors' Examination from 70 to 75. The amendment to Rule 21GG-6.01 (4) required that electrical contractors pass the certification examination with a score of 75 percent, whereas all other construction industry licensing board contractors licensed by the Department of Professional Regulation (13 other categories) were merely required to achieve a 70 percent pass score. That Section 455.217 of the Florida Statutes provides that "The board shall make rules providing for reexamination of any applicants who have failed the examination." That DPR did not and has never provided Petitioners, their attorneys or agents with their own examination booklets for the January and July 1982 Electrical Contractors' Licensing Examinations. DPR has provided Petitioners only with copies of "master" examinations for their review. That Section 455.217 of the Florida Statutes provides that the Board shall make available an examination review procedure for applicants. That the Respondents produced Notice of Destruction of Examination Booklets and Other Examination Materials dated April 14, 1982, which allegedly evidences destruction of Petitioners' examination booklets for the January 7, 1982, Examination on April 14, 1982. That the Respondents produced Notice of Destruction of Examination Booklets and Other Examination Materials dated October 15, 1982, which allegedly evidences destruction of Petitioners' examination booklets for the July 13, 1982, Examination on October 15, 1982. The findings in paragraphs 1 through 33, immediately above, are based directly on the stipulations of the parties, most of which stipulations were memorialized at the beginning of the hearing. The findings in the following paragraphs are based primarily on testimony and exhibits, but some of them are also based in whole or in part on stipulations. In the findings which follow there are certain to be at least some repetitious findings in the course of putting matters into context and making additional findings which are related to some of the stipulated findings. I have tried to avoid all unnecessary repetition, but a certain amount is necessary for clarity and a certain amount is unavoidable due to the sheer size of the task at hand. Findings on background matters and on matters relating to more than one rule Both of the Petitioners in this rule challenge proceeding are individuals who have applied to the Electrical Contractors' Licensing Board for licensure as certified electrical contractors. Both of them have been approved to sit for the licensure examination. Petitioner Harden took the Board's licensure examination on each of the following occasions: July 1981, January 1982, July 1982, and January 1983. The Board has not given him a passing grade on any of those' four examinations. Petitioner Cauthen took the Board's licensure examination on two occasions: January 1982 and July 1982. The Board has not given her a passing grade on either of those two examinations. The grade notifications received by these Petitioners show, inter alia, that Petitioner Cauthen received a grade of 73 on the July 1982 examination. Both of these Petitioners have presently pending formal proceedings under Section 120.57(1), Florida Statutes, in which they are challenging various matters related to the preparation of, administration of, scoring of, and inherent validity of the Board's January 1982 and July 1982 licensure examinations. The Electrical Contractors' Licensing Board certification examination is different from a master electrician examination. However, the two examinations are in many ways similar because of the overlap in the nature of the subject matter to be tested on both examinations. The passing score or cut score on the local master electrician examination administered by Dade County is 70 percent. Most of the questions on that examination are about the National Electric Code. Part of the Dade County master electrician exam is closed book. Prior to 1972, persons wishing to engage in electrical contracting in the state of Florida were required to be licensed by the local governments in the areas in which they sought to operate. Since 1972, persons wishing to engage in electrical contracting in the state of Florida must be licensed by a unit of local government or by the Electrical Contractors' Licensing Board. Persons who are licensed by the Electrical Contractors' Licensing Board are known as certified electrical contractors. A certified electrical contractor can engage in electrical contracting anywhere in the state of Florida without local licensure. Persons who are licensed by one or more local governments (typically a municipality or a county) are known as registered electrical contractors. Registered electrical con-tractors are licensed to engage in electrical contracting only in the geographic areas encompassed by the boundaries of the local government entities that issued their local licenses. There is, however, a certain amount of reciprocity from one local government to another. Certification as a state certified electrical contractor does not authorize the electrical contractor to work as an electrician, although a certified electrical contractor can pull permits. A person does not have to be a licensed electrician in order to become a state certified electrical contractor, although many electrical contractors are also licensed as journeyman or master electricians. Candidates for the electrical contractor licensure examination have a great $ variety in the nature and scope of their background and experience. This variety in background and experience is among the reasons which cause testing for minimum competence as an electrical contractor not to be an exact science. As between certification and registration, certification by the Electrical Contractors' Licensing Board is not a major factor in the ability to compete for business. A person who has a statewide certification can save some money by avoiding the expense of obtaining local competency cards, but as a general rule a registered electrical contractor can work just about anywhere in the state due to reciprocity. Especially, a person who has passed a Block master electrician examination or a Block local electrical contractor's examination can work just about anywhere in the state because most counties in Florida accept the Block examination. Registered electrical contractors regularly compete for business with certified electrical contractors. For example, Petitioner Cauthen's company is presently licensed in Monroe, Dade, and Broward counties, where it does a considerable amount of business. Petitioner Cauthen's company bids on lots of contracts and submits bids in competition with both registered and certified electrical contractors. Thus, the company is already in competition with certified electrical contractors. Similarly Board Member Isaacs, whose certified company does 90 percent of its work in Duval County, has regular competition from registered electrical contractors. Board Member Isaacs competes with approximately 250 electrical contractors. Of that 250, approximately 175 are registered and the others are certified. A much bigger factor in competition than the registration versus certification issue is the cost of doing business in more than one area or the cost of doing business at a location that is distant from one's primary base of operations. Only about fifty electrical contracting firms regularly compete for business over the entire state of Florida. Accordingly, a restriction on the number of persons licensed by the Board as certified electrical contractors would have minimal, if any, limitation on the competition faced by those members of the Board who are certified electrical contractors. The Electrical Contractors' Licensing Board admin-istered its own certification licensure examination from 1972 through 1980. During that period of time the Board administered the examination a total of twenty-two times. During those twenty-two sessions of Board administered examinations, a total of 824 candidates sat for the examination, of which 392 were successful. Although the percentage of candidates who were successful on a particular Board administered examination ranged from a low of 20.7 percent to a high of 78.8 percent, the average passing rate of all candidates on all twenty- two of the Board administered examinations was 47.57 percent. (By way of comparison, it is interesting to note that the percentage of candidates who are successful on the Block master electrician examination is between 30 percent and 35 percent of those who take the examination.) Further, on fifteen of the twenty-two occasions on which the Board administered the examination, the percentage of candidates who were successful was 50 percent or less. In 1979, the regulation of professions and occupations was reorganized. The Department of Professional Regulation (the Department) was created as an umbrella agency over numerous boards, including the Electrical Contractors' Licensing Board and the Construction Industry Licensing Board. The responsibility for examining applicants, which in the case of the Electrical Contractors' Licensing Board had been handled by the Board members themselves, was taken over by the Office of Examination Services (O.E.S.). Toward the end of 1980 and the beginning of 1981, O.E.S. did not like what the Board was doing and the Board did not like what O.E.S. was doing. There was a general lack of trust between the two entities, there was poor communication between the two entities, and the relationship between them at that time might best be described as estranged. Although both entities made efforts at cooperation with the other, such efforts were not always effective or well received. As a result of the poor communication between the two entities, certain personal concerns over Board actions by individuals within the O.E.S. were never communicated to the Board (and therefore could not be answered. 4/ The first examination administered by the O.E.S. was in January 1981. Of the fifty candidates who took the January 1981 examination, forty-six were successful. This was a passing rate of 92 percent. In view of the Board's experience when it was administering its own examination, the Board was understandably surprised and concerned when the percentage of candidates passing the first O.E.S. administered examination was almost twice the average passing percentage on the Board administered examinations. The Board had no evidence that the overall qualifications or capabilities of the applicants examined by O.E.S. at the January 1981 examination were any higher than the capabilities of those examined previously. To the contrary, a review of their qualifications based on the information in their applications indicated that their qualifications were substantially the same as those of previous candidates tested by the Board. Also, the admission standards for the January 1981 examination were the same as those for prior examinations. The Board concerns about the O.E.S. administered January 1981 examination included the following matters, among others, memorialized at a Board meeting discussing the examinations: The candidates had been allowed to take any reference material desired into the exami- nation. Previously the Board had allowed only reference books listed as part of the application form. There was a very high pass rate in compar- ison to all past examinations. There appeared to be too few calculation questions. The questions had been placed on the paper in sequence with the reference book materials. There was a possibility that notes were taken into the examination which would have given advantage to the candidates. The Board's Examination Committee had been under the assumption that the entire examination would be presented for review of the questions. Instead the only questions presented were ones questionable under Department criteria for measuring competency. It was explained, from past experience, that persons conducting the seminars for exams had obtained the entire exam content from persons taking the examination in the past. It had been done with a camera which was taken into the examination. The Board, when administer- ing the examination, had been very strict on what was used in performing the examination. In the last exam the security was very poor. The Board felt there were so few calculation questions on the O.E.S. administered examination in January of 1981 that the examination was not a proper or sufficient examination. This was due at least in part to the fact that O.E.S. had difficulty covering all areas of the examination with the desired number of questions because the O.E.S. bank of questions was very limited at that time. Previous Board examinations were open book with certain reference material allowed. When the Office of Examination Services took over the administration of the January 1981 exams, it sent to the candidates a form letter adapted from the Construction Industry Licensing Board. This form letter instructed the candidates that they would be able to bring into the examination any notes and other materials desired. This was contrary to the past practice of the Board and contrary to the Board's wishes. The Board was very frustrated and concerned about the circumstances which led up to the January 1981 examination because the O.E.S. had refused to allow the Board to have any input into the examination and the Board thought the O.E.S. had come up with an apprentice level examination. In the Board's opinion, the January 1981 examination prepared by O.E.S. clearly tested at a level less than that of minimum competency. And, although O.E.S. did not agree with the Board that the January examination was invalid, O.E.S. did agree that the examination was a lot easier than they had thought it would be. The Electrical Contractors' Licensing Board did not conduct any formal studies or formal analysis prior to making the 1981 amendments to Rule 21GG-6.01 changing the format of the examination and the cut score. However, the Board's Rules Committee did a great deal of work on the matter and the matter was discussed extensively at Board meetings. In this regard it is important to keep in mind that the Board members (with the exception of the two lay members) were all experienced practicing electrical contractors who were familiar with the requirements of day-to-day electrical contracting. The Board's decision to change part of the examination to closed book was due to a concern that people with no electrical knowledge could become licensed if the examination was all open book and had a low percentage of technical questions. The Board felt that more than half of the examination should be technical questions. The closed book National Electrical Code questions were for the purpose of testing what a person with experience in the field of electrical contracting should know about everyday matters. The primary, if not the sole, motivation for the Board's 1981 amendments to Rule 21GG-6.01 was the Board's concern about the quality of the O.E.S. administered examination and the Board's feeling that, due to its estranged relationship with O.E.S., the only vehicle through which it could effectively influence the quality of future examinations was through rule- making. Those amendments were not motivated by any desire on the part of the Board to restrict competition in the field of electrical contracting. During the past few years the job requirements of an electrical contractor have remained substantially the same. Block and Associates (hereinafter "Block") is a company based in Gainesville, Florida, that writes licensure examinations for contractors in various trades. Block writes such examinations for cities, counties, and states. Block has prepared electrical contracting licensure examinations for the states of Florida, Georgia, and Oklahoma, as well as for St. Johns County, Florida. Block prepares licensure examinations for just about anything that involves electricity, including such things as electrical contracting, electrical journeyman, electrical-master, low voltage, elevator, burglar alarm and fire alarm Block prepares master electrician examinations for over one hundred governmental entities. The Block electrical master exams and the Block electrical contractor exams use some of the same questions. Subsequent to the 1982 examinations which form the gravamen of these Petitioners' related proceedings under Section 120.57, Florida Statutes, the ECLB has contracted with Block and Associates to prepare the Board's certification licensure examination. The Board has never said anything to Block about wanting to achieve any particular passing rate or wanting to have any particular percentage of the candidates taking the examination achieve a passing score. The Board has never interfered with Block's autonomy in the preparation of the electrical contractor exam. All of Block's electrical contractor examinations include business questions. It is important to test for business skills because an electrical contractor who gets into business trouble may start cutting corners and cutting corners is a hazard to the public. Some of Block's electrical contractor examinations also have specific sections on safety. Block always does a post-administration statistical analysis of the examinations questions it uses. The reason for this is to find out if anything is wrong with the questions; in other words, to determine whether the questions are valid. The computer program used by Block to check the validity of its tests was prepared for Block by a professor in the College of Education at the University of Florida. Block uses a computer to conduct a validity analysis of each test it administers. Findings regarding Rule 21GG- 6.01(1)(c) of the Electrical Contractors' Licensing Board Rule 21GG-6.01(1)(c) of the Electrical Contractors' Licensing Board reads as follows (with the portion challenged by the Petitioners underscored): The areas of competency to be covered by the certification examination shall be as follows: (a) *** (c) Safety, which shall include but not be limited to, OSHA regulations, life safety codes, and the Fire Safety Code (NFPA). Prior to the December 1981 amendment which is challenged here, Rule 2100-6.01(1)(c) read as follows: The areas of competency to be covered by the certification examination shall be as follows: (a) *** (c) Safety, which shall include but not be limited to, OSHA regulations, life safety codes, and the fire safety code (NFPA 72A, B, C and D). The Board's purpose in amending the rule to delete the reference to "72A, B, C and D" was to make it clear to candidates for examination that the Board' intended to include in the safety portion of its examinations questions from portions of Volume Seven of the NFPA other than Chapters 72 A, B, C, and D of that volume. Volume Seven of the NFPA contains quite a bit of material in addition to the material included in Chapters 72 A, B, C, and D of that volume. In view of the modifying phrase "shall include but not be limited to," the amendment to delete the reference to Chapters 72 A, B, C and D was not a necessary prerequisite to the use of examination questions based on other portions of Volume Seven of the NFPA. The Board could ask the same examination questions before and after the amendment. Several weeks before each examination, the Department of Professional Regulation sent all candidates for examination, including these Petitioners, a notice to appear for the examination. The notice to appear included a list of reference books on which the examination was to be based. That list of reference books contained a specific reference to Volume Seven of the NFPA. All sixteen volumes which comprise the entire NFPA were not listed as reference books. Neither of these Petitioners demonstrated that he or she was genuinely confused or misdirected by the amendment to Rule 21GG-6.01(1)(c). Neither Petitioner contended he or she had studied all sixteen volumes of the NFPA. Instead, both Petitioners studied from the books on the reference list. Neither Petitioner has an application for the examination pending; neither claims an intention to take the examination again. Findings regarding Rule 21GG- 6.01(1), (2), (3), and (4) of the Electrical Contractors' Licensing Board At the time of the examinations which are the subject of these Petitioners' related cases under Section 120.57, Florida Statutes, Rule 21GG- 6.01(1), (2), (3), and (4) of the Electrical Contractors' Licensing Board read as follows, in pertinent part: The areas of competency to be covered by the certification examination shall be as follows: Technical, which shall include, but not be limited to, electrical calculations, estimating, designs, and electrical schematics; and which shall be divided into two (2) separate areas, one containing electrical calculations, the other containing Code-related questions not requiring calculations. General business, which shall include but not be limited to, problems relating to accounting, law, insurance, workers' compen- sation and social security; The format of the examination shall be as follows: The portions of the examination con- taining the electrical calculations and $ safety questions shall be open book. The applicant is responsible for bringing and may.. use during these portions the applicable code books, reference materials as approved by the Board, and noiseless mechanical or non- mechanical instruments he wishes to use. The portions of the examination con- taining the noncalculation technical questions and all business questions shall be closed-book. The applicant will not be permitted to use the reference materials listed in 2(a) or any other reference materials. Security measures as set forth by the Department shall be followed during both portions of the examination. The relative grading weight to be assigned to each area of competency shall be approximately as follows: Technical - 60 percent Electrical Calculations-40 percent of Test Code-related questions-20 percent of Test General business - 30 percent Safety - 10 percent An applicant shall be required to achieve a score of a general average of not less than seventy-five percent (75 percent) in order to pass the examination and be certified for licensure. There shall not be a practical or clinical examination. The language quoted above is the language of the relevant portions of the rule as it read after the March 1981 amendments to the rule. The principal amendments in March of 1981 may be summarized as follows: (1) changes which divided technical questions on the examination into two separate areas, (2) changes which made part of the examination open book and part closed book [it was previously all open book, and (3) a change in the minimum passing score from 70 percent to 75 percent. These Petitioners challenge the validity of all three of the principal amendments. As originally proposed, the March 1981 amendments to subsection (2)(a) of Rule 21GG-6.01 would have permitted a candidate to bring "any notes" to the open book portion of the examination. During the adoption process the words "and any notes" were deleted from the amendment by the Board. The words "and any notes" did not appear in Rule 21GG-6.01 prior to the 1981 amendments. Nor was it the practice of the Board prior to 1981 to permit candidates for examination to take any notes into the examination when it was administered by the Board. Findings regarding examination content and relative grading weights In November of 1983 the Electrical Contractors' Licensing Board again amended Rule 21GG-6.01(3) to change the relative grading weights of the areas of competency to be tested. After November of 1983 the relative grading weights of the areas of competency were as follows: Technical increased from 60 percent to 65 percent; electrical calculations increased from 40 percent to 45 percent; code related questions remained 20 percent of the examination; general business was decreased from 30 percent to 25 percent of the examination; and safety remained 10 percent of the examination. 5/ The licensing examination given by the Board has historically contained materials testing knowledge of electrical work and also materials testing the business qualifications of the applicants. Since the inception of the Board's licensing exam, the electrical work portions have contained electrical calculations. Throughout the time the Board has been administering licensure examinations for certification, the percentages of the examination devoted to particular subject areas have been based upon the Board's belief as to the appropriate levels of knowledge needed by a certified electrical contractor. The weighting of the examination was equitable when it was weighted 60 percent technical, 30 percent business, and 10 percent safety, and it is still equitable now that it is weighted 65 percent technical, 25 percent business, and 10 percent safety. The Board changed to the current percentages at the suggestion of Block and Associates. The Board's examination has had similar ratios for a long time, at least as far back as 1971. The proportions of the examination devoted to each particular subject matter did not change due to the March 1981 amendments. With the exception of the January 1981 examination put together by O.E.S., electrical calculations had historically been 40 percent of the test. Since the Board felt this percentage should be maintained, and since it was necessary to divide those calculations from the other portion of the technical section (since one would be tested in the closed book portion and the other in the open book portion), the division of the technical part of the exam was set out in the rule. On an electrical contractor licensure examination it is appropriate to include in the area of "technical" questions, questions dealing with the following subjects: electrical calculations, parts of estimating, and designs and electrical schematics. In the "business" portion of an electrical contractor examination, it is appropriate to include questions dealing with the following: accounting, worker compensation, and social security. Findings regarding closed book versus open book examinations Local licensing examinations had, in 1981, included a closed book portion. Local licensing examinations and state licensing examinations given in other states still include such a closed book portion. The basic Block examination has always included a closed book portion. Closed book examinations are not all that unusual in occupational and professional licensure testing. The CPA an Nursing examinations are closed book. Also, virtually all of the electrical examinations administered by Block and Associates include a closed book portion. The advantage of a closed book examination technique is that it is easier to identify the candidates who have had practical experience in the subject matter being tested. A closed book examination is a better test for field experience and for general knowledge of the subject matter being tested. Certain questions known as "anchor" questions was given both in the open book examination before the format change and in the closed book section after the format changed. The relative performance by candidates on the same questions in the two formats shows the format change not to have prejudiced the candidates. Further, an examination analysis of the results of the closed book portions of the ECLB examinations reveals that, based on candidate performance, the closed book portion of the examinations was easier than the open book portion. This also shows that the format change did not prejudice the candidates. Findings regarding "cut scores" From 1972 until the rule changes in 1981, the minimum passing score, or "cut score," set by the Board for the certification licensure examination was 70 percent. Cut scores of both 70 percent and 75 percent are very common cut scores for licensure examinations in the fields of electrical contracting, journeyman electrician, and master electrician, as well as licensure examinations in other occupations and professions. Many local licensure examinations have a 75 percent cut score. Standing alone, the cut score to be applied to a future examination is somewhat lacking in specific significance. A cut score must be considered in relation to the content of the examination to which it is to be applied. All of the testing experts who testified at the hearing agreed that setting an examination cub score is an inherently arbitrary decision. In the final analysis it is a judgment call that cannot be made with scientific exactitude and is unavoidably arbitrary to a certain extent. As explained by one of the Petitioners' testing experts, even though statistical methodology is available for the determination of cut scores, to a large extent the cut score is a random decision, the validity of which can be assessed only in relationship to the specific testing instrument to which it is applied. For example, during the course of Block's preparation of its first licensure examination in 1958 for the City of Ormond Beach, when it came time to decide what cut score to use, the five experts who were preparing the examination talked it over and decided that 75 percent was the equivalent' of a good. "C" when compared to academic grades, and they decided to use 75 percent as their cut score. It is not possible to be much more scientific or precise than that in setting cut scores, especially when one has a group of candidates of varied experience and background taking the examination. A skilled test preparer can construct an examination of equal validity within a range of cut scores from 65 percent to 80 percent if the test preparer knows at the time of constructing the examination what the cut score will be. With any cut score in that range, a skilled test preparer can raise or lower the percentage of candidates who will actually pass the examination by careful selection of the questions to be used on the examination. By using easy questions, the person preparing the examination can insure that more people will pass a test with a high cut score. Similarly, by using hard questions, the person preparing the examination can insure that fewer people will pass an examination with a low cut score. On some standardized licensure examinations the cut score is determined after the examination results are examined. In other words, the examination results are "curved" based on the performance of the group of candidates who take a particular session of the examination and the actual cut score is determined in relationship to the raw scores achieved by each particular group of candidates. This flexible method of setting cut scores seems to work well on standardized examinations which are administered to large numbers of candidates, but it is questionable whether it would work very well with small groups of candidates. The Board's examinations have historically been administered to small groups of candidates. Findings regarding the effects of changes on testing instruments. In order for a licensure examination to accomplish the purpose for which it is administered, it should, of course, be valid and reliable. Several of the Petitioners' expert witnesses described what might be characterized as the best way to make changes to an examination or as the ideal way to make changes to an examination. Nevertheless, the manner in which the Electrical Contractors' Licensing Board went about making the changes to the examination which were incorporated in the March 1981 rule changes was a reasonable and logical response to the circumstances faced by the Board. It was perhaps not the best way to have resolved matters, but it was a reasonable response. The Board's action in this regard was not arbitrary or capricious. Even the Petitioners' most persuasive expert in the field of testing conceded that if the content experts (in this case the Board members) felt that given a specific content domain and given a specific item bank the testing instrument measured less than minimum competency, a logical response would be to raise the cut score. Another logical reaction to that situation would be to change a portion of the examination from open book to closed book. Both actions would be reasonable and logical responses to that type of problem. Where there has been a modification in the format of a testing instrument, an accepted methodology for analysis of the effects of format modification is the use of repetitive questions from prior testing instruments. These repetitive questions are known as "anchor" questions. If a statistically significant deviation in performance on the anchor questions is demonstrated, the format modification requires further analysis through other statistical tools. However, if an analysis of the anchor questions shows inconclusive or mixed results on the anchor questions (such as equal or improved candidate performance on those questions), it may then be concluded that the format change is not significant to candidate performance and may be an appropriate format change relative to the profession or occupation being examined. Content validity is the most practical' measure of the validity of a licensure examination. The content validity of an examination depends on the extent to which the questions on an examination may be accepted as representative of performance within a specifically defined content domain of which the examination instrument is a sample. Content validation would be the best way to determine the validity of the examinations which were administered in 1982 pursuant to the March 1981 rule amendments. One method of establishing content validity is to have a group of experts identify the domains of knowledge that are applicable to the profession or occupation being tested. The establishment of content validity relies heavily on the opinions and judgments of people who are experts in the relevant profession or occupation. Expert judgment plays a integral part in developing the definition of the content domain to be tested. To the extent that the content domain of Boar examinations was established by the March 1981 amendments to Rule 21GG-6.01(3), the content domain was established on the basis of the expert judgment of persons who were experts in the field of electrical contracting, namely, the members of the Electrical Contractors' Licensing Board. The relative grading weights established in the 1981 amendments to Rule 21GG-6.01(3) have at least a possible correlation to electrical contractor competence. In fact, the persuasive expert testimony establishes that those relative grading weights did bear a reasonable relationship to electrical contractor competence. The current slightly different relative grading weights also bear a reasonable relationship to electrical contractor competence even though there has been no recent change in the nature of the responsibilities of electrical contractors. In this regard it must be remembered that testing is not an exact science and no test for minimum competency can ever be expected to be an exact model of actual work experience. The best that can be expected is a reasonable model. There are various accepted methodologies for the creation of standardized examinations, adherence to which tends to enhance the likelihood that the examination instrument finally produced will be valid and reliable. However, adherence to those accepted procedures is not a necessary prerequisite to creation of a valid and reliable examination. This is particularly true when one is going to be testing small groups of people with varied backgrounds instead of the massive groups of people with similar backgrounds for whom standardized tests are more typically designed. In the final analysis the only practical and reliable measure of the validity of an examination is by statistical analysis of the examination after it has been administered. Given the nature of the circumstances faced by the Board at the time of the March 1981 amendments to the examination format and cut score, given the nature of the pool of candidates to be examined, given the nature of the changes contemplated by the Board, and given the very nature of the process of testing for minimum competency--which involves perhaps as much art as it does science-- there is no study or data which would have been particularly useful to the Board in helping to determine exactly what the effect of their changes would be. Such effects can only be determined or measured with any degree of accuracy after the administration of an examination that incorporated the changes. Following the administration of such an examination, it is possible to perform a statistical analysis of all questions used on the examination and to eliminate or give credit for any questions which are shown by statistical analysis to be invalid or unreliable. This is precisely the process that is used by Block in the validation of their examinations and is an accepted testing procedure. It would have served no useful purpose to have conducted a trial run of an examination using licensed certified electrical contractors as a test group for the new examination format. First, it would be virtually impossible to try to put together an accurate cross-section of certified electrical contractors to use as a test group. Second, one would expect them all to pass the examination, so when they did so nothing of value would have been learned. Finally, the administration of such a trial run would risk the possibility of compromising examination question security. One aspect of accepted methodology for the preparation of standardized examinations is the definition of the content domain of the examination, i.e., a determination of what knowledge is essential to demonstrate that the candidates for licensure are minimally competent. This aspect of examination preparation is often accomplished by performing a formal job analysis, which is, in essence, a study of all of the usual tasks performed by a person engaged in the occupation or profession to be tested, including an evaluation of the relative importance of each of those tasks to minimum competence. The content domain can also be defined on the basis of the judgment of a group of experts in the occupation or profession to be tested. With the exception of the lay members who were added in recent years, all of the members of the Electrical Contractors' Licensing Board are, and have been, persons certified to engage in electrical contracting in the state of Florida and actively engaged in the electrical contracting business. Therefore, at all relevant times all of the professional members of the Board had extensive personal knowledge of what was involved in the practice of electrical contracting, which personal knowledge was as useful in defining content domain as would have been a formal job analysis. (In this regard it is important to note that even with the addition of lay members to the Board in recent years, the experienced professional members have continued to constitute a substantial majority of the Board.) Findings regarding Rule 21GG-6.0 of the Electrical Contractors' Licensing Board Rule 21GG-6.03 of the Electrical Contractors' Licensing Board reads as follows: An examinee is entitled to review his examination questions, answers, papers, grades and grading key used in the certification examination; however, no applicant may copy any materials provided for his review. Such review shall be conducted during regular business hours, in the presence of a representative of the Board at the Board's official headquarters. If, following the review of his exami- nation, an examinee believes that an error was made in the grading of his examination, or in the evaluation of his answers, he may request the Board to review his examination. Requests for Board review must be in writing, state with specificity the reason why review is requested, and be received within thirty (30) days after the examinee received notice that he failed the examination. (3) Upon receipt of a request for Board review, the examination shall be reviewed by the Board at the next regularly scheduled Board meeting. If it is found that an error was made, the grade received by the examinee may be adjusted to reflect the correction. The examinee shall be notified of the final decision. The Petitioners' only challenge to Rule 21GG-6.03 is predicated upon their concern that it may be misapplied to them by the Board. Findings regarding Rule 21-11.11(3) of the Department of Professional Regulation as it existed prior to the October 1982 amendments Prior to the October 1982 amendments, Rule 21-11.11(3) of the Department of Professional Regulation read as follows: (3) Examination grade reviews shall be conducted at a site designated by the Department. The candidate and his/her attorney or other qualified representative shall be allowed to attend the examination grade review. A candidate may attend only one review per examination administration. The above quoted version of the rule was in effect at the time of the Petitioners' reviews of their January and July 1982 examinations. Findings regarding Rule 21-11.11(3) of the Department of Professional Regulation, as amended in October of 198 Rule 21-11.11(3) of the Department of Professional Regulation, as amended in October of 1982, reads as follows, in pertinent part: A candidate taking a Department exam- nation may request and receive an appointment for review until such times as the exam- ination records are destroyed in accordance with Chapters 455, 119 and 267, Florida Statutes. However, unless otherwise provided by a rule of the appropriate Board within the Department, candidates may review their exam- ination for the purpose of filing objections to the examination for the Board's consideration under the following conditions and time frame: Within thirty (30) days of the date of the grade notification letter the candidate shall notify the Department of his/her desir to review the examination for the purpose of filing objections for consideration. Such review shall have been completed within the next thirty (30) days after the first thirty (30) days defined in (a) above. At the examination review, the candi- date shall be permitted to record on forms provided by the Department any and all objections to the examination the candidate desires the appropriate Board to review. Such forms shall remain in the custody of the Office of Examination Services for presen- tation to the appropriate Board at the next available Board meeting as an official agenda item. The candidate shall not copy" questions from the test booklet. The candidate may write on a separate paper in the presence of the Office of Examination Services employee, any objection or question he/she has to the written examination. The candidate shall leave the written objections and questions with the Office of Examination Services employee when he/she leaves the review room but she/she shall be permitted to leave with a form listing the question numbers he/she finds to be controversial. In the instance of a written exami- nation, all objections will be presented to the appropriate Board for consideration. Upon completion of the Board's review of written examination items the Department shall notify the reviewing candidate of the Board's decision. If the Board does not concur with the candidate's objections then the candidate will be notified in writing of this and the thirty (30) day appeal time per- iod shall begin to run from the date of this notice. Findings regarding Rule 21- 11.11(13) of the Department of Professional Regulation (later renumbered as Rule 21-11.11(3)(1)j Rule 21-11.11(13) of the Department of Professional Regulation, which was later renumbered as Rule 21-11.11(3)(1), reads as follows: If the consultant finds that the original grade was not rendered in accordance with the grading criteria, then he/she twill regrade that portion, or the entire examination, whichever is appropriate, pursuant to appli- cable statutes and rules. If it is not possible to regrade the examination, the candidate will be allowed to retake the examination at no charge. It is clear from the text of Rule 21-11.11(3)(j) that the language of Rule 21-11.11(3)(1) quoted immediately above is intended to apply only to practical examinations. It does not purport to apply to written examinations. These Petitioners took only written examinations, they have not taken any practical examinations. Findings regarding Rule 21-11.14 of the Department of Professional Regulation Rule 21-11.14 of the Department of Professional Regulation reads as follows, in pertinent part: Unless otherwise provided by a rule of the appropriate board within the Department, Security and Monitoring procedures shall be as follows: (10) Disposition of Test Booklets. Secured processing of Answer Sheets Test booklets for the state devel- oped examinations shall be disposed of and filed in accordance with the following procedure: 1. Ten copies of the examination book- lets shall be retained for ninety (90) days. This procedure shall insure that there are adequate copies of the booklets available for the board to review in their review of preliminary analysis of the examination and review sessions if review responsibilities are retained by the board. After the above defined ninety (90) day period the retention schedule shall change from ten copies to four copies. In the event any irregularity occurs during the examination with any state developed booklet, it shall be the examination supervisor's responsibility to prepare a detailed report of such irregularity and to retain the booklet in question in the secured files for a period of ninety (90) days. Destruction of examination booklets and related materials described in B.2.b., above: When in accordance with procedure, it is appropriate to destroy test booklets, it shall be the Archives's responsibility to schedule such destruction. In all instances there shall be evidenced in writing by the examination supervisor and a witness on a form which shall evidence the date of destruction and the official in charge of such destruction and a witness. When examination booklets are being prepared by the Department of Professional Regulation prior to an examination, the booklets are carefully inspected to make sure that all of the booklets are identical. Following that inspection the booklets are sealed and stored in a secure place in order to insure, among other things, that no changes are made to any of the examination booklets before they are handed out to the candidates. After an examination is given, the Department of Professional Regulation retrieves all of the examination booklets, including all booklets that were used by all of the candidates, and retains them in a secure place until the excess booklets can be destroyed. The examination supervisor selects the booklets which are to be retained from among the booklets that were not handed out to the candidates at the examination. In the normal course of events all of the examination booklets that were actually handled by the candidates at the examination are destroyed within a very few months of the date of the examination. With the exception of the examination booklet of one other candidate (which exception is not relevant to the disposition of this rule challenge proceeding), all of the examination booklets which were handed out to candidates during the examinations taken by these two Petitioners were destroyed approximately 90 days after each of the examinations. When such destruction took place, the Department retained copies of the examination booklets which had not been used by any candidate, which copies were identical to the copies that had been handed out to the candidates during the examination. As part of the examination instructions, all candidates for examination are advised not toe write anything in their examination booklets because all of the booklets used by the candidates will be shredded. They are specifically told to do all of their computations on sheets of work paper that are provided to them at the examination. All candidates are specifically told that the only things they turn in that will be saved are their answer sheets and their sheets of work paper. Findings regarding "unwritten" and "unpublished" rules of the Electrical Contractors' Licensing Board. The findings of fact immediately below relate to the eight "unwritten" and "unpublished" alleged rules of the Electrical Contractors' Licensing Board which are described in subparagraphs (F), (O), (P), (V), (X), (Y), (Z), and (FF) at pages 1 through 5 of the Petition. (The descriptions of these alleged rules are quoted at pages 2 through 6 of this Final Order.) One of these "unpublished" rules is alleged to be a rule to the effect that the Board requires more than minimum competency in order for a candidate to receive state certification and licensure as an electrical contractor. The other seven "unwritten" or "unpublished" rules of the Electrical Contractors' Licensing Board alleged in the Petition are all related in one way or another to the Board's alleged free form actions in the course of its review of the two examinations which have been challenged by these Petitioners in related proceedings under Section 120.57(1), Florida Statutes. None of the evidence regarding these matters proved the existence of an agency statement of general applicability purporting in and of itself to have the direct and consistent effect of law. For reasons which are explained in the Conclusions of Law portion of this Final Order, the evidence is insufficient to establish that the Electrical Contractors' Licensing Board has any "unwritten" or unpublished5? rules such as those described in subparagraphs (F), (O), (P), (V), (X), (Y), (Z), and (FF) of the Petition. 6/ Findings regarding "unwritten" and "unpublished" rules of the Department of Professional Regulation The findings of fact immediately below relate to the six "unwritten" and "unpublished" alleged rules of the Department of Professional Regulation which are described in subparagraphs (G), (J), (1), (L), (Q), and (R) at pages 1 through 5 of the Petition. The descriptions of these alleged rules are quoted at pages 2 through 6 of this Final Order.) These six "unwritten" or "unpublished" alleged rules of the Department of Professional Regulation are all related in one way or another to actions of functionaries of the Department in the course of performing the Department's role in the administration and review of the two examinations which have been challenged by these Petitioners in related proceedings under Section 120.57(1), Florida Statutes. None of the evidence regarding these matters proved the existence of an agency statement of general applicability purporting in and of itself to have the direct and consistent effect of law. For reasons which are explained in the Conclusions of Law portion of this Final Order, the evidence is insufficient to establish that the Department of Professional Regulation has any "unwritten" or "unpublished" rules such as those described in subparagraphs (G),(J), (I), (L), (Q), and (R).7 Findings regarding "unwritten" and "unpublished" rules of both the E1Electrical -Contractors' Licensing Board and the Department of Professional Regulation The findings of fact immediately below relate to the ten "unwritten" and "unpublished" alleged rules of both the Electrical Contractors' Licensing Board and the Department of Professional Regulation which are described in subparagraphs (M), (N), (S), (T), (U), (AA), (BB), (CC), (DD), and (EE) at pages 1 through 5 of the Petition. (The descriptions of these alleged rules are quoted at pages 2 through 6 of this Final Order.) These ten "unwritten" or "unpublished" alleged rules of both the Electrical Contractors' Licensing Board and the Department of Professional Regulation are all related in one way or another to actions allegedly taken by the Board and the Department in the performance of their respective functions related to the administration and review of the two examinations which have been challenged by these Petitioners in related proceedings under Section 120.57(1), Florida Statutes. None of the evidence regarding these matters proved the existence of an agency statement of general applicability purporting in and of itself to have the direct and consistent effect of law. For reasons which are explained in the Conclusions of Law portion of this Final Order, the evidence is insufficient to establish that both the Electrical Contractors' Licensing Board and the Department of Professional Regulation have any "unwritten" or "unpublished" rules such as those described in subparagraphs (M), (N), (S), (T), (U), (AA), (BB), (CC), (DD), and (EE) of the Petition.8
