The Issue The issue for consideration in this case is whether Petitioner should be awarded additional credit for his answer to question number 290, and thereby be given a passing grade on the Professional Engineer examination administered on October 25, 1996, in Orlando.
Findings Of Fact At all times pertinent to the issues herein, the Board of Professional Engineers (Board) was the state agency responsible for the examination and licensing of professional engineers in Florida. With the cooperation and assistance of the National Council of Examiners for Engineering and Surveying (NCEES), the Board conducts periodic examinations to test the qualifications of candidates for certification as professional engineers in this state. Such an examination was conducted in Orlando, Florida on October 25, 1996. Petitioner was a candidate at that examination. A minimum score for passing was 70. Petitioner received an overall score of 69. One of the questions posed to the candidates at that examination was question number 290, dealing with the design of a control system, which required the candidate to determine values for two parameters in such a fashion that the closed loop specifications stated as, "with K =20 the unit step response be a damped oscillation with a 10% overshoot and with a damped natural frequency of 15 rad/s" were met. In the answer to this question, the engineer has to arrive at parameters to give the desired step response within the stated percentages. According to Dr. Antonio Arroyo, an assistant professor of electrical and computer engineering at the University of Florida and an expert in electrical engineering, this subject matter is taught in a standard undergraduate controls course which is required in engineering schools nation-wide. The question in issue is a classic controls problem. The candidate is to reduce the diagram displayed in the examination question and give a closed loop description. Given that, the solution proceeds by taking the percentage of error and using it to back- track and arrive at the requested parameters, step by step. The examination is an open book examination. Because of the many formulae used in engineering, the candidate is permitted to use printed resources to assist in the solving of the problems. This formula involved in this problem is standard. Only the parameters cited in the test problem are different. In his answer to the question Petitioner cited to the page in his reference material where the solution is to be found, and he used the appropriate formula. In doing so, he could take the numbers presented in the problem and apply them to the standard problem solution contained in the reference book he had with him. It is a "plug and chug" situation wherein the candidate inserts the problem numbers into the given formula and makes the calculations. In order to take advantage of this opportunity, however, the candidate must decide how to use the information given. In this case, the problem involved a damped frequency of 15 hz and the candidate was required to calculate an undamped frequency. The Petitioner did not show that calculation in his solution, and it appears to Dr. Arroyo he missed the fact of the difference between the two frequencies. In Petitioner's solution, he listed what he saw as the data given, and though at no place did the problem show "Omega d", Petitioner put down "Omega d" but used "Omega n". In the expert opinion of Dr. Arroyo, an engineer should, at least, check his calculations. Examiners will give credit to a candidate if the candidate shows the appropriate knowledge of the concepts involved in the problem. In the instant case, Petitioner's answer to question 290 far exceeded the allowable 10% overshoot. His answer for "a" was 0.895, whereas the correct answer was 1.099. Whereas the allowable ten percent difference was .110, Petitioner’s overshoot was .204. His answer for "b" was 11.25, whereas the correct answer was 17.3. Whereas the allowable ten percent difference here was 1.73, Petitioner’s overshoot was 6.05. To Dr. Arroyo, this shows a concept error rather than a calculation error In substance, Petitioner utilized the correct formulae, but used incorrect data, and the use of the wrong data is sufficient to indicate his ignorance of the appropriate concepts. Petitioner’s expert, Dr. Garrett, who did not see the problem utilized in the examination and relied on information provided by Petitioner, concluded that Petitioner’s margin of error was within the 10% limitation. Here, notwithstanding the opinion to the contrary of Dr. Garrett, Petitioner's solution missed the authorized overshoot by a significant amount, far more than the allowable 10%. He should have known something was wrong when this happened and should have looked to see what he did wrong. In the opinion of Dr. Arroyo, the Petitioner did not adequately evaluate the problem consistent with acceptable engineering standards since the final product of his calculations did not meet the specifications of the problem. This is the purpose behind the professional certification process, and Petitioner should have recognized that his answer did not meet the required specifications. Petitioner received a score of six out of a possible ten for his solution to question 290. Dr. Arroyo is satisfied that the scoring plan of the NCEES for this problem is fair and he supports it. Petitioner’s expert, Dr. Garrett, a professional engineer and long-time professor of electrical engineering at the University of South Florida, disagrees. In his evaluation of the problem and the grading process used here, Dr. Garrett notes that problem 290 consisted of five parts, for each of which two points could be awarded. Petitioner correctly answered the first three parts and received a grade of six points. He missed part four, and part five was to use the results of parts three and four, with the proper equations, to determine the two answers required. Since Petitioner used the proper equations to figure his answer to part five, even though he did not get a correct answer to part four, which resulted in his numeric answer to part five being incorrect, Dr. Garrett is of the opinion that he should have received an additional two points for applying the proper formula in part five. Review of the scoring plan developed for this problem indicates that Petitioner met all the qualifications for award of six points, but he did not recognize the relationship of damped as opposed to undamped. He used incorrect data to arrive at "a" and "b" in that he did not identify the relationship between natural frequency and damped frequency. This is a basic problem of control systems which an undergraduate should be able to solve correctly. It is basic electrical engineering knowledge and not beyond that expected of an electrical engineer with a bachelor's degree in the field. Had Petitioner utilized the formula he used with the proper data, he would have been awarded credit for a correct answer even if his calculations were incorrect. Here, however, while Petitioner utilized the correct formula, he applied it to incorrect data, and it is this use of incorrect data which makes an award of a higher score inappropriate. The professional engineers’ examination is designed to test the individual's familiarity with engineering concepts and his ability to cast the problem into those concepts to solve the problem. Petitioner contends that his understanding of the concepts involved was correct and, therefore, even though he used the wrong figures, he should received credit for a correct answer or, at most, only 2 rather than 4 points should have been deducted. Though Petitioner utilized the correct formula for his solution to question 290, he applied the wrong values in the use of the formula. This indicates a lack of understanding of the concepts involved, and even though Petitioner used the proper formula, that formula came from the book he was permitted to use for the examination. He cannot be given full credit for copying the formula from the book. Had he used the correct values in his solution to the problem, he would have been given appropriate credit even if his calculations were wrong. After being notified of his unsuccessful exam results, Petitioner requested that his answer to question number 290 be resubmitted to NCEES for re-scoring, and this was done. By memorandum in response, dated July 10, 1997, the NCEES scorer concluded: The error in using undamped natural frequency for damped natural frequency in the examinee's solution is a major error. Whether the examinee did not recognize the function was in fact the undamped natural frequency, as given in the problem statement, or whether it was an oversight, it is still a major error since the outcome is significantly affected. The scorer, whose knowledge of the identity of the candidate was limited to a number only, recommended a score of "six" for Petitioner answer to this problem. There was no change from the initial scoring.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is recommended that the Board of Professional Engineers enter a Final Order denying Petitioner additional credit for his answer to question number 290 on the principles and practice portion of the electrical engineering examination administered for the Board of Professional Engineers on October 25 and 26, 1996. DONE AND ENTERED this 17th day of December, 1997, in Tallahassee, Leon County, Florida. ARNOLD H. POLLOCK Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (904) 488-9675 SUNCOM 278-9675 Fax Filing (904) 921-6947 Filed with the Clerk of the Division of Administrative Hearings this 17th day of December, 1997. COPIES FURNISHED: Bahman Behzadi Post Office Box 290931 Tampa, Florida 33687 R. Beth Atchison, Esquire Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0750 Lynda L. Goodgame General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0700 Angel Gonzalez Executive Director Board of Professional Engineers 1940 North Monroe Street Tallahassee, Florida 32399-0792
Findings Of Fact Petitioner herein, Kristina V. Tignor, took the Professional Engineers Examination for the State of Florida in Orlando on April 9 and 10, 1987. On July 22, 1987 she was advised by the Department of Professional Regulation's Office of Examination Services that she had failed the examination and was given a cummulative score of principles and practice of 69.1 percent. In her initial request for review and reconsideration, Petitioner objected to the points assigned to her solutions for three problems on the test, Numbers 425, 421, and 124. She contended that as a working engineer, certain criteria and assumptions must be made in approaching any engineering problem and, because the portion of the examination in issue is graded subjectively, her answered should be reconsidered and evaluated in that light. At the hearing, Petitioner contested only the grading of questions number 124 and 421, thereby accepting the grade given for question 425. With regard to Question 124, Ms. Tignor was awarded a score of 5 on her solution to this problem. The National Council of Engineering Examiners, in its Standard Scoring Plan Outline awards a "qualified" evaluation to scores from 10 down to 6 on this question. Scores from 5 to 0 are rated, "unqualified." A score of 5 indicates the applicant has failed to demonstrate adequate knowledge in one aspect of one category. Specifically, a rating of 5 in this question indicates that the examinee displayed an inadequate knowledge of weight/volume and concrete mix design. Her computations were displayed and an incomplete or erroneous solution was arrived at which gave a generally unrealistic result. Dr. Bruce A. Suprenant a civil engineer registered in four states and who teaches engineering at the University of South Florida, reviewed the question, the Petitioner's solution, the solution proposed by the examiners, and the grading scheme for this problem and found a number of illogical items in Petitioner's solution which, to him, were difficult to understand. He found several items which had no basis and which were possibly assumed. As to Part a of Petitioner's answer, a mixture of answers, (correction for moisture), which should have been in Part b, was located in Part a. As to density, the value used by Petitioner does not appear to be reasonable based on information provided in the problem. In Dr. Suprenant's opinion, there are at least three approaches to this problem. One is the water/cement ration method. Another is the weight method. The third is the absolute volume method. The water/cement ratio method would be difficult to apply here and neither Petitioner nor the examiners used it. As to the weight method, much the same problem exists. There is insufficient information provided to satisfactorily apply this method and while the examiners did not use it, Petitioner did. Petitioner's answer has a correction for moisture in the absolute volume method on the first page of the solution form at the top. The calculations by Petitioner are assumed information not known, (volume). In addition the correction for moisture in the second part of page one is included on the top of page two. It is not a part of the solution for subpart a and should not be there. Petitioner used 150 pounds per cubic foot for concrete density in her solution and this choice is not explained. Most publications utilized by engineers suggest using tables which were not provided to the examinees and it is, therefore, illogical to assume concrete density with no history for that assumption. Petitioner's answer of 5.41 cubic yards is only slightly off the suggested answer of 5.44 cubic yards but the fact that the answers are close does not justify her assumption. It might well not come so close in other cases. As to Part b of the question calling for the water/cement ratio, the corrections for moisture of fine and coarse aggregate on page one are acceptable. On the second page, a problem arises in when the correction for moisture should decrease. Petitioner got the right factor but applied it in the wrong manner. As a result, her answer to Part b of the examination question is wrong. Her answer was 4.40 as opposed to the correct answer of 4.34. This small degree of error can be attributed to the smallness of the amount in question. Were the amounts greater, the error would be greater. As to part c of the question, which deals with the cement factor in a yard of concrete, Petitioner's approach of dividing sacks of cubic yards is correct, but the cubic yard content was determined from Part a of the question, and Dr. Suprenant does not agree with how she got her solution. He therefore questions her carryover. The standard weight of a sack of concrete is 94 pounds. The individual grading Petitioner's response to Question 124 indicates she displayed inadequate knowledge and reached a solution which gives "unrealistic results." Dr. Suprenant agrees, contending that Petitioner's performance in regard to this question indicates inadequate knowledge of weight/volume relationship. She made inadequate assumptions in formulating her answer to the question. The fact that in this problem she arrived at a solution close to the correct one does not indicate that in other problems, she would achieve the same closeness using the same procedure. In his opinion, Petitioner showed some confusion regarding the basis for solving this problem and Dr. Suprenant believes that a grade of 5 as awarded by the examiner is correct. Petitioner questioned the fact that the various technical weights and volumes, such as 94 pounds in a sack of concrete, 8.33 pounds for a gallon of water, and 27 cubic feet in a cubic yard do not appear in the problem statement. This, in the opinion of Dr. Suprenant, compounds the gravity of Petitioner's deficiency. They are routine "givens" generally accepted in the practice by engineers and it would be difficult to assume that anyone familiar with the practice of engineering would use different "givens" for these specifics. Petitioner's employer, Mr. Bishop, himself a registered civil engineer in Florida since 1958, also reviewed Petitioner's solution to Question 124. He admits that on the first page of the answer sheet, Petitioner began solving the problem in an inappropriate way. Her calculations for moisture content were correct, however. On the second paged the correction factor was put in with the wrong sign and the aggregate was given the wrong factor. As a result, the answer was off. In his practice, however, the error committed by Petitioner in these regards is both minimal and acceptable. Her choice of 150 pounds per square foot is reasonable and produced a close result, and while it is true that if the project were of a greater scale, the error might be significant for a test question, as here, the error, in his opinion, is insignificant. He feels much the same way regarding the error in Part c of the examination question. While the factors used by petitioner were wrong, the process used was correct and the answer was not unreasonably incorrect for a test solution. In an examination situation, the calculations are not being done on a continuous basis, and he feels the grade of 5 awarded is unduly harsh since the error was numerical rather than operational. In his opinion, a more reasonable grade would have been a 6 or 7. Petitioner began her solution to this problem by using one similar to that used by the examiners in their publications. Shortly, however, she realized she would not get the answer she needed by doing so and abandoned her solution. She forgot to cross it out, however, and now recognizes she should have done so. She thereafter began to accomplish a series of new calculations on the first page of the answer sheet but did not necessarily utilize that data for her solution to Part a. She admits she made an error in calculation for moisture on the second page. In that calculation, she used the study manual and admits now that she should have cited the figure she used. As to Parts b and c, her use of some figures from Part a may have thrown her answer off somewhat. However, the 5 awarded her, indicating her solution was unrealistic, is, in her opinion unfair as she considers her answer to be quite realistic. The problem did not state what solution method to use and she feels her use of givens from recognized manuals such as the 150 pounds, should not be held against her. 94 pounds for a sack of cement used by the grader was also not given and her use of other accepted numbers should not, she contends, be held against her. Petitioner believes a grade of 7 would more accurately describe the quality of her answer. A 7 means that the examinee obtained an appropriate solution but chose a less than optimum approach. The solution is, therefore, awkward but nonetheless resonable. Ms. Tignor believes that while her approach may have been awkward, she achieved reasonable solution demonstrated by the fact that it was only slightly off the correct figure. Therefore, she believes a grade of 6 would be appropriate. This examination was an open book examination and Petitioner had her manuals with her. She could have easily determined the appropriate weights an "givens" from these manuals without choosing those she used. Ms. Tignor's conclusions that her results are realistic are contradicted by the Board's expert. Realistic results are, in engineering practice, not only the figure reached but also the method used in arriving at that figure. Here, though Petitioner's results are close, the approach utilized in arriving at her solution is unrealistic. Her approach showed an inadequate knowledge of weight/volume and calculations. Consequently it is found the grade is valid and was not arbitrarily assigned. According to the Standard Scoring Plan Outline, each score from 10 through 6 has an indispensable criteria that all categories must be satisfied. Since Ms. Tignor's examination response did not satisfy all categories, the best she can be given is a 5 and that award appears to be justified by the evidence presented. Question 421 was a four part drainage problem. Petitioner used as a part of her solution calculations based on a 100 year storm and this was determined by the examiners to be inappropriate. Ms. Tignor was awarded a grade of 8 and contends she was not given appropriate credit. She relates that even Mr. Smith, the Executive Director of the Board of Professional Engineers, advised her she may not have been given full credit for her answer. She was given full credit for Part a but lost two points for part c which included a calculation error to which Petitioner admits. She contends however, it was so minor, only one point should have been deducted. Were Petitioner to receive an additional one point on this question, she would pass the examination which she failed by only one point. However, this issue must be resolved on the basis of lawfully admitted evidence and Mr. Smith's comment, being unsupported hearsay evidence, cannot itself sustain the rasing of the grade. The Standard Scoring Plan Outline for this question reflects that to receive an 8, the examinee must demonstrate that all categories are satisfied, that errors are attributable to misread tables or calculating devices, and that errors would be corrected by routine checking. The results must be reasonable if not correct. For a 9, the examinee must demonstrate that all categories are satisfied; that a correct solution is arrived at but the examinee has been excessively conservative in the choice of working values; and that examinee's presentation is lacking in completeness or equations diagrams or orderly steps in solution, etc. Subqualifications for a 9 indicates that the answer is correct but that the organization of the solution is not logical. One error in calculation in any of the Parts from a to d, which does not affect the other parts of the solution, is acceptable. Mr. Kenneth Weldon, the Assistant State Drainage Engineer for the Department of Transportation, an expert in the area of drainage to which this problem relates, reviewed the question and the Petitioner's answer thereto and would award a grade of 8 to her answer. He found various numerical mathematical errors which led to the wrong solution. In addition, Petitioner made various assumptions that, though supposedly supported, were, he felt, in error through her misinterpretation. In general, none of the actual solutions she arrived at were correct. Specifically, that portion of the problem to determine the cross sectional area of the waterway for establishing normal depth flow was done incorrectly. Because the Petitioner used incorrect equations throughout the problem, the depth flow computed is high. Petitioner did no analysis to determine whether or not any of the several situations relating to flow control were pertinent. Mr. Weldon initially felt Petitioner's answer to the question merited a grade of 6. This means that the examinee knew all the proper steps but failed to interpret some of the criteria properly. He could not award her a grade of 9 which would indicate all categories were satisfied and the solution was correct, if conservative. Petitioner's solutions were incorrect. He subsequently changed his award to an 8, however, on the basis that the Petitioner's errors were attributable to a misread table or calculating device and would be corrected by routine checking. The result was reasonable, though not correct. Mr. Weldon did not like this question even though he believed it appropriate for a one-hour exam. As written, it involves establishing and making judgements beyond what someone minimally competent would be expected to do. It requires materials that are beyond what are normally available to someone taking the exam. However, Petitioner failed to make proper provision to protect herself in a case where the question is inappropriate or incomplete. If she felt something was wrong with the question, she should have clearly stated the assumption she was making to solve the problem. This was her responsibility and she failed to do so. In Mr. Weldon's opinion, Petitioner's answer might merit a grade slightly higher but not significantly higher. His reasoning is that Petitioner misinterpreted the criteria she stated for writing the problem. Her comment that the Department of Transportation uses 100 year storm criteria was incorrect even though that statement is made in outdated Department of Transportation publications. The basis for her answer is not well established or correct, or based on engineering calculations or judgement, and at best he could award no more than an 8.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is, therefore: RECOMMENDED that a Final Order be entered affirming the score awarded to Petitioner on questions 124 and 421, respectively, of the Civil Engineering Examination administered to her in April, 1987. RECOMMENDED this 10th day of June, 1988, at Tallahassee, Florida. ARNOLD H. POLLOCK, Hearing Officer Division of Administrative Hearings The Oakland Building 2009 Apalachee Parkway Tallahassee, Florida 32399-1550 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 10th day of June, 1988. APPENDIX TO RECOMMENDED ORDER, CASE NO. 87-5110 The following constitutes my specific rulings pursuant to Section 120.59(2), Florida Statutes, on all of the Proposed Findings of Fact submitted by the parties to this case. For the Petitioner None For the Respondent Accepted and incorporated herein. Accepted and incorporated herein. Accepted and incorporated except for the characterization of several assumptions as guesses. No evidence exists to support such a characterization even though they are incorrect. Accepted and incorporated herein. Accepted and incorporated herein. Accepted and incorporated herein. Accepted and incorporated herein. Accepted and incorporated herein. COPIES FURNISHED: Kristina V. Tignor, pro se 2160 North Oval Drive Sarasota, Florida 34239 H. Reynolds Sampson, Esquire Department of Professional Regulation 130 North Monroe Street Tallahassee, Florida 32399-0750 Allen R. Smith, Jr. Executive Director DPR, Board of Professional Engineers 130 North Monroe Street Tallahassee, Florida 32399-0750
The Issue The ultimate issue in this proceeding is whether proposed Florida Administrative Code Rule 61G15-21 is an invalid exercise of delegated legislative authority.
Findings Of Fact Florida Administrative Code Rule 61G15-21.004, in relevant part, states: The criteria for determining the minimum score necessary for passing the Engineering Fundamentals Examination shall be developed through the collective judgment of qualified experts appointed by NCEES to set the raw score that represents the minimum amount of knowledge necessary to pass the examination. The judges shall use a Modified Angoff Method in determining the minimally acceptable raw score necessary to pass the Fundamentals of Engineering Examination. Using the above mentioned Modified Angoff Method, the judges will indicate the probability that a minimally knowledgeable Fundamentals of Engineering examinee would answer any specific questions correctly. The probability of a correct response is then assigned to each question. Each judge will then make an estimate of the percentage of minimally knowledgeable examinees who would know the answer to each question. The totals each of the judges is added together and divided by the number of judges to determine the overall estimate of the minimum standards necessary. The minimum number of correct answers required to achieve a passing score will take into account the relative difficulty of each examination through scaling and equating each examination to the base examination. The raw score necessary to show competence shall be deemed to be a 70 on a scale of 100. A passing grade on Part Two of the examination is defined as a grade of 70 or better. The grades are determined by a group of knowledgeable professional engineers, who are familiar with engineering practice and with what is required for an applicable engineering practice and with what is required for an applicable engineering task. These professional engineers will establish a minimum passing score on each individual test item (i.e., examination problem). An Item Specific Scoring Plan (ISSP) will be prepared for each examination item based upon the NCEES standard scoring plan outline form. An ISSP will be developed by persons who are familiar with each discipline including the item author, the item scorer, and other NCEES experts. On a scale of 0-10, six (6) will be a minimum passing standard and scores between six (6) and ten (10) will be considered to be passing scores for each examination item. A score of five (5) or lower will be considered an unsatisfactory score for that item and examinee will be considered to have failed that item. To pass, an examinee must average six (6) or greater on his/her choice of eight (8) exam items, that is, the raw score must be forty- eight (48) or greater based on a scale of eighty (80). This raw score is then converted to a base 100 on which, as is noted above, a passing grade will be seventy (70). The proposed changes to Florida Administrative Code Rule 61G15-21.004, in relevant part, state: The passing grade for the Engineering Fundamentals Examination is 70 or better. The criteria for determining the minimum score necessary for passing the Engineering Fundamentals Examination shall be developed through the collective judgment of qualified experts appointed by NCEES to set the raw score that represents the minimum amount of knowledge necessary to pass the examination. The judges shall use a Modified Angoff Method in determining the minimally acceptable raw score necessary to pass the Fundamentals of Engineering Examination. Using the above mentioned Modified Angoff Method, the judges will indicate the probability that a minimally knowledgeable Fundamentals of Engineering examinee would answer any specific questions correctly. The probability of a correct response is then assigned to each question. Each judge will then make an estimate of the percentage of minimally knowledgeable examinees who would know the answer to each question. The totals each of the judges is added together and divided by the number of judges to determine the overall estimate of the minimum standards necessary. The minimum number of correct answers required to achieve a passing score will take into account the relative difficulty of each examination through scaling and equating each examination to the base examination. The raw score necessary to show competence shall be deemed to be a 70 on a scale of 100. The passing grade for the Principles and Practice Examination is 70 or better. A passing grade on Part Two of the examination is defined as a grade of 70 or better. The grades are determined by a group of knowledgeable professional engineers, who are familiar with engineering practice and with what is required for an applicable engineering practice and with what is required for an applicable engineering task. These professional engineers will establish a minimum passing score on each individual test item (i.e., examination problem). An Item Specific Scoring Plan (ISSP) will be prepared for each examination item based upon the NCEES standard scoring plan outline form. An ISSP will be developed by persons who are familiar with each discipline including the item author, the item scorer, and other NCEES experts. On a scale of 0-10, six (6) will be a minimum passing standard and scores between six (6) and ten (10) will be considered to be passing scores for each examination item. A score of five (5) or lower will be considered an unsatisfactory score for that item and examinee will be considered to have failed that item. To pass, an examinee must average six (6) or greater on his/her choice of eight (8) exam items, that is, the raw score must be forty- eight (48) or greater based on a scale of eighty (80). This raw score is then converted to a base 100 on which, as is noted above, a passing grade will be seventy (70). Petitioner resides in Tampa, Florida. On April 11, 2003, Petitioner took a national examination that Petitioner must pass to be licensed by the state as a professional engineer. On July 1, 2003, Petitioner received a letter from the Board advising Petitioner that he had received a failing grade on the examination. On July 2, 2003, Petitioner unsuccessfully requested the raw scores on his examination from a representative of the National Council of Examiners for Engineering and Surveying (NCEES). The NCEES is the national testing entity that conducts examinations and determines scores for the professional engineer examination required by the state. On July 9, 2003, Petitioner submitted a formal request to the Board for all of the raw scores related to Petitioner "and all past P.E. Exams that the Petitioner had taken." A representative of the Board denied Petitioner's request explaining that the raw scores are kept by the NCEES and "it is not their policy to release them." The Board's representative stated that the Board was in the process of adopting new rules "that were in-line with the policies of the NCEES." On July 31, 2003, Petitioner requested the Board to provide Petitioner with any statute or rule that authorized the Board to deny Petitioner's request for raw scores pursuant to Section 119.07(1)(a), Florida Statutes (2003). On the same day, counsel for the Board explained to Petitioner that the Board is not denying the request. The Board is unable to comply with the request because the Board does not have physical possession of the raw scores. Petitioner and counsel for Respondent engaged in subsequent discussions that are not material to this proceeding. On August 6, 2003, Petitioner requested counsel for Respondent to provide Petitioner with copies of the proposed rule changes that the Board intended to consider on August 8, 2003. On August 27, 2003, Petitioner filed a petition with the Board challenging existing Florida Administrative Code Rule 61G15-21.004. The petition alleged that parts of the existing rule are invalid. Petitioner did not file a challenge to the existing rule with DOAH. The Petition for Hearing states that Petitioner is filing the Petition for Hearing pursuant to Subsections 120.56(1) and (3)(b), Florida Statutes (2003). However, the statement of how Petitioner's substantial interests are affected is limited to the proposed changes to the existing rule. During the hearing conducted on January 29, 2004, Petitioner explained that he does not assert that the existing rule is invalid. Rather, Petitioner argues that the Board deviates from the existing rule by not providing examinees with copies of their raw scores and by failing to use raw scores in the determination of whether an applicant achieved a passing grade on the exam. Petitioner further argues that the existing rule benefits Petitioner by purportedly requiring the Board to use raw scores in the determination of passing grades. The elimination of that requirement in the proposed rule arguably will adversely affect Petitioner's substantial interests. The Petition for Hearing requests several forms of relief. The Petition for Hearing seeks an order granting Petitioner access to raw scores, a determination that Petitioner has met the minimum standards required under the existing rule, and an order that the Board grant a license to Petitioner. The Petition for Hearing does not request an order determining that the proposed rule changes constitute an invalid exercise of delegated legislative authority.
The Issue Whether the Petitioner, Jenny Lancett (Petitioner) is entitled to additional credit for the answers she provided to challenged examination questions. The Petitioner has challenged the State Officers Certification Examination (SOCE) for Law Enforcement Officers given in July of 2005 (DOAH Case No. 05-4544) and September of 2005 (DOAH Case No. 06-0325).
Findings Of Fact The Petitioner is an applicant for certification as a law enforcement officer. In order to be eligible to sit for the state examination, the Petitioner successfully completed an academy course that was sponsored by the Palm Beach County Sheriff’s Office. To become fully certified, the Petitioner must take and achieve a passing score on the SOCE. The Petitioner took the SOCE on July 27, 2005, but did not pass. She also took the SOCE on September 21, 2005, and did not pass. The Respondent is the state agency charged with the responsibility of administering examinations for certification for the SOCE. After the July and September examinations for the SOCE were administered and the Petitioner received failing scores, she requested and attended examination review sessions conducted at the Respondent’s headquarters in Tallahassee, Florida. The review sessions were held at the Florida Department of Law Enforcement and the Petitioner was given copies of the examination questions for which she did not provide the correct answers. The Petitioner was not told the correct answers. As to the July examination, the Petitioner timely challenged Questions 45, 56, 90, 141, 151, 156, 161, 163, 207, 227, 234, 238, and 242. That examination challenge was forwarded to the Division of Administrative Hearings on December 14, 2005, and was designated as DOAH Case No. 05- 4544. The Petitioner withdrew her challenge to Question 238. As to the September examination, the Petitioner timely challenged Questions 44, 63, 134, 160, 162, 165, 166, 178, 189, 194, 195, and 208. That examination challenge was forwarded to the Division of Administrative Hearings on January 26, 2006, and was assigned DOAH Case No. 06-0325. On February 3, 2006, in response to the Joint Motion to Consolidate, the cases were consolidated for formal hearing and rescheduled for hearing to March 21-22, 2006. The case was heard on April 5, 2006. As to each challenged examination, the applicant must answer 250 questions and achieve at least 180 correct answers. Twenty-five questions of the 250 do not count but are considered “throw-away” questions. All questions are posed in English. All questions are multiple choice and an applicant is given credit for only the correct answer. The Respondent deems the correct answer to be the best choice from among the options offered. The Petitioner is an Hispanic female who reads, writes, and speaks English. Although she inquired about accommodations, the Petitioner was not afforded any accommodations while taking the challenged examinations based upon English as her non-native language. The Commission requires that all applicants take the examination in English. Similarly, accommodations are not afforded applicants who maintain “test anxiety” as a basis for concern. All questions and answers for the challenged examinations are considered confidential as a matter of law. As to each of the questions challenged by the Petitioner, the Petitioner failed to select the correct and best option from the multiple-choice selections noted. As to each of the questions challenged by the Petitioner, the wording and options noted are clearly stated and are within the curriculum covered by the academy. Of the numerous Hispanic candidates who have successfully completed the Palm Beach County academy during Mr. Kozyra’s tenure, only two have failed to achieve a passing score on the SOCE within three attempts. The Petitioner submitted no credible evidence to support her claim that the answers she provided on the challenged examination questions were correct or that the questions in their wording or grading were flawed. The persuasive weight of the evidence was to the contrary. A question which asks the applicant to provide a response that is “most accurately” describing the situation (as did Question 56 on the July examination) means that of the choices offered only one can be considered “most accurate.” If the applicant chooses an answer that is not the “most accurate,” credit is not given. Many of the Petitioner’s responses fell into this type of erroneous response. The Petitioner simply failed to provide the “most accurate” from the selections offered. As to each of the selections chosen by the Petitioner, a clear majority of the applicants taking the examinations selected the correct option. None of the examinations questions challenged by the Petitioner were incorrectly answered by a majority of the applicants. In some instances as many as 94 percent of the test takers chose the correct answer whereas the Petitioner did not. The Respondent provided sufficient explanation and the record clearly establishes that as to each of the erroneous answers provided by this Petitioner, the correct answer (as scored by the Respondent) was the best or correct answer. The Petitioner cannot be entitled to additional credit when her answers were not correct.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that the Petitioner’s challenges to the July and September 2005 certification examinations be denied. S DONE AND ENTERED this 28th day of June, 2006, in Tallahassee, Leon County, Florida. J. D. PARRISH Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 28th day of June, 2006. COPIES FURNISHED: Michael Crews, Program Director Division of Criminal Justice Professionalism Services Department of Law Enforcement Post Office Box 1489 Tallahassee, Florida 32302 Michael Ramage, General Counsel Department of Law Enforcement Post Office Box 1489 Tallahassee, Florida 32302 Bruce A. Minnick, Esquire Minnick Law Firm Post Office Box 15588 Tallahassee, Florida 32317-5588 Linton B. Eason, Esquire Department of Law Enforcement Post Office Box 1489 Tallahassee, Florida 32302-1489
Findings Of Fact On the October 1992 examination for licensure as a Professional Engineer Petitioner received an overall grade of 69.1 on Principles and Practice. An overall grade of 70 is required to pass the examination. The examination for Professional Engineer is a national examination prepared and graded by the National Council of Engineering Examiners (NCEE). On question 124 Petitioner calculated an angle to be 25.717 degrees correctly but when this angle was used to solve a latter part of the problem it was transposed as 21.717 degrees. The calculation performed with the transposed number was correct but because of the use of the wrong number of degrees the final answer was incorrect. On question 124 Petitioner received a score of 8 out of a possible 10. Petitioner contends he should have received a score of 10. The Final Scoring Plan for problem 124 provides that a score of 10 demonstrates applicant is EXCEPTIONALLY COMPETENT (It is not necessary that the solution be perfect.) Correct approach to horizontal curve geometry and coordinate computations. All answers to the nearest plus or minus 0.01 foot. To receive a grade of 8 the scoring plan provides: MORE THAN MINIMUM BUT LESS THAN EXCEPTIONAL COMPETENCE. Generally correct approach to the problem solution, but a solution with one math error or one error in logic or a solution with answers outside of the plus or minus 0.01 foot range or correct solution to parts (b) and (c) only. It was in part (c) of the problem that Petitioner transposed the wrong angle. All of the answers produced by the approximately 1000 applicants who took question 124 were graded by one grader. When Petitioner requested this grade be reviewed it was sent back to NCEE where the Petitioner's answer again received a grade of 8. The grading system for all of the problems on the NCEE examinations are on a scale of 0-10 at two-point intervals. There are no odd numbered scores given on any question. Problem 120 which Petitioner also challenges involved calculating the cost of fill material received from two separate sources with different distances to haul, different prices per cubic yard, and with the fill having different void ratios. Petitioner's calculations were accurate except that in his calculations Petitioner added two figures together rather than subtracting one from the other as he should have done. This was done twice in solving this problem causing an error of nearly twice the cost differential analysis (from $298,000 to $580,000). Petitioner's expert witness opined that the maximum deduction of two points in problem 124 was excessive; however, he concurred that the nationwide examination prepared and graded by NCEE is the best solution to qualifying Professional Engineers.
Recommendation Based on the foregoing, it is, hereby, RECOMMENDED: That a final order be entered dismissing the challenge by Martin Marquez to the final grade he was given on the October 1992 examination for licensure as a professional engineer. DONE AND RECOMMENDED this 4th day of January, 1994, in Tallahassee, Leon County, Florida. K. N. AYERS Hearing Officer Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-1550 (904)488-9675 Filed with the Clerk of the Division of Administrative Hearings this 4th day of January, 1994. COPIES FURNISHED: Vytas J. Urba Assistant General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792 Martin Marquez 5412 Walstone Court Tampa, Florida 33624 Jack McRay Acting General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792 Angel Gonzalez Executive Director Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792
Findings Of Fact Donald A. Moyant, Petitioner, operates a contractors' exam school in Brandon, Florida, at which he preps builders to take the CILB exam for licensure as contractors. Petitioner took the exam given in June 1985 and received a score of 58.60. A score of 70.00 is required to pass this examination. The only answer that Petitioner questions is the answer he supplied to question No. 2, which related to the number of reinforcing bars required in a concrete slab of given dimensions and spacings of the bars. In Petitioner's solution to the problem he dropped the fractional bar that resulted from dividing the length of the slab by the spacing between bars. The correct solution requires the addition of a bar for this fractional spare. Petitioner's purpose in pursuing this proceeding is to obtain information to use in his Contractors Exam School, Inc. Since he would not have passed the examination even if his solution to question No. 2 had been marked correct, Petitioner is without standing to bring this action because he is doing so on behalf of the school he runs and not to allow him to receive a passing grade.
The Issue The issue in this proceeding is whether Michael Reggia meets the Florida licensure requirements for a professional engineer in the field of manufacturing engineering. The issue is specifically whether the practice and principles portion of the licensing exam was valid. Procedural Matters At the final hearing, Petitioner, Michael Reggia testified in his own behalf and presented the testimony of manufacturing engineer, Howard Bender. Petitioner's exhibits #1 and #2, letters from Martin Marietta Aerospace and Harris Corporation, were rejected as hearsay. Exhibit #3, selected pages from Fundamentals of Engineering, published by the National Council of Engineering Examiners, was admitted without objection. Respondent presented two witnesses: Cass Hurc, P.E. (by deposition, by agreement of the parties) and Allen Rex Smith, Executive Director of the Board of Professional Engineers. Respondent initially submitted four exhibits: #1 and #4 were admitted without objection, #2(a) and #2(b), were admitted over Petitioner's objection, and #3 was withdrawn. The parties requested and were given 20 days to submit post-hearing briefs and proposed orders. On September 15, 1986, Petitioner filed his arguments and summary of she testimony and evidence. Nothing was filed by Respondent.
Findings Of Fact Michael Reggia resides in Titusville and works at the Kennedy Space Center. He is licensed in the state of California as a professional engineer and has practiced in the field of manufacturing engineering. California, like Florida, does not license an individual in a particular discipline of engineering but requires that an individual select an area in which he or she will be tested. Mr. Reggia took the professional engineering license exam in Florida in October 1985. For part two of the examination, Professional Practice and Principles, he chose to be tested in his field of manufacturing engineering. He achieved a score of 64.4; in order to pass, a score of 70 is required. The examination given in Florida is a national examination produced by the National Council of Engineering Examiners (NCEE) for certification or licensure throughout the United States. The October 1985 exam was developed based upon an extensive survey study initiated by NCEE in 1979. A report of that study was published in March 1981 as "A Task Analysis of Licensed Engineers". (Respondent's exhibit #4) The primary purpose of the study was to aid NCEE in developing"... fair, meaningful, uniform, and objective standards with which to measure minimum competency for professional licensure." (exhibit #4, page E1) In drafting an exam the NCEE relies on the societies representing various engineering disciplines to submit examination problems for consideration. The Society of Manufacturing Engineers, through its professional registration committee, provides that service on behalf of the manufacturing engineers. The October 1985 examination for manufacturing engineers did not include questions relating to electrical engineering, which is Mr. Reggia's sub- area of emphasis in the area of manufacturing engineering. Since manufacturing engineering includes overlap into the basic engineering disciplines, Mr. Reggia contends the exam was one-sided and invalid as he felt it concentrated on tool designing and mechanical engineering. Some industries, particularly the aerospace industries now include a substantial number of electrical engineers on their staff. Engineering is an evolving discipline and manufacturing engineering has undergone changes with new technologies in recent years. One way of addressing the diversity and changes in the field is to provide a two-book exam that would offer the applicant a wider variety of problems from which he or she could select. This has been recommended to the NCEE by the Society of Manufacturing Engineers. Another approach, and the one utilized by the NCEE, is to conduct periodic surveys to determine the tasks which engineers are actually performing and the level of judgement required to perform the tasks effectively. It would be impossible, and perhaps inappropriate to develop an exam that would test each individual only on his or her particular expertise. In the area of manufacturing engineering the exams developed by NCEE are passed by 65- 75 percent of the candidates, a rate which is comparable to that of the mechanical engineers for their exam. Seven out of ten applicants passed the same exam which Mr. Reggia took in October 1985.
The Issue Whether Petitioner is entitled to additional credit for his solutions to four problems on the Principles and Practice of Engineering portion of the engineering licensure examination administered on October 30, 1998, by the National Council of Examiners for Engineers and Surveyors.
Findings Of Fact Based upon the evidence adduced at hearing and the record as a whole, the following findings of fact are made: On October 30, 1998, as part of his effort to obtain a Florida engineering license, Petitioner sat for the Principles and Practice of Engineering Examination (Examination). This is a national examination developed and administered by the National Council of Examiners for Engineers and Surveyors (NCEES). Petitioner chose to be tested in civil engineering. Petitioner received a raw score of 45 on the Examination. For the civil engineering specialization, a raw score of 45 converts to a score of 67. To pass the Examination, a converted score of 70 is needed. Petitioner formally requested (in writing, by letter dated March 26, 1999) that his solutions to Problems 120, 125, and 222 on the Examination be rescored. Petitioner's written request was made to Natalie Lowe of the Board, who forwarded it to the NCEES. Appended to Petitioner's letter to Ms. Lowe were two pages of "scratch paper" on which Petitioner had written during his post-examination review on March 19, 1999. On the first page were written comments he had made regarding the scoring of Problems 120 and 125. On the second page were the following written comments he had made regarding the scoring of Problems 220 and 222: 220 a, b, & c 2 parts b & c correct. Min. mark I should get[:] At least 5 instead of 2 and maybe 7. There is an error. 222 ok The NCEES's rescoring of Petitioner's solutions to Problems 120, 125, and 222 resulted in his receiving a raw score of 43 (or a converted score of 65, 5 points less than he needed to pass the Examination). The Board received the NCEES's rescoring results on May 12, 1999. The Board subsequently referred the matter to the Division to conduct an administrative hearing. At the administrative hearing that was held pursuant to the Board's referral, Petitioner challenged the grading of his solutions to Problems 120, 125, and 220 of the Examination, and indicated that he had "no dispute concerning the grading of [his solution to Problem] 222," notwithstanding that he had requested, in his March 26, 1999, letter to Ms. Lowe, that his solution to Problem 222 be rescored. Petitioner explained that he had made this request as a result of inadvertence and that he had actually intended to seek rescoring of his solution to Problem 220, not Problem 222. Problems 120, 125, and 222 were worth ten raw points each. Problem 120 contained four subparts (or requirements). Petitioner initially received four raw points for his solution to Problem 120. Rescoring did not result in any change to this score. Petitioner solved two subparts of Problem 120 correctly (subparts (a) and (b)). The solutions to the other two subparts of Problem 120 (subparts (c) and (d)), however, were incorrect inasmuch as Petitioner had neglected, in making the lateral force calculations and drawing the diagrams required by these subparts, to include the force attributable to the movement of the groundwater referred to in the problem. Therefore, in accordance with the requirements and guidelines of the NCEES scoring plan for this problem, the highest raw score that he could have received for his solution to this problem was a four, which is the score he received. Problem 125 contained three subparts (or requirements). Petitioner initially received a raw score of two for his solution to Problem 125. Upon rescoring, no change was made this raw score. Petitioner correctly solved only one of the three subparts of Problem 125 (subpart (c)). In his solution to subpart (a) of Problem 125, Petitioner did not provide, as required by this subpart, the quantities of water, cement, and aggregate necessary for the project described in the problem. Petitioner's solution to subpart (b) did not describe one of the acceptable slump increasing methods that the candidates were required describe in their solution to this subpart. Accordingly, giving Petitioner a raw score of two for his solution to Problem 125 was consistent with the requirements and guidelines of the NCEES scoring plan for this problem. Petitioner received a raw score of two for his solution to Problem 220. He did not request, in his March 26, 1999, letter to Ms. Lowe, a rescoring of his solution to this problem, and, as a result, his solution was not rescored. At the administrative hearing, Petitioner testified on his own behalf regarding the scoring of this solution and, during his testimony, contended that the score he received was too low; however, neither a copy of the problem, nor a copy of the NCEES scoring plan for this problem, was offered into evidence. Accordingly, the record is insufficient to support a finding that the score Petitioner received for his solution to Problem 220 was undeservedly low in light of the NCEES scoring plan for this problem. Petitioner initially received a raw score of eight for his solution to Problem 220. Rescoring resulted in this score being reduced two points to a six. Petitioner did not present any evidence supporting the position (which he advances in his Proposed Recommended Order) that he should have received a higher score for his solution to this problem, and, consequently, Respondent's expert, in his testimony at hearing, did not address the matter. While there were exhibits offered (by Respondent) and received into evidence relating to the scoring of Petitioner's solution to Problem 222, it is not apparent from a review of these exhibits that such scoring deviated from the requirements of the NCEES scoring plan for this problem (which was received into evidence as part of Respondent's Exhibit 12).
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that a final order be entered rejecting Petitioner's challenge to the failing score he received from the NCEES on the Principles and Practice of Engineering portion of the October 30, 1998, engineering licensure examination. DONE AND ENTERED this 20th day of December, 1999, in Tallahassee, Leon County, Florida. STUART M. LERNER Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 20th day of December, 1999.
Findings Of Fact Based on the stipulations and admissions of the parties, the exhibits received in evidence, and the testimony of the witnesses at the hearing in these consolidated cases, I make the following findings of fact. Findings based on stipulations and admissions of the parties The Petitioners, John Eugene Harden and Dova Cauthen, qualified for and were administered the January 1982 Electrical Contractors' Licensing Examination. After their examinations were graded and re-graded, Petitioners were notified that they had not received a "Passing" score of 75 or more. The Petitioners, John Eugene Harden and Dova Cauthen, qualified for and were administered the July 1982 Electrical Contractors' Licensing Examination. After their examinations were graded and re-graded, Petitioners were notified that they had not received a "Passing" score of 75 or more. Both Petitioners reside in Dade County, Florida. The Respondent Department of Professional Regulation (hereafter "DPR" or the "Department") is an agency of the State of Florida. The Respondent Electrical Contractors' Licensing Board (hereafter referred to as the "Board") is an agency of the State of Florida statutorily responsible, along with DPR, for licensing certified electrical contractors. The office of the Executive Director of the Board is located at 130 North Monroe Street, Tallahassee, Florida. The business address of Petitioner Harden is Harden Electric, 311 N.E. 8th Street, Homestead, Florida 33030. The business address of Petitioner Cauthen is 959 N.E. 79th Street, Miami, Florida 23138. Petitioner Harden specifically requested to review his January and July 1982 Electrical Contractors' Licensing Examinations. On March 25, 1982, Petitioner Harden went to Tallahassee, Florida, to review his January 1982 Licensing Examination papers, but although he requested an "examination review" he was never provided his own examination booklet or a copy thereof. During this alleged "examination review" Mr. Harden wrote written objections to the January 1982 examination on the forms provided to him based on the master copy of the examination provided to him. On August 31, 1982, Petitioner Harden went to Tallahassee, Florida, to review his July 1982 Licensing Examination papers, but although he requested an "examination review" he was never provided his own examination booklet or a copy thereof. During this alleged "examination review" Mr. Harden wrote written objections to the July 1982 examination on the forms provided to him based on the master copy of the examination provided to him. Petitioner Cauthen specifically requested an "examination review" with respect to her January and July 1982 Licensing Examination papers. On March 17, 1982, Petitioner Cauthen went to Tallahassee, Florida, to review her January 1982 Licensing Examination papers, but although she requested an "examination review" she was never provided her own examination booklet or a copy thereof. During this alleged "examination review" Ms. Cauthen wrote written objections to the January 1982 examination on the forms provided to her based on the master copy of the examination provided to her. On August 19, 1982, Petitioner Cauthen went to Tallahassee, Florida, to review her July 1982 Licensing Examination papers, but although she requested an "examination review" she was never provided her own examination booklet or a copy thereof. During this alleged "examination review" Ms. Cauthen wrote written objections to the July 1982 examination on the forms provided to her based on the master copy of the examination provided to her. Petitioners Harden and Cauthen sought Board review of their January and July 1982 Electrical Contractors' Licensing Examinations. Petitioners were advised in July 1953 that no copies of their actual booklets exist; Petitioners were advised in October 1982 that their actual booklets were shredded. With regard to question 71 on the afternoon portion of the July 1982 Licensing Examination, two of the four possible responses, "A" and "C" were credited. Candidates like Petitioner Cauthen who answered "B" received no credit for the question. DPR destroyed the Petitioners' examination booklets before the end of the two year period immediately following each of the 1982 examinations. That Section 455.217 of the Florida Statutes requires the Board "by rule" to designate areas of competency to be covered by each licensing examination. That Section 455.217 states that the Board shall "by rule specify the general areas of competency to be covered by each examination, the relative weight to be assigned in grading each area tested, and the score necessary to achieve a passing grade." That the amendment of Rule 2IGG-6.01(2) required candidates to commit to memory portions of the electrical code, accounting, law, worker's compensation rules, federal employer's tax guide, AIA General Conditions, business practices, legal and insurance requirements. In the July 1982 Electrical Contractors' Licensing Examination candidates were given one hour more time in which to take the examination than was given during the January 1982 examination. That DPR did not and has never provided Petitioners, their attorneys or agents with their own examination booklets for the January and July 1982 Electrical Contractors' Licensing Examinations. DPR has provided Petitioners only with copies of "master" examinations for their review. That the Respondents produced Notice of Destruction of Examination Booklets and Other Examination Materials dated April 14, 1982, which allegedly evidences destruction of Petitioners' examination booklets for the January 7, 1982, Examination on April 14, 1982. That the Respondents produced Notice of Destruction of Examination Booklets and Other Examination Materials dated October 15, 1982, which allegedly evidences destruction of Petitioners' examination booklets for the July 13, 1982, Examination on October 15, 1982. The findings in paragraphs 1 through 23, immediately above, are based directly on the stipulations of the parties in Case No. 84-0309R, the evidence in which has been incorporated into the record in these cases. The findings in the following paragraphs are based on testimony and exhibits, but some of them are also based in whole or in part on stipulations. In the findings which follow there are certain to be at least some repetitious findings in the course of putting matters into context and making additional findings which are related to some of the stipulated findings. Findings incorporated from Final Order in Case No. 84-0309R Both of the Petitioners in these consolidated proceedings are individuals who have applied to the Electrical Contractors' Licensing Board for licensure as certified electrical contractors. Both of them have been approved to sit for the licensure examination. Petitioner Harden took the Board's licensure examination on each of the following occasions: July 1981, January 1982, July 1982, and January 1983. The Board has not given him a passing grade on any of those four examinations. Petitioner Cauthen took the Board's licensure examination on two occasions: January 1982 and July 1982. The Board has not given her a passing grade on either of those two examinations. The Electrical Contractors' Licensing Board certification examination is different from a master electrician examination. However, the two examinations are in many ways similar because of the overlap in the nature of the subject matter to be tested on both examinations. The passing score or cut score on the local master electrician examination administered by Dade County is 70 percent. Most of the questions on that examination are about the National Electrical Code. Part of the Dade County master electrician examination is closed book. Prior to 1972, persons wishing to engage in electrical contracting in the state of Florida were required to be licensed by the local governments in the areas in which they sought to operate. Since 1972, persons wishing to engage in electrical contracting in the state of Florida must be licensed by a unit of local government or by the Electrical Contractors' Licensing Board. Persons who are licensed by the Electrical Contractors' Licensing Board are known as certified electrical contractors. A certified electrical contractor can engage in electrical contracting anywhere in the state of Florida without local licensure. Persons who are licensed by one or more local governments (typically a municipality or a county) are known as registered electrical contractors. Registered electrical contractors are licensed to engage in electrical contracting only in the geographic areas encompassed by the boundaries of the local government entities that issued their local licenses. There is, however, a certain amount of reciprocity from one local government to another. Certification as a state certified electrical contractor does not authorize the electrical contractor to work as an electrician, although a certified electrical contractor can pull permits. A person does not have to be a licensed electrician in order to become a state certified electrical contractor, although many electrical contractors are also licensed as journeyman or master electricians. Candidates for the electrical contractor licensure examination have a great variety in the nature and scope of their background and experience. This variety in background and experience is among the reasons which cause testing for minimum competence as an electrical contractor not to be an exact science. The Electrical Contractors' Licensing Board administered its own certification licensure examination from 1972 through 1980. During that period of time the Board administered the examination a total of twenty-two times. During those twenty-two sessions of Board administered examinations, a total of 824 candidates sat for the examination, of which 392 were successful. In 1979, the regulation of professions and occupations was reorganized. The Department of Professional Regulation (the Department) was created as an umbrella agency over numerous boards, including the Electrical Contractors' Licensing Board and the Construction Industry Licensing Board. The responsibility for examining applicants, which in the case of the Electrical Contractors' Licensing Board had been handled by the Board members themselves, was taken over by the Office of Examination Services (O.E.S.). Previous Board examinations were open book with certain reference material allowed. When the Office of Examination Services took over the administration of the January 1981 exams, it sent to the candidates a form letter adapted from the Construction Industry Licensing Board. This form letter instructed the candidates that they would be able to bring into the examination any notes and other materials desired. This was contrary to the past practice of the Board and contrary to the Board's wishes. The Board was very frustrated and concerned about the circumstances which led up to the January 1981 examination because the O.E.S. had refused to allow the Board to have any input into the examination and the Board thought the O.E.S. had come up with an apprentice level examination. In the Board's opinion, the January 1981 examination prepared by O.E.S. clearly tested at a level less than that of minimum competency. And although O.E.S. did not agree with the Board that the January examination was invalid, O.E.S. did agree that the examination was a lot easier than they had thought it would be. The Electrical Contractors' Licensing Board did not conduct any formal studies or formal analysis prior to making the 1981 amendments to Rule 21GG-6.01 changing the format of the examination and the cut score. However, the Board's Rules Committee did a great deal of work on the matter and the matter was discussed extensively at Board meetings. In this regard it is important to keep in mind that the Board members (with the exception of the two lay members) were all experienced practicing electrical contractors who were familiar with the requirements of day-to-day electrical contracting. The Board's decision to change part of the examination to closed book was due to a concern that people with no electrical knowledge could become licensed if the examination was all open book and had a low percentage of technical questions. The Board felt that more than half of the examination should be technical questions. The closed book National Electrical Code questions were for the purpose of testing what a person with experience in the field of electrical contracting should know about everyday matters. The primary, if not the sole, motivation for the Board's 1981 amendments to Rule 21GG-6.01 was the Board's concern about the quality of the O.E.S. administered examination and the Board's feeling that, due to its estranged relationship with O.E.S., the only vehicle through which it could effectively influence the quality of future examinations was through rulemaking. Those amendments were not motivated by any desire on the part of the Board to restrict competition in the field of electrical contracting. During the past few years the job requirements of an electrical contractor have remained substantially the same. Block and Associates (hereinafter "Block") is a company based in Gainesville, Florida, that writes licensure examinations for contractors in various trades. Block writes such examinations for cities, counties, and states. Block has prepared electrical contracting licensure examinations for the states of Florida, Georgia, and Oklahoma, as well as for St. Johns County, Florida. Block prepares licensure examinations for just about anything that involves electricity, including such things as electrical contracting, electrical journeyman, electrical master, low voltage, elevator, burglar alarm, and fire alarm. Block prepares master electrician examinations for over one hundred governmental entities. The Block electrical master exams and the Block electrical contractor exams use some of the same questions. Subsequent to the 1982 examinations which form the gravamen of these proceedings, the Board has contracted with Block and Associates to prepare the Board's certification licensure examination. The Board has never said anything to Block about wanting to achieve any particular passing rate or wanting to have any particular percentage of the candidates taking the examination achieve a passing score. The Board has never interfered with Block's autonomy in the preparation of the electrical contractor exam. All of Block's electrical contractor examinations include business questions. It is important to test for business skills because an electrical contractor who gets into business trouble may start cutting corners and cutting corners is a hazard to the public. Some of Block's electrical contractor examinations also have specific sections on safety. Block always does a post-administration statistical analysis of the examination questions it uses. The reason for this is to find out if anything is wrong with the questions; in other words, to determine whether the questions are valid. The computer program used by Block to check the validity of its tests was prepared for Block by a professor in the College of Education at the University of Florida. Block uses a computer to conduct a validity analysis of each test it administers. The Board's purpose in amending Rule 21GG- 6.01(1)(c) to delete the reference to "72A, B, C and D" was to make it clear to candidates for examination that the Board intended to include in the safety portion of its examinations questions from portions of Volume Seven of the NFPA other than Chapters 72 A, B, C, and D of that volume. Volume Seven of the NFPA contains quite a bit of material in addition to the material included in Chapters 72 A, B, C, and D of that volume. In view of the modifying phrase "shall include but not be limited to," the amendment to delete the reference to Chapters 72 A, B, C, and D was not a necessary prerequisite to the use of examination questions based on other portions of Volume Seven of the NFPA. The Board could ask the same examination questions before and after the amendment. Several weeks before each examination, the Department of Professional Regulation sent all candidates for examination, including these Petitioners, a notice to appear for the examination. The notice to appear included a list of reference books on which the examination was to be based. That list of reference books contained a specific reference to Volume Seven of the NFPA. All sixteen volumes which comprise the entire NFPA were not listed as reference books. Neither of these Petitioners demonstrated that he or she was genuinely confused or misdirected by the amendment to Rule 21GG-6.01(1)(c). Neither Petitioner contended he or she had studied all sixteen volumes of the NFPA. Instead, both Petitioners studied from the books on the reference list. In November of 1983 the Electrical Contractors' Licensing Board again amended Rule 21GG-6.01(3) to change the relative grading weights of the areas of competency to be tested. After November of 1982 the relative grading weights of the areas of competency were as follows: Technical increased from 60 percent to 65 percent; electrical calculations increased from 40 percent to 45 percent; code- related questions remained 20 percent of the examination; general business was decreased from 30 percent to 25 percent of the examination; and safety remained 10 percent of the examination. The licensing examination given by the Board has historically contained materials testing knowledge of electrical work and also materials testing the business qualifications of the applicants. Since the inception of the Board's licensing exam, the electrical work portions have contained electrical calculations. Throughout the time the Board has been administering licensure examinations for certification, the percentages of the examination devoted to particular subject areas have been based upon the Board's belief as to the appropriate levels of knowledge needed by a certified electrical contractor. The weighting of the examination was equitable when it was weighted 60 percent technical, 30 percent business, and 10 percent safety, and it is still equitable now that it is weighted 65 percent technical, 25 percent business, and 10 percent safety. The Board changed to the current percentages at the suggestion of Block and Associates. The Board's examination has had similar ratios for a long time, at least as far back as 1971. The proportions of the examination devoted to each particular subject matter did not change due to the March 1981 amendments. With the exception of the January 1981 examination put together by O.E.S., electrical calculations had historically been 40 percent of the test. Since the Board felt this percentage should be maintained, and since it was necessary to divide those calculations from the other portion of the technical section (since one would be tested in the closed book portion and the other in the open book portion), the division of the technical part of the exam was set out in the rule. On an electrical contractor licensure examination it is appropriate to include in the area of "technical" questions, questions dealing with the following subjects: electrical calculations, parts of estimating, and designs and electrical schematics. In the "business" portion of an electrical contractor examination, it is appropriate to include questions dealing with the following: accounting, worker compensation, and social security. Local licensing examinations had, in 1981, included a closed book portion. Local licensing examinations and state licensing examinations given in other states still include such a closed book portion. The basic Block examination has always included a closed book portion. Closed book examinations are not all that unusual in occupational and professional licensure testing. The CPA and nursing examinations are closed book. Also, virtually all of the electrical examinations administered by Block and Associates include a closed book portion. The advantage of a closed book examination technique is that it is easier to identify the candidates who have had practical experience in the subject matter being tested. A closed book examination is a better test for field experience and for general knowledge of the subject matter being tested. Certain questions known as "anchor" questions were given both in the open book examination before the format change and in the closed book section after the format changed. The relative performance by candidates on the same questions in the two formats shows the format change not to have prejudiced the candidates. Further, an examination analysis of the results of the closed book portions of the Board examinations reveals that, based on candidate performance, the closed book portion of the examinations was easier than the open book portion. This also shows that the format change did not prejudice the candidates. To the extent that the content domain of Board examinations was established by the March 1981 amendments to Rule 21GG-6.01(3), the content domain was established on the basis of the expert judgment of persons who were experts in the field of electrical Contracting, namely, the members of the Electrical Contractors' Licensing Board. The relative grading weights established in the 1981 amendments to Rule 2100-6.01(3) have at least a possible correlation to electrical contractor competence. In fact, the persuasive expert testimony establishes that those relative grading weights did bear a reasonable relationship to electrical contractor competence. The current slightly different relative grading weights also bear a reasonable relationship to electrical contractor competence even though there has been no recent change in the nature of the responsibilities of electrical contractors. In this regard it must be remembered that testing is not an exact science and no test for minimum competency can ever be expected to be an exact model of actual work experience. The best that can be expected is a reasonable model. There are various accepted methodologies for the creation of standardized examinations, adherence to which tends to enhance the likelihood that the examination instrument finally produced will be valid and reliable. However, adherence to those accepted procedures is not a necessary prerequisite to creation of a valid and reliable examination. This is particularly true when one is going to be testing small groups of people with varied backgrounds instead of the massive groups of people with similar backgrounds for whom standardized tests are more typically designed. In the final analysis the only practical and reliable measure of the validity of an examination is by statistical analysis of the examination after it has been administered. Given the nature of the circumstances faced by the Board at the time of the March 1981 amendments to the examination format and cut score, given the nature of the pool of candidates to be examined, given the nature of the changes contemplated by the Board, and given the very nature of the process of testing for minimum competency--which involves perhaps as much art as it does science--there is no study or data which would have been particularly useful to the Board in helping to determine exactly what the effect of their changes would be. Such effects can only be determined or measured with any degree of accuracy after the administration of an examination that incorporated the changes. Following the administration of such an examination, it is possible to perform a statistical analysis of all questions used on the examination and to eliminate or give credit for any questions which are shown by statistical analysis to be invalid or unreliable. This is precisely the process that is used by Block in the validation of their examinations and is an accepted testing procedure. It would have served no useful purpose to have conducted a trial run of an examination using licensed certified electrical contractors as a test group for the new examination format. First, it would be virtually impossible to try to put together an accurate cross-section of certified electrical contractors to use as a test group. Second, one would expect them all to pass the examination, so when they did so nothing of value would have been learned. Finally, the administration of such a trial run would risk the possibility of compromising examination question security. One aspect of accepted methodology for the preparation of standardized examinations is the definition of the content domain of the examination, i.e., a determination of what knowledge is essential to demonstrate that the candidates for licensure are minimally competent. This aspect of examination preparation is often accomplished by performing a formal job analysis, which is, in essence, a study of all of the usual tasks performed by a person engaged in the occupation or profession to be tested, including an evaluation of the relative importance of each of those tests to minimum competence. The content domain can also be defined on the basis of the judgment of a group of experts in the occupation or profession to be tested. With the exception of the lay members who were added in recent years, all of the members of the Electrical Contractors' Licensing Board are, and have been, persons certified to engage in electrical contracting in the state of Florida and actively engaged in the electrical contracting business. Therefore, at all relevant times all of the professional members of the Board had extensive personal knowledge of what was involved in the practice of electrical contracting, which personal knowledge was as useful in defining content domain as would have been a formal job analysis. (In this regard it is important to note that even with the addition of lay members to the Board in recent years, the experienced professional members have continued to constitute a substantial majority of the Board.) When examination booklets are being prepared by the Department of Professional Regulation prior to an examination, the booklets are carefully inspected to make sure that all of the booklets are identical. Following that inspection the booklets are sealed and stored in a secure place in order to insure, among other things, that no changes are made to any of the examination booklets before they are handed out to the candidates. After an examination is given, the Department of Professional Regulation retrieves all of the examination booklets, including all booklets that were used by all of the candidates, and retains them in a secure place until the excess booklets can be destroyed. The examination supervisor selects the booklets which are to be retained from among the booklets that are not handed out to the candidates at the examination. In the normal course of events all of the examination booklets that were actually handled by the candidates at the examination are destroyed within a very few months of the date of the examination. With the exception of the examination booklet of one other candidate (which exception is not relevant to the disposition of these proceedings), all of the examination booklets which were handed out to candidates during the examinations taken by these two Petitioners were destroyed approximately 90 days after each of the examinations. When such destruction took place, the Department retained copies of the examination booklets which had not been used by any candidate, which copies were identical to the copies that had been handed out to the candidates during the examination. As part of the examination instructions, all candidates for examination are advised not to write anything in their examination booklets because all of the booklets used by the candidates will be shredded. They are specifically told to do all of their computations on sheets of work paper that are provided to them at the examination. All candidates are specifically told that the only things they turn in that will be saved are their answer sheets and their sheets of work paper. The findings in paragraphs 25 through 63, immediately above, are substantially identical to certain of the findings of fact made in the Final Order in Case No. 84-0309R. They are, of course, based on the evidence received during the hearing in that case, which evidence has by stipulation been incorporated into the record in these cases. Most of the findings which follow are based on evidence received during the hearing of these examination challenge cases, bat some of them are also based on the evidence incorporated from the record in Case No. 84-0309R. The rest of the findings The examination prequalification process imposed by the Board is designed to ascertain whether an individual meets the professional criteria to attempt the examination and has no specific relationship to whether a candidate will pass or fail the examination. The January and July 1982 examinations were administered in geographical locations accessible to the Petitioners and although there were certain minor distractions and interruptions which occurred during the administration of both examinations, there were no major irregularities in the administration of either examination. Specifically, none of the irregularities in the administration of either examination was of sufficient magnitude to adversely impact the fundamental fairness of the manner in which the examinations were administered. The proof in this case is insufficient to show that the examination conditions on either occasion were such as to have interfered with the abilities of either of these Petitioners to choose correct answers on the examinations. The copies of the examination booklets which were furnished to the Petitioners at their examination reviews and thereafter were true and correct copies of the examination booklets which were given to all candidates at the time of the examinations. There is no persuasive or convincing evidence that either of these Petitioners has suffered any harm as a result of not being able to see the very same booklets that they personally used during the examination. The evidence in this regard is purely speculative. And, in any event, in view of the instructions to all candidates that their individual examination booklets would be shredded and that only their answer sheets and work paper sheets would be saved, if the Petitioners wrote useful information in their examination booklets and failed to also write that information on their work paper sheets, they were the authors of their own inconvenience and are not entitled to any relief on the basis of a matter that they could have prevented. When the Board conducts its post-examination review, it has been the policy of the Board to grant credit to all candidates for questions which are determined by the Board to be "bad items" for one reason or another. The Board has also on occasion granted credit for either of two answers when in the judgment of the Board it appeared that either of the two answers should be considered a correct answer. Both of these types of adjustments to the grading of the exams are reasonable ways to compensate for shortcomings in the examination instrument depending upon the particular circumstances of a given examination question. The examination scores achieved by the Petitioners when their scores were properly tabulated in the manner in which the scores of other candidates were tabulated are as follows: January 1982 examination: Dova Cauthen 68.0 John Harden 52.0 July 1982 examination: Dova Cauthen 73.0 John Harden 61.0 The foregoing tabulation of scores takes into consideration every reasonable manner in which examination scores could be tabulated. The foregoing tabulation ascribes the highest score possible to each candidate under all scenarios which are consistent with the greater weight of the evidence. During the course of the hearing, it was stipulated that, on the basis of proof at the hearing, both Petitioners were entitled to have certain additional points added to their grades where the proof demonstrated that the answers chosen by the Petitioners were as good as the keyed answers. On the basis of the tabulated scores plus the stipulated additions to all of those scores, both of the Petitioners would still be assigned failing grades. Accordingly, it is necessary to address the remainder of the Petitioners' contentions to determine whether they are entitled to have further points added to their scores or are otherwise entitled to licensure regardless of whether their scores can properly be raised to a passing level by the award of additional points. Findings regarding examination directions The directions which accompanied the challenged examinations are clear and to the point. They are easy to understand and consist of the type of information one would normally expect to see in the directions for a multiple choice examination. The most persuasive of the Petitioners' testing experts said the directions were good examination directions, and I so find. The Petitioners contend that, as applied, the directions are misleading and confusing because they instruct the candidates to mark only one choice for each question, but in some instances there was more than one correct answer among the choices. This is a shortcoming not of the directions, but of some of the questions and the answer choices that were given with them. In those instances in which a question does contain two correct answer choices or in which none of the answer choices are correct under any reasonable interpretation, the remedy adopted by the Respondent, which is to grant credit for additional responses and in some instances to grant credit for all responses, is an appropriate and fair resolution to any problem created by the application of the directions to a problematic question. Findings regarding "domino" questions The Petitioners have challenged a number of questions on the grounds that they are unfair or improper because they are "domino" questions. "Domino" questions are questions which require for their solution the application of information gleaned from obtaining the correct answer to one or more prior questions. Because the correct answer to one question depends upon application of the correct answer to one or more prior questions, it is argued that "domino" questions are inherently unfair inasmuch as an incorrect answer to an early question virtually guarantees an incorrect answer to any later "domino" questions. The "domino" question format is not a preferred testing format in many testing situations. Nevertheless, it appears to be a technique particularly well suited for testing for minimal competency in the field of electrical contracting because it provides a testing model which closely parallels functions electrical contractors are required to perform on a regular basis in the practice of their profession. In the practice of electrical contracting, it is common for a contractor to be required to make a series of calculations, each of which incorporates the answer obtained in one or more prior calculations. Accordingly, the "domino" question format is an appropriate format for testing candidates for licensure as electrical contractors. The "domino" question format is also used on other professional examinations in Florida. Findings regarding "Walker" questions The closed book portion of both examinations contained questions taken from an approved reference source entitled Practical Accounting and Cost Keeping for Contractors by Frank R. Walker Company, Eighth Edition. The Walker reference is a collection of materials regarding various topics related to bookkeeping and record management. The accounting information contained therein is fundamental. The Walker reference uses a number of antiquated terms and peculiar phrases. Nevertheless, most of the concepts in the Walker reference are rather straight- forward and uncomplicated. The Walker reference book could not be regarded as a very good textbook on the subject of accounting, but it is adequate for the purposes for which it was intended. Since the administration of the subject examinations, the Board has become disenchanted with the Walker book and has replaced it with another book. On a variety of different grounds, the Petitioners have challenged each and every one of the questions taken from the Walker reference. With the one exception noted below, the Petitioners' challenges to the questions taken from the Walker reference are without merit because virtually all of the questions are drawn directly from material in the Walker reference and in virtually all cases the language of the correct answer choice is the same, as or is a close paraphrase of, some explicit statement in the Walker reference. Anyone familiar with the Walker reference should not have any trouble selecting the correct answers to the Walker based questions. Furthermore, even without any familiarity with the Walker reference, a person with a reasonable amount of business experience should have been able to answer many of the questions just on the basis of a common sense application of that business experience. In sum, the Walker derived questions are fair and reasonable questions. The one exception to the foregoing is question 80 on the January closed book examination. Answer choices "C" and "D" should also be credited on this question. Accordingly, both Petitioners should receive credit for this question. Findings regarding the January "load" questions On the January 1982 examination all candidates (including the Petitioners) were given credit for all answers to a series of load questions (questions 13 through 20 of the open book portion) on that examination. For reasons which are not elucidated in the record in these cases, it was apparently determined by the Department or by the Board that those load questions were "too difficult." The Petitioners contend that because credit was given for all answers to all load questions on the January examination, the same should be done with regard to all of the load questions on the July examination. However, the Petitioners have failed to prove on what basis the January load questions were deemed to be "too difficult" and, therefore, there is insufficient evidence upon which to find whether the July load questions are "too difficult" for the same reasons. As found further below, standing alone, the July load questions do not appear to be too difficult. Findings regarding the July "load" questions Like the January exam, the July 1982 examination also contained a series of questions referred to as the "load" questions. The purpose of these questions (some of which were "domino" questions) was to require the candidates to demonstrate an ability to determine various amperages, voltages, and conduit sizes based on the content, character, and size of the buildings and equipment in a hypothetical shopping center described in an exhibit which was included in the examination booklet. The Petitioners have challenged all of the July load questions that either or both of them missed, asserting, inter alia, that the Board's solution to most of the questions is erroneous. For the reasons set forth below, I find that the Petitioners are not entitled to have any points added to their scores on the basis of their challenges to the load questions other than the one point the Respondents stipulated should be added to Petitioner Harden's score. Both Petitioners missed question 66 in the load series. Both Petitioners missed this question because they both omitted a crucial and undisputed step in their calculations. Both failed to take into account the continuous load factor. The reason for their incorrect answers had nothing to do with the issues in dispute between the experts as to how other aspects of the question should be calculated. Petitioner Harden answered question 68 correctly. Petitioner Cauthen chose an incorrect answer for reasons which are not elucidated in the record. Nevertheless, it would appear that her incorrect answer was not due to any defect in the question, because her answer choice is inconsistent with either of the answers advanced by the competing expert witnesses. Petitioner Harden answered questions 69 and 70 correctly. There is no showing of any causal connection between Petitioner Cauthen's incorrect answers to these questions and any defects in the Board's solution to the questions. To the contrary, it appears most likely that Petitioner Cauthen missed these questions by using information from her incorrect answers to prior questions. Petitioner Harden answered question 71 correctly. Although the competing experts used different methods to calculate the answer, they both came up with the same answer. Thus, there is nothing more than speculation or conjecture as to why Petitioner Cauthen missed this question. It is stipulated that Petitioner Harden should be given credit for question 72 because an alternative method of calculating the answer would result in the answer he chose. However calculated, the only arguably correct answers are the one chosen by Petitioner Harden and the one chosen by the Board and the competing experts. Thus, there is nothing more than speculation or conjecture as to why Petitioner Cauthen chose a different answer. Both Petitioners missed question 73. The experts for the opposing parties disagreed on how this question should be answered. But however it should be answered, the fact remains that neither of the Petitioners appears to have been familiar with either of the methods of solution proposed by the experts, because both chose answers that were inconsistent with those methods of solution. Thus, even assuming that the Board's solution was incorrect, there is no proof that any such shortcoming caused either of the Petitioners to miss this question. In any event, I am persuaded that the Board's solution was the better of the competing solutions offered at hearing. Both Petitioners missed questions 74 and 75. They both appear to have missed both of these questions because of choosing incorrect answers to prior questions and not because of anything improper or incorrect in questions 74 and 75. Both Petitioners missed question 76. Nothing improper or incorrect about the question affected the Petitioners' ability to select the correct answer, because under any interpretation of the correct solution to the question, the correct answer was "none of the above." Petitioner Harden answered question 77 incorrectly. Nothing improper or incorrect about this question affected his ability to select the correct answer. Although the answer depended in part on the answer to a prior question, the correct answer would be calculated using any of the solution methods proposed by the expert witnesses. Findings regarding National Electrical Code Questions Question 18 on the January closed book examination and question 20 on the July closed book examination are challenged as being "tricky" questions. While the selection of the correct answer to these questions requires close attention to detail, both questions are fair and reasonable. Question 22 on the July closed book examination contains a minor typographical error. Despite the typographical error, 84 per cent of the candidates taking the examination selected the correct answer. Accordingly, I find that the minor typographical error did not have any significant adverse impact on the ability of the candidates to understand and answer the question. Question 26 on the January closed book examination is a fair and reasonable question. It was answered correctly by 92 per cent of the candidates. Question 43 on the July closed book examination is a fair and reasonable question. The correct answer is derived directly from a specific exception to a provision of the National Electrical Code. It is reasonable to expect candidates for licensure as electrical contractors to be familiar with exceptions to the Code. Question 47 on the July closed book examination is a fair and reasonable question if credit is given for both responses "C" and "D." Both Petitioners selected response "A," so they are not entitled to additional credit for this question. Question 51 on the July closed book examination would have been easier to understand if the word "`metal" had been inserted before the word "raceway" in the stem of the question. But the evidence in this case is insufficient to establish that the omission of the word "metal" confused the Petitioners and caused them to miss this question. Their answers are consistent with their having assumed that the question involved a metal raceway, but also having simply chosen the wrong answer. Question 53 on the July closed book examination has only one correct answer, "seal or plug." The Petitioners' answer to this question is not a reasonable alternative answer. Question 78 on the January open book examination and question 25 on the July open book examination are both fair and reasonable questions. The omission of the power factor did not make it unreasonably difficult to answer these questions. Question 80 on the January open book examination and question 38 on the July open book examination are fair and reasonable questions. The fact that the question stem does not specifically state that the ranges are residential or commercial does not appear to have affected the ability of candidates to answer the question. Further, it is implicit from the available answer choices that the question contemplates residential ranges. Question 43 on the July open book examination is a fair and reasonable question if credit is given for both response "A" and "D." There is no persuasive competent substantial evidence that response "B" is an acceptable answer. Accordingly, Petitioner Cauthen is entitled to credit for this question, but Petitioner Harden is not. Question 53 on the July open book examination was miskeyed. Accordingly, Petitioner Harden should be given credit for this question. (Petitioner Cauthens original score has been computed on the basis of her having correctly answered this question.) Question 58 on the July open book examination has an ambiguous last sentence in the stem. As a result of that ambiguity, credit should be given for both responses "C" and "D." Accordingly, Petitioner Harden should be given credit for this question. Question 28 on the January open book examination has a word missing from the stem. Nevertheless, 73 per cent of the candidates were able to answer the question correctly. Accordingly, I find that the missing word did not have any significant adverse impact on the ability of the candidates to understand and answer the question. Question 41 on the January open book examination did not contain a correct answer in any of the answer choices. Accordingly, it is stipulated that Petitioner Harden should receive credit for this question. (Petitioner Cauthen's original score has been computed on the basis of her having correctly answered this question.) Question 42 on the January open book examination was miskeyed. Accordingly, Petitioner Cauthen should be given credit for this question. (Petitioner Harden's original score has been computed on the basis of his having correctly answered this question.) Question 44 on the January open book examination was miskeyed. Accordingly, Petitioner Cauthen should be given credit for this question, but Mr. Harden should not. Question 63 on the January open book examination is a fair and reasonable question. Although Petitioners contend the question is confusing, the fact remains that it was clear enough for 75 per cent of the candidates to select the correct answer. Question 43 on the January closed book examination contains a minor typographical error. Despite the typographical error, 90 per cent of the candidates taking the examination selected the correct answer. Accordingly, I find that the minor typographical error did not have any significant adverse impact on the ability of the candidates to understand and answer the question. Question 45 on the January closed book examination had four possible correct answers, depending upon which exception to Section 300-5, National Electrical Code, was applied. This question, unlike others appearing on the examinations, gave no indication in the content of the question stem as to whether the exceptions should be applied. Accordingly, all responses should be treated as correct and both Petitioners should be given credit for this question. Question 16 on the January closed book examination was miskeyed. Credit should be given for responses "A" and "B." Accordingly, Petitioner Harden should be given credit for this question. (Petitioner Cauthen's original score has been computed on the basis of her having correctly answered this question.) Question 26 on the January closed book examination is a fair and reasonable question. Although Petitioners' expert described a complex way of answering the question which would lead to a different answer, the fact remains that there is a fairly simple solution which leads to the correct answer which was used by 88 per cent of the candidates to select the correct answer. Question 69 on the January closed book examination does not have any correct answer choice. Accordingly, both Petitioners should be given credit for this question. Findings regarding financial analysis questions A major point of contention in these cases concerns the propriety of a series of questions on both the January and July examinations which can perhaps most accurately be described as financial analysis questions. This group of questions has been challenged on a variety of grounds including, (a) the candidates were not provided with a reference source for the questions, (b) the questions were too difficult for the skill level of the group being examined, (c) in order to obtain the correct answer to some questions the candidates had to guess which way the Board expected them to round their answers, and (d) that the exhibits used in conjunction with the financial analysis questions were incorrect and inadequate. With regard to the assertion that the financial analysis questions were too difficult for the skill level of the group taking the examinations, Petitioner Cauthen correctly answered 17 of 20 of this category of questions on the January examination and all 20 in this category on the July examination. Statistical analysis of the performance of all candidates on this category of questions reveals that on the January examination an average of 74.8 per cent of all candidates responding answered the financial analysis questions correctly, and on the July examination 81 per cent of all candidates responding answered the financial analysis questions correctly. These averages are significantly higher than the average percentage of correct responses achieved on the remainder of the questions on the morning portion of the two examinations. Accordingly, I find that the financial analysis questions were not too difficult for the skill level of the group being tested. Some of the financial analysis questions required the application of knowledge or information that is not specifically covered in any of the reference books listed in the notices to appear for the examinations. Petitioners contend that these questions are therefore unfair because even if they had studied all of the reference books, they would not have learned what they needed to know to answer some of these questions. The high percentage of correct answers achieved by all candidates on the financial analysis questions indicates that these questions were quite fair. All of the concepts that were involved in the solution of the financial analysis questions were concepts that an experienced businessman would normally be expected to be familiar with. All of the financial analysis questions were at a difficulty level such that a typical high school graduate who had some familiarity with bookkeeping should have been able to answer them. The unfairness contention is also dispelled by the fact that the notices to appear specifically stated that the examinations would include questions regarding financial statements, which put all candidates on notice to be prepared to answer such questions. Both of the Petitioners attended examination preparation courses, which courses included financial statement analysis in the course materials. The exhibits relating to the financial analysis questions which were included in the examination booklets contained all of the information necessary to calculate the answers to the financial analysis questions. The exhibits were fairly comprehensive and contained the type of basic "nuts and bolts" information a businessman would use for his own internal business purposes. The answer choices to several of the financial analysis questions were such as to require the candidate to round the calculated answer in order to determine the correct response. Some of the rounding functions were not in strict accordance with classical rounding principles and on some occasions the calculated answer would have to be rounded down, for example, when the classical rule would have mandated rounding up to the nearest whole number. The examination instructions did not tell the candidates what type of rounding principles they should use. The foregoing notwithstanding, irregularities in rounding do not tend to have an adverse effect on the performance of examination candidates unless the candidates are required to choose between very close answer choices. In the subject examinations there is only one question which arguably presented a rounding irregularity in conjunction with very close answer choices. This was question 32 on the January open book examination, which was missed by Petitioner Harden. However, even though it is possible that the format of the answer choices to question 32 confused Petitioner Harden and caused him to choose the wrong answer by rounding his calculated answer, it is equally possible that Petitioner Harden missed this question for some other reason. The evidence is insufficient to show that the format of question 32 on the January open book examination is what caused Petitioner Harden to miss it when so many others got it right. What the evidence does show (both through his Scores and his candid admission) is that Petitioner Harden has difficulty solving mathematical problems that involve the calculation of or application of percentages. Question 39 on the January closed book examination should be credited to Petitioner Harden for the same reasons that the identical question was credited to all candidates on the July examination. (Petitioner Cauthen's original score has been computed on the basis of her having correctly answered this question.) The evidence is insufficient to establish that the order of questions 8, 9, and 10 on the January closed book examination had any adverse effect on Petitioner Harden's ability to select the correct answer to those questions. Findings regarding safety questions In light of the language of the rule describing what as meant by "safety" questions, and the information about safety questions in the notices to appear for the examinations, there was nothing improper or unfair about the subject matter of the safety questions that appeared on the subject examinations. Findings regarding miscellaneous questions Question 23 on the January closed book examination is a fair and reasonable question. It has only one correct answer choice, which choice was correctly selected by 83 per cent of all candidates. Question 50 on the January closed book examination is a fair and reasonable question. It has only one correct answer choice, which choice was correctly selected by 92 per cent of all candidates. Question 80 on the July closed book examination is a fair and reasonable question. The question contains all the information that is necessary to select the correct answer. On the basis of the findings in the foregoing paragraphs, the tabulated scores of the Petitioners should be adjusted as follows: January 1982 Examination Dova Cauthen: Tabulated score of 68, plus addition of 3.5 points, equals final score of 71.5 John Harden Harden: Tabulated score of 2, plus addition of 4.0 points, equals final score of 56.0 July 1982 examination Dova Cauthen: Tabulated score of 73, plus addition of 0.5 points, equals final score of 73.5 John Harden: Tabulated score of 61.0, plus addition of 2.0 points, equals final score of 63.0 Findings regarding several other matters In the normal course of events, the questions on a licensure examination should be job related. The licensure examinations prepared by Block and Associates are designed to test for everyday knowledge. The Board is of the opinion that the questions asked on the subject examinations directly relate to the everyday practice of electrical contracting. The National Electrical Code is Volume 6 of the Fire Safety Code. Budgetary considerations of necessity affect the quality and nature of examinations prepared for professional licensure. For obvious reasons it would be unreasonable to spend millions of dollars to develop an examination for just a few people. Without question, the examinations at issue here are not as well constructed as the CPA examinations. Not nearly as much effort was put into the construction of these examinations as is typically put into the construction of a CPA examination. The subject examinations could perhaps even be fairly described as mediocre, at best. But on the whole, and in light of the circumstances under which the examinations were given, these examinations were reasonable. They bear a reasonable relationship to the task they seek to accomplish and they do a reasonable job of giving each candidate a fair chance to demonstrate minimal competence as an electrical contractor. Statistical analysis of the results of the January and July 1982 examinations indicates that those examinations, as adjusted to grant additional credit for some questions, are reasonably valid and reliable testing instruments. At least one person who failed one of the subject examinations has since been a successful candidate on a later examination. Where, as here, it is contended that candidates have been subjected to an invalid examination, the remedy which best addresses the competing interests of the candidates' desire for licensure and the public's need for protection is for the candidate to retake the examination.
The Issue The issue is whether Rules 21H-21.002(1) and 21H-21.004(1) are an invalid exercise of delegated legislative authority.
Findings Of Fact Ronnie F. Taylor, of Post Office Box 697, Cedar Key, Florida, is employed by the engineering firm of Ingley, Campbell, Moses and Associates of Gainesville, Florida, which engages in mechanical, electrical and plumbing engineering. Taylor has been with this engineering firm for four years and is currently a vice president in charge of production of electrical engineering documents. Prior to this employment, Taylor spent 14 years as an electrical engineer with the engineering firm of Reynolds, Smith and Hill of Jacksonville, Florida. When Taylor left Reynolds, Smith and Hill, he was the senior design engineer. Taylor served in the military as an electrician. Upon completing military service in 1967, Taylor entered Florida Junior College. He received an Associate of Science degree in Electrical Engineering Technology in 1970 from that institution. Following that degree, Taylor began employment with Reynolds, Smith and Hill, where his responsibilities included the design of electrical projects for commercial buildings, including writing specifications, making cost estimates and producing a finished product. Taylor has spent his entire career in electrical engineering and has no experience with other specialties of engineering. He has extensive experience in electrical engineering having designed and completed numerous large commercial projects. However, because Taylor is not a licensed professional engineer, a licensed professional engineer must oversee all projects during the course of design and completion and must sign and seal all completed work. Taylor is not a licensed professional engineer because he has failed to pass the Fundamentals of Engineering (FE) portion of the engineering examination. He has failed in fourteen attempts to pass the FE exam. Taylor did pass the Principles and Practices (P & P) portion of the exam in 1982. Licensure requirements specify that both sections must be passed prior to licensure. Taylor became qualified to take the engineering exam in 1977 pursuant to Section 471.21(1)(c), Florida Statutes (1977), which permitted an applicant to take the exam with "a specific record of 10 years or more of active practice in engineering work of a character indicating that the applicant is competent to be placed in responsible charge of such work." This so-called 10 year cycle permitted an applicant to qualify for the exam without the otherwise required 4- year college degree and 4 additional years of experience. In 1979, Section 471.013, Florida Statutes, was enacted, allowing persons in the final year of engineering school to take the FE exam to qualify as an engineer intern. This provision has been in effect since 1979. The FE exam, as required by Rule 21H-21.002(1), which is challenged here, includes questions on the subjects of mathematics, mathematical modeling of engineering systems, nucleonics and wave phenomena, chemistry, statistics, dynamics, mechanics of materials, fluid mechanics, thermodynamics/heat transfer, computer programming, electrical circuits, statics, structure of matter, engineering mechanics, electronics and electrical machinery. While Taylor scored highly on the subjects relating to electrical engineering, he had difficulty with other areas of the exam. The course work completed by Taylor in 1970 did not include some of these areas with which Taylor had difficulty. Taylor has had no course work in computer programming, thermodynamics, statistics, nucleonics and wave phenomena. The subjects tested in the FE exam are updated in order to test applicants on the most current information and knowledge of engineering fundamentals. Herbert A. Ingley is a licensed professional engineer and holds a Bachelors degree in Chemical Engineering, a Masters degree in Mechanical Engineering, and a Ph.D. in Mechanical Engineering with a minor in Environmental-Mathematics. He taught full time on the faculty of the University of Florida in Mechanical Engineering for 11 years. In his opinion, it is more difficult for applicants to pass the FE exam the further they are from their formal education and, therefore, applicants in the 10 year cycle have more difficulty passing the exam. According to Ingley, the requirement that persons such as Taylor wait 10 years before taking the FE exam is not logical. However, Ingley also opined that it is important for a professional engineer to have a fundamental knowledge of engineering and that there is a need to test the fundamental basics of engineering for each person who is going to become a licensed professional engineer. George Edward Rabb is a licensed professional engineer, having been licensed in 1965. He was grandfathered and therefore only had to pass the P & P exam. The FE exam was waived based on specific portions of statute and rule which waived the FE exam for persons with fifteen years experience. The waiver was only available to persons qualifying prior to November, 1970. According to Rabb, an engineer needs to have a working knowledge of fundamentals and to understand the general concepts of engineering. Robert D. Kersten, who has been the Dean of the Department of Engineering at the University of Florida for 20 years, has a Bachelors degree in Mathematics and Chemistry, a Masters degree in Civil Engineering, and a Ph.D. in Civil Engineering, Water Resource/Hydrologic Engineering. Dean Kersten has served in numerous capacities with both state and national professional associations involved in accreditation of engineers and served on the Board of Professional Engineers in Florida and on the National Council of Engineering Examiners. The FE exam is prepared by the National Council of Engineering Examiners and is designed to cover the fundamental areas essential to the basic practice of engineering. The FE exam tests both the common body of knowledge that is essential to practice in the profession and the ability to apply that knowledge. According to Dean Kersten the FE exam tests items which should be within an engineer's basic knowledge and which are necessary to communication between engineers in a design team approach to project design. Dean Kersten acknowledges that the FE exam is more difficult for applicants who lack a degree or who have been out of the academic area for a period of time, but opines that those factors do not excuse an applicant from mastering and retaining the basic fundamentals important to the practice. In fact, the FE exam is designed so that 70 percent of the applicants with-the 4- year college educational background pass the exam. Only 40 percent of the applicants in the 10 year cycle pass the exam.
