The Issue The issue is whether Petitioner is eligible to take the Principles and Practices Examination for licensure as a professional engineer.
Findings Of Fact In 1994, Petitioner passed the Secondary School Certificate Examination (a ten-year academic course) in India. Petitioner passed this high school course of study with classes in the core subjects of English, Sanskrit, Hindi, Mathematics, Science, and Social Sciences. In 1996, Petitioner passed the Higher Secondary School Certificate Examination in India. For this two-year high school course of study, Petitioner completed classes in English, Mathematics and Statistics, Physics, Chemistry, and Comprehensive Science. Petitioner completed his undergraduate degree in December 2001. He graduated from the Sardar Patel College of Engineering (SPCE), an affiliate of the University of Mumbai in Mumbai, India, with a Bachelor of Engineering Degree (Civil). The SPCE is accredited by the National Board of Accreditation of the All India Council for Technical Education (NBA-AICTE). At the time of Petitioner's graduation, the SPCE was not accredited by the Engineering Accreditation Commission of the Accreditation Board for Engineering and Technology, Inc. (ABET). For 75 years, ABET has accredited college and university programs in the United States in the following areas: (a) applied science; (b) computing; (c) engineering; and (d) technology. It is a federation of 28 professional and technical societies representing these fields. ABET accredits approximately 2,700 programs at over 550 colleges and universities nationwide. In April 2003, Petitioner passed the Engineer Intern Examination. Petitioner passed this eight-hour written examination in Ohio. In May 2003, Petitioner earned a Master of Science in Civil Engineering at the University of Toledo, Toledo, Ohio. Petitioner worked for a design engineer located in Vicksburg, Mississippi, from July 2003 to April 2004. Since May 2004, Petitioner has worked for an engineering firm located in Jacksonville, Florida. The Washington Accord, signed in 1989, is an international agreement among bodies responsible for accrediting engineering degree programs. It recognizes the substantial equivalency of programs accredited by signatories and recommends that graduates of programs accredited by any signatory be recognized by the other signatories as having met the academic requirements for entry to the practice of engineering. ABET, as a signatory of the Washington Accord, recognizes the substantial equivalency of foreign academic programs accredited by other signatory members; it does not accredit them. Further, ABET only recommends that graduates of programs from the signatories be recognized as substantially equivalent. Respondent does not follow the recommendations of ABET regarding the substantial equivalency of foreign academic programs in part because ABET and the other signatories of the Washington Accord recognize engineering technology degrees. Respondent has statutory authority to recognize engineering technology degrees only if the applicant was enrolled in a state university system prior to July 1, 1979. See § 471.013(1)(a)2., Fla. Stat. In 2007, the Washington Accord members granted provisional membership status to the NBA-AICTE. As a provisional member, the NBA-AICTE must demonstrate that the accreditation system for which it has responsibility, appears to be conceptually similar to those of the other signatories of the Washington Accord. By conferring provisional status, the signatories have indicated that they consider the provisional signatory to have the potential capability to reach full signatory status; however, the awarding of provisional status does not in any way imply a guarantee of the granting of full signatory status. April 2007, Petitioner applied to take the Principles and Practices Examination for licensure as a professional engineer. He specifically sought to be recognized as a civil engineer with proficiency in water resources. In order to show substantial equivalency pursuant to Florida Administrative Code Rule 61G15-20.007(1), Petitioner had his engineering degree from SPCE evaluated by Joseph Silny and Associates, Inc. (Silny). Respondent has approved Silny to conduct the substantial equivalency evaluations required by Florida Administrative Code Rule 61G15-20.007(3). Silny's evaluation showed that Petitioner's degree from SPCE lacked 13.59 semester credit hours of math and basic sciences, and 16 semester credit hours of humanities and social sciences. Silny concluded that Petitioner’s SPCE degree failed to meet the substantial equivalency requirements rule requirements. Petitioner submitted his transcript from the University of Toledo to Respondent for further evaluation. After reviewing the transcript, Respondent gave Petitioner credit for coursework in Numerical Analysis I and Numerical Analysis II, totaling six semester credit hours toward the math and basic science requirements. The credit reduced Petitioner's academic deficiency to 7.59 semester credit hours in math and basic science. During the hearing, Petitioner submitted transcripts and his secondary school certificates as evidence of coursework prior to his Bachelor of Science degree at SPCE. This coursework is not acceptable to meet the substantial equivalency rule requirements because they are college preparatory classes taken in high school for which Petitioner received no college credit. Many of Petitioner's high school courses cover subjects also taken in his undergraduate program, such as physics, chemistry, math, and statistics. Petitioner has already received credit for these courses that cannot be counted twice.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED: That Respondent enters a final order denying Petitioner's application to take the second part of the professional engineer examination. DONE AND ENTERED this 14th day of April, 2008, in Tallahassee, Leon County, Florida. S SUZANNE F. HOOD Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 14th day of April, 2008. COPIES FURNISHED: Rahul Parab 496 Monet Avenue Ponte Vedra, Florida 32081 Michael T. Flury, Esquire Office of the Attorney General The Capitol, Plaza Level 01 Tallahassee, Florida 32399-1050 Nancy S. Terrel, Hearing Officer Office of the General Counsel Department of Business and Professional Regulation Northwood Centre 1940 North Monroe Street Tallahassee, Florida 32399-0792 Paul J. Martin, Executive Director Patrick Creehan, Esquire Board of Professional Engineers Department of Business and Professional Regulation 2507 Callaway Road, Suite 200 Tallahassee, Florida 32303-5267 Ned Luczynski, General Counsel Department of Business and Professional Regulation Northwood Centre 1940 North Monroe Street Tallahassee, Florida 32399-0792
The Issue Whether Petitioner is entitled to additional credit for his responses to Questions 23 and 27 of the Contract Administration section of the General Contractor licensure examination administered in July 1996, and, if so, whether the additional credit would give him a passing grade. Whether Petitioner is entitled to additional credit for his responses to Questions 11, 23, and 35 of the Contract Administration section of the General Contractor licensure examination administered in April 1997 and, if so, whether the additional credit would give him a passing grade.
Findings Of Fact Petitioner took the Contract Administration section of the General Contractor’s licensure examination in July 1996 and in April 1997. Between the two exams, Petitioner passed all sections of the examination except the Contract Administration section. Petitioner’s score on the Contract Administration section of the July 1996 examination, as graded by Respondent’s Bureau of Testing, was 65. His score on the Contract Administration section of the April 1997 examination was 67.5. For both examinations, there were 40 questions on the Contract Administration section. A candidate had to achieve a score of 70 to pass that section of the examination. Because each question was equally weighted, a candidate would have to correctly answer 28 questions to earn the passing score. All questions challenged by Petitioner were multiple- choice questions where the candidate was instructed to give the best answer from four possible choices. Prior to the examinations, the candidates were given a list of approved reference materials. The candidates were permitted to refer to those reference materials while taking the examinations. Respondent’s score of 65 on the July 1996 examination was based on the Bureau of Testing’s determination that Petitioner correctly answered 26 of the 40 questions. To earn a passing grade on the Contract Administration section of the July 1996 examination, Petitioner would have to receive credit for correctly answering two additional questions. His score of 67.5 on the April 1997 was based on the determination that he correctly answered 27 of the 40 questions. To earn a passing grade on the Contract Administration section of the April 1997 examination, Petitioner would have to receive credit for correctly answering one additional question. QUESTION 23 OF THE JULY 1996 EXAM The correct answer for Question 23 of the July examination is choice “D.” Of the four possible responses, choice “D” is the best answer to the question. Petitioner’s answer to this question was choice “A.” Petitioner did not receive credit for his response to this question because he did not select the best answer. The answer selected by Petitioner would not be the most accurate and cost-effective because the methodology he selected would not detect errors made by the first person performing the computations. The challenged question is a question that a candidate for licensure should be able to answer. The challenged question is not beyond the scope of knowledge that a candidate for licensure should have. The challenged question is not ambiguous. Petitioner is not entitled to additional credit for his response to Question 23 of the July 1996 exam. QUESTION 27 OF THE JULY 1996 EXAM The correct answer for Question 27 of the July examination is choice “C.” This correct answer is supported by reference materials made available to all candidates. Petitioner’s answer to this question was choice “B.” Petitioner did not receive credit for his response to this question because he did not select the correct answer to the question. The challenged question is a question that a candidate for licensure should be able to answer. The challenged question is not beyond the scope of knowledge that a candidate for licensure should have. The challenged question is not ambiguous. Petitioner is not entitled to additional credit for his response to Question 27 of the July 1996 exam. QUESTION 11 OF THE APRIL 1997 EXAM The correct answer for Question 11 of the April 1997 examination is choice “C.” This correct answer is supported by reference materials made available to all candidates. Petitioner’s answer to this question was choice “D.” Petitioner did not receive credit for his response to this question because he did not select the correct answer to the question. The challenged question is a question that a candidate for licensure should be able to answer. The challenged question is not beyond the scope of knowledge that a candidate for licensure should have. The challenged question is not ambiguous. Petitioner is not entitled to additional credit for his response to Question 11 of the April 1997 exam. QUESTION 23 OF THE APRIL 1997 EXAM The best answer for Question 23 of the April 1997 examination is choice “C.” This correct answer is supported by reference materials made available to all candidates. Petitioner’s answer to this question was choice “A.” While there is some support in the reference material for Petitioner's answer, the greater weight of the evidence established that his choice was not the best answer. Petitioner did not receive credit for his response to this question because he did not select the best answer to the question. The challenged question is a question that a candidate for licensure should be able to answer. The challenged question is not beyond the scope of knowledge that a candidate for licensure should have. The challenged question is not ambiguous. Petitioner is not entitled to additional credit for his response to Question 23 of the April 1997 exam. QUESTION 35 OF THE APRIL 1997 EXAM The correct answer for Question 11 of the April 1997 examination is choice “C.” This correct answer is supported by reference materials made available to all candidates. Petitioner’s answer to this question was choice “D.” Petitioner did not receive credit for his response to this question because he did not select the correct answer to the question. The challenged question is a question that a candidate for licensure should be able to answer. The challenged question is not beyond the scope of knowledge that a candidate for licensure should have. The challenged question is not ambiguous. Petitioner is not entitled to additional credit for his response to Question 11 of the April 1997 exam.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that Respondent enter a Final Order that dismisses the challenges brought by Petitioner to Questions 23 and 27 on the July 1996 exam and to Questions 11, 23, and 35 of the April 1997 exam. DONE AND ENTERED this 3rd day of December, 1997, in Tallahassee, Leon County, Florida. CLAUDE B. ARRINGTON Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (904) 488-9675 SUNCOM 278-9675 Fax Filing (904) 921-6847 Filed with the Clerk of the Division of Administrative Hearings this 3rd day of December, 1997. COPIES FURNISHED: R. Beth Atchison, Esquire Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0750 Mr. Kenneth Marshall 624 Southwest 11th Court Fort Lauderdale, Florida 33315 John Preston Seiler, Esquire 2900 East Oakland Park Boulevard, No. 200 Fort Lauderdale, Florida 33306 Lynda L. Goodgame, General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792 Rodney Hurst, Executive Director Construction Industry Licensing Board 7960 Arlington Expressway, Suite 300 Jacksonville, Florida 32211-7467
The Issue Whether Petitioner is entitled to additional credit for his solutions to four problems on the Principles and Practice of Engineering portion of the engineering licensure examination administered on October 30, 1998, by the National Council of Examiners for Engineers and Surveyors.
Findings Of Fact Based upon the evidence adduced at hearing and the record as a whole, the following findings of fact are made: On October 30, 1998, as part of his effort to obtain a Florida engineering license, Petitioner sat for the Principles and Practice of Engineering Examination (Examination). This is a national examination developed and administered by the National Council of Examiners for Engineers and Surveyors (NCEES). Petitioner chose to be tested in civil engineering. Petitioner received a raw score of 45 on the Examination. For the civil engineering specialization, a raw score of 45 converts to a score of 67. To pass the Examination, a converted score of 70 is needed. Petitioner formally requested (in writing, by letter dated March 26, 1999) that his solutions to Problems 120, 125, and 222 on the Examination be rescored. Petitioner's written request was made to Natalie Lowe of the Board, who forwarded it to the NCEES. Appended to Petitioner's letter to Ms. Lowe were two pages of "scratch paper" on which Petitioner had written during his post-examination review on March 19, 1999. On the first page were written comments he had made regarding the scoring of Problems 120 and 125. On the second page were the following written comments he had made regarding the scoring of Problems 220 and 222: 220 a, b, & c 2 parts b & c correct. Min. mark I should get[:] At least 5 instead of 2 and maybe 7. There is an error. 222 ok The NCEES's rescoring of Petitioner's solutions to Problems 120, 125, and 222 resulted in his receiving a raw score of 43 (or a converted score of 65, 5 points less than he needed to pass the Examination). The Board received the NCEES's rescoring results on May 12, 1999. The Board subsequently referred the matter to the Division to conduct an administrative hearing. At the administrative hearing that was held pursuant to the Board's referral, Petitioner challenged the grading of his solutions to Problems 120, 125, and 220 of the Examination, and indicated that he had "no dispute concerning the grading of [his solution to Problem] 222," notwithstanding that he had requested, in his March 26, 1999, letter to Ms. Lowe, that his solution to Problem 222 be rescored. Petitioner explained that he had made this request as a result of inadvertence and that he had actually intended to seek rescoring of his solution to Problem 220, not Problem 222. Problems 120, 125, and 222 were worth ten raw points each. Problem 120 contained four subparts (or requirements). Petitioner initially received four raw points for his solution to Problem 120. Rescoring did not result in any change to this score. Petitioner solved two subparts of Problem 120 correctly (subparts (a) and (b)). The solutions to the other two subparts of Problem 120 (subparts (c) and (d)), however, were incorrect inasmuch as Petitioner had neglected, in making the lateral force calculations and drawing the diagrams required by these subparts, to include the force attributable to the movement of the groundwater referred to in the problem. Therefore, in accordance with the requirements and guidelines of the NCEES scoring plan for this problem, the highest raw score that he could have received for his solution to this problem was a four, which is the score he received. Problem 125 contained three subparts (or requirements). Petitioner initially received a raw score of two for his solution to Problem 125. Upon rescoring, no change was made this raw score. Petitioner correctly solved only one of the three subparts of Problem 125 (subpart (c)). In his solution to subpart (a) of Problem 125, Petitioner did not provide, as required by this subpart, the quantities of water, cement, and aggregate necessary for the project described in the problem. Petitioner's solution to subpart (b) did not describe one of the acceptable slump increasing methods that the candidates were required describe in their solution to this subpart. Accordingly, giving Petitioner a raw score of two for his solution to Problem 125 was consistent with the requirements and guidelines of the NCEES scoring plan for this problem. Petitioner received a raw score of two for his solution to Problem 220. He did not request, in his March 26, 1999, letter to Ms. Lowe, a rescoring of his solution to this problem, and, as a result, his solution was not rescored. At the administrative hearing, Petitioner testified on his own behalf regarding the scoring of this solution and, during his testimony, contended that the score he received was too low; however, neither a copy of the problem, nor a copy of the NCEES scoring plan for this problem, was offered into evidence. Accordingly, the record is insufficient to support a finding that the score Petitioner received for his solution to Problem 220 was undeservedly low in light of the NCEES scoring plan for this problem. Petitioner initially received a raw score of eight for his solution to Problem 220. Rescoring resulted in this score being reduced two points to a six. Petitioner did not present any evidence supporting the position (which he advances in his Proposed Recommended Order) that he should have received a higher score for his solution to this problem, and, consequently, Respondent's expert, in his testimony at hearing, did not address the matter. While there were exhibits offered (by Respondent) and received into evidence relating to the scoring of Petitioner's solution to Problem 222, it is not apparent from a review of these exhibits that such scoring deviated from the requirements of the NCEES scoring plan for this problem (which was received into evidence as part of Respondent's Exhibit 12).
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that a final order be entered rejecting Petitioner's challenge to the failing score he received from the NCEES on the Principles and Practice of Engineering portion of the October 30, 1998, engineering licensure examination. DONE AND ENTERED this 20th day of December, 1999, in Tallahassee, Leon County, Florida. STUART M. LERNER Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 20th day of December, 1999.
The Issue The issues in this case are whether certain questions within the June 2002 construction building contractor examination are invalid, and whether Petitioner should receive credit for certain answers scored as incorrect.
Findings Of Fact In June 2002, Petitioner sat for the construction building contractor examination. Shortly following the exam, Petitioner was advised that he incorrectly answered 17 of the 50 exam questions and did not attain the minimum passing score of 70 percent, but received a failing scaled score of 66 percent. Petitioner timely challenged the validity and scoring of eight questions, including questions 8, 14, 17, 33, 34, 38, 43, and 44. In order for Petitioner to acquire a passing score, Petitioner must prove that certain challenged questions are invalid or demonstrate that he is entitled to receive credit for his answers. Specifically, Petitioner must demonstrate that either three questions should be stricken from the exam providing Petitioner with 70.2 percent, two questions should be stricken and one answer scored as correct providing Petitioner with 70.8 percent or two answers should scored as correct providing Petitioner with 70 percent. QUESTION 8 Exam Question 8 asks, "According to AIA-A201, who determines the interest rate that the contractor can charge on due and unpaid payments?" Petitioner's expert, Mr. Uman, argues that the parties to the contract are not defined within the question and it is therefore misleading. However, the credited answer D, "all the parties must agree on the rate" is within the provided reference material and is clearly the best answer. It is not misleading and Petitioner's argument lacks merit. In addition, 89.47 percent of the test-takers correctly answered Question 8. QUESTION 14 Exam Question 14 is wordy and involves computations. It requires the test-taker to calculate the number of "labor" hours required per 100 pieces to build a wall, given certain pricing and wall construction information. Question 14 is ambiguous and confusing on its face. While the question asks for labor hours, the facts provide a fixed combined hourly cost for a mason and laborer's hour. There is no distinction made between "labor" hours and a "laborer's" hours. Mr. Collier admitted that there is some apparent confusion between "labor" costs and the "laborer's" costs. Mr. Palm further agreed and indicated that he fully understood Petitioner's rationale to divide the labor costs in half and choose answer A. Furthermore, it is clear that Petitioner's perception of the question was not unique. In fact, only 46.62 percent of the test-takers correctly answered Question 14. QUESTION 17 Exam Question 17 asks, "During the bid process, which document has priority in the event of conflicting information?" Clearly, the correct answer is B, "addenda." Petitioner's argument regarding "competitively bid projects" is without merit. Mr. Palm succinctly explained that Petitioner's selection was obviously incorrect because "plans don't change during the bid process unless there is an addenda issued." Moreover, 75.56 percent of the test-takers correctly answered Question 17. QUESTION 33 Exam Question 33 identifies a situation that where drawings differ from written specifications and where there is no legal precedent that one is more binding than the other. The question specifically calls for the best procedure according to the listed and available reference. While Mr. Uman argues that the answer does not appear within the reference material in a clear manner, the exact text of the question and answer are in fact within the material. Petitioner's argument lacks credibility. QUESTION 34 Exam Question 34 asks the test-taker "what is the EARLIEST workday for completing the masonry work?" given the number of crew, the number of hours required, and the ratio constant of the crew. Although 80.45 percent of the test-takers correctly answered Question 34, Mr. Uman argues that the question could have been answered without reference to the Builder's Guide to Accounting material and therefore, was misleading. Petitioner's argument is devoid of common sense. QUESTION 38 Exam Question 38 asks the test-taker to identify the activity that "a specialty structural contractor is qualified" to perform. Petitioner's expert, Mr. Uman, again argues that the question is misleading since the credited correct answer "perform non-structural work" is not written verbatim in the provided reference material. To the contrary however, all of the alternative choices are clearly listed in the reference material as activities specifically prohibited by specialty structure contractors. Furthermore, page 2B17 to 61G415.015 of the Contractor's Manual specifically states that: The specialty structure contractor whose services are limited shall not perform any work that alters the structural integrity of the building including but not limited to roof trusses. Respondent's experts, Mr. Collier and Mr. Palm, agree that Question 38 is clear. Moreover, 53.38 percent of test- takers correctly answered the question. While the question appears to require enhanced reasoning skills and is generally more difficult, it is not misleading. Petitioner's assertions are without merit. QUESTION 43 Exam Question 43 asks, "Which accounting method should be used by a contractor if the contractor is unable to reasonably estimate the amount of progress to date on a job or the total costs remaining to fulfill the contract?" Mr. Uman argues that the question is ambiguous and the reference material is "not terribly clear." He further alleges that when a contractor cannot estimate progress, the contractor cannot establish a "completed contract method," the credited correct answer. Respondent's experts disagree. While it is true that Mr. Palm agreed that all of the choices are accounting methods which is inconsistent with Mr. Collier's testimony, the reference material is clear. In fact, 58.65 percent of the test-takers correctly answered Question 43. Petitioner presented insufficient evidence that he should receive credit for his answer or that Question 43 should be invalidated. QUESTION 44 Exam Question 44 provides detailed information regarding a standard form contract and asks, "Based ONLY on the information given above, what is the amount of the current payment due?" In addition, however, as Mr. Uman points out, the standard form referred to in the problem was mistakenly misidentified as Form 201 instead of Form 702. While it is clear that the referenced form was mislabeled, the precise form number was incidental, unrelated to the question, and unnecessary to compute the answer. In fact, Mr. Palm explains that the problem was "just a mathematical exercise." According to Mr. Collier, the question was not misleading, and the incorrect reference was irrelevant. "It's simple math, industry knowledge." Furthermore, Petitioner's answer is clearly incorrect because "he failed to deduct the retainage." Finally, 54.89 percent of the test-takers correctly answered Question 44.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that a final order be entered invalidating only Question 14, re-computing Petitioner's examination score, and dismissing his challenge. DONE AND ENTERED this 1st day of October, 2003, in Tallahassee, Leon County, Florida. S WILLIAM R. PFEIFFER Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 1st day of October, 2003. COPIES FURNISHED: Nickolas Ekonomides, Esquire 791 Bayway Boulevard Clearwater, Florida 33767 Charles F. Tunnicliff, Esquire Department of Business and Professional Regulation 1940 North Monroe Street, Suite 60 Tallahassee, Florida 32399-2202 Nancy P. Campiglia, General Counsel Department of Business and Professional Regulation Northwood Centre 1940 North Monroe Street Tallahassee, Florida 32399-2202 Robert Crabill, Executive Director Construction Industry Licensing Board Department of Business and Professional Regulation Northwood Centre 1940 North Monroe Street Tallahassee, Florida 32399-0792
The Issue Whether Petitioner is entitled to additional credit for his responses to question numbers 21 and 24 of the Principles and Practice of Engineering Examination administered in April 1998.
Findings Of Fact Petitioner took the April 24, 1998 professional engineering licensing examination with an emphasis in civil engineering. A score of 70 is required to pass the test. Petitioner obtained a score of 69. In order to achieve a score of 70, Petitioner needs a raw score of 48. Therefore, Petitioner is in need of at least one additional raw score point. Petitioner is challenging question numbers 21 and 24. They are both multiple-choice questions and worth one point each. Exhibit 10 contains a diagram for the candidate's use in answering question numbers 21 and 24. Question 21 requires the examinee to calculate the percentage of wooded land on the diagram. The diagram contains a rectangle labeled "woodlot," and within the rectangle are three non-contiguous areas marked with schematics of trees. The Petitioner reduced the percentage of wooded area to conform to the portion of the area labeled "woodlot" marked with schematics of trees. In regard to question number 21, the Petitioner asserts that as a matter of convention, by failing to put the trees everywhere in the wooded lot, one may assume that there are trees only where there is a schematic of the trees. The Petitioner's challenge was rejected on the basis that the scorer opined that it is standard practice that drawings are only partially filled with details, and the most reasonable interpretation of the site plan drawings is that the woodlot fills the entire area enclosed by the rectangle. John Howath, a professional engineer, testified regarding accepted conventions in engineering drawings. In Howath's opinion the drawing on the examination used inconsistent methodologies and was confusing regarding whether all of the area designated by the label or "call out" of woodlot was in fact wooded. Both the Petitioner and Mr. Howath referred to drawings in the Civil Engineering Reference Manual which showed areas on drawings totally covered with visual indications of a particular material or condition. Peter Sushinsky, a professional engineer, testified as an expert for the Respondent. Mr. Sushinsky acknowledged the Petitioner's exhibits; however, Mr. Sushinsky noted that these were only a few examples of drawings that are available. Mr. Sushinsky referenced construction drawings he had seen in his practice with partial "cross-hatching" just like the diagram on the examination. In sum, Mr. Sushinsky's experience was that diagram might be totally or partially "cross-hatched." In Mr. Sushinsky's opinion it was not a bad diagram, only subject to a different interpretation by a minor group. Question number 24 asked the candidate to calculate the weir peak discharge from the catchment area using the rational formula. The Petitioner asserts the question is misleading and should read, "What is the peak discharge from the watershed?" The Petitioner bases his assertion on the ground that the "rational formula" is used to compute discharge from a watershed not a weir, as mandated by the question. The scorer did not address the Petitioner's concerns. The scorer stated, "It is clear from the item statement that the weir equation is not to be used." However, the questions ask the candidate to compute the weir discharge. Jennifer Jacobs, a professor of engineering, testified regarding the rationale formula that it was used to calculate watershed discharge and not weir discharge. All experts agreed that the rational formula is not used to compute weir discharge. The experts all agree that the question was confusing because the rational formula is not used to calculate the discharge from a weir. The Respondent's expert justifies the answer deemed correct on the basis that if one uses the rational formula and computes the watershed discharge, one of the answers provided is close to the result. The Respondent's expert calculated the watershed discharge as 230.6 cubic feet per second (cfs). The answer deemed correct was 232 cfs. The expert stated the weir attenuates flow. If the weir attenuates flow one would expect an answer less than 230.6 cfs., not an answer equal to or greater than 230.6 cfs. The amount of attenuation is based upon the physical features of the impoundment area and the mouth of the weir. Weir Attenuation varies. The only answers smaller than 230.6 are 200 or 32. Is the 232 cfs. answer wrong because it does not allow for attenuation by the weir? How much did the weir attenuate the flow? Under these facts, the question is capricious. The Respondent argues that the Petitioner didn't follow instructions while acknowledging that the "correct" answer is not the answer to the question that was asked.
Recommendation Based upon the foregoing Findings of Fact and Conclusions of Law set forth herein, it is RECOMMENDED: That the Respondent enter a final order awarding Petitioner two raw points and a passing score on the Principles and Practice of Engineering Examination. DONE AND ENTERED this 20th day of May, 1999, in Tallahassee, Leon County, Florida. STEPHEN F. DEAN Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 20th day of May, 1999. COPIES FURNISHED: Mark W. Nelson 720 Northwest 31st Avenue Gainesville, Florida 32609 Natalie A. Lowe, Esquire Board of Professional Engineers 1208 Hays Street Tallahassee, Florida 32301 Dennis Barton, Executive Director Board of Professional Engineers 1208 Hays Street Tallahassee, Florida 32301 William Woodyard, General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792
Findings Of Fact On May 23, 1975, the Petitioners herein took the general contractors licensing exam in Miami, Florida. Petitioners failed to achieve a passing score on said examination and thereafter several reviews of the examination questions and answers resulted. The exam in question was administered to approximately 659 examinees. The test was made up of 100 questions and the examinees were allotted 4 hours to complete the exam. The examinees were instructed in the examination booklet to answer as many questions as possible within the time limit and to always select the best possible answer out of the listed choices. The examinees were furnished a list of reference materials by Respondent and they were advised that the exam questions would come from some 18 odd reference books supplied on the reference lists. Petitioner Dennis Milch took the general contractor's examination administered on May 23, 1975. 2/ In preparation of the exam he took a prep course given by Cole Construction and began his preparation approximately two months prior to the exam. He received a score of 67.5 on the exam. He earned a degree from the University of South Florida and took advance construction courses at FIU where he earned a degree in marketing. His work experience consisted of serving as an apprentice carpenter for Burke Construction Company in Miami for approximately two months and as a contractor to build residential homes in Houston Texas. Milch voiced his opinion that the exam questions failed to satisfy the statutory requirements of being "objective" within the meaning of Chapter 468, Florida Statutes. Joseph Cole, the founder of Cole Construction College in 1949, testified that he had approximately 30 years experience in teaching construction, architectural and engineering courses. He had conducted various seminars for students and received a B.S. Degree from the University of Miami. He received a B.S. Degree in biochemistry and civil engineering from the University of Pittsburgh and conducted seminars at the University of Florida in Math, Physics and Engineering. He also conducted seminars at the Markowitz Engineering School. He was licensed in 1947 in Coral Gables, Miami Beach, and in Miami where he has built approximately 2,000 single-family homes, high-rise buildings and apartments. He aired his opinion that Milch missed approximately 36 questions of which approximately 24 were what he regarded as "impossible" questions. He expressed his awareness that during the morning sessions two questions were voided and credits were given to all examinees having a point value of 1 point each. Two questions were also voided from the afternoon session. Thomas H. Hebert, an associate of the testing agency which compiled the exam for the Board, i.e., Bryon, Harlow, Schaefer, Reed and Associates, explained the procedure for compiling the tests for the Board (Respondent). He stated that data is taken from Board references and an exam format is established. Examinees are tested on plan reading and estimating using standard plans for takeoff and specification requirements. The test is first administered to contractors and others who had previously passed the exam. This is done to test time limits, etc., and grading procedures. By so doing, he testified that it is possible to correct deficiencies in the exam. After the examination is compiled and is administered to the agency employees and other contractors who have passed the exam, an item analysis is compiled and computerized. There are five possible answers for each question. The exam is divided into three segments, i.e., the upper 27 percent, the mid 46 percent, and the lower 27 percent. After the tests results are in, the weighting on various questions are checked to see if large numbers of examinees "jump" one question and further to see if questions are ambiguous. If found to be or that there are two correct answers for a given question, credit is given for both answers. Thereafter, a discrimination index is compiled based on the lower and upper 27 percent. These papers are scrutinized and if there is a discrepancy in excess of .1 to .8 percent, the question is examined and a solution is derived at based on results gathered from the scrutiny. He testified further that if an exam paper is mutilated or is otherwise difficult to machine score, it is hand graded. All exam papers in which the score ranged from 0 to 30 are hand graded as are those where the score ranges between 60 to 70. Of those questions where there is no correct answer, the question is deleted and a new base is established. For example, if a question is deemed faulty, each question has a weight of 1.1 one hundredths of a point. If they have three correct answers, points are given for all three answers. It was further brought out during his testimony that it was not necessary for a contractor to pass the certification examination in order to practice contracting in Florida. Evidence reveals that there are two kinds of licenses issued by the Board, i.e., registration and certification. The registration process only requires compliance with local requirements and the filing of a form with the Board, which may be the passing of a local competency exam or simply obtaining a local occupational license. The Certification method is optional and if the contractor passes the certification examination, it is unnecessary for him to take any local examinations. After going over various questions missed by Petitioner Dennis Milch, Petitioners argue that the scope of the Board Certification Examination included questions affecting the business of contracting as well as the technical aspects of for example how to nail two boards together to make a safe structure. Florida Statutes Chapter 468.106(2)(a) provides for an examination covering knowledge of basic principles of contracting and construction. Chapter 468.101 declares the purpose of Chapter 468 and states in pertinent part that "any person desiring to obtain a certificate to engage in the business shall be required to establish his competency and qualifications." Hence, the legislature has covered the business of contracting as well as the theory of construction. This serves the purpose of Chapter 468 by making it safer for owners to contract with the contractors and to have assurances that no liens will be placed on their property by subcontractors, that the owners are safe from suit from work that was done on the job, that the payments made on the construction will not be diverted and that the contractor understands his obligations. This requires general knowledge of the mechanics lien law, basic contract law and workers compensation law, all of which were tested by the subject examination. Respecting Petitioners' argument that they were denied certain constitutional guarantees when they were instructed by Respondent to select the best possible answer but that after the test was administered and Respondent determined that many questions had no best choice, the Board failed to delete such questions from the exam, it was noted that after the Respondent discovered that several test questions were deemed acceptable but that the answers offered did not meet the tests of selecting the best possible answer, adjustments were made. In other words, there was no single best possible answer for approximately four questions. Rather than deleting the entire question, Respondent permitted those examinees who selected either the answer originally preferred by the Board or one of the later adopted alternate answers to achieve full credit for such questions and answers. The statute (Chapter 468, F.S.) mandates that the examination be an "objective" written examination. The criteria of objectivity is not met where the examining body is granted the discretion to accept alternate answers to a given question. A "best" answer is something different from an acceptable answer. To give the Board discretion to accept alternate answers would authorize a substitution of standards which is' not permitted by Chapter 468, F.S. Once subjectivity comes into play, Respondent becomes vested with almost unbridled discretion in deciding who shall become a certified general contractor. This was prohibited by the legislature by requiring objectivity in setting a uniform minimum test grade. As relates to Petitioner Milch, it was noted that a subsequent review of his exam resulted in his being awarded a half credit for his answer to question number 39 and the Board determined that after review answers B and A were both correct. The net result of this was that his overall score was 68. A review of the court cases revealed that Florida courts have not been involved in the minute details of how examination grades or points are awarded. See the cases of State ex rel. Topp v. Board of Electrical Examiners, 101 So.2d 583 (Fla. App. 1st 1968), and State ex real. Lane v. Dade County 258 So.2d (347 Fla. App. 3rd, 1972). These cases generally show that unless there is a clear abuse of discretion, courts shall not substitute their judgment for the agencies as to how examinations are graded. Petitioners also submit that once regrading had commenced, the Board should have deleted all questions with wrong answers or more than one equally acceptable answer, distributed the weight of the deleted questions in proportionate fashion of the remaining questions and considered passing to be 70 percent of the total points available, rather than 70 cumulative points. During the hearing, Petitioners failed to show how they were injured by the difference in the award of the points for questions deleted. A wrong without damage does not constitute a good cause of action. Based on the evidence presented it appears that the Petitioners are treated the same as all other examinees. Since the Petitioners have failed to establish that if the assignment of points were different, they would have passed the examination, this argument is moot. Petitioners also alleges that they were denied certain constitutional protections by Respondent's failure to adopt and promulgate uniform rules and regulations concerning preparation, administration and review of licensing examinations. Florida Statutes, Chapter 468, requires Respondent to conduct its affairs pursuant to Florida Statutes Chapter 120. The Administrative Procedure Acts set out specific procedures to be followed by State agencies in adopting, promulgating and enforcing rules. Statutory authority governing the granting of a license should be strictly followed. In this case, there is no evidence of the existence of any unlawful rule or regulation adopted by Respondent to govern any of the variety of issues concerning the licensing of general contractors. Petitioners also submit that the Board should be required to promulgate and enforce rules concerning examinations and appeals or results thereof and cite the reasons for the actions it takes prior to its review of the examination. The Board is not required to adopt rules and regulations in every area in which it is authorized to act by statute. Where the statute is clear, there is no requirement or reason by the Board to adopt rules. Here, the Board provided the applicants with a chance to examine the questions, their papers, grades and to complain if they wished about the questions either individually or at board meetings with the possibility that the fairness of the questions could be resolved quickly and informally and if necessary, as in this case, without the full ponoply of an administrative hearing. By so doing, the Board was clearly following its statutory duty to provide the applicants with a chance to see their examination papers and grades. (F.S. Chapter 466.110). Based on the above, it is concluded that Respondent compiled the May 23, 1975 examination based on objective standards. When the Board determined that certain questions were defective either because there was more than one answer or for other reasons, the Board reviewed said questions and credited those examinees who failed to properly answer the question. By so doing, Petitioners were treated the same as all examinees who took the exam. Based on the record evidence, it further appears that all the questions meet the statutory tests of being objective and the Board's determination that a cumulative score of 70 percent is necessary to successfully obtain a certification, was not shown by any competent or substantial evidence to be an abuse of discretion. It is therefore recommended that the agency's action be sustained and that the petition filed herein be dismissed.
Recommendation Based on the foregoing findings of fact and conclusions of law, it is therefore recommended that the agency's actions be affirmed and the petition filed herein be DISMISSED. DONE and ENTERED this 17th day of January, 1977, in Tallahassee, Florida. JAMES E. BRADWELL, Hearing Officer Division of Administrative Hearings Room 530, Carlton Building Tallahassee, Florida 32304 (904) 488-9675
The Issue Whether Petitioner is entitled to additional credit for his solutions to three problems on the Principles and Practice of Engineering portion of the engineering licensure examination administered on October 29, 1999, by the National Council of Examiners for Engineers and Surveyors.
Findings Of Fact Based upon the evidence adduced at hearing and the record as a whole, the following findings of fact are made: On October 29, 1999, as part of his effort to obtain a Florida engineering license, Petitioner sat for the Principles and Practice of Engineering Examination (Examination). This is a national examination developed and administered by the National Council of Examiners for Engineers and Surveyors (NCEES). Petitioner chose to be tested in mechanical engineering. Petitioner received a raw score of 47 on the Examination. For the mechanical engineering specialization, a raw score of 47 converts to a score of 69. To pass the Examination, a converted score of 70 is needed. Petitioner formally requested (in writing, by letter dated March 13, 2000) that his solutions to Problems 141, 144, and 147 on the Examination be rescored. Petitioner's written request was made to the Board's "Legal Section," which forwarded it to the NCEES. The NCEES's rescoring of Petitioner's solutions to Problems 141, 144, and 147 resulted in his receiving no additional points. The Board received the NCEES's rescoring results on or about April 25, 1999. After receiving a letter from Petitioner (dated May 3, 2000) requesting a "formal hearing," the Board referred the matter to the Division. Problems 141, 144, and 147 were worth ten raw points each. Petitioner received four raw points for his solution to Problem 141. In his solution to Problem 141, Petitioner failed to take into consideration bending stresses and loads. Therefore, in accordance with the requirements and guidelines of the NCEES scoring plan for this problem, the highest raw score that he could have received for his solution to this problem was a four, which is the score he received. Petitioner received a raw score of two for his solution to Problem 144. In rescoring Petitioner's solution to this problem, the NCEES rescorer made the following "comments": A correct solution [to this problem] must include an energy balance on the open feedwater heater to determine the fraction of flow through turbine T1 that is extracted and taken to the open feedwater heater. a correct equation for determining the specific work developed by the two turbines on the basis of one pound entering turbine T1. The equation the examinee has written assumes the same flow through both turbines. determination of the mass rate of flow (m1) at the inlet to turbine T1. This is determined by dividing the net power by the specific net work. determining the rate at which heat is added in the steam generator and reheater. finally, dividing the rate at which heat is added in the steam generator by the heating value times 0.75 with the appropriate conversion factors. The examinee has used the new power (200 MW or 200 x 105)as the rate at which heat is added in the steam generator and reheater. This is incorrect. The scoring plan states 2 RUDIMENTARY KNOWLEDGE . . . OR-(3) determines tons/day = Wnet/7650, Wnet = (h1 - h2) + (h3 - h4) This is what the examinee has done. Based on the scoring plan and the above analysis, a score of 2 is recommended. There has been no showing that the foregoing "analysis" was in any way flawed or that application of the requirements and guidelines of the NCEES scoring plan for this problem should have resulted in Petitioner receiving a raw score higher than two for his solution to Problem 144. Petitioner received a raw score of four for his solution to Problem 147. In rescoring Petitioner's solution to this problem, the NCEES rescorer made the following "comments": The examinee used an incorrect temperature difference in [his] calculation of the heat transferred by convection and radiation from the outer surface of the pipe. Most of the examinee's work for requirement (b) was not needed. In doing that unnecessary work, however, [he] made two significant errors: 1. [He] evaluated a radiation exchange between the steam inside the pipe and the environment surrounding the pipe. The pipe shields the environment surrounding the pipe from the steam. 2. The examinee's equation "Total heat Loss = Conductive + Radiation" is not satisfactory. In attempting to evaluate the heat transfer from the insul[a]ted pipe, [he] assumed that the outer surface heat transfer coefficient was very high; 3.0 is not high. The examinee made no attempt to evaluate the payback period for the insulation. There has been no showing that the foregoing analysis was in any way flawed. For the errors made by Petitioner in his solution to Problem 147, a 50% "grade reduction" was warranted pursuant to the "error analysis" portion of the NCEES scoring plan for this problem. 1/ The remaining portions of the scoring plan for Problem 147 provided as follows: 10: Essentially complete and correct solution. May have one or two minor math, data, or chart reading errors. . . . Grade of 8: A grade of 8 will result from having any combination of the above listed errors which causes a grade reduction between 10% and 50%. A Grade of 6: A grade of 6 will result from having any combination of the above listed errors which causes a grade reduction between 30% and 50%. Grade of 4: 2/ A grade of 4 will result from having any combination of the above listed errors which causes a grade reduction between 50% and 70%. Grade of 2: A grade of 2 will result from having any combination of the above listed errors which causes a grade reduction between 70% and 90%. Grade of Zero: Nothing presented that warrants a grade of at least 10%. It is unclear from a reading of the NCEES scoring plan for Problem 147 whether a grade reduction of 50% should result in a raw score of four or six. The plan is ambiguous in this regard. While it may be reasonable to interpret the plan as requiring that a raw score of six be given where there is a grade reduction of 50%, the plan is also reasonably susceptible to the interpretation that a 50% grade reduction should result in a raw score of four, the score Petitioner received for his solution to Problem 147. It therefore cannot be said that the scoring of his solution to this problem was inconsistent with the problem's scoring plan, as reasonably construed.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that a final order be entered rejecting Petitioner's challenge to the failing score he received from the NCEES on the Principles and Practice of Engineering portion of the October 29, 1999, engineering licensure examination. DONE AND ENTERED this 13th day of October, 2000, in Tallahassee, Leon County, Florida. STUART M. LERNER Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 13th day of October, 2000.
Findings Of Fact Based on the stipulations and admissions of the parties, the exhibits received in evidence, and the testimony of the witnesses at the hearing in this case, I make the following findings of fact. Findings based on stipulations and admissions of the parties The Petitioners, John Eugene Harden and Dova Cauthen, qualified for and were administered the January 1982 Electrical Contractors' Licensing Examination. After their examinations were graded and regraded, Petitioners were notified that they had not received a "Passing" score of 75 or more. The Petitioners, John Eugene Harden and Dova Cauthen, qualified for and were administered the July 1982 Electrical Contractors' Licensing Examination. After their examinations were graded and regraded, Petitioners were notified that they had not received a "Passing" score of 75 or more. Both Petitioners reside in Dade County, Florida. The Respondent Department of Professional Regulation (hereafter DPR or the "Department") is an agency of the State of Florida. The Respondent Electrical Contractors' Licensing Board (hereafter referred to as the "Board") is an agency of the State of Florida statutorily responsible, along with DPR, for licensing certified electrical contractors. The Office of the Executive Director of the Board is located at 130 North Monroe Street, Tallahassee, Florida. The business address of Petitioner Harden is Harden Electric, 311 N.E. 8th Street, Homestead, Florida 33030. The business address of Petitioner Cauthen is 959 N.E. 79th Street, Miami, Florida 33138. 8. On March 20, 1981, Rules 21GG-6.01 (2), 21GG-6.01 (3), 21GG-6.01 (4), of the Rules of the Electrical Contractors' Licensing Board, were amended. Rule 21GG-6.01 (2) altered the format of the licensing examination so that the formerly entirely open book examination became a part open book, part closed book examination. Rule 21GG-6.01 (4) raised the passing grade on the licensing examination from 70 to 75. Rule 21GG-6.01 (3) altered the technical format. On December 3, 1981, Susan Tully, counsel to the Electrical Contractors' Licensing Board, caused a change to be made in Rule 21GG-6.01 (1) (c) of the Rules of the Electrical Contractors' Licensing Board. This change was made without formal notice or informal notice to the public and without a vote of the Electrical Contractors' Licensing Board, although the change was discussed at a Board meeting. None of the procedures set forth in Section 120.54 of the Florida Statutes were followed in amending this Rule. Liz Cloud was the Bureau Chief of the Division of Elections, Bureau of Administrative Code, Department of State. The address of the Bureau is Room 1802, The Capitol Building, Tallahassee, Florida. The change in Rule 21GG-6.01 (1)(c) eliminated parts 72 A, B, C and D and indicated that the entire Fire Safety Code (and not just the aforementioned parts) would be a subject of the Electrical Contractors' Licensing Examinations. In fact, the reason for the alleged "technical change" was that the Board intended and tested examinees in the January and July 1982 Licensing Examinations on materials in parts of the Fire Safety Code in addition to those contained in 72 A, B, C and D of the Fire Safety Code. Petitioner Harden specifically requested to review his January and July 1982 Electrical Contractors' Licensing Examinations. On March 25, 1982, Petitioner Harden went to Tallahassee, Florida to review his January 1982 Licensing Examination papers, but although he requested an "examination review" he was never provided with his own examination booklet or a copy thereof. During this alleged "examination review" Mr. Harden wrote written objections to the January 1982 examination on the forms provided to him based on the master copy of the examination provided to him. On August 31, 1982, Petitioner Harden went to Tallahassee, Florida to review his July 1982 Licensing Examination papers, but although he requested an "examination review" he was never provided with his own examination booklet or a copy thereof. During this alleged "examination review" Mr. Harden wrote written objections to the July 1982 examination on the forms provided to him based on the master copy of the examination provided to him. Petitioner Cauthen specifically requested an "examination review" with respect to her January and July 1982 Licensing Examination papers. On March 17, 1982, Petitioner Cauthen went to Tallahassee, Florida to review her January 1982 Licensing Exami-nation papers, but although she requested an "examination review" she was never provided with her own examination booklet or a copy thereof. During this alleged "examination review" Ms. Cauthen wrote written objections to the January 1982 examination on the forms provided to her based on the master copy of the examination provided to her. On August, 19, 1982 Petitioner Cauthen went to Tallahassee, Florida to review her July 1982 Licensing Examination papers, but although she requested an "examination review" she was never provided with her own examination booklet or a copy thereof. During this alleged "examination review" Ms. Cauthen wrote written objections to the July 1982 examination on the forms provided to her based on the master copy of the examination provided to her. Petitioners Harden and Cauthen sought Board review of their January and July 1982 Electrical Contractors' Licensing Examinations. Petitioners were advised in July 1983 that no copies of their actual booklets exist; Petitioners were advised in October 1983 that their actual booklets were shredded. At the November 8, 1982, meeting of the Board, Ms. Ida Cameron representing DPR, presented to the Board a package of information (assembled in package form) for the Board to consider during the examination review. With regard to Question Number 71 on the afternoon portion of the July 1982 Licensing Examination, two of the four possible responses, "A" and "C" were credited. Candidates like Petitioner Cauthen who answered "B" received no credit for the question. Petitioners Harden and Cauthen have requested Chapter 120.57 hearings with respect to their January and July 1982 licensing examinations and the review, grading and agency action with respect to same. Although request for production was specifically made for said packages of information with respect to the January 1982 and July 1982 licensing examinations in February, 1983, no packages have been produced to date. DPR destroyed the Petitioners' examination booklets before the end of the two year period immediately following each of the 1982 examinations. That Section 455.217 of the Florida Statutes requires the Board "by rule" to designate areas of competency to be covered by each licensing examination. That Section 455.217 states that the Board shall "by rule specify the general areas of competency to be covered by each examination, the relative weight to be assigned in grading each area tested, and the score necessary to achieve a passing grade." That the amendment of Rule 21GG-6.01 (2) required candidates to commit to memory portions of the electrical code, accounting, law, worker's compensation rules, federal employer's tax guide, A1A General Conditions, business practices; legal and insurance requirements. In the July 1982 Electrical Contractors' Licensing Examination candidates were given one hour more time in which to take the examination than was given during the January 1982 examination. Rule 21GG-6.01 (4) was amended in conjunction with Rules 21GG-6.01 (1)(a), (2) and (3) at a board hearing in March, 1981. Rule 21GG-6.01 (4) raised the passing score on the January 1982 Electrical Contractors' Examination and the July 1982 Electrical Contractors' Examination from 70 to 75. The amendment to Rule 21GG-6.01 (4) required that electrical contractors pass the certification examination with a score of 75 percent, whereas all other construction industry licensing board contractors licensed by the Department of Professional Regulation (13 other categories) were merely required to achieve a 70 percent pass score. That Section 455.217 of the Florida Statutes provides that "The board shall make rules providing for reexamination of any applicants who have failed the examination." That DPR did not and has never provided Petitioners, their attorneys or agents with their own examination booklets for the January and July 1982 Electrical Contractors' Licensing Examinations. DPR has provided Petitioners only with copies of "master" examinations for their review. That Section 455.217 of the Florida Statutes provides that the Board shall make available an examination review procedure for applicants. That the Respondents produced Notice of Destruction of Examination Booklets and Other Examination Materials dated April 14, 1982, which allegedly evidences destruction of Petitioners' examination booklets for the January 7, 1982, Examination on April 14, 1982. That the Respondents produced Notice of Destruction of Examination Booklets and Other Examination Materials dated October 15, 1982, which allegedly evidences destruction of Petitioners' examination booklets for the July 13, 1982, Examination on October 15, 1982. The findings in paragraphs 1 through 33, immediately above, are based directly on the stipulations of the parties, most of which stipulations were memorialized at the beginning of the hearing. The findings in the following paragraphs are based primarily on testimony and exhibits, but some of them are also based in whole or in part on stipulations. In the findings which follow there are certain to be at least some repetitious findings in the course of putting matters into context and making additional findings which are related to some of the stipulated findings. I have tried to avoid all unnecessary repetition, but a certain amount is necessary for clarity and a certain amount is unavoidable due to the sheer size of the task at hand. Findings on background matters and on matters relating to more than one rule Both of the Petitioners in this rule challenge proceeding are individuals who have applied to the Electrical Contractors' Licensing Board for licensure as certified electrical contractors. Both of them have been approved to sit for the licensure examination. Petitioner Harden took the Board's licensure examination on each of the following occasions: July 1981, January 1982, July 1982, and January 1983. The Board has not given him a passing grade on any of those' four examinations. Petitioner Cauthen took the Board's licensure examination on two occasions: January 1982 and July 1982. The Board has not given her a passing grade on either of those two examinations. The grade notifications received by these Petitioners show, inter alia, that Petitioner Cauthen received a grade of 73 on the July 1982 examination. Both of these Petitioners have presently pending formal proceedings under Section 120.57(1), Florida Statutes, in which they are challenging various matters related to the preparation of, administration of, scoring of, and inherent validity of the Board's January 1982 and July 1982 licensure examinations. The Electrical Contractors' Licensing Board certification examination is different from a master electrician examination. However, the two examinations are in many ways similar because of the overlap in the nature of the subject matter to be tested on both examinations. The passing score or cut score on the local master electrician examination administered by Dade County is 70 percent. Most of the questions on that examination are about the National Electric Code. Part of the Dade County master electrician exam is closed book. Prior to 1972, persons wishing to engage in electrical contracting in the state of Florida were required to be licensed by the local governments in the areas in which they sought to operate. Since 1972, persons wishing to engage in electrical contracting in the state of Florida must be licensed by a unit of local government or by the Electrical Contractors' Licensing Board. Persons who are licensed by the Electrical Contractors' Licensing Board are known as certified electrical contractors. A certified electrical contractor can engage in electrical contracting anywhere in the state of Florida without local licensure. Persons who are licensed by one or more local governments (typically a municipality or a county) are known as registered electrical contractors. Registered electrical con-tractors are licensed to engage in electrical contracting only in the geographic areas encompassed by the boundaries of the local government entities that issued their local licenses. There is, however, a certain amount of reciprocity from one local government to another. Certification as a state certified electrical contractor does not authorize the electrical contractor to work as an electrician, although a certified electrical contractor can pull permits. A person does not have to be a licensed electrician in order to become a state certified electrical contractor, although many electrical contractors are also licensed as journeyman or master electricians. Candidates for the electrical contractor licensure examination have a great $ variety in the nature and scope of their background and experience. This variety in background and experience is among the reasons which cause testing for minimum competence as an electrical contractor not to be an exact science. As between certification and registration, certification by the Electrical Contractors' Licensing Board is not a major factor in the ability to compete for business. A person who has a statewide certification can save some money by avoiding the expense of obtaining local competency cards, but as a general rule a registered electrical contractor can work just about anywhere in the state due to reciprocity. Especially, a person who has passed a Block master electrician examination or a Block local electrical contractor's examination can work just about anywhere in the state because most counties in Florida accept the Block examination. Registered electrical contractors regularly compete for business with certified electrical contractors. For example, Petitioner Cauthen's company is presently licensed in Monroe, Dade, and Broward counties, where it does a considerable amount of business. Petitioner Cauthen's company bids on lots of contracts and submits bids in competition with both registered and certified electrical contractors. Thus, the company is already in competition with certified electrical contractors. Similarly Board Member Isaacs, whose certified company does 90 percent of its work in Duval County, has regular competition from registered electrical contractors. Board Member Isaacs competes with approximately 250 electrical contractors. Of that 250, approximately 175 are registered and the others are certified. A much bigger factor in competition than the registration versus certification issue is the cost of doing business in more than one area or the cost of doing business at a location that is distant from one's primary base of operations. Only about fifty electrical contracting firms regularly compete for business over the entire state of Florida. Accordingly, a restriction on the number of persons licensed by the Board as certified electrical contractors would have minimal, if any, limitation on the competition faced by those members of the Board who are certified electrical contractors. The Electrical Contractors' Licensing Board admin-istered its own certification licensure examination from 1972 through 1980. During that period of time the Board administered the examination a total of twenty-two times. During those twenty-two sessions of Board administered examinations, a total of 824 candidates sat for the examination, of which 392 were successful. Although the percentage of candidates who were successful on a particular Board administered examination ranged from a low of 20.7 percent to a high of 78.8 percent, the average passing rate of all candidates on all twenty- two of the Board administered examinations was 47.57 percent. (By way of comparison, it is interesting to note that the percentage of candidates who are successful on the Block master electrician examination is between 30 percent and 35 percent of those who take the examination.) Further, on fifteen of the twenty-two occasions on which the Board administered the examination, the percentage of candidates who were successful was 50 percent or less. In 1979, the regulation of professions and occupations was reorganized. The Department of Professional Regulation (the Department) was created as an umbrella agency over numerous boards, including the Electrical Contractors' Licensing Board and the Construction Industry Licensing Board. The responsibility for examining applicants, which in the case of the Electrical Contractors' Licensing Board had been handled by the Board members themselves, was taken over by the Office of Examination Services (O.E.S.). Toward the end of 1980 and the beginning of 1981, O.E.S. did not like what the Board was doing and the Board did not like what O.E.S. was doing. There was a general lack of trust between the two entities, there was poor communication between the two entities, and the relationship between them at that time might best be described as estranged. Although both entities made efforts at cooperation with the other, such efforts were not always effective or well received. As a result of the poor communication between the two entities, certain personal concerns over Board actions by individuals within the O.E.S. were never communicated to the Board (and therefore could not be answered. 4/ The first examination administered by the O.E.S. was in January 1981. Of the fifty candidates who took the January 1981 examination, forty-six were successful. This was a passing rate of 92 percent. In view of the Board's experience when it was administering its own examination, the Board was understandably surprised and concerned when the percentage of candidates passing the first O.E.S. administered examination was almost twice the average passing percentage on the Board administered examinations. The Board had no evidence that the overall qualifications or capabilities of the applicants examined by O.E.S. at the January 1981 examination were any higher than the capabilities of those examined previously. To the contrary, a review of their qualifications based on the information in their applications indicated that their qualifications were substantially the same as those of previous candidates tested by the Board. Also, the admission standards for the January 1981 examination were the same as those for prior examinations. The Board concerns about the O.E.S. administered January 1981 examination included the following matters, among others, memorialized at a Board meeting discussing the examinations: The candidates had been allowed to take any reference material desired into the exami- nation. Previously the Board had allowed only reference books listed as part of the application form. There was a very high pass rate in compar- ison to all past examinations. There appeared to be too few calculation questions. The questions had been placed on the paper in sequence with the reference book materials. There was a possibility that notes were taken into the examination which would have given advantage to the candidates. The Board's Examination Committee had been under the assumption that the entire examination would be presented for review of the questions. Instead the only questions presented were ones questionable under Department criteria for measuring competency. It was explained, from past experience, that persons conducting the seminars for exams had obtained the entire exam content from persons taking the examination in the past. It had been done with a camera which was taken into the examination. The Board, when administer- ing the examination, had been very strict on what was used in performing the examination. In the last exam the security was very poor. The Board felt there were so few calculation questions on the O.E.S. administered examination in January of 1981 that the examination was not a proper or sufficient examination. This was due at least in part to the fact that O.E.S. had difficulty covering all areas of the examination with the desired number of questions because the O.E.S. bank of questions was very limited at that time. Previous Board examinations were open book with certain reference material allowed. When the Office of Examination Services took over the administration of the January 1981 exams, it sent to the candidates a form letter adapted from the Construction Industry Licensing Board. This form letter instructed the candidates that they would be able to bring into the examination any notes and other materials desired. This was contrary to the past practice of the Board and contrary to the Board's wishes. The Board was very frustrated and concerned about the circumstances which led up to the January 1981 examination because the O.E.S. had refused to allow the Board to have any input into the examination and the Board thought the O.E.S. had come up with an apprentice level examination. In the Board's opinion, the January 1981 examination prepared by O.E.S. clearly tested at a level less than that of minimum competency. And, although O.E.S. did not agree with the Board that the January examination was invalid, O.E.S. did agree that the examination was a lot easier than they had thought it would be. The Electrical Contractors' Licensing Board did not conduct any formal studies or formal analysis prior to making the 1981 amendments to Rule 21GG-6.01 changing the format of the examination and the cut score. However, the Board's Rules Committee did a great deal of work on the matter and the matter was discussed extensively at Board meetings. In this regard it is important to keep in mind that the Board members (with the exception of the two lay members) were all experienced practicing electrical contractors who were familiar with the requirements of day-to-day electrical contracting. The Board's decision to change part of the examination to closed book was due to a concern that people with no electrical knowledge could become licensed if the examination was all open book and had a low percentage of technical questions. The Board felt that more than half of the examination should be technical questions. The closed book National Electrical Code questions were for the purpose of testing what a person with experience in the field of electrical contracting should know about everyday matters. The primary, if not the sole, motivation for the Board's 1981 amendments to Rule 21GG-6.01 was the Board's concern about the quality of the O.E.S. administered examination and the Board's feeling that, due to its estranged relationship with O.E.S., the only vehicle through which it could effectively influence the quality of future examinations was through rule- making. Those amendments were not motivated by any desire on the part of the Board to restrict competition in the field of electrical contracting. During the past few years the job requirements of an electrical contractor have remained substantially the same. Block and Associates (hereinafter "Block") is a company based in Gainesville, Florida, that writes licensure examinations for contractors in various trades. Block writes such examinations for cities, counties, and states. Block has prepared electrical contracting licensure examinations for the states of Florida, Georgia, and Oklahoma, as well as for St. Johns County, Florida. Block prepares licensure examinations for just about anything that involves electricity, including such things as electrical contracting, electrical journeyman, electrical-master, low voltage, elevator, burglar alarm and fire alarm Block prepares master electrician examinations for over one hundred governmental entities. The Block electrical master exams and the Block electrical contractor exams use some of the same questions. Subsequent to the 1982 examinations which form the gravamen of these Petitioners' related proceedings under Section 120.57, Florida Statutes, the ECLB has contracted with Block and Associates to prepare the Board's certification licensure examination. The Board has never said anything to Block about wanting to achieve any particular passing rate or wanting to have any particular percentage of the candidates taking the examination achieve a passing score. The Board has never interfered with Block's autonomy in the preparation of the electrical contractor exam. All of Block's electrical contractor examinations include business questions. It is important to test for business skills because an electrical contractor who gets into business trouble may start cutting corners and cutting corners is a hazard to the public. Some of Block's electrical contractor examinations also have specific sections on safety. Block always does a post-administration statistical analysis of the examinations questions it uses. The reason for this is to find out if anything is wrong with the questions; in other words, to determine whether the questions are valid. The computer program used by Block to check the validity of its tests was prepared for Block by a professor in the College of Education at the University of Florida. Block uses a computer to conduct a validity analysis of each test it administers. Findings regarding Rule 21GG- 6.01(1)(c) of the Electrical Contractors' Licensing Board Rule 21GG-6.01(1)(c) of the Electrical Contractors' Licensing Board reads as follows (with the portion challenged by the Petitioners underscored): The areas of competency to be covered by the certification examination shall be as follows: (a) *** (c) Safety, which shall include but not be limited to, OSHA regulations, life safety codes, and the Fire Safety Code (NFPA). Prior to the December 1981 amendment which is challenged here, Rule 2100-6.01(1)(c) read as follows: The areas of competency to be covered by the certification examination shall be as follows: (a) *** (c) Safety, which shall include but not be limited to, OSHA regulations, life safety codes, and the fire safety code (NFPA 72A, B, C and D). The Board's purpose in amending the rule to delete the reference to "72A, B, C and D" was to make it clear to candidates for examination that the Board' intended to include in the safety portion of its examinations questions from portions of Volume Seven of the NFPA other than Chapters 72 A, B, C, and D of that volume. Volume Seven of the NFPA contains quite a bit of material in addition to the material included in Chapters 72 A, B, C, and D of that volume. In view of the modifying phrase "shall include but not be limited to," the amendment to delete the reference to Chapters 72 A, B, C and D was not a necessary prerequisite to the use of examination questions based on other portions of Volume Seven of the NFPA. The Board could ask the same examination questions before and after the amendment. Several weeks before each examination, the Department of Professional Regulation sent all candidates for examination, including these Petitioners, a notice to appear for the examination. The notice to appear included a list of reference books on which the examination was to be based. That list of reference books contained a specific reference to Volume Seven of the NFPA. All sixteen volumes which comprise the entire NFPA were not listed as reference books. Neither of these Petitioners demonstrated that he or she was genuinely confused or misdirected by the amendment to Rule 21GG-6.01(1)(c). Neither Petitioner contended he or she had studied all sixteen volumes of the NFPA. Instead, both Petitioners studied from the books on the reference list. Neither Petitioner has an application for the examination pending; neither claims an intention to take the examination again. Findings regarding Rule 21GG- 6.01(1), (2), (3), and (4) of the Electrical Contractors' Licensing Board At the time of the examinations which are the subject of these Petitioners' related cases under Section 120.57, Florida Statutes, Rule 21GG- 6.01(1), (2), (3), and (4) of the Electrical Contractors' Licensing Board read as follows, in pertinent part: The areas of competency to be covered by the certification examination shall be as follows: Technical, which shall include, but not be limited to, electrical calculations, estimating, designs, and electrical schematics; and which shall be divided into two (2) separate areas, one containing electrical calculations, the other containing Code-related questions not requiring calculations. General business, which shall include but not be limited to, problems relating to accounting, law, insurance, workers' compen- sation and social security; The format of the examination shall be as follows: The portions of the examination con- taining the electrical calculations and $ safety questions shall be open book. The applicant is responsible for bringing and may.. use during these portions the applicable code books, reference materials as approved by the Board, and noiseless mechanical or non- mechanical instruments he wishes to use. The portions of the examination con- taining the noncalculation technical questions and all business questions shall be closed-book. The applicant will not be permitted to use the reference materials listed in 2(a) or any other reference materials. Security measures as set forth by the Department shall be followed during both portions of the examination. The relative grading weight to be assigned to each area of competency shall be approximately as follows: Technical - 60 percent Electrical Calculations-40 percent of Test Code-related questions-20 percent of Test General business - 30 percent Safety - 10 percent An applicant shall be required to achieve a score of a general average of not less than seventy-five percent (75 percent) in order to pass the examination and be certified for licensure. There shall not be a practical or clinical examination. The language quoted above is the language of the relevant portions of the rule as it read after the March 1981 amendments to the rule. The principal amendments in March of 1981 may be summarized as follows: (1) changes which divided technical questions on the examination into two separate areas, (2) changes which made part of the examination open book and part closed book [it was previously all open book, and (3) a change in the minimum passing score from 70 percent to 75 percent. These Petitioners challenge the validity of all three of the principal amendments. As originally proposed, the March 1981 amendments to subsection (2)(a) of Rule 21GG-6.01 would have permitted a candidate to bring "any notes" to the open book portion of the examination. During the adoption process the words "and any notes" were deleted from the amendment by the Board. The words "and any notes" did not appear in Rule 21GG-6.01 prior to the 1981 amendments. Nor was it the practice of the Board prior to 1981 to permit candidates for examination to take any notes into the examination when it was administered by the Board. Findings regarding examination content and relative grading weights In November of 1983 the Electrical Contractors' Licensing Board again amended Rule 21GG-6.01(3) to change the relative grading weights of the areas of competency to be tested. After November of 1983 the relative grading weights of the areas of competency were as follows: Technical increased from 60 percent to 65 percent; electrical calculations increased from 40 percent to 45 percent; code related questions remained 20 percent of the examination; general business was decreased from 30 percent to 25 percent of the examination; and safety remained 10 percent of the examination. 5/ The licensing examination given by the Board has historically contained materials testing knowledge of electrical work and also materials testing the business qualifications of the applicants. Since the inception of the Board's licensing exam, the electrical work portions have contained electrical calculations. Throughout the time the Board has been administering licensure examinations for certification, the percentages of the examination devoted to particular subject areas have been based upon the Board's belief as to the appropriate levels of knowledge needed by a certified electrical contractor. The weighting of the examination was equitable when it was weighted 60 percent technical, 30 percent business, and 10 percent safety, and it is still equitable now that it is weighted 65 percent technical, 25 percent business, and 10 percent safety. The Board changed to the current percentages at the suggestion of Block and Associates. The Board's examination has had similar ratios for a long time, at least as far back as 1971. The proportions of the examination devoted to each particular subject matter did not change due to the March 1981 amendments. With the exception of the January 1981 examination put together by O.E.S., electrical calculations had historically been 40 percent of the test. Since the Board felt this percentage should be maintained, and since it was necessary to divide those calculations from the other portion of the technical section (since one would be tested in the closed book portion and the other in the open book portion), the division of the technical part of the exam was set out in the rule. On an electrical contractor licensure examination it is appropriate to include in the area of "technical" questions, questions dealing with the following subjects: electrical calculations, parts of estimating, and designs and electrical schematics. In the "business" portion of an electrical contractor examination, it is appropriate to include questions dealing with the following: accounting, worker compensation, and social security. Findings regarding closed book versus open book examinations Local licensing examinations had, in 1981, included a closed book portion. Local licensing examinations and state licensing examinations given in other states still include such a closed book portion. The basic Block examination has always included a closed book portion. Closed book examinations are not all that unusual in occupational and professional licensure testing. The CPA an Nursing examinations are closed book. Also, virtually all of the electrical examinations administered by Block and Associates include a closed book portion. The advantage of a closed book examination technique is that it is easier to identify the candidates who have had practical experience in the subject matter being tested. A closed book examination is a better test for field experience and for general knowledge of the subject matter being tested. Certain questions known as "anchor" questions was given both in the open book examination before the format change and in the closed book section after the format changed. The relative performance by candidates on the same questions in the two formats shows the format change not to have prejudiced the candidates. Further, an examination analysis of the results of the closed book portions of the ECLB examinations reveals that, based on candidate performance, the closed book portion of the examinations was easier than the open book portion. This also shows that the format change did not prejudice the candidates. Findings regarding "cut scores" From 1972 until the rule changes in 1981, the minimum passing score, or "cut score," set by the Board for the certification licensure examination was 70 percent. Cut scores of both 70 percent and 75 percent are very common cut scores for licensure examinations in the fields of electrical contracting, journeyman electrician, and master electrician, as well as licensure examinations in other occupations and professions. Many local licensure examinations have a 75 percent cut score. Standing alone, the cut score to be applied to a future examination is somewhat lacking in specific significance. A cut score must be considered in relation to the content of the examination to which it is to be applied. All of the testing experts who testified at the hearing agreed that setting an examination cub score is an inherently arbitrary decision. In the final analysis it is a judgment call that cannot be made with scientific exactitude and is unavoidably arbitrary to a certain extent. As explained by one of the Petitioners' testing experts, even though statistical methodology is available for the determination of cut scores, to a large extent the cut score is a random decision, the validity of which can be assessed only in relationship to the specific testing instrument to which it is applied. For example, during the course of Block's preparation of its first licensure examination in 1958 for the City of Ormond Beach, when it came time to decide what cut score to use, the five experts who were preparing the examination talked it over and decided that 75 percent was the equivalent' of a good. "C" when compared to academic grades, and they decided to use 75 percent as their cut score. It is not possible to be much more scientific or precise than that in setting cut scores, especially when one has a group of candidates of varied experience and background taking the examination. A skilled test preparer can construct an examination of equal validity within a range of cut scores from 65 percent to 80 percent if the test preparer knows at the time of constructing the examination what the cut score will be. With any cut score in that range, a skilled test preparer can raise or lower the percentage of candidates who will actually pass the examination by careful selection of the questions to be used on the examination. By using easy questions, the person preparing the examination can insure that more people will pass a test with a high cut score. Similarly, by using hard questions, the person preparing the examination can insure that fewer people will pass an examination with a low cut score. On some standardized licensure examinations the cut score is determined after the examination results are examined. In other words, the examination results are "curved" based on the performance of the group of candidates who take a particular session of the examination and the actual cut score is determined in relationship to the raw scores achieved by each particular group of candidates. This flexible method of setting cut scores seems to work well on standardized examinations which are administered to large numbers of candidates, but it is questionable whether it would work very well with small groups of candidates. The Board's examinations have historically been administered to small groups of candidates. Findings regarding the effects of changes on testing instruments. In order for a licensure examination to accomplish the purpose for which it is administered, it should, of course, be valid and reliable. Several of the Petitioners' expert witnesses described what might be characterized as the best way to make changes to an examination or as the ideal way to make changes to an examination. Nevertheless, the manner in which the Electrical Contractors' Licensing Board went about making the changes to the examination which were incorporated in the March 1981 rule changes was a reasonable and logical response to the circumstances faced by the Board. It was perhaps not the best way to have resolved matters, but it was a reasonable response. The Board's action in this regard was not arbitrary or capricious. Even the Petitioners' most persuasive expert in the field of testing conceded that if the content experts (in this case the Board members) felt that given a specific content domain and given a specific item bank the testing instrument measured less than minimum competency, a logical response would be to raise the cut score. Another logical reaction to that situation would be to change a portion of the examination from open book to closed book. Both actions would be reasonable and logical responses to that type of problem. Where there has been a modification in the format of a testing instrument, an accepted methodology for analysis of the effects of format modification is the use of repetitive questions from prior testing instruments. These repetitive questions are known as "anchor" questions. If a statistically significant deviation in performance on the anchor questions is demonstrated, the format modification requires further analysis through other statistical tools. However, if an analysis of the anchor questions shows inconclusive or mixed results on the anchor questions (such as equal or improved candidate performance on those questions), it may then be concluded that the format change is not significant to candidate performance and may be an appropriate format change relative to the profession or occupation being examined. Content validity is the most practical' measure of the validity of a licensure examination. The content validity of an examination depends on the extent to which the questions on an examination may be accepted as representative of performance within a specifically defined content domain of which the examination instrument is a sample. Content validation would be the best way to determine the validity of the examinations which were administered in 1982 pursuant to the March 1981 rule amendments. One method of establishing content validity is to have a group of experts identify the domains of knowledge that are applicable to the profession or occupation being tested. The establishment of content validity relies heavily on the opinions and judgments of people who are experts in the relevant profession or occupation. Expert judgment plays a integral part in developing the definition of the content domain to be tested. To the extent that the content domain of Boar examinations was established by the March 1981 amendments to Rule 21GG-6.01(3), the content domain was established on the basis of the expert judgment of persons who were experts in the field of electrical contracting, namely, the members of the Electrical Contractors' Licensing Board. The relative grading weights established in the 1981 amendments to Rule 21GG-6.01(3) have at least a possible correlation to electrical contractor competence. In fact, the persuasive expert testimony establishes that those relative grading weights did bear a reasonable relationship to electrical contractor competence. The current slightly different relative grading weights also bear a reasonable relationship to electrical contractor competence even though there has been no recent change in the nature of the responsibilities of electrical contractors. In this regard it must be remembered that testing is not an exact science and no test for minimum competency can ever be expected to be an exact model of actual work experience. The best that can be expected is a reasonable model. There are various accepted methodologies for the creation of standardized examinations, adherence to which tends to enhance the likelihood that the examination instrument finally produced will be valid and reliable. However, adherence to those accepted procedures is not a necessary prerequisite to creation of a valid and reliable examination. This is particularly true when one is going to be testing small groups of people with varied backgrounds instead of the massive groups of people with similar backgrounds for whom standardized tests are more typically designed. In the final analysis the only practical and reliable measure of the validity of an examination is by statistical analysis of the examination after it has been administered. Given the nature of the circumstances faced by the Board at the time of the March 1981 amendments to the examination format and cut score, given the nature of the pool of candidates to be examined, given the nature of the changes contemplated by the Board, and given the very nature of the process of testing for minimum competency--which involves perhaps as much art as it does science-- there is no study or data which would have been particularly useful to the Board in helping to determine exactly what the effect of their changes would be. Such effects can only be determined or measured with any degree of accuracy after the administration of an examination that incorporated the changes. Following the administration of such an examination, it is possible to perform a statistical analysis of all questions used on the examination and to eliminate or give credit for any questions which are shown by statistical analysis to be invalid or unreliable. This is precisely the process that is used by Block in the validation of their examinations and is an accepted testing procedure. It would have served no useful purpose to have conducted a trial run of an examination using licensed certified electrical contractors as a test group for the new examination format. First, it would be virtually impossible to try to put together an accurate cross-section of certified electrical contractors to use as a test group. Second, one would expect them all to pass the examination, so when they did so nothing of value would have been learned. Finally, the administration of such a trial run would risk the possibility of compromising examination question security. One aspect of accepted methodology for the preparation of standardized examinations is the definition of the content domain of the examination, i.e., a determination of what knowledge is essential to demonstrate that the candidates for licensure are minimally competent. This aspect of examination preparation is often accomplished by performing a formal job analysis, which is, in essence, a study of all of the usual tasks performed by a person engaged in the occupation or profession to be tested, including an evaluation of the relative importance of each of those tasks to minimum competence. The content domain can also be defined on the basis of the judgment of a group of experts in the occupation or profession to be tested. With the exception of the lay members who were added in recent years, all of the members of the Electrical Contractors' Licensing Board are, and have been, persons certified to engage in electrical contracting in the state of Florida and actively engaged in the electrical contracting business. Therefore, at all relevant times all of the professional members of the Board had extensive personal knowledge of what was involved in the practice of electrical contracting, which personal knowledge was as useful in defining content domain as would have been a formal job analysis. (In this regard it is important to note that even with the addition of lay members to the Board in recent years, the experienced professional members have continued to constitute a substantial majority of the Board.) Findings regarding Rule 21GG-6.0 of the Electrical Contractors' Licensing Board Rule 21GG-6.03 of the Electrical Contractors' Licensing Board reads as follows: An examinee is entitled to review his examination questions, answers, papers, grades and grading key used in the certification examination; however, no applicant may copy any materials provided for his review. Such review shall be conducted during regular business hours, in the presence of a representative of the Board at the Board's official headquarters. If, following the review of his exami- nation, an examinee believes that an error was made in the grading of his examination, or in the evaluation of his answers, he may request the Board to review his examination. Requests for Board review must be in writing, state with specificity the reason why review is requested, and be received within thirty (30) days after the examinee received notice that he failed the examination. (3) Upon receipt of a request for Board review, the examination shall be reviewed by the Board at the next regularly scheduled Board meeting. If it is found that an error was made, the grade received by the examinee may be adjusted to reflect the correction. The examinee shall be notified of the final decision. The Petitioners' only challenge to Rule 21GG-6.03 is predicated upon their concern that it may be misapplied to them by the Board. Findings regarding Rule 21-11.11(3) of the Department of Professional Regulation as it existed prior to the October 1982 amendments Prior to the October 1982 amendments, Rule 21-11.11(3) of the Department of Professional Regulation read as follows: (3) Examination grade reviews shall be conducted at a site designated by the Department. The candidate and his/her attorney or other qualified representative shall be allowed to attend the examination grade review. A candidate may attend only one review per examination administration. The above quoted version of the rule was in effect at the time of the Petitioners' reviews of their January and July 1982 examinations. Findings regarding Rule 21-11.11(3) of the Department of Professional Regulation, as amended in October of 198 Rule 21-11.11(3) of the Department of Professional Regulation, as amended in October of 1982, reads as follows, in pertinent part: A candidate taking a Department exam- nation may request and receive an appointment for review until such times as the exam- ination records are destroyed in accordance with Chapters 455, 119 and 267, Florida Statutes. However, unless otherwise provided by a rule of the appropriate Board within the Department, candidates may review their exam- ination for the purpose of filing objections to the examination for the Board's consideration under the following conditions and time frame: Within thirty (30) days of the date of the grade notification letter the candidate shall notify the Department of his/her desir to review the examination for the purpose of filing objections for consideration. Such review shall have been completed within the next thirty (30) days after the first thirty (30) days defined in (a) above. At the examination review, the candi- date shall be permitted to record on forms provided by the Department any and all objections to the examination the candidate desires the appropriate Board to review. Such forms shall remain in the custody of the Office of Examination Services for presen- tation to the appropriate Board at the next available Board meeting as an official agenda item. The candidate shall not copy" questions from the test booklet. The candidate may write on a separate paper in the presence of the Office of Examination Services employee, any objection or question he/she has to the written examination. The candidate shall leave the written objections and questions with the Office of Examination Services employee when he/she leaves the review room but she/she shall be permitted to leave with a form listing the question numbers he/she finds to be controversial. In the instance of a written exami- nation, all objections will be presented to the appropriate Board for consideration. Upon completion of the Board's review of written examination items the Department shall notify the reviewing candidate of the Board's decision. If the Board does not concur with the candidate's objections then the candidate will be notified in writing of this and the thirty (30) day appeal time per- iod shall begin to run from the date of this notice. Findings regarding Rule 21- 11.11(13) of the Department of Professional Regulation (later renumbered as Rule 21-11.11(3)(1)j Rule 21-11.11(13) of the Department of Professional Regulation, which was later renumbered as Rule 21-11.11(3)(1), reads as follows: If the consultant finds that the original grade was not rendered in accordance with the grading criteria, then he/she twill regrade that portion, or the entire examination, whichever is appropriate, pursuant to appli- cable statutes and rules. If it is not possible to regrade the examination, the candidate will be allowed to retake the examination at no charge. It is clear from the text of Rule 21-11.11(3)(j) that the language of Rule 21-11.11(3)(1) quoted immediately above is intended to apply only to practical examinations. It does not purport to apply to written examinations. These Petitioners took only written examinations, they have not taken any practical examinations. Findings regarding Rule 21-11.14 of the Department of Professional Regulation Rule 21-11.14 of the Department of Professional Regulation reads as follows, in pertinent part: Unless otherwise provided by a rule of the appropriate board within the Department, Security and Monitoring procedures shall be as follows: (10) Disposition of Test Booklets. Secured processing of Answer Sheets Test booklets for the state devel- oped examinations shall be disposed of and filed in accordance with the following procedure: 1. Ten copies of the examination book- lets shall be retained for ninety (90) days. This procedure shall insure that there are adequate copies of the booklets available for the board to review in their review of preliminary analysis of the examination and review sessions if review responsibilities are retained by the board. After the above defined ninety (90) day period the retention schedule shall change from ten copies to four copies. In the event any irregularity occurs during the examination with any state developed booklet, it shall be the examination supervisor's responsibility to prepare a detailed report of such irregularity and to retain the booklet in question in the secured files for a period of ninety (90) days. Destruction of examination booklets and related materials described in B.2.b., above: When in accordance with procedure, it is appropriate to destroy test booklets, it shall be the Archives's responsibility to schedule such destruction. In all instances there shall be evidenced in writing by the examination supervisor and a witness on a form which shall evidence the date of destruction and the official in charge of such destruction and a witness. When examination booklets are being prepared by the Department of Professional Regulation prior to an examination, the booklets are carefully inspected to make sure that all of the booklets are identical. Following that inspection the booklets are sealed and stored in a secure place in order to insure, among other things, that no changes are made to any of the examination booklets before they are handed out to the candidates. After an examination is given, the Department of Professional Regulation retrieves all of the examination booklets, including all booklets that were used by all of the candidates, and retains them in a secure place until the excess booklets can be destroyed. The examination supervisor selects the booklets which are to be retained from among the booklets that were not handed out to the candidates at the examination. In the normal course of events all of the examination booklets that were actually handled by the candidates at the examination are destroyed within a very few months of the date of the examination. With the exception of the examination booklet of one other candidate (which exception is not relevant to the disposition of this rule challenge proceeding), all of the examination booklets which were handed out to candidates during the examinations taken by these two Petitioners were destroyed approximately 90 days after each of the examinations. When such destruction took place, the Department retained copies of the examination booklets which had not been used by any candidate, which copies were identical to the copies that had been handed out to the candidates during the examination. As part of the examination instructions, all candidates for examination are advised not toe write anything in their examination booklets because all of the booklets used by the candidates will be shredded. They are specifically told to do all of their computations on sheets of work paper that are provided to them at the examination. All candidates are specifically told that the only things they turn in that will be saved are their answer sheets and their sheets of work paper. Findings regarding "unwritten" and "unpublished" rules of the Electrical Contractors' Licensing Board. The findings of fact immediately below relate to the eight "unwritten" and "unpublished" alleged rules of the Electrical Contractors' Licensing Board which are described in subparagraphs (F), (O), (P), (V), (X), (Y), (Z), and (FF) at pages 1 through 5 of the Petition. (The descriptions of these alleged rules are quoted at pages 2 through 6 of this Final Order.) One of these "unpublished" rules is alleged to be a rule to the effect that the Board requires more than minimum competency in order for a candidate to receive state certification and licensure as an electrical contractor. The other seven "unwritten" or "unpublished" rules of the Electrical Contractors' Licensing Board alleged in the Petition are all related in one way or another to the Board's alleged free form actions in the course of its review of the two examinations which have been challenged by these Petitioners in related proceedings under Section 120.57(1), Florida Statutes. None of the evidence regarding these matters proved the existence of an agency statement of general applicability purporting in and of itself to have the direct and consistent effect of law. For reasons which are explained in the Conclusions of Law portion of this Final Order, the evidence is insufficient to establish that the Electrical Contractors' Licensing Board has any "unwritten" or unpublished5? rules such as those described in subparagraphs (F), (O), (P), (V), (X), (Y), (Z), and (FF) of the Petition. 6/ Findings regarding "unwritten" and "unpublished" rules of the Department of Professional Regulation The findings of fact immediately below relate to the six "unwritten" and "unpublished" alleged rules of the Department of Professional Regulation which are described in subparagraphs (G), (J), (1), (L), (Q), and (R) at pages 1 through 5 of the Petition. The descriptions of these alleged rules are quoted at pages 2 through 6 of this Final Order.) These six "unwritten" or "unpublished" alleged rules of the Department of Professional Regulation are all related in one way or another to actions of functionaries of the Department in the course of performing the Department's role in the administration and review of the two examinations which have been challenged by these Petitioners in related proceedings under Section 120.57(1), Florida Statutes. None of the evidence regarding these matters proved the existence of an agency statement of general applicability purporting in and of itself to have the direct and consistent effect of law. For reasons which are explained in the Conclusions of Law portion of this Final Order, the evidence is insufficient to establish that the Department of Professional Regulation has any "unwritten" or "unpublished" rules such as those described in subparagraphs (G),(J), (I), (L), (Q), and (R).7 Findings regarding "unwritten" and "unpublished" rules of both the E1Electrical -Contractors' Licensing Board and the Department of Professional Regulation The findings of fact immediately below relate to the ten "unwritten" and "unpublished" alleged rules of both the Electrical Contractors' Licensing Board and the Department of Professional Regulation which are described in subparagraphs (M), (N), (S), (T), (U), (AA), (BB), (CC), (DD), and (EE) at pages 1 through 5 of the Petition. (The descriptions of these alleged rules are quoted at pages 2 through 6 of this Final Order.) These ten "unwritten" or "unpublished" alleged rules of both the Electrical Contractors' Licensing Board and the Department of Professional Regulation are all related in one way or another to actions allegedly taken by the Board and the Department in the performance of their respective functions related to the administration and review of the two examinations which have been challenged by these Petitioners in related proceedings under Section 120.57(1), Florida Statutes. None of the evidence regarding these matters proved the existence of an agency statement of general applicability purporting in and of itself to have the direct and consistent effect of law. For reasons which are explained in the Conclusions of Law portion of this Final Order, the evidence is insufficient to establish that both the Electrical Contractors' Licensing Board and the Department of Professional Regulation have any "unwritten" or "unpublished" rules such as those described in subparagraphs (M), (N), (S), (T), (U), (AA), (BB), (CC), (DD), and (EE) of the Petition.8
The Issue Whether Petitioner is entitled to additional credit for her solution to Problem 120 on the Principles and Practice of Engineering portion of the engineering licensure examination administered on April 14, 2000, by the National Council of Examiners for Engineers and Surveyors.
Findings Of Fact Based upon the evidence adduced at hearing and the record as a whole, the following findings of fact are made: On April 14, 2000, as part of her effort to obtain a Florida engineering license, Petitioner sat for the Principles and Practice of Engineering Examination (Examination). This is a national examination developed and administered by the National Council of Examiners for Engineers and Surveyors (NCEES). Petitioner chose to be tested in civil engineering. Petitioner received a raw score of 46 on the Examination. For the civil engineering specialization, a raw score of 46 converts to a score of 68. To pass the Examination, a converted score of 70 is needed. Petitioner formally requested that her solution to Problem 120 on the Examination be rescored. Petitioner's written request was referred to the NCEES. The NCEES's rescoring of Petitioner's solution to Problem 120 resulted in her receiving no additional points. The Board received the NCEES's rescoring results on or about December 5, 2000. After receiving a letter from Petitioner (dated December 14, 2000) requesting a "formal hearing," the Board referred the matter to the Division. Problem 120 was worth ten raw points. Petitioner received six raw points for her solution to Problem 120. In her solution to Problem 120, Petitioner failed to properly take into consideration the height of the water table, did not compute the factor of safety for load-bearing capacity in the manner required, and made an arithmetic mistake. Therefore, in accordance with the requirements and guidelines of the NCEES's scoring plan for this problem, the highest raw score that she could have received for her solution to this problem was a six, which is the score she received. In rescoring Petitioner's solution to this problem, the NCEES rescorer made the following "comments": The examinee made three errors. The solution approved by the Civil Engineering Exam committee called for a correction in requirement (a) for the mid height water table. The examinee ignored this correction. A two point grade reduction is called for. The examinee made a numerical error in evaluating the bearing capacity equation. This error called for a one point grade reduction. In evaluating the factor of safety the examinee added an erroneous load factor. A two point grade reduction is called for. With a total of five grade points lost a final grade of six is called for. SCORER'S RECOMMENDATION: Recommended score = six There has been no showing that the rescorer's analysis was in any way flawed.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that a final order be entered rejecting Petitioner's challenge to the failing score she received from the NCEES on the Principles and Practice of Engineering portion of the April 14, 2000, engineering licensure examination. DONE AND ENTERED this 16th day of April, 2001, in Tallahassee, Leon County, Florida. STUART M. LERNER Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 16th day of April, 2001.
Findings Of Fact L.B. Thanki received a degree in Civil Engineering at the University of Durham at Kings College, Newcastle Upon Tyne in the United Kingdom in 1956. Petitioner received a batchelor of law degree from Sardar Patel University (India) in 1967. This degree is the equivalent of two years study in law. The degree obtained from the University of Durham is not the equivalent of the degree received from an ABET approved university in the United States because it lacks 16 credit hours in Humanities and Social Sciences. Petitioner presented no evidence that his degree from the University of Durham or the curriculum he completed at any other university included the missing 16 hours in Humanities and Social Sciences. Petitioner presented a certificate (which was not offered into evidence) that he had completed a course in computer services meeting the board's evidentiary requirements of computer skills.
Recommendation Based on foregoing Findings of Fact and Conclusions of Law, it is recommended that a Final Order be entered denying Petitioner's application for licensure by examination as an engineering intern. RECOMMENDED this 10th day of May, 1991, in Tallahassee, Leon County, Florida. N. AYERS Hearing Officer Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-1550 (904)488-9675 Filed with the Clerk of the Division of Administrative Hearings this 10th day of May, 1991. COPIES FURNISHED: B. Thanki 1106 East Hillsborough Avenue Tampa, Florida 33604 Edwin A. Bayo, Esquire Assistant Attorney General Department of Legal Affairs The Capitol, Suite LL04 Tallahassee, Florida 32399-1050 Carrie Flynn, Acting Executive Director Florida Board of Professional Engineers Northwood Centre, Suite 60 1940 North Monroe Street Tallahassee, Florida 32399-0755 Jack L. McRay, General Counsel Department of Professional Regulation Northwood Centre, Suite 60 1940 North Monroe Street Tallahassee, Florida 32399-0792
