The Issue The issues in this case are whether Petitioner, Gary P. Santoro (“Petitioner” or “Mr. Santoro”), undeservedly received a failed grade on the Construction Business and Finance Examination (“Examination”) for licensure as an air-conditioning contractor; whether any questions on the examination had more than one correct answer; whether the examination is unfair; whether there is transparency in the examination review process; and whether the examination grading process is arbitrary and capricious.
Findings Of Fact Mr. Santoro took the Examination on November 16, 2018. Petitioner failed the Examination because he scored less than 70 percent correct. The Examination contains 125 questions, 120 of which are scored. The other five are not scored and are considered “pilot” questions for potential use on future examinations. In order to pass the Examination, a candidate must obtain a score of at least 70 percent. All scored questions on the Examination are weighted equally. As a result of failing to pass the Examination, Petitioner was notified of his results. All questions on the Examination had a single correct answer. Cynthia Woodley, Ph.D., employed by Professional Testing, Inc. (“PTI”), as the chief operating officer, is an expert in psychometrics and exam development. She holds a master’s degree in vocational education and a doctorate in curriculum and instruction with a specialization in measurement. Her current position calls for her to manage a number of licensure and certification exam programs. She explained at length how specific questions become part of a professional licensure exam. To develop questions, her company brings in any number of subject matter experts, people actually employed in the professions being tested, and they help develop subject matter questions for a particular exam. That was the process used for development of the Examination in this matter. Once the subject matter experts are trained in exam question writing techniques, they write questions, which are reviewed by other subject matter experts to determine whether the questions are fair and understandable enough to be answered by prospective test takers. Generally, five subject matter experts review each question before it makes its way onto an exam. PTI measures the “P value” of the questions by determining what percent of individuals taking a given exam answer a particular question correctly. For example, a P value of .90 means that 90 percent of the people taking the exam answered a particular question correctly. PTI looks for a wide range of P values in its exam questions. If a P value is too low, say .40, the company might reexamine that question to determine whether it should be removed from future exams since fewer than half the people taking the exam answered it correctly. The business and finance portion of the exam is given to all contractors, regardless of their specialty, with the exception of pool service contractors. Here, Petitioner, a HVAC contractor was administered the same Examination as plumbing contractors, electrical contractors, general contractors, etc. Each of the 120 questions on the exam in this case was equally weighted. There were also five pilot questions inserted into the exam, which did not count towards the total score, but were included as test questions for future exams. Petitioner provided hearsay documents regarding computer hacking and computer glitches associated with some exams administered around the United States. However, he did not connect the articles submitted into evidence to the exam administered in this case or any exam administered by the Department in Florida. Dr. Woodley was familiar with the allegations of computer glitches in testing, but testified that the problems were with K-12 testing in schools, not with professional licensure exams, such as administered by the Department. Therefore, since the hearsay evidence was not linked to the exam at issue or similar professional licensure exams given in Florida, it is entitled to no weight in arriving at the decision in this case. Question BF 1290 has a single correct answer, which is answer “C.” Petitioner selected answer “B.” Petitioner was unable to demonstrate that the answer he selected was correct. Question BF 0473 has a single correct answer, which is answer “A.” Petitioner selected answer “C.” This question asks for an answer of general applicability. Petitioner’s claim that his answer is equally correct is based on a narrow exception in law. Accordingly, Petitioner was not able to demonstrate that the answer he selected was correct. Question BF 0162 has a single correct answer, which is answer “B.” Petitioner selected answer “C.” Petitioner was unable to demonstrate that the answer he selected was correct. Question BF 1691 has a single correct answer, which is answer “C.” Petitioner selected answer “D.” Petitioner was unable to demonstrate that the answer he selected was correct. Petitioner was unable to submit sufficient evidence to show that the Examination is unfair, that there is insufficient transparency in the examination review process, or that the examination grading process is arbitrary and capricious. Accordingly, he cannot prevail in his challenge to the Examination. Petitioner testified that he took and passed the HVAC contractors special license examination on his first attempt. He has taken the Examination on numerous occasions and is yet to be successful. He testified he studied hard for every administration of the exam, but just cannot reach the finish line successfully. While that is unfortunate, the evidence does not support that his failure to succeed on the Examination is the fault of the exam itself or of the Department either in its contracting to have the exam created or in the administration of the exam. From the way he conducted himself at hearing, Petitioner appears to be an intelligent, diligent, and successful individual in his HVAC business. For some unknown reason he has been unable to successfully complete the Examination. His persistence in retaking the Examination multiple times is admirable and should ultimately pay off with his successful passage of the Examination.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that the Construction Industry Licensing Board enter a final order upholding the Department’s Amended Grade Report finding that Petitioner failed to achieve a passing score on the Construction Business and Finance Examination, which he took on November 16, 2018. DONE AND ENTERED this 23rd day of August, 2019, in Tallahassee, Leon County, Florida. S ROBERT S. COHEN Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 23rd day of August, 2019. COPIES FURNISHED: Thomas G. Thomas, Esquire Department of Business and Professional Regulation 2601 Blair Stone Road Tallahassee, Florida 32399-2202 (eServed) Gary Peter Santoro Hometown Air & Services 8229 Blaikie Court Sarasota, Florida 34240-8323 (eServed) Ray Treadwell, General Counsel Office of the General Counsel Department of Business and Professional Regulation 2601 Blair Stone Road Tallahassee, Florida 32399-2202 (eServed) Daniel Biggins, Executive Director Construction Industry Licensing Board Department of Business and Professional Regulation 2601 Blair Stone Road Tallahassee, Florida 32399 (eServed) Halsey Beshears, Secretary Department of Business and Professional Regulation 2601 Blair Stone Road Tallahassee, Florida 32399-2202 (eServed)
Findings Of Fact The Petitioner completed an academic program in accounting at the University of South Florida in March, 1976. She applied to sit for the May, 1976 Certified Public Accountant's examination, and paid her fee. There are four sections to the examination: Auditing, Law, Theory, and Practice. At the May, 1976 examination the Petitioner passed the Law section, but failed the sections on Auditing, Theory and Practice. Accordingly, under the Board's rules, the Petitioner was not credited with having passed any sections of the examination, and needed to take the entire test again. She applied to sit for the November, 1976 examination, paid her application fee, and sat for the examination. On this occasion she passed the Theory and Practice sections of the examination but failed the Auditing and Law sections. Under the Board's rules the Petitioner at this juncture was credited with having passed the Theory and Practice sections, and would be allowed to sit for the next three consecutive examinations in order to pass the remaining two sections. She applied to sit for the May, 1977 examination, paid her fee and sat for the examination. She passed the Law section and failed Auditing. At this juncture she needed to pass only the Auditing section, and had two examinations within which to accomplish that. She applied to sit for the November, 1977 examination. The deadline for making application was September 1, 1977. The Petitioner, through her own mistake, was lake in making application, and her application was rejected. She was not permitted to sit for the November examination. She did timely apply for the May, 1978 examination. She again failed the Auditing section with a score of 69. Under the Board's rule her application for certification as a CPA was considered she would need to being again the testing process, without being credited with having passed any sections. She applied for a regrading of the May, 1978 examination. The examination was regraded, but her score was not changed. The Petitioner is seeking, through this proceeding, an opportunity to retake the Auditing section of the examination, while continuing to receive credit for having passed the Law, Theory, and Practice sections. Under the Board's interpretation of its rules, she would not receive credit for having passed the sections, but would need to begin the testing procedure as a new applicant.
The Issue Whether the Petitioner, Jenny Lancett (Petitioner) is entitled to additional credit for the answers she provided to challenged examination questions. The Petitioner has challenged the State Officers Certification Examination (SOCE) for Law Enforcement Officers given in July of 2005 (DOAH Case No. 05-4544) and September of 2005 (DOAH Case No. 06-0325).
Findings Of Fact The Petitioner is an applicant for certification as a law enforcement officer. In order to be eligible to sit for the state examination, the Petitioner successfully completed an academy course that was sponsored by the Palm Beach County Sheriff’s Office. To become fully certified, the Petitioner must take and achieve a passing score on the SOCE. The Petitioner took the SOCE on July 27, 2005, but did not pass. She also took the SOCE on September 21, 2005, and did not pass. The Respondent is the state agency charged with the responsibility of administering examinations for certification for the SOCE. After the July and September examinations for the SOCE were administered and the Petitioner received failing scores, she requested and attended examination review sessions conducted at the Respondent’s headquarters in Tallahassee, Florida. The review sessions were held at the Florida Department of Law Enforcement and the Petitioner was given copies of the examination questions for which she did not provide the correct answers. The Petitioner was not told the correct answers. As to the July examination, the Petitioner timely challenged Questions 45, 56, 90, 141, 151, 156, 161, 163, 207, 227, 234, 238, and 242. That examination challenge was forwarded to the Division of Administrative Hearings on December 14, 2005, and was designated as DOAH Case No. 05- 4544. The Petitioner withdrew her challenge to Question 238. As to the September examination, the Petitioner timely challenged Questions 44, 63, 134, 160, 162, 165, 166, 178, 189, 194, 195, and 208. That examination challenge was forwarded to the Division of Administrative Hearings on January 26, 2006, and was assigned DOAH Case No. 06-0325. On February 3, 2006, in response to the Joint Motion to Consolidate, the cases were consolidated for formal hearing and rescheduled for hearing to March 21-22, 2006. The case was heard on April 5, 2006. As to each challenged examination, the applicant must answer 250 questions and achieve at least 180 correct answers. Twenty-five questions of the 250 do not count but are considered “throw-away” questions. All questions are posed in English. All questions are multiple choice and an applicant is given credit for only the correct answer. The Respondent deems the correct answer to be the best choice from among the options offered. The Petitioner is an Hispanic female who reads, writes, and speaks English. Although she inquired about accommodations, the Petitioner was not afforded any accommodations while taking the challenged examinations based upon English as her non-native language. The Commission requires that all applicants take the examination in English. Similarly, accommodations are not afforded applicants who maintain “test anxiety” as a basis for concern. All questions and answers for the challenged examinations are considered confidential as a matter of law. As to each of the questions challenged by the Petitioner, the Petitioner failed to select the correct and best option from the multiple-choice selections noted. As to each of the questions challenged by the Petitioner, the wording and options noted are clearly stated and are within the curriculum covered by the academy. Of the numerous Hispanic candidates who have successfully completed the Palm Beach County academy during Mr. Kozyra’s tenure, only two have failed to achieve a passing score on the SOCE within three attempts. The Petitioner submitted no credible evidence to support her claim that the answers she provided on the challenged examination questions were correct or that the questions in their wording or grading were flawed. The persuasive weight of the evidence was to the contrary. A question which asks the applicant to provide a response that is “most accurately” describing the situation (as did Question 56 on the July examination) means that of the choices offered only one can be considered “most accurate.” If the applicant chooses an answer that is not the “most accurate,” credit is not given. Many of the Petitioner’s responses fell into this type of erroneous response. The Petitioner simply failed to provide the “most accurate” from the selections offered. As to each of the selections chosen by the Petitioner, a clear majority of the applicants taking the examinations selected the correct option. None of the examinations questions challenged by the Petitioner were incorrectly answered by a majority of the applicants. In some instances as many as 94 percent of the test takers chose the correct answer whereas the Petitioner did not. The Respondent provided sufficient explanation and the record clearly establishes that as to each of the erroneous answers provided by this Petitioner, the correct answer (as scored by the Respondent) was the best or correct answer. The Petitioner cannot be entitled to additional credit when her answers were not correct.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that the Petitioner’s challenges to the July and September 2005 certification examinations be denied. S DONE AND ENTERED this 28th day of June, 2006, in Tallahassee, Leon County, Florida. J. D. PARRISH Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 28th day of June, 2006. COPIES FURNISHED: Michael Crews, Program Director Division of Criminal Justice Professionalism Services Department of Law Enforcement Post Office Box 1489 Tallahassee, Florida 32302 Michael Ramage, General Counsel Department of Law Enforcement Post Office Box 1489 Tallahassee, Florida 32302 Bruce A. Minnick, Esquire Minnick Law Firm Post Office Box 15588 Tallahassee, Florida 32317-5588 Linton B. Eason, Esquire Department of Law Enforcement Post Office Box 1489 Tallahassee, Florida 32302-1489
The Issue The issues presented are: (1) whether or not Respondent wrongfully eliminated materials from the Candidate/Petitioner during the April 19, 1990 engineering examination, and if so, (2) whether the Candidate/Petitioner received a failing grade because the materials were wrongfully eliminated.
Findings Of Fact The Petitioner (#100021) received a score of 69.0 on the Professional Engineer Fundamentals Examination given April 19, 1990. A minimum passing score was 70.0 on the examination which is written by National Council of Engineering Examiners and graded by Education Testing Service. (Transcript Pages 36 and 39) Prior to the April 1990 examination, the Board sent each candidate a letter, dated December 18, 1989 (Exhibit P-1) (Transcript Page 9 and 12), which said, "No review publications directed principally toward sample questions and their solutions of engineering problems are permitted in the examination room." (Transcript Page 31). The candidates were also provided with a "Candidate Information Booklet" dated January 1990 (Exhibit R-1, Transcript Page 77). The booklet states on page 14, "No books with contents directed toward sample questions or solutions of engineering problems are permitted in the examination room." (Transcript Pages 77 and 96). Petitioner, who also took the October 1989 examination had received notice at that examination that the Board of Engineers intended to change the procedure allowing reference materials in the examination. (Transcript Page 89 and Respondent's Exhibit 2.) The Board of Professional Engineers advised the examination supervisor and proctors that no engineering "review" materials would be allowed in the examination although engineering "reference" materials could be brought into and used for the examination. However the books which were excluded included books without "review" in the title, books with "reference" in the title, and books which contained problems and solutions. Before the examination began Deena Clark, an examination supervisor, read over a loud speaker system names of books that would not be permitted (Transcript Page 81). Practice examination and solution manuals were not allowed for use by engineering candidates (Transcript Pages 93 and 94). Schram's outlines and other materials were also excluded (Transcript Page 91). Also excluded was Lindeburg's 6th edition, "Engineering In Training Review Manual." (Transcript Pages 16 and 79). This decision was verified by the Board before the examination began (Transcript Page 81). After the examination had begun, Ms. Clark announced that the candidates could put certain copyrighted materials in a three-ring binder and use them which had been excluded earlier (Transcript Page 85). This was in response to candidates who needed economics tables for the examination However, no time was provided the candidate to prepare these references and only one minute was added to the examination time. (Transcript Page 85). Petitioner did not bring any economic tables to the examination site except those contained in books which were not allowed in the examination. (Transcript Page 19). Petitioner did not remove the economic tables and permitted references from the Lindeburg's review manual until lunch and these tables were not available to him on the morning examination. (Transcript Pages 22 and 88). Of the six engineering economics questions on the morning portion for the examination, the candidate correctly answered four. No data was provided on the nature of these questions. The Candidate correctly answered 53 questions in the morning (weighted x 1) and 23 questions in the afternoon (weighted x 2) for a total of 99 weighted required points. He answered eight questions correctly in the "addition" portion of the examination. The table for eight additional questions correct in the "Scoring Information Booklet" used in determining the candidates final grade shows the adjusted equated score was 126 and his scaled score was 69. (Page 21 of booklet). The value of each economics question converted to final scoring scale was enough that passage of one economics question would have resulted in passage of the examination. The exclusion of certain materials from the examination was arbitrary and capricious and was done by a few individuals without any stated objective standard published by the board. Further, the board knew before the examination which books were to be excluded and could have notified examinees of the exact items to be excluded. The Board's generally poor handling of this matter is exemplified in announcing after the examination had begun that items previously excluded could be used if placed in a ring binder but not allowing any time to prepare such materials. (Tx. pgs., 74-80, 84-86, and 91-97) The Petitioner would have used several tables which were excluded if the announcement had been made before the morning examination began with time to put the items in acceptable form. After notifications in October 1989, December 1989, and January 1990, Petition admitted that he did not call the Board of Professional Engineers to ask for guidance on books that would not be allowed on the April 1990 examination (Transcript Page 29). However, a final decision on books to be excluded was not made until approximately two weeks before the examination. The Petitioner did not show that the two questions which he missed on the Engineering Economics portion of the morning examination were missed for lack of the tables. The examination is a national examination and there is no evidence that the requirements and limits established by the Board in Florida were applicable nationwide. To alter the national instructions locally potentially adversely affects Florida results.
Recommendation Based upon the foregoing findings of fact and conclusions of law, it is recommended that the Petitioner be permitted to take the examination without charge on one occasion. RECOMMENDED this 27th day of March, 1991, in Tallahassee, Florida. STEPHEN F. DEAN Hearing Officer Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, FL 32399-1550 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 27th day of March, 1991. 1/ The general information provided to examinees by the State Board regarding the values of questions on the examination and scoring it misleading or inaccurate because neither the weighted required score nor the adjusted score was 48% of 80, 280, or any other number related to the scaled score of 70. The manner in which these values are associated with the scale score of 70 is contrary to the Board's explanation and is not self evident. This is a potential problem if the matter were formally challenged, and it appears the Board needs to reassess its procedures and instructions. APPENDIX TO RECOMMENDED ORDER, CASE NO. 90-5728 The Petitioner did not submit proposed findings. The Respondent submitted proposed findings which were read and considered. The following proposed findings were adopted or reject for the reasons stated: Adopted. Issue not fact. - 4. Rejected. Preliminary statement not fact. 5. -12. Adopted. Rejected. Preliminary statement not fact. Rejected as irrelevant. Rejected as preliminary statement. Adopted. Adopted. COPIES FURNISHED: Alan K. Garman Civil-Tech, Inc. 3573 Commercial Way Street B Spring Hill, FL 34606 William F. Whitson, Law Clerk Department of Professional Regulation 1940 North Monroe Street Tallahassee, FL 32399-0792 Rex Smith Executive Director Board of Professional Engineers Department of Professional Regulation 1940 North Monroe Street Tallahassee, FL 32399-0792 Jack McRay, General Counsel Department of Professional Regulation 1940 North Monroe Street Tallahassee, FL 32399-0792
Findings Of Fact Test Anxiety at the February 1994 General Contractor's Examination The General Contractor's Examination used in the licensing of individuals as general contractors is divided into three parts, each of which may be passed separately. Credit for passing any one portion of the examination is good for a period of only one year. An applicant for a license as a General Contractor is limited in taking any part of the examination to three times per year. If an applicant fails to pass all three parts of the examination in one year, therefore, the applicant must commence the examination process anew, as if none of the portions of the examination had been passed. On February 17, 1994, William T. Davenport, sat for the Contract Administration portion of the exam for the third time within a one-year period. At each of the two examinations taken earlier, he had passed one portion. The Contract Administration portion was the only part he had not yet passed. Mr. Davenport was anxious as he waited to take the exam. He fully realized that if he did not pass the Contract Administration portion he would have to start the examination process all over again. His concern was generated not only from the perspective of delay and inconvenience attendant to having to repeat the entire exam process, but also from the perspective of cost. Repeating the process would require him to pay the full exam fee after having paid already a full examination fee for each of three examinations in the year of his attempt to successfully complete the examination requirements for licensure. Different Approved Reference Lists The Construction Industry Licensing Board approves reference materials that applicants may use during the examination and issues a list of those materials periodically. The Reference List for the February examination covered the period between November 1, 1993 and February 28, 1994. On that list was a book Mr. Davenport brought with him to the examination: Practical Mathematics, 3rd Edition, Copyright 1972. As Mr. Davenport sat nervously awaiting commencement of the exam, a proctor removed Practical Mathematics from his desk indicating that use of the book was not allowed. Mr. Davenport relates his response to the incident in this way, (Tr. 26.) The anxiety was very high at that point. My concern was try to get through the exam and, when the book was taken away from me, I panicked, to be blunt. I just totally panicked. Unlike the testing period from November 1, 1993 to February 28, 1994, Practical Mathematics was not on the Reference List for the next period of testing, from March 1, through June 30, 1994. The later list, issued January 13, had been out for over a month at the time of the exam. It is likely the proctor removed the book in mistaken reliance on the later list. As it turned out, the proctor returned the book to Mr. Davenport either shortly before the examination commenced. As he did so, the proctor commented, "Well, I don't know." (Tr. 25.) Mr. Davenport did not use the book during the exam because, "[a]t that point, I was reluctant to use the book ... I didn't want the test to be invalidated and I didn't want to be challenged." Id. Not using the book proved to be critical to whether Mr. Davenport passed the Contract Administration portion of the exam. He scored a 68. A passing grade is 70. Had Mr. Davenport answered correctly question number 3, which was worth four points, he would have received a 72, a passing grade. The question involves applying a percentage. Mr. Davenport could not remember whether in obtaining a percentage it is necessary to multiply or divide. Practical Mathematics has a chapter on percentages. The chapter teaches that multiplication is the arithmetic method to use when obtaining a percentage. But Mr. Davenport guessed that division should be used. He divided by the percentage and, therefore, chose an incorrect answer. Question number 3 on the exam is one of the questions that Mr. Davenport challenged originally: A 2-man crew has consistently worked at a labor performance standard ratio of 0.85 to 1. They are selected for a job requiring 60 (standard time) man-hours to perform. They will NOT work more than 8 hours per workday. NO work will be done on Saturdays or Sundays. There are NO holidays during the time the work will be performed. According to Builder's Guide to Accounting, if the job must be finished NO later than Friday afternoon at 5:00 p.m., what is the last day that they could be scheduled to start the job? Thursday of the previous week Friday of the previous week Monday of the same week Tuesday of the same week Respondent's. Ex. No. 2. The correct answer is "(D) Tuesday of the same week." The Department's expert witness explained that the correct answer is reached by way of an algebraic formula. The formula is: "the labor performance standard ratio = x (the unknown) divided by the standard time man hours" and then that answer is divided by 2 since the crew is a 2 man crew. Applied to the problem, the formula is: .85/1 = x/60, with x, once known, divided by 2. Using the formula, the calculation goes as follows: .85/1 = x/60; multiplying both sides of the equation by 60, .85(60) = x; carrying out the arithmetic calculation, x equals 51; 51/2 = 25.5. It takes 25.5 hours, therefore, for the crew to complete the job. If, as the problem states, the job must be completed by Friday at 5 p.m. and the crew works 8 hours a day, then it will take the crew 3 full days and 1.5 additional hours to complete the job. Working backward from Friday, the crew will work 8 hours on Friday, 8 hours on Thursday, and 8 hours on Wednesday for a total of 24 hours. The crew must start on Tuesday of the same week to work the additional 1.5 hours required to complete the job. One does not need to use algebra, however, to solve the problem. One can simply obtain the number of actual hours needed to complete the job by applying 85 percent (the crew's labor performance standard) to the number of standard time manhours called for by the job, in this case, 60. Here is where Mr. Davenport needed Practical Mathematics. Not knowing whether to obtain the actual hours by multiplying .85 times 60 or dividing .85 into 60, he guessed, in error, division. Through the use of division, it appears incorrectly that the number of manhours needed is 70.588. Divided by two, to take into account that there are two members of the crew, it would take the crew 35.294 hours. If it took the crew 35 hours and a fraction to complete the job on time, the crew would need to start on Monday of the same week. "(C) Monday of the same week," is the answer chosen by Mr. Davenport. Other Distractions and the Site of the Exam The examination was conducted in the Tallahassee office of the National Assessment Institute (NAI). The NAI was under contract to the Department as the vendor to conduct the exam. At the time of the examination, the exam site was a room approximately forty feet by thirteen feet four inches in size. It contained nine tables, each 18 inches by eight feet. The tables were spaced 35 inches from each other. At each table were two straight-backed chairs. There were seven candidates present for the examination. The first and last tables were unoccupied. Each of the seven candidates were allotted 34 square feet of floor space to be occupied by the candidate, the chair, the table and materials used in the exam. On the east wall of the exam room were three plate glass fixed windows. Two of the windows are 48 inches long by 36 inches tall and one is 36 inches long by 31 inches tall. The glass is one-eighth inch thick non- commercial grade. The windows are acceptable under NAI guidelines. On the other side of the windows is a workroom that measured 15.25 by 17.5 feet. Through these windows the examination supervisor seated in the work room can monitor the performance of the proctors in the exam room and see the candidates as they take the exam. The candidates, seated to the left of the workroom, do not face the workroom. Rather, their right side is exposed to the workroom. If the tables are numbered 1 through 9 on Respondent's Ex. No. 1, beginning with 1 on the side of the room marked on the exhibit as "N" or north, candidates who were seated at tables 4 through 8 were directly exposed to the workroom windows. Mr. Davenport was seated at one of the tables exposed to the workroom windows, most likely table 3, 4 or 5, that is, one of the center 3 tables. During the exam, he could see employees through the windows moving in the workroom and hear noise from the workroom. There were four employees who were present at one time or another in the workroom. Three of these employees were also engaged in proctoring the examination. In addition to the visual diversion posed by the four employees in the workroom, Mr. Davenport could hear sounds emanating from the room. The doors to the workroom, open so that the exam supervisor seated in the workroom could hear what occurred in the exam room, also allowed sounds from the workroom into the exam room. The source of the sound was the printer working, the four workers conversing from time to time and other noises associated with an office work environment. Sight of the employees and noise from the workroom prevented petitioner from fully concentrating on the exam. Mr. Davenport was also distracted by the activities of the proctors while in the exam room. During the four hours he sat for the exam, three of the four employees he observed in the workroom were also acting as proctors. They left the workroom in a rotation in order to spell each other. During their shifts as proctors, the three monitored the exam room. Mr. Davenport felt distracted by the coming and goings of the three as they rotated in and out of the room. Although there was a table designated for the proctors at a corner outside the workroom across from table 3, they rarely sat there. They sat at one of the empty tables or walked beside the seated candidates, all the time carrying out the function of a proctor: observing the candidates during examination. The FCILB Examination Administration Manual, applicable to the February General Contractor's exam, details the responsibilities of proctors in sixteen separate counts. No. 13 reads: Proctors observe at all times and move quietly about the room. Proctors do not disturb or distract candidates during the examination. If speaking is necessary, a proctor needs to be quiet and brief as possible. Proctors avoid asking candidates to move chairs to get around them, standing too close or directly behind candidates, or rustling papers and talking to other proctors in the vicinity. Petitioner's Ex. No. 2, FCILB Examination Administration Manual, p. 2-5. Movement of the proctors was necessary during the exam because of its open-book format. It is incumbent on the proctors of an open-book exam to insure that candidates do not copy questions form the examination into their reference materials. Other Candidates Reactions to the NAI's Tallahassee Office Among the three proctors the day of the exam was Ms. Jean Love. Ms. Love is also the Office Manager of the NAI's Tallahassee Office. She has worked for NAI for over two years. Before that she worked for eight and one-half years with the Department in examination services, during which she administered exams, including acting as a proctor for exams. In addition to the daily operations of the office, she oversees the administration of examinations, a function she fulfilled at the February General Contractor's exam this year. Ms. Love did not see any unusual or distracting activities on the part of the other two proctors and did not undertake any activities, in her opinion, that would have violated any of the responsibilities of proctors, including those quoted, above, from the FCILB Examination Administration Manual. The activities during the exam in the workroom, undertaken under Ms. Love's supervision, were normal activities undertaken every day at the NAI Tallahassee Office during and outside of times of examinations. Aside from typical office activities, such as conducting telephone conversations, scheduling candidates for tests, and doing paperwork that included hand-folding documentation, there was no unusual activity the day of the exam. The only event in the workroom that contributed at all to the sound of normally quiet office activity was the validation of a single candidate's check. No complaints about noise in the workroom during the February General Contractor's Exam were registered with the NAI Tallahassee Office. Nor did any of the candidates that day complain about the activities of the proctors. Ms. Love did not learn of Mr. Davenport's complaint until after he filed his challenge to the examination questions. While a proctor may have from time to time stood near Mr. Davenport as he took the exam, none of the proctors hovered over him or, in Ms. Love's opinion, did anything that would distract the average candidate. No complaint during the examination was made by Mr. Davenport. He did not complain about inability to concentrate on the exam until after he received the exam results. During Ms. Love's two years at the NAI Tallahassee office, no candidate, prior to Mr. Davenport, had ever complained about the testing environment for any reason. The comments she has received from candidates following exams have been solely complimentary. Over the last two years, the office has administered between 15 and 20 tests per month. Complimentary comments are made, on average, by one candidate per test. In the last two years the office has received, at a minimum, well in excess of 350 compliments on the testing environment from candidates. In contrast, Mr. Davenport's complaint stands alone as the only complaint about the office testing environment in the last two years at the NAI's Tallahassee Office.
Recommendation It is, accordingly, RECOMMENDED, in the alternative: That petitioner's request for reexamination or a passing grade on the "Contract Administration, Division I" portion of the General Contractor's examination administered in February 1994 be DENIED; or, in the alternative, If the Construction Industry Licensing Board is willing to overlook the petitioner's failure to challenge his grade specifically on the appropriation of the book before the examination in the petition for formal hearing and the Department's legitimate objection to the presentation of evidence on the issue, that petitioner be allowed to sit for reexamination and, if he passes the Contract Administration portion of the exam, be credited with passing the other two portions of the exam as well as if all three portions had been passed in one year. DONE and ENTERED this 1st day of December, 1994. DAVID M. MALONEY Hearing Officer Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-1550 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 1st day of December, 1994. APPENDIX Petitioner's proposed findings of fact Nos. 3, 4, 5, 6, 9, 10, 12, and 13 are adopted, in substance, insofar as material. With respect to petitioner's proposed finding of fact No. 1, the first and last sentences are adopted. The remainder of the proposed finding, and in particular the reference to Linda Chaffin, test proctor, is rejected because it is either not supported by the evidence or argumentative in nature rather than factual. Petitioner's proposed finding of fact No. 2 is rejected as against the weight of the evidence. Petitioner's proposed finding of fact Nos. 7 and 8 are rejected as unsupported by the evidence. With respect to petitioner's proposed finding of fact No. 11, the reference to Linda Chaffin is rejected. Ms. Chaffin was not identified by the evidence as the proctor who removed the book from petitioner prior to the exam. Respondent's proposed findings of fact are adopted, in substance, insofar as material. COPIES FURNISHED: William T. Davenport 336 14th Avenue, North Jacksonville, FL 32250 William W. Woodyard Assistant General Counsel D B P R 1940 North Monroe Street Tallahassee, FL 32399-0750 Richard Hickok, Executive Director C I L B 7960 Arlington Expy., Ste. 300 Jacksonville, FL 32311-7467 Jack McRay General Counsel D B P R 1940 North Monroe Street Tallahassee, FL 32399-0750
Findings Of Fact The Respondent, Leo Y. Lewis, from 1969 to date had been employed by the State of Florida as Internal Audit Coordinator for the Department of Health and Rehabilitative Services. Respondent applied for and was issued a reciprocal certificate by the State Board of Accountancy on January 28, 1972. The reciprocal certificate was issued under the provisions of Florida Statutes, Section 473.201. 473.201 Certificates granted to holders from other states Upon application, the board shall issue a reciprocal certificate to the holder of a valid, unrevoked certificate issued by, or under the authority of, another state or political subdivision of the United States only if: The original certificate was secured as the result of a written examination which in the judgment of the board was the equivalent of the examination given by the board at the time the applicant passed such written examination; provided, however, that if such written exami- nation did not include the subject of ethics the application shall not be denied for that reason. In any event, the board shall require all applicants to take and pass the same current open book examination in ethics given to applicants for the written examination in the state before issuance of a reciprocal certificate; The applicant meets all the requirements in effect, at the time of the application, for applicants to take the Florida examination; The applicant is a resident of and domiciled in the state; The applicant intends to enter into the full-time, year-round practice of public accounting in the state; and The board is otherwise fully satisifed as to the moral and technical fitness of applicant." The Board now seeks to revoke the reciprocal certificate issued to Respondent for failure to comply with Section 473.201(2), Florida Statutes, "(2) Failure of the holder of a certificate issued under this section to be domiciled in and practice public accounting on a substantially full-time basis in the state for a continuous period of three years immediately after issuance of much certificate shall be prima facie evidence of the lack of the requisite intent and sufficient grounds for revocation of the reciprocal certificate issued by the board.", contending that Respondent has not been practicing public accounting for the three years immediately subsequent to the issuance of his certificate. There was no contention by the Board in its original revocation procedures that the Respondent has failed to meet all of the requirements of Section 473.201(1) and (2), Florida Statute, with the exception of the contention that Respondent has not practiced "public accountancy". At the hearing and in the Memorandum of Law there is an argument by the Petitioner that the Respondent did not actually intend to practice public accounting at the time that he received a reciprocal certificate. There is no showing that any fraud was involved and the grounds upon which the motion for revocation of Respondent's license is on the ground that Mr. Lewis did not engage in the practice of public accounting on substantially full-time basis for a period of three years after issuance of his reciprocal certificate. His intent at the time he received his reciprocal certificate is not necessary to the determination of the issue at hand. Petitioner contends: That Respondent is not in the practice of public accounting; That Section 473.08(2)(c) applies only to the Auditor General and tlie Chief Auditor of the Public Service Commission and that Section 473.08(2)(c), infra, is an exception to the general requirement that persons who have taken and passed an examination and become CPA's must have one year of public accounting experience or equivalent education in order to obtain their CPA certificates; and That the exception is not a recognition that either the Auditor General or the Chief Auditor of the Public Service Cotmission is engaged in the practice of public accounting. Respondent contends: That he is in fact a Florida practitioner engaged in the practice of public accounting; That the legislature and the Board have recognized that works similar to that which the Respondent performs constitutes the practice of public accounting. Said statute as passed by the legislature is Section 473.08, Florida Statutes. "473.08 Qualifications of applicant for examination; certificates to successful examinees; standards A person who qualifies to take such examination pursuant to provisions of subsection (1) and who takes and passes such examination shall receive a certificate as a certified public accountant issued by the board and shall be permitted to practice public accounting in this state if he:... Shall have completed one year of employment in the office of a Florida practitioner or an out-of-state practitioner or shall have successfully completed an additional one year accounting course at an accredited college or university. However: Any person employed as an accountant in the accounting department of the Florida public service commission who is qualified to take an examination for the purpose of determining whether or not such person shall be permitted to practice in this state as a certified public accountant, and who takes and passes such examination, shall be entitled to receive a certificate under the provisions of authorizing practice in this state as a certified public accountant upon completion of one year of accounting work for said commission under the supervision and direction of a certified public accountant serving as director of commission's accounting department. Any person employed as an accountant or post auditor on the staff of the auditor general who is qualified to take an examination for the purpose of determining whether or not such person shall be permitted to practice in this state as a certified public accountant, and who takes and passes such examination, shall be entitled to receive a certificate, under the provisions of this authorizing such person to practice in this state as a certified public accountant upon the completion of one year of experience as an accountant or post auditor under the supervision and direction of a certified public accountant serving as auditor general. Proof of compliance with the provisions of this introductory paragraph of this subsection shall be established in such form as is prescribed by the board;. The Florida Statutes do not clearly define the term "public accounting", but the definitions in Section 473.011 are helpful. "473.011 Definitions of terms used in chapter whenever the terms `certified public or public accountant are used in this chapter, except as used in subsection (3), they shall be deemed and construed to mean a person holding a certificate to practice as such, issued by the state under this chapter, or any law of the state heretofore in force and effect. [(3) not applicable] whenever the term `Florida practitioner' is used in this chapter it shall be deemed and construed to mean a certified public accountant or public accountant, as defined in subsection (1), engaged in the practice of public accounting in Florida, whether as an individual, a partner or employee of a partner- ship, or a stockholder, officer, or employee of a professional corporation. It shall also be deemed and construed to mean a partnership or professional corporation of Florida practitioners. (5) Thenever the term `public accounting is used in this chapter, it shall be deemed and construed to mean: All services offered to or performed for the public by a Florida practitioner or an out of state practitioner involving the use of accounting skills, specifically including, but not limited to, management services, and All services offered to or performed for the public by any other person, acting as an individual, as a partner or employee of a partnership, as a stockholder, officer or employee of a professional corporation, or as an officer or employee of any other corporation, involving the use of accounting skills, except as follows: (not applicable) Under the definition section of Rule 21A-1.10 of the Florida Administrative Code, practice of or practicing public accounting is defined as: "Practice of, or practicing public accounting. 'Practice of, or practicing public accounting' shall be deemed and construed to mean offering to perform, performing, or holding oneself out as being qualified to perform, those services described in Section 473.011(5), F.S." [supra] as: Rule 21A-1.11 of the Florida Administrative Code defines practitioner "'Practitioner' shall be defined as either a Florida practitioner (as defined in Section 473.011(2), F.S.), or an out of-state practitioner (as defined in Section 473.011(3), F.S.), engaged in the practice of public accounting in Florida under a special permit." Black's Law Dictionary, Revised Fourth Edition, defines public as: "PUBLIC, adj. Pertaining to a state, nation, or community; proceeding from, relating to, or affecting the whole body of people or an entire community. Open to all; notorious. Common to all or many; general; open to common use. Morgan v. Cree, 46 Vt. 786, 14 Am.Rep. 640; Crane v. Waters, C.C.Mass., 10 F. 621. Belonging to the people at large; relating to or affecting the whole people of a state, nation, or community; not limited or restricted to any particular class of the community. People v. Powell, 280 Mich. 699, 274 N.W. 372, 373, 111 A.L.R. 721." Black's Law Dictionary, Revised Fourth Edition, defines accounting as: "ACCOUNTING. An act or system of making up or settling accounts; a statement of account, or a debit and credit in financial transactions. Kansas City v. Burns, 137 Kan. 905, 22 P.2d 444." The Hearing Officer finds: That the work the Respondent does in his position as Internal Audit Coordinator for the Department of Health and Rehabilitative Services involves the use of accounting skills and includes management services and that he is a Florida practitioner as defined in Chapter 473, Florida Statutes, and in Chapter 21A-I, Florida Administrative Code. Petitioner contends that the Respondent is a state employee and therefore cannot be independent of his client However, the evidence shows that much of the work performed by Respondent and his staff are for agencies and organizations outside the Department of Health and Rehabilitative Services which employs Respondent. There is no employee-employer relationship between Respondent and these organizations. Further, no showing has been made that the employer, the Department of Health and Rehabilitative Services, attempts to or has ever directed the work of Respondent or in any way exercised control over the audits of Respondent. Respondent is in the practice of public accounting in his capacity as Internal Audit Coordinator.
Recommendation Dismiss the petition of the State Board of Accountancy. DONE and ORDERED this 24th day of May, 1976. DELPHENE C. STRICKLAND Hearing Officer Division of Administrative Hearings The Carlton Building, Room 530 Tallahassee, Florida 32304 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 24th day of May, 1976.
The Issue Whether Petitioner is entitled to a passing score on the pharmacology/ocular disease portion of the optometry licensing examination administered August 3, 2000.
Findings Of Fact Pursuant to Chapter 456, Florida Statutes, Respondent is the agency of the State of Florida that develops, administers, scores, and reports scores for licensure examinations, such as the examination at issue in this proceeding. The Board of Optometry is created as a part of Respondent by Section 463.003(1), Florida Statutes. Pursuant to Section 456.013(4), Florida Statutes, this Recommended Order is to be forwarded to the Board of Optometry, which will enter a Final Order. Section 463.006(1), Florida Statutes, provides that anyone seeking licensure as an optometrist must pass a licensure examination. Section 463.006(2), Florida Statutes, provides, in part, as follows: The examination shall consist of the appropriate subjects, including applicable state laws and rules and general and ocular pharmacology with emphasis on the topical application and side effects of ocular pharmaceutical agents. . . . The optometry licensing examination consists of four separate examinations, one of which is the pharmacology examination. A candidate cannot be licensed as an optometrist in Florida until he or she passes all four examinations. In 1999, Petitioner passed three of the four examinations, but he failed the pharmacology examination. Petitioner retook the pharmacology examination on August 3, 2000. Pursuant to Section 456.017(2), Florida Statutes, and Rule 64B13-4.002, Florida Administrative Code, Petitioner did not have to retake the three portions of the licensure examination he passed in 1999. A candidate who fails a licensure examination has the right to review the examination material to determine whether he or she wants to file a challenge to the grading of the examination. Pertinent to this proceeding, Section 456.017(2), Florida Statutes, requires the following of Respondent: . . . provide procedures for applicants who fail an examination to review their examination questions, answers, papers, grades, and grading key for the questions the candidate answered incorrectly or, if not feasible, the parts of the examination failed. . . . Respondent is required to maintain the examination material by Section 456.017(3), Florida Statutes, which provides as follows: (3) For each examination developed or administered by the department or a contracted vendor, an accurate record of each applicant's examination questions, answers, papers, grades, and grading key shall be kept for a period of not less than 2 years immediately following the examination, and such record shall thereafter be maintained or destroyed as provided in chapters 119 and 257. This subsection does not apply to national examinations approved and administered pursuant to this section. A candidate is not allowed to retain a copy of the examination material or to make any copy thereof. Rule 64B13- 4.003, Florida Administrative Code, provides as follows: (3) An applicant is entitled to review his examination questions, answers, papers, grades and grading key used in the state examination for licensure; however, no applicant may copy any materials provided for his review. . . . A candidate has the right to a second review of the examination material in order to prepare for an administrative hearing. The candidate's attorney can participate in this second review. Rule 64B-1.009(1), Florida Administrative Code, provides, in pertinent part, as follows: (1) After the candidate's petition, which is a written statement requesting a hearing pursuant to 120.57, Florida Statutes, and setting out the information required under rule 28-106.201 of the Florida Administrative Code, has been filed, the candidate, and/or the candidate's attorney shall be permitted to review the examination questions and answers at the department's headquarters for the purpose of preparing for the administrative hearing, as specified in board rule or by the department when there is no board. . . . The examination at issue in this proceeding was not a national examination. Respondent maintains its master examination item bank for the optometry examination by computer. 1/ Typically, an examination booklet for a particular examination is printed from that computer item bank only when the booklet is needed for a legitimate purpose, such as an examination, a review, or a hearing. Once the booklet has served its purpose, it is destroyed. A psychometrician and three consulting optometrists usually proofread the contents of a newly printed examination booklet before it is used for an examination. The pharmacology examination at issue in this proceeding consisted of different case histories, each of which described a patient’s presenting condition and pertinent medical history. Each case history was followed by five questions with multiple choice answers. Candidates were instructed to select the best answer to each question from the multiple choice answers provided in the examination booklet. Respondent printed Booklet D from its master examination item bank for use as an exhibit in this proceeding. Booklets A, B, and C were not available for use as exhibits. Following his review of the examination material on November 7, 2000, Petitioner filed the Petition that underpins this proceeding. Question 74 required a candidate to select the best treatment for a patient based on the patient's case history. The candidate had 7 possible answers, lettered A - G, from which to choose. Each of the choices was a prescription medicine. In discussing Question 74, the Petition alleged that according to the answer key, the best answer was a certain topical steroid, which was choice F on Booklet D. That assertion is wrong. Choice E, not choice F, was the choice identified by the answer key as being the best answer to Question 74. Petitioner's response to Question 74 on August 3, 2000, was choice B. In discussing Question 44, the Petition alleged that according to the case history, a particular diagnostic procedure had not been performed on the patient. The last sentence of the case history for this question in Booklet D reflected the results of the diagnostic procedure that Petitioner alleged was not performed. Petitioner reviewed the examination material, including Booklet C, to prepare for the final hearing in this proceeding on February 28, 2001. Petitioner testified at the final hearing that the medication identified by Respondent as being the best response (choice E in Booklets C and D) to Question 74 was not an available answer in Booklets A and B. Petitioner testified at the final hearing that the last sentence of the case history for Question 44 in Booklets C and D had been omitted from Booklets A and B. Petitioner continued to assert that his responses to Questions 44, 74, and 75 were the best responses as those questions were presented to him when he took the examination. Lee Skinner, a psychometrician employed by Respondent, supervised the administration of the pharmacology examination at issue in this proceeding. Mr. Skinner and three consulting optometrists proofread the examination booklets used for the August 3, 2000, pharmacology examination. Mr. Skinner testified that Booklet A was identical in all material respects to Booklet D and that the alleged omissions did not exist. Consistent with Respondent’s policies, the hard copy of Booklet A was destroyed following the administration of the examination. Petitioner's answer sheet and the notes he took during the examination were preserved and were admitted as exhibits. Consistent with Respondent's examination review policies, Petitioner was not permitted to retain a copy of or make notes as to Booklet A, B, or C. For reasons that cannot be attributed to him, Petitioner’s testimony as to the alleged omissions in Booklets A and B could not be corroborated because hard copies of the examination booklets at issue were not available. 2/ Because Petitioner could not have a copy of or make notes from the examination booklets, he had to rely on his memory when preparing the underlying Petition and in testifying. Mr. Skinner’s testimony that there were no material differences between Booklet A and Booklet D is credible and persuasive. Petitioner failed to prove the alleged discrepancies between Booklet A and Booklet D. A score of 70% is needed to pass the pharmacology examination. Petitioner's score on the pharmacology examination administered August 3, 2000, was a failing score of 68.5%. Each of the three questions at issue is worth 0.75%. Petitioner would have to receive credit for a correct answer to at least two of the three questions at issue in this proceeding to attain the additional 1.5% he needs to pass the examination. The case history for Question 44 contained all the information necessary for a candidate to select the correct answer. On August 3, 2000, Petitioner did not select the best answer to Question 44. Consequently, he is not entitled to additional credit for his answer to that question. The case history for Questions 74 and 75 contained all the information necessary for a candidate to select the correct answer. Petitioner received no credit for his answer to Question 74 because he did not select the best answer to that question. Question 75 required the candidate to select the correct dosage and manner of administration of the medicine that was the best answer to Question 74. Petitioner's incorrect answer to Question 74 caused him to miss Question 75. Petitioner received no credit for his answer to Question 75 because he did not select the correct answer to that question. In addition to proofreading an examination booklet, a psychometrician typically reviews all answers to a licensure examination to make sure that no question was invalid. A question is considered invalid if 30% or fewer candidates select the answer identified by Respondent as being the best answer. Mr. Skinner reviewed all answers to Questions 44, 74, and 75 to determine whether an abnormal number of candidates missed each question. Based on the number of candidates that correctly responded to the three questions at issue compared to the incorrect answers, Mr. Skinner opined that each of the three questions was a valid examination question. 3/ Petitioner failed to establish a basis to disqualify Questions 44, 74, or 75.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that Respondent enter a final order dismissing Petitioner's challenge to the grading of his responses to Questions 44, 74, and 75 of the pharmacology examination administered August 3, 2000. DONE AND ENTERED this 14th day of June, 2001, in Tallahassee, Leon County, Florida. CLAUDE B. ARRINGTON Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 14th day of June, 2001.
Findings Of Fact Based upon all of the evidence, the following findings of fact are determined: On April 19, 1990, petitioner, Don R. Blackburn, was a candidate on the engineering intern portion of the professional engineer examination given in Miami, Florida. The test was administered by the Department of Professional Regulation (DPR) on behalf of respondent, Board of Professional Engineers (Board). On July 25, 1990, the Board issued a written uniform grade notice advising petitioner that he had received a grade of 66 on the examination. A grade of 70 is necessary to pass this part of the examination. By letter dated August 15, 1990, petitioner requested a formal hearing to contest his score. In his letter, Blackburn generally contended that the examination was unfairly administered because certain books were allowed to be used by some but not all candidates, untrained proctors were given the authority to scan review materials and determine which could or could not be used by the candidates, and because of the chaos and confusion that occurred during the examination, he was unable to attain a score that he otherwise would have been able to achieve. Blackburn is an engineer for Lee County and is seeking to pass the engineering intern portion of the examination. A passing grade on that portion is a prerequisite to sitting on the second part of the professional engineer examination. He has taken the examination on a number of occasions and has gradually improved his score to just short of passing. Indeed, on the October 1989 examination, Blackburn scored a 69, or just one point less than the required 70. Prior to the April 1990 examination, the engineering intern portion of the professional engineer examination was an unrestricted open book examination. This meant candidates could use any and all reference and review materials during the examination. Beginning with the April 1990 examination, the Board imposed certain restrictions on the use of review materials. As early as October 9, 1989, the Board's executive director sent a memorandum to all candidates on the October 1989 examination, including Blackburn, concerning the new restrictions. The memorandum stated in part: Please be advised of certain restrictions listed in the Candidate Information Booklet which will not be implemented until the April 1990 examination. These restrictions are found in the "Examination Administration Information" section and are concerning the following two areas: * * * 2. Books or information containing sample questions or engineering problems may also be brought provided they are bound. Again, the new restrictions listed in the Candidate Information Booklet regarding the above two areas WILL NOT be implemented until the April 1990 examination. All candidates on the April 1990 examination were given a Candidate Information Booklet prepared in January 1990 by DPR's Bureau of Examination Services. On pages 13 and 14 of the booklet was found the following information: This is an open book examination. Candidates may use textbooks, handbooks, notes, and reference materials which are bound, copyrighted and printed. The term "bound" refers to material that is bound permanently, hard or paperback stitched or glued, or spiral, plastic or three-ringed bound. The printed material must remain contained (bound) in its cover during the entire examination. No writing tablets, unbound tablets or unbound "loose notes" will be allowed. No books with contents directed toward sample questions or solutions of engineering problems are permitted in the examination room. Examinees are not permitted to exchange reference materials or aids during the examination. (Emphasis in original) What the emphasized language meant is that "review" manuals, which contain problems and solutions, were prohibited from use during the examination while "reference" books were not. However, the booklet did not list the specific names of published materials that would be permitted or excluded. In order to ascertain which books he might use on the next examination, on March 27, 1990, Blackburn telephoned the Board in Tallahassee and spoke with a female employee named "B. J." who advised him that "review publications directed principally towards the solution of engineering problems" would be excluded. When asked if "Lindeburg's Sixth Edition" would be authorized, B. J. told Blackburn she wasn't sure and that it would be left up to the proctors in the room. She did say, however, that a review manual authored by Schaum could be used. The engineering intern examination in April 1990 was administered in two separate rooms at the Radisson Hotel in Miami, Florida. Blackburn was in a "very large" upstairs room with approximately thirty other candidates while a similar number took the examination in a downstairs room. The examination in the upstairs room began at 8:43 a.m. after various instructions were read to the candidates by the examination supervisor, Jeannie Smith, a veteran of twenty years in proctoring and supervising professional examinations. According to Smith, there was "considerable confusion" concerning which books could be used by the candidates, particularly since this was the first examination given with the new restrictions. She also acknowledged that there was "chaos" prior to the beginning of the examination and that this was, "extremely upsetting" to the examinees. However, before the examination began, Smith announced on a microphone the names of certain books which the Board had given her that were either prohibited or could be used by candidates. She further advised that if candidates had any questions they were to come to a bulletin board by the microphone where she had posted Xerox copies of the covers of various books. If a book could be used, it had the word "YES" printed on the cover while a "NO" was printed on those covers of books that could not be used. 1/ It is noted that only one cover sheet with a "YES" was posted, that being the Civil Engineering Reference Manual, Fourth Edition, Michael R. Lindeburg. However, at least three candidates who took the examination that morning, including petitioner, did not see the posted materials nor hear the invitation for candidates to come to the bulletin board. One book in issue that was specifically prohibited was Engineer In Training Review Manual, Sixth Edition, Michael R. Lindeburg, which contained 378 solved problems, and thus fell within the general prohibition of review manuals described on page 14 of the Candidate Information Booklet. However, those candidates who had the Seventh Edition of the same book were allowed to keep and use that manual even though it contained 422 solved problems, or some 44 more solved problems than was contained in the prohibited Sixth Edition. By allowing those students having the Seventh Edition to use the same even though it contained "review" materials, DPR violated the instructions contained in the Candidate Information Booklet and gave an advantage to those candidates not enjoyed by others, including petitioner. In addition, at least one other candidate in the upstairs group was allowed to use a prohibited review manual (Schaum's Outline Series, Theory and Problems of Electric Power Systems) but still that candidate did not attain a passing grade. Petitioner also contended that candidates taking the examination in the downstairs room were allowed to use language dictionaries during the morning part of the examination while those upstairs could not. 2/ Petitioner's contention is grounded upon hearsay evidence and accordingly it is found that no competent proof to support this claim was submitted. However, there was obviously some confusion over this matter because, after receiving complaints of this nature from two candidates, Smith telephoned the Board's offices in Tallahassee during the lunch break to ascertain whether such books could be used. Upon learning that they could not, she advised the upstairs group at the beginning of the afternoon session that dictionaries were not allowed. Blackburn also established that during the examination proctors went from desk to desk examining the materials that each candidate had in his possession. If a candidate had what the proctor perceived to be a book containing solutions to problems, the candidate was told to put the book on the floor. In the alternative, she candidates were told that if they tore the offending pages out of the book, they could continue using the remaining materials. Petitioner has complained that the proctors were not engineers and they were untrained in determining whether a book was acceptable or not. The Board has conceded that engineers do not proctor examinations but asserted that they are intelligent enough to determine whether books fall within the proscribed category. According to Blackburn's proctor at the examination, George Walton, a retired Coast Guard captain and engineering graduate of the Coast Guard Academy, he relied upon the list of approved and disapproved books supplied by the Board prior to the examination in determining whether materials would be excluded or not. Walton also stated that if he examined a book and found it contained solutions, he would disallow the same unless the offending pages were removed. A DPR expert in testing and measurements, Dr. Joseph A. Klock, examined the pass/fail rate for the examination taken by Blackburn and compared that rate to the October 1989 examination rate. Doctor Klock found no significant difference in the two rates and concluded that there was no statistically significant difference in performance of candidates over those time periods despite the confusion which occurred during the April 1990 examination. Blackburn did not present any evidence to show that if he had used the Seventh Edition of the Engineer In Training Review Manual, he would have been able to achieve more points on a particular problem and thus would have had a passing grade. Blackburn's principal complaint was that he had spent many hours preparing for the examination in question, that he was forced to guess which books to bring to the examination, and because of the confusion and chaos that took place at the beginning of the examination as well as his awareness that others were using a review manual with solved problems, it was impossible for him to give his best effort on the examination.
Recommendation Based on the foregoing findings of fact and conclusions of law, it is RECOMMENDED that petitioner's request to receive a passing grade on the April 1990 professional engineer examination be DENIED. However, petitioner should be entitled to retake the next examination at no charge. DONE and ENTERED this 28th day of November, 1990, in Tallahassee, Florida. DONALD R. ALEXANDER Hearing Officer Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, FL 32399-1550 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 28th day of November, 1990.
Findings Of Fact On May 23, 1975, the Petitioners herein took the general contractors licensing exam in Miami, Florida. Petitioners failed to achieve a passing score on said examination and thereafter several reviews of the examination questions and answers resulted. The exam in question was administered to approximately 659 examinees. The test was made up of 100 questions and the examinees were allotted 4 hours to complete the exam. The examinees were instructed in the examination booklet to answer as many questions as possible within the time limit and to always select the best possible answer out of the listed choices. The examinees were furnished a list of reference materials by Respondent and they were advised that the exam questions would come from some 18 odd reference books supplied on the reference lists. Petitioner Dennis Milch took the general contractor's examination administered on May 23, 1975. 2/ In preparation of the exam he took a prep course given by Cole Construction and began his preparation approximately two months prior to the exam. He received a score of 67.5 on the exam. He earned a degree from the University of South Florida and took advance construction courses at FIU where he earned a degree in marketing. His work experience consisted of serving as an apprentice carpenter for Burke Construction Company in Miami for approximately two months and as a contractor to build residential homes in Houston Texas. Milch voiced his opinion that the exam questions failed to satisfy the statutory requirements of being "objective" within the meaning of Chapter 468, Florida Statutes. Joseph Cole, the founder of Cole Construction College in 1949, testified that he had approximately 30 years experience in teaching construction, architectural and engineering courses. He had conducted various seminars for students and received a B.S. Degree from the University of Miami. He received a B.S. Degree in biochemistry and civil engineering from the University of Pittsburgh and conducted seminars at the University of Florida in Math, Physics and Engineering. He also conducted seminars at the Markowitz Engineering School. He was licensed in 1947 in Coral Gables, Miami Beach, and in Miami where he has built approximately 2,000 single-family homes, high-rise buildings and apartments. He aired his opinion that Milch missed approximately 36 questions of which approximately 24 were what he regarded as "impossible" questions. He expressed his awareness that during the morning sessions two questions were voided and credits were given to all examinees having a point value of 1 point each. Two questions were also voided from the afternoon session. Thomas H. Hebert, an associate of the testing agency which compiled the exam for the Board, i.e., Bryon, Harlow, Schaefer, Reed and Associates, explained the procedure for compiling the tests for the Board (Respondent). He stated that data is taken from Board references and an exam format is established. Examinees are tested on plan reading and estimating using standard plans for takeoff and specification requirements. The test is first administered to contractors and others who had previously passed the exam. This is done to test time limits, etc., and grading procedures. By so doing, he testified that it is possible to correct deficiencies in the exam. After the examination is compiled and is administered to the agency employees and other contractors who have passed the exam, an item analysis is compiled and computerized. There are five possible answers for each question. The exam is divided into three segments, i.e., the upper 27 percent, the mid 46 percent, and the lower 27 percent. After the tests results are in, the weighting on various questions are checked to see if large numbers of examinees "jump" one question and further to see if questions are ambiguous. If found to be or that there are two correct answers for a given question, credit is given for both answers. Thereafter, a discrimination index is compiled based on the lower and upper 27 percent. These papers are scrutinized and if there is a discrepancy in excess of .1 to .8 percent, the question is examined and a solution is derived at based on results gathered from the scrutiny. He testified further that if an exam paper is mutilated or is otherwise difficult to machine score, it is hand graded. All exam papers in which the score ranged from 0 to 30 are hand graded as are those where the score ranges between 60 to 70. Of those questions where there is no correct answer, the question is deleted and a new base is established. For example, if a question is deemed faulty, each question has a weight of 1.1 one hundredths of a point. If they have three correct answers, points are given for all three answers. It was further brought out during his testimony that it was not necessary for a contractor to pass the certification examination in order to practice contracting in Florida. Evidence reveals that there are two kinds of licenses issued by the Board, i.e., registration and certification. The registration process only requires compliance with local requirements and the filing of a form with the Board, which may be the passing of a local competency exam or simply obtaining a local occupational license. The Certification method is optional and if the contractor passes the certification examination, it is unnecessary for him to take any local examinations. After going over various questions missed by Petitioner Dennis Milch, Petitioners argue that the scope of the Board Certification Examination included questions affecting the business of contracting as well as the technical aspects of for example how to nail two boards together to make a safe structure. Florida Statutes Chapter 468.106(2)(a) provides for an examination covering knowledge of basic principles of contracting and construction. Chapter 468.101 declares the purpose of Chapter 468 and states in pertinent part that "any person desiring to obtain a certificate to engage in the business shall be required to establish his competency and qualifications." Hence, the legislature has covered the business of contracting as well as the theory of construction. This serves the purpose of Chapter 468 by making it safer for owners to contract with the contractors and to have assurances that no liens will be placed on their property by subcontractors, that the owners are safe from suit from work that was done on the job, that the payments made on the construction will not be diverted and that the contractor understands his obligations. This requires general knowledge of the mechanics lien law, basic contract law and workers compensation law, all of which were tested by the subject examination. Respecting Petitioners' argument that they were denied certain constitutional guarantees when they were instructed by Respondent to select the best possible answer but that after the test was administered and Respondent determined that many questions had no best choice, the Board failed to delete such questions from the exam, it was noted that after the Respondent discovered that several test questions were deemed acceptable but that the answers offered did not meet the tests of selecting the best possible answer, adjustments were made. In other words, there was no single best possible answer for approximately four questions. Rather than deleting the entire question, Respondent permitted those examinees who selected either the answer originally preferred by the Board or one of the later adopted alternate answers to achieve full credit for such questions and answers. The statute (Chapter 468, F.S.) mandates that the examination be an "objective" written examination. The criteria of objectivity is not met where the examining body is granted the discretion to accept alternate answers to a given question. A "best" answer is something different from an acceptable answer. To give the Board discretion to accept alternate answers would authorize a substitution of standards which is' not permitted by Chapter 468, F.S. Once subjectivity comes into play, Respondent becomes vested with almost unbridled discretion in deciding who shall become a certified general contractor. This was prohibited by the legislature by requiring objectivity in setting a uniform minimum test grade. As relates to Petitioner Milch, it was noted that a subsequent review of his exam resulted in his being awarded a half credit for his answer to question number 39 and the Board determined that after review answers B and A were both correct. The net result of this was that his overall score was 68. A review of the court cases revealed that Florida courts have not been involved in the minute details of how examination grades or points are awarded. See the cases of State ex rel. Topp v. Board of Electrical Examiners, 101 So.2d 583 (Fla. App. 1st 1968), and State ex real. Lane v. Dade County 258 So.2d (347 Fla. App. 3rd, 1972). These cases generally show that unless there is a clear abuse of discretion, courts shall not substitute their judgment for the agencies as to how examinations are graded. Petitioners also submit that once regrading had commenced, the Board should have deleted all questions with wrong answers or more than one equally acceptable answer, distributed the weight of the deleted questions in proportionate fashion of the remaining questions and considered passing to be 70 percent of the total points available, rather than 70 cumulative points. During the hearing, Petitioners failed to show how they were injured by the difference in the award of the points for questions deleted. A wrong without damage does not constitute a good cause of action. Based on the evidence presented it appears that the Petitioners are treated the same as all other examinees. Since the Petitioners have failed to establish that if the assignment of points were different, they would have passed the examination, this argument is moot. Petitioners also alleges that they were denied certain constitutional protections by Respondent's failure to adopt and promulgate uniform rules and regulations concerning preparation, administration and review of licensing examinations. Florida Statutes, Chapter 468, requires Respondent to conduct its affairs pursuant to Florida Statutes Chapter 120. The Administrative Procedure Acts set out specific procedures to be followed by State agencies in adopting, promulgating and enforcing rules. Statutory authority governing the granting of a license should be strictly followed. In this case, there is no evidence of the existence of any unlawful rule or regulation adopted by Respondent to govern any of the variety of issues concerning the licensing of general contractors. Petitioners also submit that the Board should be required to promulgate and enforce rules concerning examinations and appeals or results thereof and cite the reasons for the actions it takes prior to its review of the examination. The Board is not required to adopt rules and regulations in every area in which it is authorized to act by statute. Where the statute is clear, there is no requirement or reason by the Board to adopt rules. Here, the Board provided the applicants with a chance to examine the questions, their papers, grades and to complain if they wished about the questions either individually or at board meetings with the possibility that the fairness of the questions could be resolved quickly and informally and if necessary, as in this case, without the full ponoply of an administrative hearing. By so doing, the Board was clearly following its statutory duty to provide the applicants with a chance to see their examination papers and grades. (F.S. Chapter 466.110). Based on the above, it is concluded that Respondent compiled the May 23, 1975 examination based on objective standards. When the Board determined that certain questions were defective either because there was more than one answer or for other reasons, the Board reviewed said questions and credited those examinees who failed to properly answer the question. By so doing, Petitioners were treated the same as all examinees who took the exam. Based on the record evidence, it further appears that all the questions meet the statutory tests of being objective and the Board's determination that a cumulative score of 70 percent is necessary to successfully obtain a certification, was not shown by any competent or substantial evidence to be an abuse of discretion. It is therefore recommended that the agency's action be sustained and that the petition filed herein be dismissed.
Recommendation Based on the foregoing findings of fact and conclusions of law, it is therefore recommended that the agency's actions be affirmed and the petition filed herein be DISMISSED. DONE and ENTERED this 17th day of January, 1977, in Tallahassee, Florida. JAMES E. BRADWELL, Hearing Officer Division of Administrative Hearings Room 530, Carlton Building Tallahassee, Florida 32304 (904) 488-9675
