The Issue The ultimate issue in this proceeding is whether proposed Florida Administrative Code Rule 61G15-21 is an invalid exercise of delegated legislative authority.
Findings Of Fact Florida Administrative Code Rule 61G15-21.004, in relevant part, states: The criteria for determining the minimum score necessary for passing the Engineering Fundamentals Examination shall be developed through the collective judgment of qualified experts appointed by NCEES to set the raw score that represents the minimum amount of knowledge necessary to pass the examination. The judges shall use a Modified Angoff Method in determining the minimally acceptable raw score necessary to pass the Fundamentals of Engineering Examination. Using the above mentioned Modified Angoff Method, the judges will indicate the probability that a minimally knowledgeable Fundamentals of Engineering examinee would answer any specific questions correctly. The probability of a correct response is then assigned to each question. Each judge will then make an estimate of the percentage of minimally knowledgeable examinees who would know the answer to each question. The totals each of the judges is added together and divided by the number of judges to determine the overall estimate of the minimum standards necessary. The minimum number of correct answers required to achieve a passing score will take into account the relative difficulty of each examination through scaling and equating each examination to the base examination. The raw score necessary to show competence shall be deemed to be a 70 on a scale of 100. A passing grade on Part Two of the examination is defined as a grade of 70 or better. The grades are determined by a group of knowledgeable professional engineers, who are familiar with engineering practice and with what is required for an applicable engineering practice and with what is required for an applicable engineering task. These professional engineers will establish a minimum passing score on each individual test item (i.e., examination problem). An Item Specific Scoring Plan (ISSP) will be prepared for each examination item based upon the NCEES standard scoring plan outline form. An ISSP will be developed by persons who are familiar with each discipline including the item author, the item scorer, and other NCEES experts. On a scale of 0-10, six (6) will be a minimum passing standard and scores between six (6) and ten (10) will be considered to be passing scores for each examination item. A score of five (5) or lower will be considered an unsatisfactory score for that item and examinee will be considered to have failed that item. To pass, an examinee must average six (6) or greater on his/her choice of eight (8) exam items, that is, the raw score must be forty- eight (48) or greater based on a scale of eighty (80). This raw score is then converted to a base 100 on which, as is noted above, a passing grade will be seventy (70). The proposed changes to Florida Administrative Code Rule 61G15-21.004, in relevant part, state: The passing grade for the Engineering Fundamentals Examination is 70 or better. The criteria for determining the minimum score necessary for passing the Engineering Fundamentals Examination shall be developed through the collective judgment of qualified experts appointed by NCEES to set the raw score that represents the minimum amount of knowledge necessary to pass the examination. The judges shall use a Modified Angoff Method in determining the minimally acceptable raw score necessary to pass the Fundamentals of Engineering Examination. Using the above mentioned Modified Angoff Method, the judges will indicate the probability that a minimally knowledgeable Fundamentals of Engineering examinee would answer any specific questions correctly. The probability of a correct response is then assigned to each question. Each judge will then make an estimate of the percentage of minimally knowledgeable examinees who would know the answer to each question. The totals each of the judges is added together and divided by the number of judges to determine the overall estimate of the minimum standards necessary. The minimum number of correct answers required to achieve a passing score will take into account the relative difficulty of each examination through scaling and equating each examination to the base examination. The raw score necessary to show competence shall be deemed to be a 70 on a scale of 100. The passing grade for the Principles and Practice Examination is 70 or better. A passing grade on Part Two of the examination is defined as a grade of 70 or better. The grades are determined by a group of knowledgeable professional engineers, who are familiar with engineering practice and with what is required for an applicable engineering practice and with what is required for an applicable engineering task. These professional engineers will establish a minimum passing score on each individual test item (i.e., examination problem). An Item Specific Scoring Plan (ISSP) will be prepared for each examination item based upon the NCEES standard scoring plan outline form. An ISSP will be developed by persons who are familiar with each discipline including the item author, the item scorer, and other NCEES experts. On a scale of 0-10, six (6) will be a minimum passing standard and scores between six (6) and ten (10) will be considered to be passing scores for each examination item. A score of five (5) or lower will be considered an unsatisfactory score for that item and examinee will be considered to have failed that item. To pass, an examinee must average six (6) or greater on his/her choice of eight (8) exam items, that is, the raw score must be forty- eight (48) or greater based on a scale of eighty (80). This raw score is then converted to a base 100 on which, as is noted above, a passing grade will be seventy (70). Petitioner resides in Tampa, Florida. On April 11, 2003, Petitioner took a national examination that Petitioner must pass to be licensed by the state as a professional engineer. On July 1, 2003, Petitioner received a letter from the Board advising Petitioner that he had received a failing grade on the examination. On July 2, 2003, Petitioner unsuccessfully requested the raw scores on his examination from a representative of the National Council of Examiners for Engineering and Surveying (NCEES). The NCEES is the national testing entity that conducts examinations and determines scores for the professional engineer examination required by the state. On July 9, 2003, Petitioner submitted a formal request to the Board for all of the raw scores related to Petitioner "and all past P.E. Exams that the Petitioner had taken." A representative of the Board denied Petitioner's request explaining that the raw scores are kept by the NCEES and "it is not their policy to release them." The Board's representative stated that the Board was in the process of adopting new rules "that were in-line with the policies of the NCEES." On July 31, 2003, Petitioner requested the Board to provide Petitioner with any statute or rule that authorized the Board to deny Petitioner's request for raw scores pursuant to Section 119.07(1)(a), Florida Statutes (2003). On the same day, counsel for the Board explained to Petitioner that the Board is not denying the request. The Board is unable to comply with the request because the Board does not have physical possession of the raw scores. Petitioner and counsel for Respondent engaged in subsequent discussions that are not material to this proceeding. On August 6, 2003, Petitioner requested counsel for Respondent to provide Petitioner with copies of the proposed rule changes that the Board intended to consider on August 8, 2003. On August 27, 2003, Petitioner filed a petition with the Board challenging existing Florida Administrative Code Rule 61G15-21.004. The petition alleged that parts of the existing rule are invalid. Petitioner did not file a challenge to the existing rule with DOAH. The Petition for Hearing states that Petitioner is filing the Petition for Hearing pursuant to Subsections 120.56(1) and (3)(b), Florida Statutes (2003). However, the statement of how Petitioner's substantial interests are affected is limited to the proposed changes to the existing rule. During the hearing conducted on January 29, 2004, Petitioner explained that he does not assert that the existing rule is invalid. Rather, Petitioner argues that the Board deviates from the existing rule by not providing examinees with copies of their raw scores and by failing to use raw scores in the determination of whether an applicant achieved a passing grade on the exam. Petitioner further argues that the existing rule benefits Petitioner by purportedly requiring the Board to use raw scores in the determination of passing grades. The elimination of that requirement in the proposed rule arguably will adversely affect Petitioner's substantial interests. The Petition for Hearing requests several forms of relief. The Petition for Hearing seeks an order granting Petitioner access to raw scores, a determination that Petitioner has met the minimum standards required under the existing rule, and an order that the Board grant a license to Petitioner. The Petition for Hearing does not request an order determining that the proposed rule changes constitute an invalid exercise of delegated legislative authority.
The Issue Whether Petitioner is entitled to additional credit for his responses to Question No. 130 of the Principles and Practice of Engineering portion of the engineering licensure examination administered on April 23, 1999, by the National Council of Examiners for Engineers and Surveyors.
Findings Of Fact Petitioner, Dennis Vann (Petitioner), is an applicant for licensure as a professional engineer in the State of Florida. On April 23, 1999, Petitioner sat for the Principles and Practice Engineering Examination portion of the engineer licensure examinations. This is a national examination developed, controlled, and administered by the National Council of Examiners for Engineering and Surveying (NCEES). Petitioner received a raw score of 45 on this examination. For the electrical engineering discipline, a raw score of 45 results in a converted score of 67. A minimum converted score of 70 is required to pass this examination. A raw score of 48 results in a converted score of 70. Therefore, Petitioner needs an additional 3 raw score points to earn a passing score on the examination. Petitioner challenged the scoring of Question No. 130 on the examination and formally requested the NCEES to rescore his solutions to the question. The NCEES rescored Question No. 130 and determined that Petitioner was not entitled to any additional points for Question No. 130. For Question No. 130, the maximum score achievable was Petitioner received a score of 4 on that item. The NCEES developed and used an Item Specific Scoring Plan (ISSP) for each examination question. Question No. 130 was scored by the NCEES according to the ISSP for that question. Question No. 130 contains two subparts, which require the examinee to address four discrete requirements. Petitioner correctly calculated the bus current (requirement 3). However, Petitioner failed to properly calculate the busway loading and determination of adequacy (requirement 1), the bus impedance (requirement 2), and percent voltage drop (requirement 4). Petitioner's response to Question No. 130 was initially assigned a score of 4. However, if graded correctly, that response would have resulted in a score of 6. The credible testimony of Respondent's expert was that under the ISSP for Question No. 130, Petitioner is entitled to a score of 6 for his response. With a score of 6 for Question No. 130, Petitioner's raw score is increased to 47. A raw score of 47 results in a converted score of 69. Even with the 2 additional points awarded to Petitioner's response to Question No. 130, his score on the professional engineering licensure examination is still below 70 and is not a passing score. Question No. 130 provides all the necessary information for an examinee to solve the problem. Moreover, Question No. 130 is properly designed to test an examinee's competence in electrical engineering.
Recommendation Based upon the foregoing Findings of Fact and Conclusions of Law, it is hereby RECOMMENDED that a final order be entered concluding that Petitioner is entitled to a score of 6 points for his response to Question No. 130, and recalculating Petitioner's total score on the examination on the basis of that conclusion. DONE AND ENTERED this 21st day of April, 2000, in Tallahassee, Leon County, Florida. CAROLYN S. HOLIFIELD Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 21st day of April, 2000. COPIES FURNISHED: Dennis Vann Post Office box 23054 Tampa, Florida 33623 William H. Hollimon, Esquire Ausley & McMullen 227 South Calhoun Street Post Office Box 391 Tallahassee, Florida 32301-1884 Dennis Barton, Executive Director Department of Business and Professional Regulation Board of Professional Engineers 1208 Hays Street Tallahassee, Florida 32301 Natalie A. Lowe, Esquire Florida Engineers Management Corporation 1208 Hays Street Tallahassee, Florida Barbara D. Auger, General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399
The Issue Is Petitioner entitled to one additional point on the October 1996 Professional Civil Engineer Examination so as to achieve a passing score for licensure in Florida?
Findings Of Fact Petitioner took the Civil Engineer Examination given in October 1996. The Department of Business and Professional Regulation's Bureau of Testing notified Petitioner by Examination Grade Report dated February 17, 1997, that she had earned a score of 69.00 on the Civil Engineer Examination. The minimum passing score for the Civil Engineer Examination is 70.00. Petitioner timely requested formal hearing and challenged only Question 120, for which she received no points. Petitioner is trained as a materials engineer. Question 120 is a soils and foundation problem outside her concentrated area of study. It is an open book examination question. Petitioner selected the correct equation from the applicable manual, but acknowledged that she solved the variables of that equation incorrectly. The National Council of Examiners for Engineering and Surveying (NCEES) produced, distributed, and was responsible for grading the examinations. Petitioner contended that the examiner who graded her answer sheet applied different criteria than the examination criteria published by the NCEES. Petitioner further contended that since one criterion her grader actually used was merely to "write the correct equation," she should be awarded at least one point on that basis. However, a comparison of the actual grader's handwritten "summary" on Petitioner's Solution Pamphlet (Respondent's Exhibit 3) and the NCEES's Solutions and Scoring Plan (Respondent's Exhibit 2) does not bear out Petitioner's theory. It is clear that out of five possible parts of the question, which five parts total two points' credit each, merely selecting the correct equation from an open text would not amount to two points, or even one point, credit. I accept as more competent, credible and persuasive the testimony of Eugene N. Beauchamps, the current Chairman of the NCEES Examination Policy Committee and a Florida licensed Professional Engineer, that the grader's "summary" describes what he actually reviewed in Petitioner's written solution to Question 120 rather than establishing one or more different grading criteria. In order to receive a score of two on Question 120, the candidate was required to demonstrate any one of five requirements listed in the NCEES Solution and Scoring Plan for "2-Rudimentary Knowledge." The first requirement in the NCEES Solution and Scoring Plan (Respondent's Exhibit 2) for receiving a score of two points is, "Determines effective overburden stress at mid- depth of clay layer." The remaining four NCEES scoring criteria required that the examinee: Computes the change in effective stress at mid- depth of the clay layer due to placement of the fill. Computes the primary consolidation settlement, based on a change in effective stress, due to the fill surcharge. Evaluates the Average Degree of Consolidation and the Time Factor. Determines the waiting period after fill placement recognizing the existence of double-drained conditions. In order to gain two more points (total 4 points) so as to demonstrate "More Than Rudimentary Knowledge But Insufficient to Demonstrate Minimum Competence," Petitioner would have to have met two of the five bulleted criteria. For two more points (total 6 points) for "Minimum Competence," Petitioner would have had to score three bullets. For two more points (total 8 points) for "More than Minimum But Less Than Exceptional Competence," Petitioner would have had to score four bullets. Finally, to attain "Exceptional Competence" for 10 total points, Petitioner would have had to score all five bullets. In the first correct equation for answering Question 120, "p sub zero" (p naught) equals the present effective overburden pressure, which represents what clay was present before anything was put on top of the clay layer. "P" equals the total pressure acting at mid-height of the consolidating clay layer or the pressure of the dirt and the water in the dirt. "H" equals the thickness of the consolidating clay layer. Petitioner's solution for the first bullet, "determining the effective overburden stress at mid-depth of clay layer," indicated p sub zero (p naught) as the "present effective overburden pressure," but it incorrectly calculated p sub zero equaling 125 pounds multiplied by 13 feet. This is incorrect because the effective overburden pressure would not include 13 feet of fill. The 13 feet of fill is not part of p sub zero, the present effective overburden pressure. Petitioner's solution for the first bullet, also multiplied water, represented by 62.4, by 12, which is incorrect. She should have used a multiplier of 10 to receive credit for this problem. The grader indicated the correct equation was used incorrectly by Petitioner because of the two foregoing incorrect calculations. The equation, as Petitioner stated it, was correct and her multiplication was correct. Her solution identified P sub zero as present effective overburden pressure but present effective overburden pressure would not include the fill. Petitioner had the correct equation for the present effective overburden pressure and her mathematics were correct. However, she did not use the consolidation equation correctly, not obtaining the correct percentage of primary consolidation. As stated, the problem did not consider the fill as part of the present effective overburden pressure. Her solution also contained the correctly written time rate of settlement equation but failed to use it, and no waiting period was determined. The practical result of Petitioner's error could range from a cracked building to a collapsed building, depending upon the degree of error to site and materials.
Recommendation Upon the foregoing findings of fact and conclusions of law, it is RECOMMENDED that the Department of Business and Professional Regulation enter a Final Order denying Petitioner's challenge and affirming her score as one point below passing. RECOMMENDED this 3rd day of March, 1998, in Tallahassee, Leon County, Florida. ELLA JANE P. DAVIS Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 Filed with the Clerk of the Division of Administrative Hearings this 3rd day of March, 1998. COPIES FURNISHED: Susan E. Wilson 3581 Jose Terrace Jacksonville, Florida 32217 R. Beth Atchison Assistant General Counsel Department of Business and Profession Regulation 1940 North Monroe Street Tallahassee, Florida 32399 Angel Gonzalez, Executive Director Department of Business and Profession Regulation 1940 North Monroe Street Tallahassee, Florida 32399 Lynda L. Goodgame General Counsel Department of Business and Profession Regulation 1940 North Monroe Street Tallahassee, Florida 32399
The Issue Whether Petitioner is eligible for licensure by endorsement as a professional engineer and/or waiver of Part I of the engineering licensing examination.
Findings Of Fact From 1969 through 1974, Petitioner attended the Lipetsk Branch of the Moscow Institute of Steel and Alloys. In 1974, Petitioner graduated from the Lipetsk Polytechnical Institute (Institute) in Russia, with a degree in industrial and civil engineering. The degree in civil engineering earned by Petitioner is equivalent to a bachelor of science degree in civil engineering in the United States. Upon graduation from the Institute, Petitioner commenced her professional employment as an engineer on August 30, 1974, at the LIPTSKGRAZHDANPROEKT Design Institute. In January 1976, Petitioner was employed as a professional engineer at the State Design Institute DNEPRPROEKKTSTALKONSTRUKSIYA (DNEPR), where she was continuously employed until leaving the Soviet Union in December 1993. There are no specific licensing or registration requirements in Russia with respect to engineers. Therefore, after earning a degree in industrial and civil engineering, Petitioner could work as an engineer in Russia without taking any professional examination. However, in 1979, Petitioner took an examination in Russia that allowed her to sign her own drawings and calculations. During Petitioner's tenure at the DNEPR, she achieved the status of Senior Engineer in 1986; was promoted to the position of Category II Engineer for Steel Structures in 1988; was promoted to the post of Category I Engineer for Steel Structures in 1990; and was elevated to the position of Leading Engineer in 1991. To achieve the status of Category II Engineer for Steel Structures and Category I Engineer for Steel Structures at DNEPR, Petitioner had to take an examination in 1988 and in 1990, respectively. The promotion to each of these positions was predicated upon Petitioner's passing these examinations and demonstrating expertise in the areas of economics, chemistry, mathematics, physics, building materials, corrosion prevention, resistance of materials, and construction mechanics. As a result of passing the examinations in 1988 and 1990, Petitioner was not only promoted, but also received salary increases. Petitioner believes that the two examinations she took in Russia in 1988 and in 1990, while working at the DNEPR were substantially equivalent to the Fundamentals Examination. However, no evidence was presented to support this claim. The Fundamentals Examination is one component of the engineering licensing examination, and is designed to assess whether an individual is qualified to practice in this state as an engineer intern. This examination is usually taken either in the applicant's last year in engineering school or shortly after graduation. With regard to format, the Fundamentals Examination is an eight-hour examination and consists of 120 multiple-choice questions. The Principles and Practice Examination is the second part of the engineering licensing examination and is taken after successful completion of the Fundamentals Examination. Oscar E. Olsen, a structural engineer and owner of O.E. Olsen and Associates, a structural engineering firm, is currently Petitioner's employer. Mr. Olsen, who is generally familiar with the Fundamentals Examination, testified that the list of subjects covered on the two examinations taken by Petitioner in 1988 and 1990, coincide with the subject matter on the Fundamentals Examination. Mr. Olsen further testified that it appeared to him that the two examinations taken by Petitioner were comparable to the Fundamental Examinations required in Florida. Notwithstanding his testimony that the exams taken by Petitioner are substantially equivalent to the Fundamentals Examination, Mr. Olsen admitted that he has never seen or reviewed the examinations taken by Petitioner while she was in Russia. It is impossible to render a reasonable opinion as to whether the two examinations taken by Petitioner in Russia are substantially equivalent to the Fundamentals Examination, where the only information provided with regard to the former is a list of subject areas covered. Such a list gives no indication of the depth and specific content of the subject matter on the examinations; the difficulty of the examinations; the passing scores; the number and format of the questions; and the length of the examinations.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is recommended that Respondent, the Board of Professional Engineers, enter a Final Order denying Petitioner's request for waiver of Part I, the Fundamentals Examination, and for licensure by endorsement. DONE AND ENTERED this 9th day of December, 1997, in Tallahassee, Leon County, Florida. CAROLYN S. HOLIFIELD Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (904) 488-9675 SUNCOM 278-9675 Fax Filing (904) 921-6847 Filed with the Clerk of the Division of Administrative Hearings this 9th day of December, 1997. COPIES FURNISHED: Murray Silverstein, Esquire Powell, Carney, Hayes, and Silverstein One Plaza, Suite 1210 St. Petersburg, Florida 33731-1689 Edwin A. Bayo Assistant Attorney General Office of the Attorney General The Capitol, Plaza 01 Tallahassee, Florida 32399-1050 Angel Gonzalez Executive Director Board of Professional Engineers Northwood Centre 1940 North Monroe Street Tallahassee, Florida 32399-0755 Lynda L. Goodgame, General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792
Findings Of Fact In June, 1987, petitioner, J. Frank Smith, was an examinee on Division (C) of the professional architectural examination. Successful completion of the examination is a prerequisite to the issuance of a license as an architect by respondent, Department of Professional Regulation, Board of Architecture (Board or Department). Petitioner had previously received passing grades on all other parts of the examination. By notice dated October 9, 1987, petitioner was advised by the Department's Office of Examination Services that he had received a "Fail" score on Division (C). After participating in an informal grade review session concerning his examination score, 1/ Smith requested a formal hearing by letter dated December 2, 1987. Although the letter did not identify any alleged errors in the manner in which the examination was graded, Smith's concerns were brought out during final hearing. Division (C) is a twelve hour sketch problem involving building design considerations. On the examination in question, the candidates were required to design a downtown athletic club including a site and first floor plan, a second floor plan, elevations and building section. To receive a passing score, the candidate had to prepare "minimally acceptable solutions" in the following major categories: (a) program requirements, (b) design logic, (c) code compliance, (d) technical aspects, and (e) completeness and clarity of presentation, adherence to test instructions, or required drawing(s) missing. By rule, the Board has established a passing score to be a 3 or 4 while a fail score is 2, 1 or 0. In order to pass, a candidate is required to receive at least two passing scores from the graders. For the June, 1987 examination, approximately 150 professional architects were used to grade the candidates' examinations in two regional locations. The graders were experienced architects who had graded examinations in prior years. Before actually grading the solutions, the graders were given one-half day training sessions which consisted of receiving instructions in the grading process, reviewing the grading criteria and practicing on sample solutions that had already been graded by master jurists. The instruction was given by the director of examinations development for the National Council of Architecture Registration Boards (NCARB). The NCARB is the organization that prepares the examination questions. The live grading began on the afternoon of the first day of the examination. The graders used a holistic grading approach which meant they reviewed the entire examination and made a pass-fail judgment. If a failing grade was assigned, the grader was required to check up to three so-called "weakness boxes" on the grading sheet to indicate in which areas the grader found the candidate to be deficient. A grader was also allowed to make brief written comments next to the checked boxes to elaborate on the perceived weaknesses. After being graded by the first grader, the examination was given to a second and third grader, both of whom were unaware of the scores assigned by the others. If a mixed score occurred, that is, two 2's and a 3 or two 3's and a 2, a master jurist then graded the examination. Therefore, it was possible for the same examination to be graded four times before a final score was assigned. In Smith's case, he initially received two 2's and a 3, or an overall fail score. A master jurist reviewed the solutions and assigned a grade of 2. Thus, Smith did not pass Division (C). Grader 1217-153 assigned a score of 2 and noted that Smith was weak in category M, or completeness and clarity of presentation, adherence to test instructions, or required drawing(s) missing. Grader 1232-155 assigned a score of 2 and checked off areas A, G and K which meant Smith was weak in the areas of program requirements, code compliance and technical aspects, respectively. Finally, Grader 9929-075 concluded that a score of 2 was appropriate since he believed Smith was weak in areas D, E and G. Areas D and E pertained to design logic (circulation) and design logic (spatial relationships/ proportions/adjacencies) while Area G, as noted above, related to technical aspects. The fourth grader assigned a passing score of 3. At hearing, Smith testified on his own behalf and generally reviewed his solution pointing out what he perceived to be his strengths and weaknesses. As he saw it, he attained a passing score. Smith is a graduate, with high honors, of the University of Florida School of Architecture in 1976 and received a master's degree in architecture from the university of Illinois in 1983. He is president of a construction firm in Fort Lauderdale that designs and constructs various projects, both large and small, and his partner is a registered architect. In addition to his own results, Smith offered into evidence the solutions prepared by five other candidates who received passing scores on the June, 1987 examination. Smith then attempted to point out that, even though their solutions contained various flaws, the candidates nonetheless received passing grades. Expert witness Butt, who is a registered architect, was a long time professor and former chairman of the School of Architecture at the university of Florida. He continues to do consulting work for the Board. Butt has been a grader on many examinations and was a master jurist as recently as 1987. Butt regraded Smith's examination and concluded that a score of 2 was appropriate. This was because he noted major deficiencies in the areas of design logic (circulation), code compliance, and completeness, clarity and presentation of the solution as to the structural system. More specifically Smith failed to adequately address certain public circulation questions on the building's first floor. This raised serious concerns as to petitioner's design logic and compliance with building codes. In addition, the structural system, which was required to be shown as part of the transverse section, was incomplete and contained several errors. Further, no structural system was shown on the west side of the building. Other difficulties included poor circulation around the monumental staircase, questionable design logic as to fire safety requirements in front of the building and the use of two hour rated firewalls throughout the entire building even though the fire code required only one hour rated firewalls. Butt's testimony as to the appropriate grade to be assigned Smith's examination is found to be more credible than that of Smith and is hereby accepted. Although several errors were noted on the other solutions of passing candidates, those errors were not so serious in nature as to require a failing grade. Further, such errors were solvable without major modifications in the design intent of the subject, and the candidates had completed all program requirements on the examination.
Recommendation Based on the foregoing findings of fact and conclusions of law, it is RECOMMENDED that the petition of J. Frank Smith to receive a passing score on Division (C) of the June, 1987 professional architectural examination be DENIED. DONE AND ORDERED this 28th day of September, 1988, in Tallahassee, Leon County, Florida. DONALD R. ALEXANDER Hearing Officer Division of Administrative Hearings The Oakland Building 2009 Apalachee Parkway Tallahassee, Florida 32399-1550 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 28th day of September, 1988.
The Issue Whether Petitioner's challenge to the failing grade he received on the contract administration portion of the October 2000 General Contractor Examination should be sustained.
Findings Of Fact Based upon the evidence adduced at hearing and the record as a whole, the following findings of fact are made: Petitioner sat for the contract administration portion of the Florida certification examination for general contractors administered in October 2000 (Contract Administration Examination). The Contract Administration Examination consisted of 60 multiple-choice questions of equal value, worth a total of 100 points. To attain a passing score on the Contract Administration Examination, candidates needed to receive a total of 70 points. Of the 378 candidates who took the Contract Administration Examination, 156 received passing scores. Petitioner was not among this group of successful candidates. He received a failing score of 66.67 on the examination. Question 2 of the Contract Administration Examination was a clear and unambiguous multiple-choice question that required the candidate to determine, based upon the information given, on what workday (not calendar day) the pouring of concrete footings for a residential construction project would begin. There was only one correct answer to this question. Approximately 50 percent of the candidates chose this correct response. Petitioner chose another answer that was clearly incorrect because it represented the calendar day (not the workday) on which the pouring would begin. He therefore appropriately received no credit for his answer. Question 9 of the Contract Administration Examination was a clear and unambiguous multiple-choice question that fairly tested the candidate's knowledge of the requirements of Section 489.113(3), Florida Statutes, which provides as follows: A contractor shall subcontract all electrical, mechanical, plumbing, roofing, sheet metal, swimming pool, and air- conditioning work, unless such contractor holds a state certificate or registration in the respective trade category, however: A general, building, or residential contractor, except as otherwise provided in this part, shall be responsible for any construction or alteration of a structural component of a building or structure, and any certified general contractor or certified underground utility and excavation contractor may perform clearing and grubbing, grading, excavation, and other site work for any construction project in the state. Any certified building contractor or certified residential contractor may perform clearing and grubbing, grading, excavation, and other site work for any construction project in this state, limited to the lot on which any specific building is located. A general, building, or residential contractor shall not be required to subcontract the installation, or repair made under warranty, of wood shingles, wood shakes, or asphalt or fiberglass shingle roofing materials on a new building of his or her own construction. A general contractor shall not be required to subcontract structural swimming pool work. A general contractor, on new site development work, site redevelopment work, mobile home parks, and commercial properties, shall not be required to subcontract the construction of the main sanitary sewer collection system, the storm collection system, and the water distribution system, not including the continuation of utility lines from the mains to the buildings. A general contractor shall not be required to subcontract the continuation of utility lines from the mains in mobile home parks, and such continuations are to be considered a part of the main sewer collection and main water distribution systems. A solar contractor shall not be required to subcontract minor, as defined by board rule, electrical, mechanical, plumbing, or roofing work so long as that work is within the scope of the license held by the solar contractor and where such work exclusively pertains to the installation of residential solar energy equipment as defined by rules of the board adopted in conjunction with the Electrical Contracting Licensing Board. No general, building, or residential contractor certified after 1973 shall act as, hold himself or herself out to be, or advertise himself or herself to be a roofing contractor unless he or she is certified or registered as a roofing contractor. There was only one correct answer to this question. Approximately 65 percent of the candidates chose this correct response. Petitioner chose another answer that was clearly incorrect inasmuch as a newly licensed general contractor is not free, pursuant to Section 489.113(3)(b), Florida Statutes, to install or repair wood shake roofs on existing buildings constructed by other contractors. Petitioner therefore appropriately received no credit for his answer. Question 29 of the Contract Administration Examination was a clear and unambiguous multiple-choice question that fairly tested the candidate's ability to calculate, based upon the information given, the cost of delivering 28,000 lineal feet of #5 bars of reinforcing steel. There was only one correct answer to this question . Approximately 67 percent of the candidates chose this correct response. Petitioner chose another answer that was clearly incorrect. He therefore appropriately received no credit for his answer. Question 38 of the Contract Administration Examination was a clear and unambiguous multiple-choice question that fairly tested the candidate's ability to distinguish between unit price contracts and other types of contracts, including lump sum contracts. Approximately 82 percent of the candidates chose this correct response. Petitioner chose another answer that was clearly incorrect. He therefore appropriately received no credit for his answer.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that a final order be entered rejecting Petitioner's challenge to the failing score he received on the contract administration portion of the October 2000 certification examination for general contractors. DONE AND ENTERED this 11th day of May, 2001, in Tallahassee, Leon County, Florida. STUART M. LERNER Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 11th day of May, 2001. COPIES FURNISHED: Michael Richards 3802 Lakewood Road Lake Worth, Florida 33461 Charles F. Tunnicliff, Esquire Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-2202 Kathleen O'Dowd, Executive Director Construction Industry Licensing Board Department of Business and Professional Regulation 7960 Arlington Expressway, Suite 300 Jacksonville, Florida 32211-7467 Hardy L. Roberts III, General Counsel Department of Business and Professional Regulation Northwood Centre 1940 North Monroe Street Tallahassee, Florida 32399-2202
The Issue The issue presented is whether Mr. Lord should be granted additional credit for his answer to question number 144 on the April 1990 Professional Engineer licensure examination.
Findings Of Fact Mr. Lord (Candidate #301402) received a score of 66.3 percent on the April 20, 1991, Principals and Practice portion of the Professional Engineer examination. A minimum passing score was 70.0 percent. Mr. Lord challenged the scoring of his response to question number 144. Question number 144 is an essay question involving an assembly line problem where four separate stations are used to assemble a product in sequence. A fifth station can assist in maximizing the number of finished products produced per hour, and is capable of performing all operations. The correct answer to question number 144 was 100 products per hour, while Mr. Lord's answer was 25 pieces per hour. Petitioner received a score of 2 (out of a possible 10) points on question number 144. This was based on the scoring plan developed for the exam by the National Council of Examiners for Engineering and Surveying. Mr. Lord used a method of averaging station assembly times to determine the maximum average number of products each station could produce. The averaging method gave a solution which did not identify the central issue presented by the essay question: identifying and eliminating the bottlenecks in production. Mr. Lord also made an assumption that the initial four stations could do all operations, thus defining the model inaccurately. This misreading allowed Mr. Lord to use an averaging methodology. Mr. Granata, the Department's expert, testified that it is a coincidence of the numbers that if you multiply Respondent's answer (25) by four (the initial number of machines) you get the Board's answer (100). Mr. Greenbaum, Petitioner's expert witness, testified that Petitioner's answer is "unique" and that he, as an expert, would have answered the question using a methodology similar to the one developed by the Department's expert, Mr. Granata, and by the NCEE (National Council of Examiners for Engineering).
Recommendation Based upon the foregoing Findings of Fact and Conclusions of Law, it is recommended that the challenge to the grading of Mr. Lord's response to question 144 on the April 1990 Professional Engineer licensure examination be dismissed. RECOMMENDED this 14th day of March, 1991, at Tallahassee, Florida. WILLIAM R. DORSEY, JR. Hearing Officer Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-1550 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 14th day of March, 1991. COPIES FURNISHED: William F. Whitson, Law Clerk Department of Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792 Curtis Lord 1416A Old Lystra Road Chapel Hill, NC 27514 Rex Smith, Executive Director Department of Professional Regulation Board of Professional Engineers 1940 North Monroe Street Tallahassee, Florida 32399-0792 Jack McRay, General Counsel Department of Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792
The Issue The primary issue is whether the process used by the Department of Education (Department) for evaluating and ranking the proposals submitted in response to Request For Proposal (RFP) 99-03 for the Florida Comprehensive Assessment Test (FCAT) administration contract was contrary to the provisions of the RFP in a way that was clearly erroneous, contrary to competition, arbitrary, or capricious.
Findings Of Fact The RFP for the FCAT describes a five stage process for evaluating proposals. In Stage I, the Department’s Purchasing Office determined whether a proposal contained certain mandatory documents and statements and was sufficiently responsive to the requirements of the RFP to permit a complete evaluation. Stage II involved the Department’s evaluation of a bidder’s corporate qualifications to determine whether the bidder has the experience and capability to do the type of work that will be required in administering the FCAT. Stage III was the Department’s evaluation of a bidder’s management plan and production proposal. In Stage IV, the Department evaluated a bidder’s cost proposal. Stage V involved the ranking of proposals based on points awarded in Stages II-IV. If a proposal did not meet the requirements at any one stage of the evaluation process, it was not to be evaluated in the following stage. Instead, it was to be disqualified from further consideration. Stages II and III of the evaluation process were conducted by an evaluation team comprised of six Department employees: Dr. Debby Houston, Ms. Lynn Joszefczyk, Dr. Peggy Stillwell, Dr. Cornelia Orr, Dr. Laura Melvin, and Ms. Karen Bennett. Dr. Thomas Fisher, head of the Department’s Assessment and Evaluation Services Section, and Dr. Mark Heidorn, Administrator for K-12 Assessment Programs within the Department’s Assessment and Evaluation Services Section, served as non-voting co-chairs of the evaluation team. The focus of this proceeding is Stage II of the evaluation process addressing a bidder’s corporate qualifications. RFP Provisions Regarding Corporate Qualification The FCAT administration contractor will be required to administer tests to approximately one and a half million students each year in a variety of subject areas at numerous grade levels. The FCAT program involves a complex set of interrelated work activities requiring specialized human resources, technological systems and procedures. The FCAT must be implemented annually within limited time periods. The FCAT administration contractor must meet critical deadlines for the delivery of test materials to school districts and the delivery of student scores prior to the end of the school year. In developing the RFP, the Department deliberately established a set of minimum requirements for corporate qualifications that a bidder was to demonstrate in order for its proposal to be eligible for further evaluation. The purpose of the RFP’s minimum corporate qualifications requirements was to limit bidding to qualified vendors who have demonstrated prior experience in successfully administering large-scale assessment projects like the FCAT, thereby providing the Department with some degree of assurance that the winning bidder could successfully administer the FCAT. The instructions to bidders regarding the minimum requirements for corporate qualifications are contained in RFP Section 10, which gives directions on proposal preparation. Section 10.1, which lists certain mandatory documents and statements to be included in the bidder’s proposal, requires that a transmittal letter contain "[a] statement certifying that the bidder has met the minimum corporate qualifications as specified in the RFP." These "minimum corporate qualifications" are set forth in RFP Appendix J. RFP Section 10.2 identifies what a bidder is required to include in its proposal with respect to corporate qualifications. The first paragraph of Section 10.2 directs a bidder generally to describe its qualifications and experience performing tasks similar to those that it would perform in administering the FCAT, in order to demonstrate that the bidder is qualified where it states: Part II of a bidder’s proposal shall be entitled Corporate Qualifications. It shall provide a description of the bidder’s qualifications and prior experience in performing tasks similar to those required in this RFP. The discussion shall include a description of the bidder’s background and relevant experience that qualifies it to provide the products and services required by the RFP. RFP Section 10.2, however, is not limited to a directive that qualifications and past experience be described generally. Instead, Section 10.2, also communicates, in plain and unambiguous terms, that there are specific minimum corporate qualifications a bidder must demonstrate: The minimum expectations for corporate qualifications and experience are shown in Appendix J. There are two separate sets of factors, one set of eight for the developmental contractor and another set of nine for the administration contractor. Bidders must demonstrate their Corporate Qualifications in terms of the factors that are applicable to the activities for which a bid is being submitted -- development or administration. For each criterion, the bidder must demonstrate that the minimum threshold of experience has been achieved with prior completed projects. (Emphasis added.) Moreover, Section 10.2 singles out for emphasis, in relation to the administration component of the RFP, the importance placed on a bidder’s ability to demonstrate experience processing a large volume of tests: The [bidder’s prior completed] projects must have included work tasks similar to those described herein, particularly in test development or processing a comparable number of tests. The bidder will provide a description of the contracted services; the contract period; and the name, address, and telephone number of a contact person for each of the contracting agencies. This description shall (1) document how long the organization has been providing similar services; (2) provide details of the bidder’s experience relevant to the services required by this RFP; and (3) describe the bidder’s other testing projects, products, and services that are similar to those required by this RFP. (Emphasis added.) The Department thus made clear its concern that bidders demonstrate experience with large-scale projects. RFP Appendix J sets forth nine different criteria (C1 through C9) for the administration contractor. As stated in RFP Section 10.2, "[f]or each criterion, the bidder must demonstrate that the minimum threshold of experience has been achieved with prior completed projects . . . ." (emphasis added). Appendix J contains a chart which lists for each criterion: (1) a summary of the related FCAT work task, (2) the detailed criteria for the bidder’s experience related to that work task, and (3) the necessary documentation a bidder must provide. Criterion C4 and Criterion C6 include work tasks that involve the use of image-based scoring technology. C4 and C6 are the only corporate qualifications criteria at issue in this proceeding. RFP Provisions Involving Corporate Qualifications for Image-Based Scoring "Handscoring" is the test administration activity in which open-ended or performance-based student responses are assessed. This practice involves a person reading something the student has written as part of the test, as distinguished from machine scoring multiple choice responses (i.e., the filled-in "bubbles" on an answer sheet). There are two types of handscoring: (1) paper-based handscoring, and (2) image-based handscoring. Paper-based handscoring requires that a student response paper be sent to a reader, who then reviews the student’s response as written on the paper and enters a score on a separate score sheet. Image-based handscoring involves a scanned image of the student’s response being transmitted to a reader electronically. The student’s response is then projected on a computer screen, where the reader reviews it and assigns a score using the computer. The RFP requires that the reading and math portions of the FCAT be handscored on-line using imaging technology beginning with the February 2000 FCAT administration. The RFP provides that the writing portion of the FCAT may be handscored using either the paper-based method or on-line imaging technology during the February 2000 and 2001 FCAT administrations. However, on-line image-based scoring of the writing portion of the FCAT is required for all FCAT administrations after February 2001. An image-based scoring system involves complex computer technology. William Bramlett, an expert in designing and implementing large-scale imaging computer systems and networks, presented unrefuted testimony that an image-based scoring system will be faced with special challenges when processing large volumes of tests. These challenges involve the need to automate image quality control, to manage the local and wide area network load, to assure adequate server performance and storage requirements, and to manage the work flow in a distributed environment. In particular, having an image-based scoring system process an increasing volume of tests is not simply a matter of adding more components. Rather, the system’s basic software architecture must be able to understand and manage the added elements and volume involved in a larger operation. According to Bramlett, there are two ways that the Department could assess the ability of a bidder to perform a large- scale, image-based scoring project such as the FCAT from a technological perspective: (1) have the bidder provide enough technological information about its system to be able to model or simulate the system and predict its performance for the volumes involved, or (2) require demonstrated ability through completion of prior similar projects. Dr. Mark Heidorn, Administrator for Florida’s K-12 Statewide Assessment Programs, was the primary author of RFP Sections 1-8, which describe the work tasks for the FCAT -- the goods and services vendors are to provide and respond to in their technical proposals. Dr. Heidorn testified that in the Department’s testing procurements involving complex technology, the Department has never required specific descriptions of the technology to be used. Instead, the Department has relied on the bidder’s experience in performing similar projects. Thus, the RFP does not specifically require that bidders describe in detail the particular strategies and approaches they intend to employ when designing and implementing an image-based scoring system for FCAT. Instead, the Department relied on the RFP requirements calling for demonstrated experience as a basis to understand that the bidder could implement such an image-based scoring system. Approximately 717,000 to 828,000 student tests will be scored annually by the FCAT administration contractor using imaging technology. The RFP, however, does not require that bidders demonstrate image-based scoring experience at that magnitude. Instead, the RFP requires bidders to demonstrate only a far less demanding minimum level of experience using image-based scoring technology. Criterion C4 and Criterion C6 in Appendix J of the RFP each require that a bidder demonstrate prior experience administering "a minimum of two" assessment programs using imaged- based scoring that involved "at least 200,000 students annually." The requirements for documenting a "minimum of two" programs or projects for C4 and C6 involving "at least 200,000 students annually" are material because they are intended to provide the Department with assurance that the FCAT administration contractor can perform the large-scale, image-based scoring requirements of the contract from a technological perspective. Such experience would indicate that the bidder would have been required to address the sort of system issues described by Bramlett. Dr. Heidorn testified that the number 200,000 was used in C4 and C6 "to indicate the level of magnitude of experience which represented for us a comfortable level to show that a contractor had enough experience to ultimately do the project that we were interested in completing." Dr. Fisher, who authored Appendix J, testified that the 200,000 figure was included in C4 and C6 because it was a number judged sufficiently characteristic of large-scale programs to be relevant for C4 and C6. Dr. Fisher further testified that the Department was interested in having information that a bidder’s experience included projects of a sufficient magnitude so that the bidder would have experienced the kinds of processing issues and concerns that arise in a large-scale testing program. The Department emphasized this specific quantitative minimum requirement in response to a question raised at the Bidder’s Conference held on November 13, 1998: Q9: In Appendix J, the criteria for evaluating corporate quality for the administration operations C4, indicates that the bidder must have experience imaging as indicated. Does this mean that the bid [sic] must bid for using [sic] imaging technology for reading and mathematics tests? A: Yes. The writing assessment may be handscored for two years, and then it will be scored using imaging technology. To be responsive, a bid must be for imaging. The corporate experience required (200,000 students annually for which reports were produced in three months) could be the combined experience of the primary contractor and the subcontractors. (Emphasis added.) Criterion C4 addresses the RFP work tasks relating to handscoring, including both the image-based handscoring of the reading and math portions of the FCAT for all administrations and the writing portions of the FCAT for later administrations. The "Work Task" column for C4 in Appendix J of the RFP states: Design and implement efficient and effective procedures for handscoring student responses to performance tasks within the limited time constraints of the assessment schedule. Handscoring involves image-based scoring of reading and mathematics tasks for all administrations and writing tasks for later administrations at secure scoring sites. Retrieve and score student responses from early district sample schools and deliver required data to the test development contractor within critical time periods for calibration and scaling. The "Necessary Documentation" column for C4 in Appendix J states: Bidder must document successful completion of a minimum of two performance item scoring projects for statewide assessment programs during the last four years for which the bidder was required to perform as described in the Criteria column. (Emphasis added.) The "Criteria" column for C4 in Appendix J, like the related work tasks in the RFP, addresses both image-based handscoring of reading and math, as well as paper-based or image- based handscoring of writing. In connection with all handscoring work tasks, "[t]he bidder must demonstrate completion of test administration projects for a statewide program for which performance items were scored using scoring rubrics and associated scoring protocols." With respect to the work tasks for handscoring the reading and math portions of the FCAT, "[t]he bidder must demonstrate completion of statewide assessment programs involving scoring multiple-choice and performance items for at least 200,000 students annually for which reports were produced in three months." In addition, for the reading and math work tasks, "[e]xperience must been shown in the use of imaging technology and hand-scoring student written responses with completion of scoring within limited time restrictions." This provision dealing with "imaging technology" experience self-evidently addresses the reading and math components, because separate language addresses imaging experience in connection with the writing component. The relevant handscoring experience for the reading and math aspects of the program is experience using image-based technology. By contrast, with respect to the work tasks for scoring the writing portions of the FCAT, "the bidder must also demonstrate completion of statewide assessment programs involving paper-based or imaged scoring student responses to writing assessment prompts for at least 200,000 students annually for which reports were produced in three months." (Emphasis added.) Criterion C6 addresses work tasks relating to designing and implementing systems for processing, scanning, imaging and scoring student responses to mixed-format tests within limited time constraints. The "Work Task" column for C6 in RFP Appendix J states: Design and implement systems for the processing, scanning, imaging, and scoring of student responses to test forms incorporating both multiple-choice and constructed response items (mixed-format) within the limited time constraints of the assessment schedule. Scoring of student responses involves implementation of IRT scoring tables and software provided by the development contractor within critical time periods. The "Necessary Documentation" column for C6 in Appendix J states: Bidder must document successful completion of a minimum of two test administration projects for statewide assessment programs during the last four years in which the bidder was required to perform as described in the Criteria column. (Emphasis added.) The Criteria column for C6 in Appendix J states: The bidder must demonstrate completion of test administration projects for statewide assessment programs or other large-scale assessment programs that required the bidder to design and implement systems for processing, scanning, imaging, and scoring responses to mixed-format tests for at least 200,000 students annually for which reports were produced in three months. Experience must be shown in use of imaging student responses for online presentation to readers during handscoring. (Emphasis added.) RFP Provisions Per Corporate Qualifications The procedure for evaluating a bidder’s corporate qualifications is described in RFP Section 11.3: The Department will evaluate how well the resources and experience described in each bidder’s proposal qualify the bidder to provide the services required by the provisions of this RFP. Consideration will be given to the length of time and the extent to which the bidder and any proposed subcontractors have been providing services similar or identical to those requested in this RFP. The bidder’s personnel resources as well as the bidder’s computer, financial, and other technological resources will be considered in evaluating a bidder’s qualifications to meet the requirements of this RFP. Client references will be contacted and such reference checks will be used in judging a bidder’s qualifications. The criteria to be used to rate a bidder’s corporate qualifications to meet the requirements of this RFP are shown in Appendix J and will be applied as follows: * * * Administrative Activities. Each of the nine administration activities criteria in Appendix J will be individually rated by members of the evaluation team. The team members will use the rating scale shown in Figure 1 below. Individual team members will review the bidder’s corporate qualifications and rate the response with a rating of one to five. The ratings across all evaluators for each factor will be averaged, rounded to the nearest tenth, and summed across all criteria. If each evaluator assigns the maximum number of points for each criterion, the total number of points will be 45. To meet the requirements of Stage II, the proposal must achieve a minimum rating of 27 points and have no individual criterion for which the number of points averaged across evaluators and then rounded is less than 3.0. Each proposal that receives a qualifying score based on the evaluation of the bidder’s qualifications will be further evaluated in Stage III. Figure 1 Evaluation Scale for Corporate Qualifications 5 Excellent 4 3 Satisfactory 2 1 Unsatisfactory The bidder has demonstrated exceptional experience and capability to perform the required tasks. The bidder has demonstrated that it meets an acceptable level of experience and capability to perform the required tasks. The bidder either has not established its corporate qualifications or does not have adequate qualifications. RFP Section 11.3 provides that each of the nine corporate qualifications criteria for administration operations in Appendix J (C1 through C9) will be individually rated by the six members of the evaluation team using a scale of one to five. A rating of three is designated as "satisfactory" which means that "[t]he bidder has demonstrated that it meets an acceptable level of experience and capability to perform the required tasks." In order to be further evaluated, Section 11.3 provides that there must be no individual corporate qualifications criterion for which the bidder’s proposal receives a score less than 3.0 (average points across evaluators). Dr. Fisher, the primary author of Section 11.3 of the RFP, referred to the 3.0 rating as the "cut score." (Emphasis added.) The RFP’s clear and unambiguous terms thus establish the "minimum threshold" of experience that a bidder "must demonstrate" in its proposal for Criterion C1 through Criterion C9. The "minimum threshold" of experience that a bidder must demonstrate for each criterion is described in Appendix J of the RFP. If a proposal failed to demonstrate that the bidder meets the minimum threshold of experience for a particular criterion in Appendix J, the bidder obviously would not have demonstrated "that it meets an acceptable level of experience and capability to perform the required tasks." Thus, in that setting, an evaluator was to have assigned the proposal a rating of less than "satisfactory," or less than three, for that criterion. (Emphasis added.) The fact that a score less than "3" was expected for -- and would eliminate -- proposals that did not demonstrate the "minimum threshold" of experience does not render meaningless the potential scores of "1" and "2." Those scores may reflect the degree to which a bidder’s demonstrated experience was judged to fall below the threshold. Although some corporate capability minimums were stated quantitatively (i.e., "minimum of two," or "at least 200,000"), others were open to a more qualitative assessment (i.e., "large-scale," "systems," or "reports"). Moreover, a proposal that included demonstrated experience in some manner responsive to each aspect of Appendix J might nevertheless be assigned a score of less than "3," based on how an evaluator assessed the quality of the experience described in the proposal. By the terms of the RFP, however, an average score across evaluators of less than 3 represented essentially a decision that the minimum threshold of experience was not demonstrated. Had the Department truly intended Appendix J to reflect only general targets or guidelines, there were many alternative ways to communicate such an intent without giving mandatory direction about what bidders "must demonstrate" or without establishing quantitative minimums (i.e. "a minimum of two," or "at least 200,000"). RFP Appendix K, for instance, sets forth the evaluation criteria for technical proposals in broad terms that do not require the bidder to provide anything in particular. Even within Appendix J, other than in Criterion C4 and Criterion C6, bidders were to show experience with "large-scale" projects rather than experience at a quantified level. Pursuant to the RFP’s plain language, in order to meet the "minimum threshold" of experience for Criterion C4 and Criterion C6, a bidder "must demonstrate," among other things, successful completion of a "minimum of two" projects, each involving the use of image-based scoring technology in administering tests to "at least 200,000 students annually." Department’s Evaluation of Corporate Qualifications In evaluating Harcourt’s proposal, the Department failed to give effect to the plain RFP language stating that a bidder "must document" successful completion of a "minimum of two" testing projects involving "at least 200,000 students annually" in order to meet the "minimum threshold" of experience for C4 and C6. Dr. Fisher was the primary author of Sections 10, 11 and Appendix J of the RFP. He testified that during the Stage II evaluation of corporate qualifications, the evaluation team applied a "holistic" approach, like that used in grading open-ended written responses in student test assessments. Under the holistic approach that Dr. Fisher described, each member of the evaluation team was to study the proposals, compare the information in the proposals to everything contained in Appendix J, and then assign a rating for each criterion in Appendix J based on "how well" the evaluator felt the proposal meets the needs of the agency. Notwithstanding Dr. Fisher’s present position, the RFP’s terms and their context demonstrate that the minimum requirements for corporate qualifications are in RFP Appendix J. During the hearing, Dr. Fisher was twice asked to identify language in the RFP indicating that the Department would apply a "holistic" approach when evaluating corporate qualifications. Both times, Dr. Fisher was unable to point to any explicit RFP language putting bidders on notice that the Department would be using a "holistic" approach to evaluating proposals and treating the Appendix J thresholds merely as targets. In addition, Dr. Fisher testified that the Department did not engage in any discussion at the bidders’ conference about the evaluation method that was going to be used other than drawing the bidders’ attention to the language in the RFP. As written, the RFP establishes minimum thresholds of experience to be demonstrated. Where, as in the RFP, certain of those minimum thresholds are spelled out in quantitative terms that are not open to interpretation or judgment, it is neither reasonable nor logical to rate a proposal as having demonstrated "an acceptable level of experience" when it has not demonstrated the specified minimum levels, even if other requirements with which it was grouped were satisfied. The plain RFP language unambiguously indicates that an analytic method, not a "holistic" method, will be applied in evaluating corporate qualifications. Dr. Fisher acknowledged that, in an assessment using an analytic method, there is considerable effort placed up front in deciding the specific factors that will be analyzed and those factors are listed and explained. Dr. Fisher admitted that the Department went into considerable detail in Appendix J of the RFP to explain to the bidders the minimums they had to demonstrate and the documentation that was required. In addition, Dr. Orr, who served as a member of the evaluation team and who herself develops student assessment tests, stated that in assessments using the "holistic" method there is a scoring rubric applied, but that rubric does not contain minimum criteria like those found in the RFP for FCAT. The holistic method applied by the Department ignores very specific RFP language which spells out minimum requirements for corporate qualifications. Harcourt’s Corporate Qualifications for C4 and C6 Harcourt’s proposal lists the same three projects administered by Harcourt for both Criterion C4 and Criterion C6: the Connecticut Mastery Test ("CMT"), the Connecticut Academic Performance Test ("CAPT") and the Delaware Student Testing Program ("DSTP"). Harcourt’s proposal also lists for Criterion C4 projects administered by its proposed scoring subcontractors, Measurement Incorporated ("MI") and Data Recognition Corporation ("DRC"). However, none of the projects listed for MI or DRC involve image- based scoring. Thus, the MI and DRC projects do not demonstrate any volume of image-based scoring as required by C6 and by the portion of C4 which relates to the work task for the imaged-based scoring of the math and reading portions of the FCAT. Harcourt’s proposal states that "[a]pproximately 35,000 students per year in grade 10 are tested with the CAPT." Harcourt’s proposal states that "[a]pproximately 120,000 students per year in grades 4, 6 and 8 are tested with the CMT." Harcourt’s proposal states that "[a]pproximately 40,000 students in grades 3, 5, 8, and 10" are tested with the DSTP. Although the descriptions of the CMT and the CAPT in Harcourt’s proposal discuss image-based scoring, there is nothing in the description of the DSTP that addresses image-based scoring. There is no evidence that the evaluators were ever made aware that the DSTP involved image-based scoring. Moreover, although the Department called the Delaware Department of Education ("DDOE") as a reference for Harcourt’s development proposal, the Department did not discuss Harcourt’s administration of the DSTP (including whether the DSTP involves image-based scoring) with the DDOE. Harcourt overstated the number of students tested in the projects it referenced to demonstrate experience with image-based scoring. Harcourt admitted at hearing that, prior to submitting its proposal, Harcourt had never tested 120,000 students with the CMT. In fact, the total number of students tested by Harcourt on an annual basis under the CMT has ranged from 110,273 in the 1996- 97 school year to 116,679 in the 1998-99 school year. Harcourt also admitted at hearing that, prior to submitting its proposal, Harcourt had never tested 35,000 students in grade 10 with the CAPT. Instead, the total number of grade 10 students tested by Harcourt on an annual basis with the CAPT ranged from 30,243 in 1997 to 31,390 in 1998. In addition, Harcourt admitted at hearing that, prior to submitting its proposal, it had conducted only one "live" administration of the DSTP (as distinguished from field testing). That administration of the DSTP involved only 33,051, not 40,000, students in grades 3, 5, 8 and 10. Harcourt itself recognized that "field tests" of the DSTP are not responsive to C4 and C6, as evidenced by Harcourt’s own decision not to include in its proposal the number of students field tested under the DSTP. Even assuming that the numbers in Harcourt’s proposal are accurate, and that the description of the DSTP in Harcourt’s proposal reflected image-based scoring, Harcourt’s proposal on its face does not document any single project administered by Harcourt for C4 or C6 involving image-based testing of more than 120,000 students annually. When the projects are aggregated, the total number of students claimed as tested annually still does not reach the level of "at least 200,000;" it comes to only 195,000, and it reaches that level only once due to the single administration of the DSTP. Moreover, even if that 195,000 were considered "close enough" to the 200,000 level required, it was achieved only one time, while Appendix J plainly directs that there be a minimum of two times that testing at that level has been performed. The situation worsens for Harcourt when using the true numbers of students tested under the CMT, CAPT, and DSTP, because Harcourt cannot document any single image-based scoring project it has administered involving testing more than 116,679 students annually. Moreover, when the true numbers of students tested are aggregated, the total rises only to 181,120 students tested annually on one occasion, and no more than 141,663 tested annually on any other occasion. Despite this shortfall from the minimum threshold of experience, under the Department’s holistic approach the evaluators assigned Harcourt’s proposal four ratings of 3.0 and two ratings of 4.0 for C4, for an average of 3.3 on C4; and five ratings of 3.0 and one rating of 4.0 for C6, for an average of 3.2 on C6. Applying the plain language of the RFP in Sections 10 and 11 and Appendix J, Harcourt did not demonstrate that it meets an acceptable level of experience and capability for C4 or C6, because Harcourt did not satisfy the minimum threshold for each criterion by demonstrating a minimum of two prior completed projects involving image-based scoring requiring testing of at least 200,000 students annually. Harcourt’s proposal should not have received any rating of 3.0 or higher on C4 or C6 and should have been disqualified from further evaluation due to failure to demonstrate the minimum experience that the Department required in order to be assured that Harcourt can successfully administer the FCAT program. NCS’s Compliance With RFP Requirements Even though the NCS proposal did not meet all of the mandatory requirements, and despite the requirement of Section 11.2 that the proposal be automatically disqualified under such circumstances, the Department waived NCS’s noncompliance as a minor irregularity. The factors in C4 and C6 were set, minimal requirements with which NCS did not comply. For example, one of the two programs NCS submitted in response to Criteria C4 and C6 was the National Assessment of Educational Progress program ("NAEP"). NAEP, however, is not a "statewide assessment program" within the meaning of that term as used in Criteria C4 and C6. Indeed, NCS admitted that NAEP is not a statewide assessment program and that, without consideration of that program, NCS’s proposal is not responsive to Criteria C4 and C6 because NCS has not have submitted the required proof of having administered two statewide assessment programs. This error cannot be cured by relying on the additional experience of NCS’s subcontractor because that experience does not show that its subcontractor produced reports within three months, and so such experience does not demonstrate compliance with Criteria C4. The Department deliberately limited the competition for the FCAT contract to firms with specified minimum levels of experience. As opined at final hearing, if the Department in the RFP had announced to potential bidders that the type of experience it asked vendors to describe were only targets, goals and guidelines, and that a failure to demonstrate target levels of experience would not be disqualifying, then the competitive environment for this procurement would have differed since only 2.06 evaluation points (out of a possible 150) separated the NCS and Harcourt scores. Dr. Heidorn conceded that multiple companies with experience in different aspects of the FCAT program -- a computer/imaging company and a firm experienced in educational testing -- might combine to perform a contract like the FCAT. Yet, that combination of firms would be discouraged from bidding because they could not demonstrate the minimum experience spelled out in the RFP. Language in the RFP, indicating the "holistic" evaluation that was to be applied, could have resulted in a different field of potential and actual bidders.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is recommended that Respondent, State of Florida, Department of Education, enter a Final Order rejecting the bids submitted by Harcourt and NCS for the administration component of the RFP. The Department should then seek new proposals. DONE AND ENTERED this 25th day of May, 1999, in Tallahassee, Leon County, Florida. DON W. DAVIS Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 25th day of May, 1999. COPIES FURNISHED: Karen D. Walker, Esquire Holland and Knight, LLP Post Office Drawer 810 Tallahassee, Florida 32302 Mark D. Colley, Esquire Holland and Knight, LLP Suite 400 2100 Pennsylvania Avenue, Northwest Washington, D.C. 20037 Charles S. Ruberg, Esquire Department of Education The Capitol, Suite 1701 Tallahassee, Florida 32399-0400 Paul R. Ezatoff, Jr., Esquire Christopher B. Lunny, Esquire Katz, Kutter, Haigler, Alderman, Bryant and Yon, P.A. 106 East College Avenue, Suite 1200 Tallahassee, Florida 32302-7741 Tom Gallagher Commissioner of Education Department of Education The Capitol, Plaza Level 08 Tallahassee, Florida 32399-0400 Michael H. Olenick, General Counsel Department of Education The Capitol, Suite 1701 Tallahassee, Florida 32399-0400
The Issue The issue to be resolved is whether Petitioner is entitled to additional credit for her response to question nos. 122 and 222 of the civil engineering examination administered on October 31, 1997.
Findings Of Fact On October 31, 1997, Petitioner took the civil professional engineering licensing examination. A score of 70 is required to pass the test. Petitioner obtained a score of 69. Petitioner challenged the scoring of question nos. 122 and 222. As part of the examination challenge process, Petitioner's examination was returned to the National Council of Examiners for Engineering and Surveying where it was re-scored. In the re-score process, the grader deducted points from Petitioner's original score. Petitioner was given the same raw score of 6 on question number 122; however, on question number 222 her raw score of 4 was reduced to a 2. Petitioner needed a raw score of 48 in order to achieve a passing score of 70; she needed at least three additional raw score points to obtain a passing raw score of 48. Petitioner is entitled to a score of 6 on problem number 122. The solution and scoring plan for that problem required the candidate to obtain a culvert size in the range of 21-36 inches. The Petitioner incorrectly answered 3.1 feet or 37.2 inches. She is not entitled to additional credit for problem number 122 because she answered the question with the wrong size culvert. Problem number 122 required the candidate to use a predevelopment peak flow of 40 cubic feet per second (cfs). Petitioner used 58.33 cfs. She chose the maximum flow rather than the predevelopment peak flow. In solving problem number 122, Petitioner chose a design headwater depth of 4.8 feet. The correct solution required a design headwater depth of 5.7 feet. Petitioner made another mistake in problem number 122; she failed to check the water depth in the downstream swale. Petitioner concedes she was given sufficient information to solve problem number 122. She understood what the question was asking of her. She admits that she did not compute the critical depth of the water and that she did not complete the solution. Question number 222 had three parts. The candidate was required to determine the footing size, to select the reinforcing steel, and to provide a sketch for a concrete column located along the edge of a building. Petitioner understood the question and was provided enough information to solve the problem. Petitioner correctly checked the footing size as required by the first part; however, she did not select the reinforcing steel or show the required sketch. Therefore, Petitioner did not complete enough of the problem to qualify for a score of 4 points. She is entitled to a score of 2 points. The examination questions at issue here were properly designed to test the candidate's competency in solving typical problems in real life. The grader (re-scorer) utilized the scoring plan correctly. Petitioner has been in the United States for approximately eleven years. She lived in Romania before she came to the United States. In Romania, Petitioner used only the metric system in her professional work. While she has used the English system since moving to the United States, Petitioner is more familiar with the metric system. The Principles and Practice examination is an open-book examination. Petitioner took a book entitled the Fundamentals of Engineering Reference Handbook to the examination. When the proctor examined her books, she told the Petitioner she was not permitted to keep the handbook. The proctor took the handbook from the Petitioner. Petitioner protested the confiscation of her reference book because she had used the same book in two previous tests. About ten minutes later, the proctor's supervisor returned the book to Petitioner. Petitioner's book was returned at least ten minutes before the test began. She was permitted to use the book during the test. There is no persuasive evidence that the proctor's mistake in temporarily removing Petitioner's reference book caused her to be so upset that she failed the test. Candidates were not permitted to study their books prior to the beginning of the examination. Petitioner may have been nervous when the test began. However, Petitioner received a perfect score of ten points on the first problem she worked, problem number 121.
Recommendation Based upon the findings of fact and conclusions of law, it is RECOMMENDED that the Board of Professional Engineers enter a Final Order confirming Petitioner's score on the examination and dismissing the Petitioner's challenge. DONE AND ENTERED this 13th day of January, 1999, in Tallahassee, Leon County, Florida. SUZANNE F. HOOD Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 13th day of January, 1999. COPIES FURNISHED: Natalie A. Lowe, Esquire Board of Professional Engineers 1208 Hays Street Tallahassee, Florida 32301 William Bruce Muench, Esquire 438 East Monroe Street Jacksonville, Florida 32202 Lynda L. Goodgame, General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792 Dennis Bartin, President Florida Engineers Management Corporation 1940 North Monroe Street Tallahassee, Florida 32399-0792
The Issue Whether Respondent, Department of Transportation’s (“DOT”), notice of intent to award a contract to Intervenor, Blue Ray’z Heating and Air Conditioning, LLC (“Blue”), for maintenance, repair, installation, and replacement of heating, ventilation, and air conditioning (“HVAC”) equipment and components located at various facilities along Florida’s Turnpike System, is contrary to DOT’s governing statutes, rules, or the bid specifications, and contrary to competition, clearly erroneous, or arbitrary and capricious.
Findings Of Fact DOT is an agency of the state of Florida tasked with procuring the construction of all roads designated as part of the State Highway System, the State Park Road System, or any roads placed under DOT’s supervision by law. On March 10, 2017, DOT published its bid solicitation for ITB-DOT-16/17-8017-AC (the “ITB”), seeking bids from contractors for maintenance, repair, installation, and replacement of HVAC equipment and components at various facilities along Florida’s Turnpike (SR 91) milepost 172.0 to milepost 312.0; Southern Connector (SR 417) milepost 0.0 to milepost 6.5; Beachline Expressway (SR 528) milepost 0.0 to milepost 8.4; Seminole Expressway (SR 417) milepost 37.7 to milepost 55.0; and Daniel Webster Western Beltway (SR 429) milepost 0.9 to milepost 11.0. The scope of work of the ITB requires all labor, materials, and incidentals necessary to provide maintenance and repair of 232 HVAC units located at 65 facilities along Florida’s Turnpike System. The contract is for one year, with three one- year renewal periods. The 65 facilities span the distance from Wildwood in the north at milepost 304 to past Yeehaw Junction in the south to milepost 172, and from a westernmost point on State Road 429 at milepost 11 (Orlando area) to the easternmost section of State Road 417. Under the ITB, the vendor is required to conduct bimonthly preventative maintenance services on each HVAC unit; a total of five visits per site, per year. The vendor is also required during the first month of the contract and any subsequent annual renewal periods to conduct one annual preventative maintenance service. The annual maintenance is typically more extensive than the 60-day maintenance. However, the bi-monthly and annual maintenance services require, on average, 30 minutes for each of the 232 HVAC units. The vendor is also required to provide unscheduled, emergency services to diagnose problems and make necessary repairs of units that are not operating properly. An unscheduled repair could take several hours to complete, and there have been occasions where more than one unit needed a repair at the same time. The vendor must be available 24 hours a day, seven days a week, 52 weeks a year, to provide unscheduled, emergency services. Most air conditioning work is treated as an emergency, which requires the vendor to respond within three hours. The ITB includes specifications, schedules, a list of facilities, and other materials. Section 9.1 of the ITB requires bidders to meet certain minimum qualifications, including demonstrating the experience necessary to satisfactorily perform the services within the scope of work. Of particular relevance to the instant case is the following language on page 12 of the ITB: Certification of Experience The organized business enterprise (e.g. corporation, LLC or sole proprietorship) shall have been licensed and actively involved in the type of business requested for a minimum of three (3) years. Prior experience shall specifically be related to HVAC maintenance, repair, installation and replacement services of commercial facilities similar in size, technical scope, and volume of work to that specified in the Scope of Work for this Contract. Submit documentation of the work experience with the bid package. Pursuant to section 9.2, “[f]ailure by the bidder to provide the above item(s) will constitute a non-responsive determination. Bids found to be non-responsive will not be considered.” The Certification of Experience requirement is also referenced on page 2 of the ITB, followed by this statement: “The Department will review carefully to determine if the Vendor is responsive, responsible and qualified in the area of work contemplated by this Contract.” A two-page “Certification of Experience Documentation” form is located on pages 23 and 24 of the ITB. The form reiterates, in pertinent part: Prior experience shall specifically be related to the technical scope and volume of work to that specified in the scope of work for this Contract. Submit documentation of the work experience with the bid package. The Department will review carefully to determine if the Vendor(s) is responsive, responsible and qualified in the area of work contemplated by this Contract. Describe your work experience in detail for the minimum period required, beginning with your current or most recent project. Use a separate block to describe each project. (Print out additional copies of the form or attach additional sheets as necessary.) The form includes many lines, spaces, and separate blocks for bidders to provide the client names, addresses, and telephone numbers; dates of service; dollar value of each project; client project manager for each project; and a description of each project. The form includes enough lines and spaces for bidders to provide the required information for up to seven clients. The bidders are directed to attach additional sheets as necessary. The deadline for submission of bids was Tuesday, April 4, 2017, at 2:30 p.m. On April 4, 2017, DOT received and opened bids from four vendors in response to the ITB, which revealed the following bid tabulation prices: (1) All Seasons ($158,446.00); (2) Blue ($128,630.00); (3) Kenyon & Partners, LLC ($279,183.00); and (4) Florida Drawbridges, Inc. ($331,183.00). On April 4, 2017, DOT posted a notice of intent to award the contract for the work described in the ITB to Blue. Notwithstanding the requirement for each bidder to demonstrate prior experience “specifically . . . related to HVAC maintenance, repair, installation and replacement services of commercial facilities similar in size, technical scope, and volume of work to that specified in the Scope of Work for this Contract,” Blue failed to demonstrate such experience in the Certification of Experience Documentation form. The first client listed on Blue’s Certification of Experience Documentation form is the Florida Department of Juvenile Justice (“DJJ”). With regard to this client, Blue indicated service dates of June 2015 to present and a project dollar value of $5,000.00 per year. As to the project description, Blue stated: “Provide HVAC Maintenance, Repair & Service @ Orange Regional Juvenile Detention Center.” At hearing, Mr. Davis, the sole owner and operator of Blue, conceded the work involved a total of only 12 HVAC units, in two DJJ buildings, and at the same location. The second client listed is Florida Environmental Compliance Corp. (“FECC”). With regard to this client, Blue indicated service dates of 2012 to present and no dollar value for the project was provided. As to the project description, Blue stated: “HVAC Maintenance, Repair & Service. Installation.” At hearing, Mr. Davis conceded the work for FECC involved a total of only eight HVAC units at two locations (Polk County–two units, and Orlando-six units). The third client listed is DOT. With regard to this client, Blue indicated service dates of July through August 2015, and the dollar value of the project was $21,300.00. As to the project description, Blue stated: “Installation of Recorder Room Ductless split system.” At hearing, Mr. Davis conceded the work for DOT involved the installation of only three HVAC units in three locations at a cost of $7,100.00 each. The fourth client listed is CVS. With regard to this client, Blue indicated service dates of July through August 2014, and the dollar value of the project was $17,000.00. As to the project description, Blue stated: “Installation of Mini split system in CVS store clinics.” At hearing, Mr. Davis conceded the work for CVS involved the installation of only two HVAC units at one store location.2/ DOT did not review Blue’s Certification of Experience Documentation form to determine whether Blue demonstrated the necessary qualifications and experience required by the ITB. At hearing, none of the witnesses presented by DOT could testify as to Blue’s demonstration of prior experience, specifically related to HVAC maintenance, repair, installation, and replacement services of commercial facilities similar in size, technical scope, and volume of work to that specified in the scope of work. Leonard Robinson, a contract manager for Jacobs Engineering, the consultant for the project, testified he was involved only “to a very small extent” in the review of Blue’s bid. Mr. Robinson testified that “the only thing I had to do was to award the bid per the lowest bidder as written in the contract and state the reason why I’m doing so and also sign the Conflict of Interest form. That is it.” Mr. Robinson did not review Blue’s qualifications or past experience, and he could not say that Blue could perform the work described in the ITB. Alan Chua is the capital improvement procurement administrator for Jacobs Engineering. Mr. Chua evaluated Blue’s bid to determine whether it corresponded to the engineer’s estimate. He testified he reviewed the Certification of Experience Documentation form, but only as to the dates and to the limited extent necessary to determine whether the projects listed by Blue satisfied the three years of experience required by the contract. Mr. Chua conceded he did not review the project descriptions for the four referenced clients identified on the form, and he could not say whether any of the four projects identified on the form were similar in volume or technical scope to that specified in the ITB. Trisa Thomas is a contracts development manager for Jacobs Engineering. Ms. Thomas testified she reviewed the Certification Experience Documentation form. Ms. Thomas testified she reviewed the dates to determine whether the projects listed by Blue satisfied the three years or more of experience required by the contract. She further testified she reviewed the dates, project manager, and project descriptions. As to the project descriptions, Ms. Thomas testified she reviewed the type of work Blue did for the company to see if it was related to the work specified in the ITB. However, she did not consider the volume of Blue’s work. Significantly, Ms. Thomas testified: Q: Okay. All right. Do you know why the dollar value of the project is on the Certification of Experience Documentation? A: Just to get an idea what they--how much they probably performed. Q: So would that help you get an idea of the volume of work that they performed for that client? A: I’m not even really sure if that played a factor. Q: And when you say, I’m not sure if that played a factor, what--what are you referring to? A: Well I guess what I’m saying is, I’m-- that’s really not where my concentration is. I’m looking at the years, the project description, what they--the type of work they performed for the--for that agency, and if the references are credible or, you know, satisfactory. Q: Okay. And when you say they’re satisfactory, what do you look at to determine if they’re satisfactory? A: Based on the service that they provided to them, the vendor will let us know how pleased they were with the work that they provided to their agency. Q: All right. Looking at this Joint Exhibit 5, pages 5 and 6, which places did Blue Ray’z perform HVAC maintenance, repair, installation and repair services that were of the same technical scope as the Invitation to Bid? A: I see on the Certification of Experience where they provided HVAC repair and service installation. Also I see where they provided installation of a recorder room. But the volume, there’s--I don’t--I wouldn’t be able to determine that. Q: And let me ask my question again. So which places did Blue Ray’z perform HVAC maintenance, repair, installation and replacement services that were of the similar technical scope? A: I don’t know. Q: What maintenance, repair and installation or replacement services did Blue Ray’z provide to another vendor that was similar to the volume of work specified in the Invitation to Bid? A: The volume work? I don’t know. Q: Do you know if the bid by Blue Ray’z was reviewed by anybody to determine if the experience was similar in size, technical scope and volume of work specified in the Scope of Work? A: I don’t know. Tr., Vol. I, pp. 121-123. Santiago Alvarez is the facilities and telecommunications administrator for the Turnpike Enterprise, which is part of DOT. Mr. Alvarez testified he did not review the Certification of Experience Documentation form in any detail. He just looked to confirm the documents were included in the bid package. Sheree Merting is the contract services administrator for DOT’s Turnpike Enterprise. Ms. Merting testified she was not involved in the evaluation of Blue’s bid. Amanda Cruz is a contract analyst for DOT. She did not review Blue’s bid to determine whether Blue was qualified by having the experience required by the ITB. DOT established specific requirements for the ITB to determine responsiveness and then failed to determine if Blue had prior work experience specifically related to HVAC maintenance, repair, installation, and replacement services of commercial facilities similar in size, technical scope, and volume of work to that specified in the scope of work. DOT awarded the bid to Blue because it was the lowest bidder, without considering whether Blue demonstrated prior experience specifically related to HVAC maintenance, repair, installation, and replacement services of commercial facilities similar in size, technical scope, and volume of work to that specified in the scope of work. DOT awarded the bid to Blue because it was the lowest bidder, even though Blue failed to demonstrate in the Certification of Experience Documentation form that it had prior experience specifically related to HVAC maintenance, repair, installation, and replacement services of commercial facilities similar in size, technical scope, and volume of work to that specified in the scope of work. In sum, Blue is not a responsive and responsible vendor, and DOT’s proposed action is contrary to the bid specifications, clearly erroneous, contrary to competition, arbitrary and capricious.3/
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that Respondent, Department of Transportation, enter a final order rescinding the proposed award to Intervenor, Blue Ray’z Heating and Air Conditioning, LLC. DONE AND ENTERED this 28th day of August, 2017, in Tallahassee, Leon County, Florida. S DARREN A. SCHWARTZ Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 28th day of August, 2017.
