The Issue Whether Petitioner is eligible for licensure by endorsement as a professional engineer and/or waiver of Part I of the engineering licensing examination.
Findings Of Fact From 1969 through 1974, Petitioner attended the Lipetsk Branch of the Moscow Institute of Steel and Alloys. In 1974, Petitioner graduated from the Lipetsk Polytechnical Institute (Institute) in Russia, with a degree in industrial and civil engineering. The degree in civil engineering earned by Petitioner is equivalent to a bachelor of science degree in civil engineering in the United States. Upon graduation from the Institute, Petitioner commenced her professional employment as an engineer on August 30, 1974, at the LIPTSKGRAZHDANPROEKT Design Institute. In January 1976, Petitioner was employed as a professional engineer at the State Design Institute DNEPRPROEKKTSTALKONSTRUKSIYA (DNEPR), where she was continuously employed until leaving the Soviet Union in December 1993. There are no specific licensing or registration requirements in Russia with respect to engineers. Therefore, after earning a degree in industrial and civil engineering, Petitioner could work as an engineer in Russia without taking any professional examination. However, in 1979, Petitioner took an examination in Russia that allowed her to sign her own drawings and calculations. During Petitioner's tenure at the DNEPR, she achieved the status of Senior Engineer in 1986; was promoted to the position of Category II Engineer for Steel Structures in 1988; was promoted to the post of Category I Engineer for Steel Structures in 1990; and was elevated to the position of Leading Engineer in 1991. To achieve the status of Category II Engineer for Steel Structures and Category I Engineer for Steel Structures at DNEPR, Petitioner had to take an examination in 1988 and in 1990, respectively. The promotion to each of these positions was predicated upon Petitioner's passing these examinations and demonstrating expertise in the areas of economics, chemistry, mathematics, physics, building materials, corrosion prevention, resistance of materials, and construction mechanics. As a result of passing the examinations in 1988 and 1990, Petitioner was not only promoted, but also received salary increases. Petitioner believes that the two examinations she took in Russia in 1988 and in 1990, while working at the DNEPR were substantially equivalent to the Fundamentals Examination. However, no evidence was presented to support this claim. The Fundamentals Examination is one component of the engineering licensing examination, and is designed to assess whether an individual is qualified to practice in this state as an engineer intern. This examination is usually taken either in the applicant's last year in engineering school or shortly after graduation. With regard to format, the Fundamentals Examination is an eight-hour examination and consists of 120 multiple-choice questions. The Principles and Practice Examination is the second part of the engineering licensing examination and is taken after successful completion of the Fundamentals Examination. Oscar E. Olsen, a structural engineer and owner of O.E. Olsen and Associates, a structural engineering firm, is currently Petitioner's employer. Mr. Olsen, who is generally familiar with the Fundamentals Examination, testified that the list of subjects covered on the two examinations taken by Petitioner in 1988 and 1990, coincide with the subject matter on the Fundamentals Examination. Mr. Olsen further testified that it appeared to him that the two examinations taken by Petitioner were comparable to the Fundamental Examinations required in Florida. Notwithstanding his testimony that the exams taken by Petitioner are substantially equivalent to the Fundamentals Examination, Mr. Olsen admitted that he has never seen or reviewed the examinations taken by Petitioner while she was in Russia. It is impossible to render a reasonable opinion as to whether the two examinations taken by Petitioner in Russia are substantially equivalent to the Fundamentals Examination, where the only information provided with regard to the former is a list of subject areas covered. Such a list gives no indication of the depth and specific content of the subject matter on the examinations; the difficulty of the examinations; the passing scores; the number and format of the questions; and the length of the examinations.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is recommended that Respondent, the Board of Professional Engineers, enter a Final Order denying Petitioner's request for waiver of Part I, the Fundamentals Examination, and for licensure by endorsement. DONE AND ENTERED this 9th day of December, 1997, in Tallahassee, Leon County, Florida. CAROLYN S. HOLIFIELD Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (904) 488-9675 SUNCOM 278-9675 Fax Filing (904) 921-6847 Filed with the Clerk of the Division of Administrative Hearings this 9th day of December, 1997. COPIES FURNISHED: Murray Silverstein, Esquire Powell, Carney, Hayes, and Silverstein One Plaza, Suite 1210 St. Petersburg, Florida 33731-1689 Edwin A. Bayo Assistant Attorney General Office of the Attorney General The Capitol, Plaza 01 Tallahassee, Florida 32399-1050 Angel Gonzalez Executive Director Board of Professional Engineers Northwood Centre 1940 North Monroe Street Tallahassee, Florida 32399-0755 Lynda L. Goodgame, General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792
The Issue The primary issue is whether the process used by the Department of Education (Department) for evaluating and ranking the proposals submitted in response to Request For Proposal (RFP) 99-03 for the Florida Comprehensive Assessment Test (FCAT) administration contract was contrary to the provisions of the RFP in a way that was clearly erroneous, contrary to competition, arbitrary, or capricious.
Findings Of Fact The RFP for the FCAT describes a five stage process for evaluating proposals. In Stage I, the Department’s Purchasing Office determined whether a proposal contained certain mandatory documents and statements and was sufficiently responsive to the requirements of the RFP to permit a complete evaluation. Stage II involved the Department’s evaluation of a bidder’s corporate qualifications to determine whether the bidder has the experience and capability to do the type of work that will be required in administering the FCAT. Stage III was the Department’s evaluation of a bidder’s management plan and production proposal. In Stage IV, the Department evaluated a bidder’s cost proposal. Stage V involved the ranking of proposals based on points awarded in Stages II-IV. If a proposal did not meet the requirements at any one stage of the evaluation process, it was not to be evaluated in the following stage. Instead, it was to be disqualified from further consideration. Stages II and III of the evaluation process were conducted by an evaluation team comprised of six Department employees: Dr. Debby Houston, Ms. Lynn Joszefczyk, Dr. Peggy Stillwell, Dr. Cornelia Orr, Dr. Laura Melvin, and Ms. Karen Bennett. Dr. Thomas Fisher, head of the Department’s Assessment and Evaluation Services Section, and Dr. Mark Heidorn, Administrator for K-12 Assessment Programs within the Department’s Assessment and Evaluation Services Section, served as non-voting co-chairs of the evaluation team. The focus of this proceeding is Stage II of the evaluation process addressing a bidder’s corporate qualifications. RFP Provisions Regarding Corporate Qualification The FCAT administration contractor will be required to administer tests to approximately one and a half million students each year in a variety of subject areas at numerous grade levels. The FCAT program involves a complex set of interrelated work activities requiring specialized human resources, technological systems and procedures. The FCAT must be implemented annually within limited time periods. The FCAT administration contractor must meet critical deadlines for the delivery of test materials to school districts and the delivery of student scores prior to the end of the school year. In developing the RFP, the Department deliberately established a set of minimum requirements for corporate qualifications that a bidder was to demonstrate in order for its proposal to be eligible for further evaluation. The purpose of the RFP’s minimum corporate qualifications requirements was to limit bidding to qualified vendors who have demonstrated prior experience in successfully administering large-scale assessment projects like the FCAT, thereby providing the Department with some degree of assurance that the winning bidder could successfully administer the FCAT. The instructions to bidders regarding the minimum requirements for corporate qualifications are contained in RFP Section 10, which gives directions on proposal preparation. Section 10.1, which lists certain mandatory documents and statements to be included in the bidder’s proposal, requires that a transmittal letter contain "[a] statement certifying that the bidder has met the minimum corporate qualifications as specified in the RFP." These "minimum corporate qualifications" are set forth in RFP Appendix J. RFP Section 10.2 identifies what a bidder is required to include in its proposal with respect to corporate qualifications. The first paragraph of Section 10.2 directs a bidder generally to describe its qualifications and experience performing tasks similar to those that it would perform in administering the FCAT, in order to demonstrate that the bidder is qualified where it states: Part II of a bidder’s proposal shall be entitled Corporate Qualifications. It shall provide a description of the bidder’s qualifications and prior experience in performing tasks similar to those required in this RFP. The discussion shall include a description of the bidder’s background and relevant experience that qualifies it to provide the products and services required by the RFP. RFP Section 10.2, however, is not limited to a directive that qualifications and past experience be described generally. Instead, Section 10.2, also communicates, in plain and unambiguous terms, that there are specific minimum corporate qualifications a bidder must demonstrate: The minimum expectations for corporate qualifications and experience are shown in Appendix J. There are two separate sets of factors, one set of eight for the developmental contractor and another set of nine for the administration contractor. Bidders must demonstrate their Corporate Qualifications in terms of the factors that are applicable to the activities for which a bid is being submitted -- development or administration. For each criterion, the bidder must demonstrate that the minimum threshold of experience has been achieved with prior completed projects. (Emphasis added.) Moreover, Section 10.2 singles out for emphasis, in relation to the administration component of the RFP, the importance placed on a bidder’s ability to demonstrate experience processing a large volume of tests: The [bidder’s prior completed] projects must have included work tasks similar to those described herein, particularly in test development or processing a comparable number of tests. The bidder will provide a description of the contracted services; the contract period; and the name, address, and telephone number of a contact person for each of the contracting agencies. This description shall (1) document how long the organization has been providing similar services; (2) provide details of the bidder’s experience relevant to the services required by this RFP; and (3) describe the bidder’s other testing projects, products, and services that are similar to those required by this RFP. (Emphasis added.) The Department thus made clear its concern that bidders demonstrate experience with large-scale projects. RFP Appendix J sets forth nine different criteria (C1 through C9) for the administration contractor. As stated in RFP Section 10.2, "[f]or each criterion, the bidder must demonstrate that the minimum threshold of experience has been achieved with prior completed projects . . . ." (emphasis added). Appendix J contains a chart which lists for each criterion: (1) a summary of the related FCAT work task, (2) the detailed criteria for the bidder’s experience related to that work task, and (3) the necessary documentation a bidder must provide. Criterion C4 and Criterion C6 include work tasks that involve the use of image-based scoring technology. C4 and C6 are the only corporate qualifications criteria at issue in this proceeding. RFP Provisions Involving Corporate Qualifications for Image-Based Scoring "Handscoring" is the test administration activity in which open-ended or performance-based student responses are assessed. This practice involves a person reading something the student has written as part of the test, as distinguished from machine scoring multiple choice responses (i.e., the filled-in "bubbles" on an answer sheet). There are two types of handscoring: (1) paper-based handscoring, and (2) image-based handscoring. Paper-based handscoring requires that a student response paper be sent to a reader, who then reviews the student’s response as written on the paper and enters a score on a separate score sheet. Image-based handscoring involves a scanned image of the student’s response being transmitted to a reader electronically. The student’s response is then projected on a computer screen, where the reader reviews it and assigns a score using the computer. The RFP requires that the reading and math portions of the FCAT be handscored on-line using imaging technology beginning with the February 2000 FCAT administration. The RFP provides that the writing portion of the FCAT may be handscored using either the paper-based method or on-line imaging technology during the February 2000 and 2001 FCAT administrations. However, on-line image-based scoring of the writing portion of the FCAT is required for all FCAT administrations after February 2001. An image-based scoring system involves complex computer technology. William Bramlett, an expert in designing and implementing large-scale imaging computer systems and networks, presented unrefuted testimony that an image-based scoring system will be faced with special challenges when processing large volumes of tests. These challenges involve the need to automate image quality control, to manage the local and wide area network load, to assure adequate server performance and storage requirements, and to manage the work flow in a distributed environment. In particular, having an image-based scoring system process an increasing volume of tests is not simply a matter of adding more components. Rather, the system’s basic software architecture must be able to understand and manage the added elements and volume involved in a larger operation. According to Bramlett, there are two ways that the Department could assess the ability of a bidder to perform a large- scale, image-based scoring project such as the FCAT from a technological perspective: (1) have the bidder provide enough technological information about its system to be able to model or simulate the system and predict its performance for the volumes involved, or (2) require demonstrated ability through completion of prior similar projects. Dr. Mark Heidorn, Administrator for Florida’s K-12 Statewide Assessment Programs, was the primary author of RFP Sections 1-8, which describe the work tasks for the FCAT -- the goods and services vendors are to provide and respond to in their technical proposals. Dr. Heidorn testified that in the Department’s testing procurements involving complex technology, the Department has never required specific descriptions of the technology to be used. Instead, the Department has relied on the bidder’s experience in performing similar projects. Thus, the RFP does not specifically require that bidders describe in detail the particular strategies and approaches they intend to employ when designing and implementing an image-based scoring system for FCAT. Instead, the Department relied on the RFP requirements calling for demonstrated experience as a basis to understand that the bidder could implement such an image-based scoring system. Approximately 717,000 to 828,000 student tests will be scored annually by the FCAT administration contractor using imaging technology. The RFP, however, does not require that bidders demonstrate image-based scoring experience at that magnitude. Instead, the RFP requires bidders to demonstrate only a far less demanding minimum level of experience using image-based scoring technology. Criterion C4 and Criterion C6 in Appendix J of the RFP each require that a bidder demonstrate prior experience administering "a minimum of two" assessment programs using imaged- based scoring that involved "at least 200,000 students annually." The requirements for documenting a "minimum of two" programs or projects for C4 and C6 involving "at least 200,000 students annually" are material because they are intended to provide the Department with assurance that the FCAT administration contractor can perform the large-scale, image-based scoring requirements of the contract from a technological perspective. Such experience would indicate that the bidder would have been required to address the sort of system issues described by Bramlett. Dr. Heidorn testified that the number 200,000 was used in C4 and C6 "to indicate the level of magnitude of experience which represented for us a comfortable level to show that a contractor had enough experience to ultimately do the project that we were interested in completing." Dr. Fisher, who authored Appendix J, testified that the 200,000 figure was included in C4 and C6 because it was a number judged sufficiently characteristic of large-scale programs to be relevant for C4 and C6. Dr. Fisher further testified that the Department was interested in having information that a bidder’s experience included projects of a sufficient magnitude so that the bidder would have experienced the kinds of processing issues and concerns that arise in a large-scale testing program. The Department emphasized this specific quantitative minimum requirement in response to a question raised at the Bidder’s Conference held on November 13, 1998: Q9: In Appendix J, the criteria for evaluating corporate quality for the administration operations C4, indicates that the bidder must have experience imaging as indicated. Does this mean that the bid [sic] must bid for using [sic] imaging technology for reading and mathematics tests? A: Yes. The writing assessment may be handscored for two years, and then it will be scored using imaging technology. To be responsive, a bid must be for imaging. The corporate experience required (200,000 students annually for which reports were produced in three months) could be the combined experience of the primary contractor and the subcontractors. (Emphasis added.) Criterion C4 addresses the RFP work tasks relating to handscoring, including both the image-based handscoring of the reading and math portions of the FCAT for all administrations and the writing portions of the FCAT for later administrations. The "Work Task" column for C4 in Appendix J of the RFP states: Design and implement efficient and effective procedures for handscoring student responses to performance tasks within the limited time constraints of the assessment schedule. Handscoring involves image-based scoring of reading and mathematics tasks for all administrations and writing tasks for later administrations at secure scoring sites. Retrieve and score student responses from early district sample schools and deliver required data to the test development contractor within critical time periods for calibration and scaling. The "Necessary Documentation" column for C4 in Appendix J states: Bidder must document successful completion of a minimum of two performance item scoring projects for statewide assessment programs during the last four years for which the bidder was required to perform as described in the Criteria column. (Emphasis added.) The "Criteria" column for C4 in Appendix J, like the related work tasks in the RFP, addresses both image-based handscoring of reading and math, as well as paper-based or image- based handscoring of writing. In connection with all handscoring work tasks, "[t]he bidder must demonstrate completion of test administration projects for a statewide program for which performance items were scored using scoring rubrics and associated scoring protocols." With respect to the work tasks for handscoring the reading and math portions of the FCAT, "[t]he bidder must demonstrate completion of statewide assessment programs involving scoring multiple-choice and performance items for at least 200,000 students annually for which reports were produced in three months." In addition, for the reading and math work tasks, "[e]xperience must been shown in the use of imaging technology and hand-scoring student written responses with completion of scoring within limited time restrictions." This provision dealing with "imaging technology" experience self-evidently addresses the reading and math components, because separate language addresses imaging experience in connection with the writing component. The relevant handscoring experience for the reading and math aspects of the program is experience using image-based technology. By contrast, with respect to the work tasks for scoring the writing portions of the FCAT, "the bidder must also demonstrate completion of statewide assessment programs involving paper-based or imaged scoring student responses to writing assessment prompts for at least 200,000 students annually for which reports were produced in three months." (Emphasis added.) Criterion C6 addresses work tasks relating to designing and implementing systems for processing, scanning, imaging and scoring student responses to mixed-format tests within limited time constraints. The "Work Task" column for C6 in RFP Appendix J states: Design and implement systems for the processing, scanning, imaging, and scoring of student responses to test forms incorporating both multiple-choice and constructed response items (mixed-format) within the limited time constraints of the assessment schedule. Scoring of student responses involves implementation of IRT scoring tables and software provided by the development contractor within critical time periods. The "Necessary Documentation" column for C6 in Appendix J states: Bidder must document successful completion of a minimum of two test administration projects for statewide assessment programs during the last four years in which the bidder was required to perform as described in the Criteria column. (Emphasis added.) The Criteria column for C6 in Appendix J states: The bidder must demonstrate completion of test administration projects for statewide assessment programs or other large-scale assessment programs that required the bidder to design and implement systems for processing, scanning, imaging, and scoring responses to mixed-format tests for at least 200,000 students annually for which reports were produced in three months. Experience must be shown in use of imaging student responses for online presentation to readers during handscoring. (Emphasis added.) RFP Provisions Per Corporate Qualifications The procedure for evaluating a bidder’s corporate qualifications is described in RFP Section 11.3: The Department will evaluate how well the resources and experience described in each bidder’s proposal qualify the bidder to provide the services required by the provisions of this RFP. Consideration will be given to the length of time and the extent to which the bidder and any proposed subcontractors have been providing services similar or identical to those requested in this RFP. The bidder’s personnel resources as well as the bidder’s computer, financial, and other technological resources will be considered in evaluating a bidder’s qualifications to meet the requirements of this RFP. Client references will be contacted and such reference checks will be used in judging a bidder’s qualifications. The criteria to be used to rate a bidder’s corporate qualifications to meet the requirements of this RFP are shown in Appendix J and will be applied as follows: * * * Administrative Activities. Each of the nine administration activities criteria in Appendix J will be individually rated by members of the evaluation team. The team members will use the rating scale shown in Figure 1 below. Individual team members will review the bidder’s corporate qualifications and rate the response with a rating of one to five. The ratings across all evaluators for each factor will be averaged, rounded to the nearest tenth, and summed across all criteria. If each evaluator assigns the maximum number of points for each criterion, the total number of points will be 45. To meet the requirements of Stage II, the proposal must achieve a minimum rating of 27 points and have no individual criterion for which the number of points averaged across evaluators and then rounded is less than 3.0. Each proposal that receives a qualifying score based on the evaluation of the bidder’s qualifications will be further evaluated in Stage III. Figure 1 Evaluation Scale for Corporate Qualifications 5 Excellent 4 3 Satisfactory 2 1 Unsatisfactory The bidder has demonstrated exceptional experience and capability to perform the required tasks. The bidder has demonstrated that it meets an acceptable level of experience and capability to perform the required tasks. The bidder either has not established its corporate qualifications or does not have adequate qualifications. RFP Section 11.3 provides that each of the nine corporate qualifications criteria for administration operations in Appendix J (C1 through C9) will be individually rated by the six members of the evaluation team using a scale of one to five. A rating of three is designated as "satisfactory" which means that "[t]he bidder has demonstrated that it meets an acceptable level of experience and capability to perform the required tasks." In order to be further evaluated, Section 11.3 provides that there must be no individual corporate qualifications criterion for which the bidder’s proposal receives a score less than 3.0 (average points across evaluators). Dr. Fisher, the primary author of Section 11.3 of the RFP, referred to the 3.0 rating as the "cut score." (Emphasis added.) The RFP’s clear and unambiguous terms thus establish the "minimum threshold" of experience that a bidder "must demonstrate" in its proposal for Criterion C1 through Criterion C9. The "minimum threshold" of experience that a bidder must demonstrate for each criterion is described in Appendix J of the RFP. If a proposal failed to demonstrate that the bidder meets the minimum threshold of experience for a particular criterion in Appendix J, the bidder obviously would not have demonstrated "that it meets an acceptable level of experience and capability to perform the required tasks." Thus, in that setting, an evaluator was to have assigned the proposal a rating of less than "satisfactory," or less than three, for that criterion. (Emphasis added.) The fact that a score less than "3" was expected for -- and would eliminate -- proposals that did not demonstrate the "minimum threshold" of experience does not render meaningless the potential scores of "1" and "2." Those scores may reflect the degree to which a bidder’s demonstrated experience was judged to fall below the threshold. Although some corporate capability minimums were stated quantitatively (i.e., "minimum of two," or "at least 200,000"), others were open to a more qualitative assessment (i.e., "large-scale," "systems," or "reports"). Moreover, a proposal that included demonstrated experience in some manner responsive to each aspect of Appendix J might nevertheless be assigned a score of less than "3," based on how an evaluator assessed the quality of the experience described in the proposal. By the terms of the RFP, however, an average score across evaluators of less than 3 represented essentially a decision that the minimum threshold of experience was not demonstrated. Had the Department truly intended Appendix J to reflect only general targets or guidelines, there were many alternative ways to communicate such an intent without giving mandatory direction about what bidders "must demonstrate" or without establishing quantitative minimums (i.e. "a minimum of two," or "at least 200,000"). RFP Appendix K, for instance, sets forth the evaluation criteria for technical proposals in broad terms that do not require the bidder to provide anything in particular. Even within Appendix J, other than in Criterion C4 and Criterion C6, bidders were to show experience with "large-scale" projects rather than experience at a quantified level. Pursuant to the RFP’s plain language, in order to meet the "minimum threshold" of experience for Criterion C4 and Criterion C6, a bidder "must demonstrate," among other things, successful completion of a "minimum of two" projects, each involving the use of image-based scoring technology in administering tests to "at least 200,000 students annually." Department’s Evaluation of Corporate Qualifications In evaluating Harcourt’s proposal, the Department failed to give effect to the plain RFP language stating that a bidder "must document" successful completion of a "minimum of two" testing projects involving "at least 200,000 students annually" in order to meet the "minimum threshold" of experience for C4 and C6. Dr. Fisher was the primary author of Sections 10, 11 and Appendix J of the RFP. He testified that during the Stage II evaluation of corporate qualifications, the evaluation team applied a "holistic" approach, like that used in grading open-ended written responses in student test assessments. Under the holistic approach that Dr. Fisher described, each member of the evaluation team was to study the proposals, compare the information in the proposals to everything contained in Appendix J, and then assign a rating for each criterion in Appendix J based on "how well" the evaluator felt the proposal meets the needs of the agency. Notwithstanding Dr. Fisher’s present position, the RFP’s terms and their context demonstrate that the minimum requirements for corporate qualifications are in RFP Appendix J. During the hearing, Dr. Fisher was twice asked to identify language in the RFP indicating that the Department would apply a "holistic" approach when evaluating corporate qualifications. Both times, Dr. Fisher was unable to point to any explicit RFP language putting bidders on notice that the Department would be using a "holistic" approach to evaluating proposals and treating the Appendix J thresholds merely as targets. In addition, Dr. Fisher testified that the Department did not engage in any discussion at the bidders’ conference about the evaluation method that was going to be used other than drawing the bidders’ attention to the language in the RFP. As written, the RFP establishes minimum thresholds of experience to be demonstrated. Where, as in the RFP, certain of those minimum thresholds are spelled out in quantitative terms that are not open to interpretation or judgment, it is neither reasonable nor logical to rate a proposal as having demonstrated "an acceptable level of experience" when it has not demonstrated the specified minimum levels, even if other requirements with which it was grouped were satisfied. The plain RFP language unambiguously indicates that an analytic method, not a "holistic" method, will be applied in evaluating corporate qualifications. Dr. Fisher acknowledged that, in an assessment using an analytic method, there is considerable effort placed up front in deciding the specific factors that will be analyzed and those factors are listed and explained. Dr. Fisher admitted that the Department went into considerable detail in Appendix J of the RFP to explain to the bidders the minimums they had to demonstrate and the documentation that was required. In addition, Dr. Orr, who served as a member of the evaluation team and who herself develops student assessment tests, stated that in assessments using the "holistic" method there is a scoring rubric applied, but that rubric does not contain minimum criteria like those found in the RFP for FCAT. The holistic method applied by the Department ignores very specific RFP language which spells out minimum requirements for corporate qualifications. Harcourt’s Corporate Qualifications for C4 and C6 Harcourt’s proposal lists the same three projects administered by Harcourt for both Criterion C4 and Criterion C6: the Connecticut Mastery Test ("CMT"), the Connecticut Academic Performance Test ("CAPT") and the Delaware Student Testing Program ("DSTP"). Harcourt’s proposal also lists for Criterion C4 projects administered by its proposed scoring subcontractors, Measurement Incorporated ("MI") and Data Recognition Corporation ("DRC"). However, none of the projects listed for MI or DRC involve image- based scoring. Thus, the MI and DRC projects do not demonstrate any volume of image-based scoring as required by C6 and by the portion of C4 which relates to the work task for the imaged-based scoring of the math and reading portions of the FCAT. Harcourt’s proposal states that "[a]pproximately 35,000 students per year in grade 10 are tested with the CAPT." Harcourt’s proposal states that "[a]pproximately 120,000 students per year in grades 4, 6 and 8 are tested with the CMT." Harcourt’s proposal states that "[a]pproximately 40,000 students in grades 3, 5, 8, and 10" are tested with the DSTP. Although the descriptions of the CMT and the CAPT in Harcourt’s proposal discuss image-based scoring, there is nothing in the description of the DSTP that addresses image-based scoring. There is no evidence that the evaluators were ever made aware that the DSTP involved image-based scoring. Moreover, although the Department called the Delaware Department of Education ("DDOE") as a reference for Harcourt’s development proposal, the Department did not discuss Harcourt’s administration of the DSTP (including whether the DSTP involves image-based scoring) with the DDOE. Harcourt overstated the number of students tested in the projects it referenced to demonstrate experience with image-based scoring. Harcourt admitted at hearing that, prior to submitting its proposal, Harcourt had never tested 120,000 students with the CMT. In fact, the total number of students tested by Harcourt on an annual basis under the CMT has ranged from 110,273 in the 1996- 97 school year to 116,679 in the 1998-99 school year. Harcourt also admitted at hearing that, prior to submitting its proposal, Harcourt had never tested 35,000 students in grade 10 with the CAPT. Instead, the total number of grade 10 students tested by Harcourt on an annual basis with the CAPT ranged from 30,243 in 1997 to 31,390 in 1998. In addition, Harcourt admitted at hearing that, prior to submitting its proposal, it had conducted only one "live" administration of the DSTP (as distinguished from field testing). That administration of the DSTP involved only 33,051, not 40,000, students in grades 3, 5, 8 and 10. Harcourt itself recognized that "field tests" of the DSTP are not responsive to C4 and C6, as evidenced by Harcourt’s own decision not to include in its proposal the number of students field tested under the DSTP. Even assuming that the numbers in Harcourt’s proposal are accurate, and that the description of the DSTP in Harcourt’s proposal reflected image-based scoring, Harcourt’s proposal on its face does not document any single project administered by Harcourt for C4 or C6 involving image-based testing of more than 120,000 students annually. When the projects are aggregated, the total number of students claimed as tested annually still does not reach the level of "at least 200,000;" it comes to only 195,000, and it reaches that level only once due to the single administration of the DSTP. Moreover, even if that 195,000 were considered "close enough" to the 200,000 level required, it was achieved only one time, while Appendix J plainly directs that there be a minimum of two times that testing at that level has been performed. The situation worsens for Harcourt when using the true numbers of students tested under the CMT, CAPT, and DSTP, because Harcourt cannot document any single image-based scoring project it has administered involving testing more than 116,679 students annually. Moreover, when the true numbers of students tested are aggregated, the total rises only to 181,120 students tested annually on one occasion, and no more than 141,663 tested annually on any other occasion. Despite this shortfall from the minimum threshold of experience, under the Department’s holistic approach the evaluators assigned Harcourt’s proposal four ratings of 3.0 and two ratings of 4.0 for C4, for an average of 3.3 on C4; and five ratings of 3.0 and one rating of 4.0 for C6, for an average of 3.2 on C6. Applying the plain language of the RFP in Sections 10 and 11 and Appendix J, Harcourt did not demonstrate that it meets an acceptable level of experience and capability for C4 or C6, because Harcourt did not satisfy the minimum threshold for each criterion by demonstrating a minimum of two prior completed projects involving image-based scoring requiring testing of at least 200,000 students annually. Harcourt’s proposal should not have received any rating of 3.0 or higher on C4 or C6 and should have been disqualified from further evaluation due to failure to demonstrate the minimum experience that the Department required in order to be assured that Harcourt can successfully administer the FCAT program. NCS’s Compliance With RFP Requirements Even though the NCS proposal did not meet all of the mandatory requirements, and despite the requirement of Section 11.2 that the proposal be automatically disqualified under such circumstances, the Department waived NCS’s noncompliance as a minor irregularity. The factors in C4 and C6 were set, minimal requirements with which NCS did not comply. For example, one of the two programs NCS submitted in response to Criteria C4 and C6 was the National Assessment of Educational Progress program ("NAEP"). NAEP, however, is not a "statewide assessment program" within the meaning of that term as used in Criteria C4 and C6. Indeed, NCS admitted that NAEP is not a statewide assessment program and that, without consideration of that program, NCS’s proposal is not responsive to Criteria C4 and C6 because NCS has not have submitted the required proof of having administered two statewide assessment programs. This error cannot be cured by relying on the additional experience of NCS’s subcontractor because that experience does not show that its subcontractor produced reports within three months, and so such experience does not demonstrate compliance with Criteria C4. The Department deliberately limited the competition for the FCAT contract to firms with specified minimum levels of experience. As opined at final hearing, if the Department in the RFP had announced to potential bidders that the type of experience it asked vendors to describe were only targets, goals and guidelines, and that a failure to demonstrate target levels of experience would not be disqualifying, then the competitive environment for this procurement would have differed since only 2.06 evaluation points (out of a possible 150) separated the NCS and Harcourt scores. Dr. Heidorn conceded that multiple companies with experience in different aspects of the FCAT program -- a computer/imaging company and a firm experienced in educational testing -- might combine to perform a contract like the FCAT. Yet, that combination of firms would be discouraged from bidding because they could not demonstrate the minimum experience spelled out in the RFP. Language in the RFP, indicating the "holistic" evaluation that was to be applied, could have resulted in a different field of potential and actual bidders.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is recommended that Respondent, State of Florida, Department of Education, enter a Final Order rejecting the bids submitted by Harcourt and NCS for the administration component of the RFP. The Department should then seek new proposals. DONE AND ENTERED this 25th day of May, 1999, in Tallahassee, Leon County, Florida. DON W. DAVIS Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 25th day of May, 1999. COPIES FURNISHED: Karen D. Walker, Esquire Holland and Knight, LLP Post Office Drawer 810 Tallahassee, Florida 32302 Mark D. Colley, Esquire Holland and Knight, LLP Suite 400 2100 Pennsylvania Avenue, Northwest Washington, D.C. 20037 Charles S. Ruberg, Esquire Department of Education The Capitol, Suite 1701 Tallahassee, Florida 32399-0400 Paul R. Ezatoff, Jr., Esquire Christopher B. Lunny, Esquire Katz, Kutter, Haigler, Alderman, Bryant and Yon, P.A. 106 East College Avenue, Suite 1200 Tallahassee, Florida 32302-7741 Tom Gallagher Commissioner of Education Department of Education The Capitol, Plaza Level 08 Tallahassee, Florida 32399-0400 Michael H. Olenick, General Counsel Department of Education The Capitol, Suite 1701 Tallahassee, Florida 32399-0400
The Issue The issue for consideration in this case is whether Petitioner, Andrew Netupsky, should be granted licensure by endorsement as a professional engineer in Florida.
Findings Of Fact At all times, pertinent to the issues herein, Respondent, Florida Engineers Management Corporation (FEMC), was the state agency responsible for the licensing of professional engineers and for the regulation of the engineering profession in Florida. Petitioner, Andrew Netupsky, is a professional engineer licensed in the Province of British Columbia, Canada. He graduated from the University of British Columbia in 1971 with a Bachelor of Applied Science in Civil Engineering and was licensed as a Registered Professional Engineer (Structural) in British Columbia in 1973. He was subsequently licensed by endorsement as a Registered Professional Engineer (Structural) in Arizona in 1987. The State of Arizona accepted Petitioner's scholastic examinations as a valid basis for licensure. The State of Florida does not. Petitioner's engineering education in Canada was accredited by the Canadian Counsel of Professional Engineers (CCPE). The accrediting body within the United States is the Engineering Accreditation Commission of the Accreditation Board for Engineering and Technology (EAC/ABET). By virtue of a mutual agreement between the accrediting bodies in the United States and Canada, Petitioner's Canadian schooling and accreditation is deemed equivalent to an EAC/ABET accredited engineering program. Accreditation and schooling are not at issue in this proceeding. Sometime before November 15, 1999, Petitioner applied to the Florida Board of Professional Engineers (Board) for licensure by endorsement as a professional engineer in Florida. By letter dated November 22, 1999, the Board notified Petitioner that his application had been deemed administratively complete and that it would be presented for review to the Board's Application Review Committee (Committee) on December 7, 1999. Though the educational programs in Canada and Florida are similar, the methods and requirements for licensure are not. In Canada, a successful graduate of an engineering program does not take a subsequent comprehensive examination as a condition of licensure. Canadian engineering graduates are permitted to practice upon graduation from an engineering course of study. After the graduate has obtained four years of acceptable engineering experience, he or she takes a Principles and Practices examination which is, in effect, an examination on the laws and rules of practice. The subject matter examinations taken by the candidate while in engineering school serve as the required technical examination. In Florida, candidates for licensure as a Professional Engineer take two examinations in addition to the course material examinations given in engineering school. One of these is the Fundamentals examination, also known as the Engineer In Training/EIT examination. It is an examination of the candidate's broad knowledge of mathematics, general sciences, and engineering sciences. This examination is generally taken shortly before or upon graduation from a baccalaureate program in engineering school. Upon successful practice of that examination, and upon four years of acceptable engineering experience, the candidate may sit for the Principles and Practice examination, also known as the Professional Engineer or PE examination. This latter examination tests the candidate's knowledge in a specific discipline. Both the Fundamentals examination and the Principles and Practice examination are prepared and provided to the states by the National Council of Examiners for Engineers and Surveyors (NCEES). Petitioner took the examinations required for each of the mathematics, general science, and engineering courses required by his course of study as an engineering student in Canada, a total of 25 separate course examinations. In 1973 he also took the Professional Practice examination administered by the Association of Professional Engineers and Geo-scientists of British Columbia (APEGBC). At no time did he take either examination prepared by NCEES. Petitioner contends the course examinations he took in engineering school and the Professional Practice examination he took after graduation are equivalent to the NCEES examination process administered to candidates for licensing as professional engineers in Florida, and that he should be licensed by endorsement as a professional engineer in this state. In support of Petitioner's contention, he presented the testimony of Dr. Leslie Tremaine Russell, of Halifax, Nova Scotia, a mechanical engineer in practice since 1955, and past chairman of the Canadian Council of Professional Engineers (CCPE) and the Canadian Engineering Accreditation Board (CEAB). Dr. Russell has worked as a design engineer and has served as a university professor of engineering, being designated as Professor Emeritus for the past nine years. He has reviewed the engineering education programs in all Canadian provinces except Nova Scotia, Quebec, and Saskatchewan, and works in harmony with the ABET, the American equivalent to the CEAB to ensure that the engineering education programs in the two countries are as similar as is possible. As in the United States, the CCPE gains all its powers from the various provincial licensing bodies. All provinces require graduation from an accredited engineering program, and accreditation of the programs is done at the request of the university presenting the program. All students who seek to graduate from an accredited engineering program in Canada must meet all requirements of the accrediting board. Before a candidate may be licensed as an engineer in Canada, he or she must both be a graduate of an accredited engineering program and have four years' experience in the practice of engineering. The CEAB monitors engineering programs at all Canadian engineering schools to ensure that the programs are adequate to guarantee uniformity and safety. If so, the graduates are not required to take other comprehensive examinations. The engineering department at the University of British Columbia has been accredited since 1965. While the Canadian and United States programs are similar, Dr. Russell concedes that similar is not identical. An international agreement forged under the North American Free Trade Act in 1995, and applying to engineering education and licensing in Canada, the United States, and Mexico, calls for reciprocal accreditation of individuals with a four- year degree from an accredited institution and eight years of satisfactory experience. This agreement was signed by NCEES, ABET, and NSPE in the United States, but it requires ratification by the individual licensing bodies in order to become effective. The agreement has been ratified by all provinces in Canada, but only by Texas in the United States. Dr. Russell, in his many years of experience, has taken both eight-hour examinations, such as the NCEES examinations, and shorter ones as well. He is not convinced that the NCEES examinations, being made up of true/false and multiple choice questions, are particularly reflective of a candidate's actual knowledge. They are somewhat superficial. On the other hand, a university examination requires a full issue analysis. Dr. Russell is comfortable with the conclusions that the Canadian university subject examinations, which Petitioner took while in engineering school, are the equivalent of the NCEES examination on the same subject matter. Dr. Melvin W. Anderson, currently Dean of the College of Engineering at the University of South Florida, also reviewed the examinations presented by Petitioner and agreed that many had questions that were typical of the questions asked on final examinations in university engineering courses. However, some of those courses claimed by Petitioner to be fundamentals of engineering courses, such as those in English, Russian, Surveying, and Biology, have little to do with engineering science and are not subjects covered by the NCEES Fundamentals examination. Dr. Anderson also opined that the university examinations taken by Petitioner cannot be considered equal to the NCEES examinations. Further, according to Dr. Anderson, the NCEES examinations are developed by a panel of experts and administered nation-wide by state licensing bodies. The Fundamentals portion of the examination is taken by university seniors or graduates and deal with engineering and fundamental science and mathematics. The Principles and Practice examination is administered after four years of practice and deals with problem solving. Scores on the NCEES examinations are based on national assessment by practicing engineers who establish a cut score which is based on the assessed difficulty of the examination. The cut score is then equated to a passing score. When the examination is taken, it is "blind graded" which avoids any potential for the grader to be influenced by knowledge of the examinee. While the quality of engineering education in Canada has been shown to be the equivalent of that in the United States, and while the licensing process in Canada has been shown to be comprehensive and effective, the differences in the systems, in particularity the requirement for a comprehensive examination as required in the United States, render the systems different and not equivalent.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is recommended that the Board of Professional Engineers enter a Final Order denying Petitioner's application for licensure by endorsement as a professional engineer in Florida. DONE AND ENTERED this 4th day of October, 2000, in Tallahassee, Leon County, Florida. ARNOLD H. POLLOCK Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6947 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 4th day of October, 2000. COPIES FURNISHED: Andrew Netupsky 1275 West 75th Avenue Vancouver, British Columbia Canada V6P 3G4 Edwin A. Bayo, Esquire Office of the Attorney General Department of Legal Affairs The Capitol, Plaza Level-01 Tallahassee, Florida 32399-1050 Dennis Barton, Executive Director Board of Professional Engineers Florida Engineers Management Corporation 1208 Hays Street Tallahassee, Florida 32301 Natalie A. Lowe, Esquire Florida Engineers Management Corporation 1208 Hays Street Tallahassee, Florida 32802-1900 Barbara D. Auger, General Counsel Department of Business and Professional Regulation Northwood Centre 1940 North Monroe Street Tallahassee, Florida 32399-0792
The Issue At issue is whether respondent committed the offense alleged in the administrative complaint and, if so, what disciplinary action should be taken.
Findings Of Fact Petitioner, Department of Business and Professional Regulation, Division of Real Estate (Department), is a state government licensing and regulatory agency charged with the duty and responsibility to prosecute administrative complaints pursuant to the laws of the State of Florida, in particular Section 20.165, Florida Statutes, Chapters 120, 455, and 475, Florida Statutes, and the rules promulgated pursuant thereto. Respondent, Anita Ruth Green, is now and has been since November 17, 1986, a licensed real estate salesperson in the State of Florida, having been issued license number 0484830. On or about October 7, 1994, respondent signed and submitted to the Department an application (“renewal notice”) to renew her real estate license. Such application contained the following affirmation: AFFIRMATION OF ELIGIBILITY FOR LICENSE RENEWAL I hereby affirm that I have met all of the requirements for license renewal set forth by the Department of Business and Professional Regulation and/or the professional regulatory board indicated on the reverse side of this notice. I understand that, within the upcoming licensure period, if my license number is selected for audit by the Department and/or professional regulatory board, I may be required to submit proof that I have met all applicable license renewal requirements. I understand that proof may be required by the Department of Business and Professional Regulation and/or professional regulatory board at any time and that it is my responsibility to maintain all documentation supporting my affirmation of eligibility for license renewal. I further understand that failure to comply with such requirements is in violation of the rules and statutes governing my profession and subjects me to possible disciplinary action and, further, that any false statements herein is in violation of section 455.227, Florida Statutes, subjecting me to disciplinary action as well as those penalties provided below. I affirm that those statements are true and correct and recognize that providing false information may result in disciplinary action on my license and/or criminal prosecution as provided in section 455.2275, Florida Statutes. Respondent’s application was approved and her license was renewed for the October 1, 1994, to September 30, 1996, licensure period. At the time respondent submitted her application, she knew that successful completion of 14 hours of continuing education for real estate salespersons was a requirement for license renewal. Respondent was likewise aware of her responsibility to retain proof of her compliance with the continuing education requirements for at least 2 years following the end of the renewal period for which the education was claimed. By letter of June 6, 1995, the Department advised respondent that her license number had been selected to audit for compliance with continuing education requirements at the time of her most recent license renewal, and requested that she furnish proof of compliance within 21 days. Such proof was not forthcoming, and by letter of July 24, 1995, the Department reminded respondent of such failing and accorded her 10 days to submit evidence of the satisfactory completion of 14 hours of continuing education necessary to support the renewal of her license for the October 1, 1994, to September 30, 1996, licensure period. To date, no documentation has been submitted to substantiate respondent’s compliance with such requirement. At hearing, respondent acknowledged the lack of any documentation to support her affirmation of having met the continuing education requirement for license renewal; however, she averred she did successfully complete 14 hours of continuing education through a correspondence course in or about September 1994, but apparently inadvertently disposed of her certificate of completion when disposing of unneeded paperwork in preparation for a move. She could not, however, recall the name of the real estate school, the course she completed, the date of completion (although she did observe it was “probably” within 30 days of the renewal date) or her score (although she did observe it was “not less than 79 percent”). Moreover, although ostensibly paying for such course by check, respondent failed to produce any such documentation at hearing. In explanation for such failing, respondent stated she did not retain checks or other banking documents “that far back,” and decided not to secure a copy from her bank because it was “too expensive.” The actual cost or expense for that service was not, however, articulated at hearing. Having considered the proof, it must be concluded that respondent’s contention, absent any corroborative support, that she successfully completed the continuing education requirement before renewal is not compelling. In so concluding, it is observed that when the Department first requested documentation from respondent only nine months had elapsed since she ostensibly took the course. Certainly, respondent, who was articulate and displayed above-average intelligence at hearing, would have recalled at that time, had she taken a course, the name of the real estate school, the course she completed, or her score. Moreover, after such nominal lapse of time, it is most unlikely she would have disposed of not only her checks for the relevant time period, but also other banking documents, including her check registry. In sum, respondent’s failure to produce a certificate of completion or other credible proof of having satisfied the continuing education requirement compels the conclusion that she did not satisfy her continuing education requirement prior to renewal and that her affirmation of compliance was false.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that a final order be rendered finding respondent guilty of having violated Section 475.25(l)(m), Florida Statutes, and revoking respondent’s license, subject to the opportunity to reapply for licensure five years after the date of filing of the final order in this case. DONE AND ENTERED this 6th day of January, 1997, in Tallahassee, Leon County, Florida. WILLIAM J. KENDRICK Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32301-3060 (904) 488-9675 SUNCOM 278-9675 Filed with the Clerk of the Division of Administrative Hearings this 6th day of January, 1997. COPIES FURNISHED: Christine M. Ryall, Esquire Division of Real Estate Post Office Box 1900 Orlando, Florida 32802-1900 Ms. Anita Ruth Green Pace 321 Sandtree Drive Palm Beach Gardens, Florida 33410 Henry M. Solares, Director Division of Real Estate Post Office Box 1900 Orlando, Florida 32802-1900 Lynda L. Goodgame, General Counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792
The Issue Whether Petitioner meets the requirements of Section 471.015(3), Florida Statutes (2007), for licensure as a professional engineer by endorsement.
Findings Of Fact Petitioner attended Old Dominion University, Norfolk, Virginia, from fall of 1983, to May 16, 1987. He graduated May 16, 1987, with a Bachelor of Science Degree in Engineering Technology, with a major in Mechanical Engineering Technology. There is no affirmative evidence that Old Dominion University's curriculum demonstrates a deficient level of competence necessary to practice engineering in the State of Florida in the capacity of a Professional Engineer to protect public health and safety. There is no affirmative evidence of conditions unique to the State of Florida that warrant a level of competence beyond that demonstrated by Petitioner's Degree in Engineering Technology. Petitioner passed the National Council of Examiners for Engineering and Surveying Examination Part I (NCEES) Fundamentals of Engineering examination on April 11, 1987. Petitioner passed the NCEES Principles and Practices examination on October 27, 1995. Petitioner received a professional engineering license to practice in the Commonwealth (State) of Virginia on January 30, 1996. Petitioner received a professional engineering license to practice in the State of Alabama on May 30, 2003. Petitioner received a professional engineering license to practice in the State of Texas in 2005. Petitioner received a professional engineering license to practice in the State of Wisconsin in 2005. Petitioner applied for a Florida professional engineering license by endorsement on July 12, 2007. Petitioner has over four years' active engineering experience, meeting the requirements set forth in Section 471.013(1)(a), Florida Statutes. There is no evidence that the Florida Board of Professional Engineers requested supplemental information beyond that required by the Respondent's Application for Licensure by Endorsement, but Petitioner had every opportunity to present evidence in the present de novo proceeding. The Notice of Denial issued by the Florida Board of Professional Engineers on January 14, 2008, reads, in pertinent part: The Applicant does not satisfy the Education requirements of Chapter [sic] 471.015 that incorporates by reference Chapter [sic] 471.013 Florida Statutes. Your application failed to meet requirements of Section 471.013 (1) (a) F.S. Under this provision of the law, you must evidence a degree from an EAC/ABET accredited engineering program. You hold a Bachelor of Science in Engineering Technology Degree from Old Dominion University, Norfolk, Virginia. The Applicant does not have a Board approved degree.
Recommendation Based on the foregoing Findings of Facts and Conclusions of Law, it is RECOMMENDED that the Florida Board of Professional Engineers enter a final order denying Petitioner's application for licensure as a professional engineer by endorsement. DONE AND ENTERED this 28th day of July, 2008, in Tallahassee, Leon County, Florida. S ELLA JANE P. DAVIS Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 28th day of July, 2008.
The Issue The ultimate issue in this proceeding is whether proposed Florida Administrative Code Rule 61G15-21 is an invalid exercise of delegated legislative authority.
Findings Of Fact Florida Administrative Code Rule 61G15-21.004, in relevant part, states: The criteria for determining the minimum score necessary for passing the Engineering Fundamentals Examination shall be developed through the collective judgment of qualified experts appointed by NCEES to set the raw score that represents the minimum amount of knowledge necessary to pass the examination. The judges shall use a Modified Angoff Method in determining the minimally acceptable raw score necessary to pass the Fundamentals of Engineering Examination. Using the above mentioned Modified Angoff Method, the judges will indicate the probability that a minimally knowledgeable Fundamentals of Engineering examinee would answer any specific questions correctly. The probability of a correct response is then assigned to each question. Each judge will then make an estimate of the percentage of minimally knowledgeable examinees who would know the answer to each question. The totals each of the judges is added together and divided by the number of judges to determine the overall estimate of the minimum standards necessary. The minimum number of correct answers required to achieve a passing score will take into account the relative difficulty of each examination through scaling and equating each examination to the base examination. The raw score necessary to show competence shall be deemed to be a 70 on a scale of 100. A passing grade on Part Two of the examination is defined as a grade of 70 or better. The grades are determined by a group of knowledgeable professional engineers, who are familiar with engineering practice and with what is required for an applicable engineering practice and with what is required for an applicable engineering task. These professional engineers will establish a minimum passing score on each individual test item (i.e., examination problem). An Item Specific Scoring Plan (ISSP) will be prepared for each examination item based upon the NCEES standard scoring plan outline form. An ISSP will be developed by persons who are familiar with each discipline including the item author, the item scorer, and other NCEES experts. On a scale of 0-10, six (6) will be a minimum passing standard and scores between six (6) and ten (10) will be considered to be passing scores for each examination item. A score of five (5) or lower will be considered an unsatisfactory score for that item and examinee will be considered to have failed that item. To pass, an examinee must average six (6) or greater on his/her choice of eight (8) exam items, that is, the raw score must be forty- eight (48) or greater based on a scale of eighty (80). This raw score is then converted to a base 100 on which, as is noted above, a passing grade will be seventy (70). The proposed changes to Florida Administrative Code Rule 61G15-21.004, in relevant part, state: The passing grade for the Engineering Fundamentals Examination is 70 or better. The criteria for determining the minimum score necessary for passing the Engineering Fundamentals Examination shall be developed through the collective judgment of qualified experts appointed by NCEES to set the raw score that represents the minimum amount of knowledge necessary to pass the examination. The judges shall use a Modified Angoff Method in determining the minimally acceptable raw score necessary to pass the Fundamentals of Engineering Examination. Using the above mentioned Modified Angoff Method, the judges will indicate the probability that a minimally knowledgeable Fundamentals of Engineering examinee would answer any specific questions correctly. The probability of a correct response is then assigned to each question. Each judge will then make an estimate of the percentage of minimally knowledgeable examinees who would know the answer to each question. The totals each of the judges is added together and divided by the number of judges to determine the overall estimate of the minimum standards necessary. The minimum number of correct answers required to achieve a passing score will take into account the relative difficulty of each examination through scaling and equating each examination to the base examination. The raw score necessary to show competence shall be deemed to be a 70 on a scale of 100. The passing grade for the Principles and Practice Examination is 70 or better. A passing grade on Part Two of the examination is defined as a grade of 70 or better. The grades are determined by a group of knowledgeable professional engineers, who are familiar with engineering practice and with what is required for an applicable engineering practice and with what is required for an applicable engineering task. These professional engineers will establish a minimum passing score on each individual test item (i.e., examination problem). An Item Specific Scoring Plan (ISSP) will be prepared for each examination item based upon the NCEES standard scoring plan outline form. An ISSP will be developed by persons who are familiar with each discipline including the item author, the item scorer, and other NCEES experts. On a scale of 0-10, six (6) will be a minimum passing standard and scores between six (6) and ten (10) will be considered to be passing scores for each examination item. A score of five (5) or lower will be considered an unsatisfactory score for that item and examinee will be considered to have failed that item. To pass, an examinee must average six (6) or greater on his/her choice of eight (8) exam items, that is, the raw score must be forty- eight (48) or greater based on a scale of eighty (80). This raw score is then converted to a base 100 on which, as is noted above, a passing grade will be seventy (70). Petitioner resides in Tampa, Florida. On April 11, 2003, Petitioner took a national examination that Petitioner must pass to be licensed by the state as a professional engineer. On July 1, 2003, Petitioner received a letter from the Board advising Petitioner that he had received a failing grade on the examination. On July 2, 2003, Petitioner unsuccessfully requested the raw scores on his examination from a representative of the National Council of Examiners for Engineering and Surveying (NCEES). The NCEES is the national testing entity that conducts examinations and determines scores for the professional engineer examination required by the state. On July 9, 2003, Petitioner submitted a formal request to the Board for all of the raw scores related to Petitioner "and all past P.E. Exams that the Petitioner had taken." A representative of the Board denied Petitioner's request explaining that the raw scores are kept by the NCEES and "it is not their policy to release them." The Board's representative stated that the Board was in the process of adopting new rules "that were in-line with the policies of the NCEES." On July 31, 2003, Petitioner requested the Board to provide Petitioner with any statute or rule that authorized the Board to deny Petitioner's request for raw scores pursuant to Section 119.07(1)(a), Florida Statutes (2003). On the same day, counsel for the Board explained to Petitioner that the Board is not denying the request. The Board is unable to comply with the request because the Board does not have physical possession of the raw scores. Petitioner and counsel for Respondent engaged in subsequent discussions that are not material to this proceeding. On August 6, 2003, Petitioner requested counsel for Respondent to provide Petitioner with copies of the proposed rule changes that the Board intended to consider on August 8, 2003. On August 27, 2003, Petitioner filed a petition with the Board challenging existing Florida Administrative Code Rule 61G15-21.004. The petition alleged that parts of the existing rule are invalid. Petitioner did not file a challenge to the existing rule with DOAH. The Petition for Hearing states that Petitioner is filing the Petition for Hearing pursuant to Subsections 120.56(1) and (3)(b), Florida Statutes (2003). However, the statement of how Petitioner's substantial interests are affected is limited to the proposed changes to the existing rule. During the hearing conducted on January 29, 2004, Petitioner explained that he does not assert that the existing rule is invalid. Rather, Petitioner argues that the Board deviates from the existing rule by not providing examinees with copies of their raw scores and by failing to use raw scores in the determination of whether an applicant achieved a passing grade on the exam. Petitioner further argues that the existing rule benefits Petitioner by purportedly requiring the Board to use raw scores in the determination of passing grades. The elimination of that requirement in the proposed rule arguably will adversely affect Petitioner's substantial interests. The Petition for Hearing requests several forms of relief. The Petition for Hearing seeks an order granting Petitioner access to raw scores, a determination that Petitioner has met the minimum standards required under the existing rule, and an order that the Board grant a license to Petitioner. The Petition for Hearing does not request an order determining that the proposed rule changes constitute an invalid exercise of delegated legislative authority.
The Issue Whether Petitioner is entitled to credit for his answers to questions 42 and 81 of the morning session of the Fundamentals of Engineering Examination portion of the engineering licensure examination given on April 15, 2000.
Findings Of Fact Worku took the Fundamentals of Engineering Examination portion of the examination for licensure to practice as an engineer intern on April 15, 2000. The examination is a national multiple-choice examination developed and administered by the National Council of Examiners for Engineers and Surveyors (NCEES). The examination is divided into a morning session and an afternoon session. The questions in the morning session are worth one raw point each. The questions in the afternoon session are worth two raw points each. Worku challenged questions 42 and 81, which were on the morning session of the examination. Worku received 56 raw points for the morning session and 52 raw points for the afternoon session for a total raw score of 108 on the examination. Based on the NCEES' Score Conversion Table, a raw score of 108 converts to a score of 69. A converted score of 70, which equates to a raw score of 109-113, is a passing score. Question 81 asked the examinee to identify the geometric shape that was given by an equation provided in the question. Each examinee was given a reference manual during the examination. The manual contains general formulas for the types of geometric shapes listed as possible answers to question 81. The equation given in question 81 was for a specific shape and was not listed among the general formulas in the reference manual. Worku felt that because the general equation was not used that the equation was stated incorrectly. However, the equation was stated correctly. The equation differed from the equation listed in the reference manual because it was for a special shape of the geometric figure. Worku did not answer question 81 correctly. Question 42 dealt with recrystallization as it relates to metal. The question asks the examinee to pick the answer which explains the reference to the term "recrystallization" in the question. Worku contends that there are two correct answers to question 42 and that the answer which he provided is one of the correct answers. The answer which Worku provided is not a correct answer. It refers to the process of annealing, which is the process of decreasing the toughness of a metal. Recrystallization can be a part, but is not always part of annealing. Recrystallization and annealing are not synonymous terms; thus Worku is not entitled to credit for question 42.
Recommendation Based on the foregoing Findings of Fact and Conclusions of Law, it is RECOMMENDED that a Final Order be entered finding that Amanuel Worku failed the Engineering Fundamentals Examination with a score of 69. DONE AND ENTERED this 5th day of January, 2001, in Tallahassee, Leon County, Florida. ___________________________________ Susan B. Kirkland Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 5th day of January, 2001. COPIES FURNISHED: Amanuel Worku 18492 Northwest 52nd Path Miami, Florida 33055 Douglas Sunshine, Esquire Florida Engineers Management Corporation 1208 Hays Street Tallahassee, Florida 32301 Natalie A. Lowe, Executive Director Board of Professional Engineers 1208 Hays Street Tallahassee, Florida 32301 Barbara D. Auger, General counsel Department of Business and Professional Regulation 1940 North Monroe Street Tallahassee, Florida 32399-0792
Findings Of Fact Petitioner Samuel J. Pomeranz holds a "Rank 2" certificate issued by the Florida Department of Education. Petitioner Samuel J. Pomeranz obtained an advanced certificate in Educational Administration and Supervision in June 1970, from City College of New York. He obtained a Bachelor of Arts Degree in 1957 and a Master of Arts Degree in Education in 1959. Petitioner was licensed as a teacher in the State of New York and served as head of Curriculum Development in a senior high school in New York, New York. At the time of the hearing, he had not taught school in the State of Florida. Petitioner applied for a "Rank 1A" teaching certificate from the Respondent Department of Education Certification Section, but certification as "Rank 1A" was denied. Florida Administrative Code Rule 6A-4.049(1)(b) 1. requires that an applicant hold a "sixth year postmaster's level degree." Applicant received a certificate rather than a degree at the conclusion of his postmaster's work.
Recommendation Affirm the Respondent's action in denying Petitioner's request for "Rank 1A" certificate. DONE and ORDERED this 22nd day of October, 1976 in Tallahassee, Florida. COPIES FURNISHED: William L. Boyd, Esquire Post Office Box 5617 Tallahassee, Florida 32303 Gene T. Sellers, Esquire State Board of Education Knott Building Tallahassee, Florida 32304 DELPHENE C. STRICKLAND Hearing Officer Division of Administrative Hearings Room 530 Carlton Building Tallahassee, Florida 32304 (904) 488-9675
The Issue The issue in this case is whether the Petitioner's academic record meets the academic requirements that are prerequisites to taking the Fundamentals of Engineering examination.
Findings Of Fact The Petitioner submitted an application for approval to take the Fundamentals of Engineering examination. The Petitioner studied engineering at the Tongji University in the People's Republic of China from 1991 to 1995. He majored in Building Engineering and was awarded the degree of Bachelor of Engineering on July 10, 1995. Beginning in September of 1995, the Petitioner studied engineering at the graduate level at Tongji University. His graduate studies lasted until April of 1998, at which time he was awarded the degree of Master of Engineering with a major in Structural Engineering. During the course of his graduate studies at Tongji University from September of 1995 until April of 1998, the Petitioner completed a total of 38 semester credit hours. Those semester credit hours included the following courses with their indicated semester credit hours: Applied Statistics 2 credit hours Numerical Analysis 3 credit hours The courses titled Applied Statistics and Numerical Analysis are both higher mathematics courses. In the fall of 2000, the Petitioner began further graduate studies in engineering at Auburn University. He studied at Auburn University through the spring of 2002. The courses taken by the Petitioner at Auburn University included the following, with the indicated number of semester credit hours: Advanced Structural Analysis 3 credit hours Advanced Stress Analysis 3 credit hours Structural Dynamics I 3 credit hours Finite Element Methods in Structural Mechanics 3 credit hours The course titled Finite Element Methods in Structural Mechanics is a higher mathematics class. The other three Auburn courses listed immediately above, if not pure mathematics courses, are certainly courses which involve the application of advanced principles of mathematics. To successfully complete such courses, a person would have to be well-grounded in higher mathematics. In the fall of 2002, the Petitioner transferred to the University of Florida where he continued his graduate studies in engineering. On December 20, 2003, the University of Florida awarded the Petitioner the degree of Master of Engineering with a major in Civil Engineering. Pursuant to Florida Administrative Code Rule 61G15- 20.007 the Petitioner submitted his educational credentials to an educational evaluator approved by the Board. The evaluator selected by the Petitioner was Josef Silny & Associates, Inc. (Silny). Following its evaluation of the Petitioner's educational credentials, Silny prepared a Report of Evaluation of Educational Credentials (Silny Report) dated June 15, 2005. The Silny Report reached the conclusion that the Petitioner's undergraduate education at Tongji University was not the equivalent of a degree in engineering earned from a program approved by ABET. Silny was of the view that the Petitioner's undergraduate course of study at Tongji University was not equivalent because his curriculum was deficient five semester credit hours in higher mathematics and basic sciences and was deficient one semester credit hour in humanities and social sciences.1 The conclusions reached in the Silny Report were based on an evaluation of the Petitioner's undergraduate course work at Tongji University from 1991 to 1995. The Silny Report did not take into consideration any of the courses taken by the Petitioner during his graduate studies at Tongji University from 1995 to 1998, during his graduate studies at Auburn University from 2000 to 2002, or during his graduate studies at the University of Florida from 2002 to 2003. During his undergraduate engineering studies at Tongji University, the Petitioner completed 36 semester hour credits of course work in the areas of higher mathematics and basic sciences. Silny is of the opinion that semester credit hours completed at Tongji University represent less study than semester credit hours completed at an accredited engineering school in a university in the United States of America. Specifically, Silny is of the opinion that semester credit hours completed at Tongji University are the equivalent of only 75 percent of semester credit hours earned in accredited engineering programs in the United States of America. Accordingly, when Silny evaluated the Petitioner's undergraduate education credentials, Silny multiplied the 36 semester credit hours the Petitioner had completed at Tongji University in the areas of higher mathematics and basic sciences by a factor of 0.75, and concluded that those 36 semester credit hours were equivalent to only 27 semester credit hours at an accredited engineering program in the United States of America.2 Florida Administrative Code Rule 61G15-20.007 includes the following requirements regarding applicants with degrees from foreign institutions: Applicants having degrees from foreign institutions shall be required to document “substantial equivalency” to the 2002 ABET Accreditation Yearbook for Accreditation Cycle Ended September 30, 2002 engineering criteria. This document is hereby incorporated by reference. In order to document “substantial equivalency” to an ABET accredited engineering program, the applicant must demonstrate: 32 college credit hours of higher mathematics and basic sciences. The hours of mathematics must be beyond algebra and trigonometry and must emphasize mathematical concepts and principles rather than computation. Courses in probability and statistics, differential calculus, integral calculus, and differential equations are required. Additional courses may include linear algebra, numerical analysis, and advanced calculus. As for the hours in basic sciences, courses in general chemistry and calculus-based general physics are required, with at least a two semester (or equivalent) sequence of study in either area. Additional basic sciences courses may include life sciences (biology), earth sciences (geology), and advanced chemistry or physics. Computer skills and/or programming courses cannot be used to satisfy mathematics or basic science requirements. 16 college credit hours in humanities and social sciences. Examples of traditional courses in this area are philosophy, religion, history, literature, fine arts, sociology, psychology, political science, anthropology, economics, and no more than 6 credit hours of languages other than English or other than the applicant’s native language. Courses in technology and human affairs, history of technology, professional ethics and social responsibility are also acceptable. Courses such as accounting, industrial management, finance, personnel administration, engineering economics and military training are not acceptable. Courses which instill cultural values are acceptable, while routine exercises of personal craft are not. 48 college credit hours of engineering science and engineering design. Courses in this area have their roots in mathematics and basic sciences but carry knowledge further toward creative application. Examples of traditional engineering science courses are mechanics, thermodynamics, electrical and electronic circuits, materials science, transport phenomena, and computer science (other than computer programming skills). Courses in engineering design stress the establishment of objectives and criteria, synthesis, analysis, construction, testing, and evaluation. In order to promote breadth, at least one engineering course outside the major disciplinary area is required. In addition, evidence of attainment of appropriate laboratory experience, competency in English, and understanding of the ethical, social, economic and safety considerations of engineering practice must be presented. As for competency in English, transcripts of course work completed, course content syllabi, testimonials from employers, college level advanced placement tests, Test of English as a Foreign Language (TOEFL) scores of at least 550 in the paper- based version, or 213 in the computer-based version, will be accepted as satisfactory evidence.
Recommendation On the basis of the foregoing findings of fact and conclusions of law, it is RECOMMENDED that a Final Order be issued concluding that the Petitioner has met the requirements of Florida Administrative Code Rule 61G15-20.007, and is eligible to take the Fundamentals of Engineering examination. DONE AND ENTERED this 29th day of March, 2006, in Tallahassee, Leon County, Florida. S MICHAEL M. PARRISH Administrative Law Judge Division of Administrative Hearings The DeSoto Building 1230 Apalachee Parkway Tallahassee, Florida 32399-3060 (850) 488-9675 SUNCOM 278-9675 Fax Filing (850) 921-6847 www.doah.state.fl.us Filed with the Clerk of the Division of Administrative Hearings this 29th day of March, 2006.
Findings Of Fact Petitioner is a licensed architect in the State of Michigan. He began working in this field in 1964 as a designer-draftsman. He later served as a job captain, supervising draftsmen and designers. He began practicing architecture as a principal in February, 1974, and has been active as an architect since that time. Petitioner began his education in architecture at the University of Oklahoma in 1960, but did not obtain an architectural degree. He began graduate studies at the University of Detroit in 1974 and was awarded a Masters of Architecture Degree in December, 1975. Petitioner was originally registered as an architect in Michigan on February 2, 1975. He obtained his registration without an architectural degree on the basis of his training, experience and national examination results.
Recommendation Based on the foregoing, it is RECOMMENDED: That Respondent enter a Final Order affirming its denial of Petitioner's application for licensure by endorsement. DONE and ENTERED this 28th day of June, 1983, in Tallahassee, Florida. R. T. CARPENTER, Hearing Officer Division of Administrative Hearings The Oakland Building 2009 Apalachee Parkway Tallahassee, Florida 32301 (904) 488-9675 Filed with the Clerk of the Division of Administrative Hearings this 28th day of June, 1983.
