vs. ) Case No. 99-4738
) DEPARTMENT OF BUSINESS AND ) PROFESSIONAL REGULATION, ) FLORIDA ENGINEERS MANAGEMENT ) CORPORATION, )
Respondent. )
Robert E. Meale, Administrative Law Judge of the Division of Administrative Hearings, conducted the final hearing in Tampa, Florida, on March 3, 2000.
For Respondent: William H. Hollimon
The issue is whether Petitioner is entitled to a passing grade on the Principles and Practice of Engineering portion of the engineering licensure examination administered on April 23, 1999, by the National Council of Examiners for Engineers and Surveyors.
By Examination Grade Report mailed on September 10, 1999, Respondent informed Petitioner that he scored a 69 on the Mechanical Engineer Exam. By letter dated November 1, 1999, Petitioner timely requested a formal hearing.
At the hearing, each party called one witness. Petitioner offered into evidence four exhibits, and Respondent offered into evidence 14 exhibits. All exhibits were admitted. At the request of Respondent, the Administrative Law Judge has sealed Respondent Exhibits 7-10.
The court reporter filed the Transcript on April 13, 2000.
Petitioner has a bachelor of science degree in mechanical engineering from California State University in Sacramento. He has worked as a mechanical engineer for Long and Associates in Tampa for ten years. His responsibilities include the calculation of cooling and heating loads and design of heating, ventilating, and air-conditioning (HVAC) systems. He has ten additional years' engineering experience in his home country of Iran, which he left in 1978.
Petitioner challenges the scoring of two essay questions on the Principles and Practice Engineering Examination (Exam) administered on April 23, 1999. By stipulation, the parties agree that Petitioner would pass the Exam if he receives an additional credit of one raw point.
Petitioner challenges Questions 146 and 147. Question
146 asks whether an existing rooftop heat pump, as specified, has the capacity to supply 100 percent outside air makeup to specified space, under specified usage, during the summer and winter. The question tells the applicant to ignore solar heat gain.
Question 147 asks the applicant to determine the overall heat transfer coefficient and the exit temperature of water being heated, assuming a clean shell-and-tube heat exchanger of a specified capacity and subject to specified usage.
Scoresheets for both questions describe the standards for points to be awarded to answers. The purpose of these descriptions is to normalize the scoring that is performed by different scorers. The scoresheet descriptions address points in two-point increments: each scoresheet describes the requirements for 10, 8, 6, 4, and 2 points, but not odd-numbered points.
For Question 146, an answer receiving ten points must be "[e]xceptionally competent." The description for a maximum of 10 points states:
Correct evaluation of both winter and summer conditions, with only minor math error is required. A complete and correct solution which demonstrates knowledge, skills, and abilities with no conceptual errors is sought. Correct use of equations or the psychrometric chart is imperative. A minor error in reading the chart is acceptable.
For eight points, an answer must demonstrate "more than minimum, but less than exceptional competence." The description for eight points states:
The solution is essentially correct with one of the following errors:
Incorrect application of load equations (Winter or Summer) including structural heat or cooling load, people, lights, and equipment,
Incorrect application of the fresh air load,
Incorrect conclusion of whether the unit is satisfactory for both winter or [sic] summer.
For six points, the answer must demonstrate "minimum competence," with any two of the three errors listed in the preceding paragraph. For four points, the answer must demonstrate "more than rudimentary knowledge but insufficient to demonstrate minimum competence," with all three of the errors listed in the preceding paragraph.
For two points, the answer must demonstrate "rudimentary knowledge," with a presentation of any one of the "concepts defined above," but without an attempted "solution." For zero points, the answer must demonstrate "no knowledge," with a presentation of "[n]othing significant . . . to indicate knowledge of the problem."
For Question 147, the descriptions omit the labels, such as "exceptionally competent," present in the scoresheet descriptions for Question 146. However, the descriptions for Question 147 are detailed.
For ten points, the answer must demonstrate "[c]orrect analysis." The description states that the answer "[m]ay have minor errors (sloppy chart reading or decimal error)."
For eight points, the answer may demonstrate three types of error: errors valued at 20 percent, errors valued at 10 percent, and errors with a variable value.
For errors valued at 20 percent, the description allows three sources of error: a "units error," "incorrect evaluation of LMTD [log-mean temperature difference]," or "incorrect calculation of q (use of LMTD instead of (T)fluid or vice- versa)." valued at no more than 20 percent and four types of error valued at no more than 10 percent, and one error having a variable value.
For errors valued at ten percent, the description allows four sources of error: "no heat exchanger correction factor used (or uses F = 1.0)," "incorrect use of the fouling factor," "incorrect evaluation of the NTU [Number of Transfer Units] or effectiveness," or "failure to evaluate the final temperature of the water being heated."
For errors with variable value, the description allows one source of error: "incomplete solution."
For six points, the answer may demonstrate an error valued at 40 percent, if from the following two sources: "failure to realize that fouling causes a change in LMTD" or "a
score resulting from any combination of the above listed errors adding up to 40 percent."
For four points, the answer may demonstrate any combination of the above-listed errors valued at no more than 60 percent.
For two points, the answer may demonstrate "[v]ery little progress toward a solution," or any combination of the above-listed errors valued at no more than 80 percent.
For zero points, the answer must demonstrate "[n]o significant work."
The National Council of Examiners for Engineering and Surveying (NCEES) scored all of the Exams. The NCEES scorer assigned Respondent four points for his answer to Question 146 and two points for his answer to Question 147.
Petitioner appealed the two scores, and an NCEES scorer rescored them the same. For Respondent's answer to Question 146, the NCEES scorer noted:
The examinee did not consider at all the added heat gain in the summer and the added heat loss in the winter due to the 4000 cfm ventilation load. These are egregious errors. In addition the 40,000 Btu/hr equipment load was not included, and the examinee did not neglect the solar heat gain as stated in the problem (which was done to simplify the problem). This is a total of four errors.
The examinee did not meet the criteria for minimum competence, hence only a score of 4 was earned.
For the answer to Question 147, the NCEES scorer noted:
The examinee has made the following mistakes and/or omissions.
The examinee used an incorrect equation to determine the log-mean temperature difference (LMTD). It cannot be a parallel flow heat exchanger as assumed by the examinee since the exit temperature of the cold stream would be at a higher temperature than the exit temperature of the hot stream for parallel flow. Also, see the diagram given with the exam question.
Examinee has assumed a correction factor, F, of 1.0. This is an incorrect value.
Units on calculated Q value of 10,080 are incorrect. Units would be Btu/min, not Btu/hr. Also, mass rate of flow was given as 10,000 lb/hr.
LMTD)dirty LMTD)clean. Need to use this Effectiveness-NTU method to determine Tc,o.
No indication of knowing the need for Cmin and Tmin to determine Q for dirty conditions.
Based on above analysis and the scoring plan, a score of 2 is recommended.
In asking the examinee to assess the capacity of the rooftop HVAC unit, Question 146 asks the examinee to evaluate the summer loads and the winter loads.
The description for eight points appears to list only three possible errors. However, paragraphs a) and c) each describe two possible errors because of the division of the problem into winter and summer situations. Paragraph a) explicitly identifies two errors: the incorrect application of load equations--in the winter or summer. Despite its illogical
linkage of "both" with the disjunctive, "or," paragraph c) also identifies two possible errors: the capacity of the unit for wintertime heating and the capacity of the unit for summertime cooling.
Paragraph b) is ambiguous regarding whether it identifies one or two errors. Although the fresh air load is clearly relevant to heating and cooling tabulations, paragraph b), unlike paragraphs a) and c), omits mention of the two seasons.
Militating in favor of an interpretation of paragraph
b) as identifying two errors, Question 146 clearly identifies two seasons for solution. Also, the fresh air load is almost equal to the total of the other heating loads for summertime cooling and about five times greater than the total of the other cooling loads for wintertime heating.
On the other hand, paragraphs a) and c) mention the two seasons and paragraph b) does not. Also, the importance of the fresh air load is already reflected in the fact that its omission means (at least) one error under paragraph b) and almost always means another error for the wintertime result under paragraph c).
On balance, the better interpretation is to limit paragraph b) to one error. Respondent has already received the benefit of the doubt in the interpretation that the two paragraphs, a) and c), establish four errors. Given the high likelihood, as in this case, that the omission of the fresh air
load will also cause an error under paragraph c), the interpretation that paragraph b) identifies two (and, thus, probably three) errors is unfair and unsupported by the record.
Petitioner does not dispute that he incorrectly omitted the fresh air load, which is paragraph b). Petitioner's attribution of this omission to "exam stress" is irrelevant. For the reasons already noted, this omission, although a flaw in his summertime and wintertime tabulations, constitutes only one error.
Due to the omission of the fresh air load from his wintertime calculations, Petitioner also incorrectly concluded that the HVAC unit is adequate for wintertime heating, which is paragraph c). This is Petitioner's second error.
However, Petitioner correctly concluded that the HVAC unit is adequate for summertime cooling. Respondent's contention that this conclusion is guesswork is irrelevant. The simple fact is that Paragraph c) does not justify the finding of an error as long as the conclusion is correct.
Perhaps the NCEES may wish to revise paragraph c) in future tests. However, the assertion of guesswork is relevant only to a determination whether an answer is entitled to eight points. For eight points, the description effectively precludes guesswork by requiring, in addition to no more than one of the possible five errors, an "essentially correct solution." An
"essentially correct solution" would contain sufficient information as to preclude the possibility of guesswork.
In any event, the NCEES elected not to impose the requirement of an "essentially correct solution" for the "minimum competence" required for a score of six points. The sole requirement for six points is that the examinee not make more than two of the five possible errors described in the description for eight points. For this reason, much of the testimony of Respondent's expert concerning the overall inadequacy of Petitioner's answer is irrelevant to the issue of whether Petitioner is entitled to six points.
The testimony of Respondent's expert that Petitioner's summertime conclusion is the product of guesswork is also unsupported by the record. As noted below, Petitioner's analysis of summertime loads addresses all load components, other than the fresh air load, and does so in a substantially accurate manner.
The issue concerning Question 146 therefore turns on whether Petitioner committed any error regarding paragraph a). If he committed even one error, he would have committed three of the five described errors and would be entitled only to the four points that he was awarded. If he committed no errors, he would
have committed only two of the five described errors and would be entitled to six points, which would earn him a passing grade.
Petitioner correctly accounted for the wintertime heating loads or credits attributable to the roof, walls, and
equipment. Although he ignored the impacts of the lights and people, Petitioner adequately explained his exclusion of these items, in accordance with generally accepted engineering practices. Petitioner thus did not commit an error in his answer to the wintertime application of the load equations.
The preceding statement is unaffected by the recognition that Petitioner's answer ignores the largest single factor in wintertime heating: fresh outside air. This significant omission cost Petitioner two errors: one under paragraph b) where Petitioner ignored the fresh air load and one under paragraph c) where, due to the size of the fresh air load, Petitioner erroneously concluded that the HVAC unit was sufficient for wintertime heating. Under the scoring scheme of the Exam, Petitioner did not commit a third error under paragraph
a) due to this omission of fresh outside air.
Petitioner correctly accounted for the summertime cooling loads attributable to the people, lights, equipment, roof, and walls. Respondent's expert admits that Petitioner correctly tabulated the cooling loads for the people, lights, and equipment, but contends that Petitioner incorrectly tabulated the cooling loads for the roof and walls.
Petitioner calculated excessive cooling loads for the roof and walls. His calculations were 14,377.5 Btu/hr for the roof and 2700 Btu/hr for the walls. The correct calculations were 6390 Btu/hr for the roof and 1200 Btu/hr for the walls.
However, this discrepancy is attributable to Petitioner's failure to ignore the effects of solar heat gain.
The purpose of the direction to ignore the effects of solar heat gain was to simplify the problem. In determining whether Petitioner's answer correctly applied the load equations, the issue is the application of the loading factors for the roof and walls, not that Petitioner overly complicated his answer by adjusting these factors for solar gain. Petitioner thus did not commit an error in his answer to the summertime application of the load equations.
The record does not permit a comprehensive finding as to the source of the error in the original scoring of Petitioner's answer to Question 146, but it does in the rescoring and the testimony of Respondent's expert witness. Incorrectly identifying two possible errors identified by paragraph b), the NCEES scorer found two errors in Petitioner's failure to consider the fresh air load in summertime cooling or wintertime heating. This should have been only one error.
The NCEES scorer then identifies as another error Petitioner's failure to account for the equipment cooling load. This is a mistake by the scorer because, as Respondent's expert witness admitted, Petitioner's answer includes the impact of the equipment in the summer and winter.
The NCEES scorer then finds a fourth error in Respondent's failure to ignore solar heat gain, as instructed by
Question 146. There are two problems with this assertion. First, the direction to ignore solar heat gain was to simplify the problem, so it would not appear that an examinee's disregarding of the direction should necessarily result in the
loss of credit. More importantly, the failure to disregard solar heat gain is not one of the five errors identified in the description for eight points. The NCEES scorer improperly failed to proceed to analyze Petitioner's summertime load tabulations.
Respondent's expert recognized that Petitioner's answer accounted for the equipment in both seasons. However, in finding six errors in Petitioner's answer to Question 146 (Respondent Exhibit 12), Respondent's expert makes several errors. Respondent's expert repeats the error of the NCEES scorer concerning Petitioner's failure to ignore solar heat gain in his summertime load tabulations. For the reasons already noted, Respondent's expert discredits Petitioner's correct conclusion that the HVAC unit is adequate for summer use. Respondent then improperly counts as another error Petitioner's failure to address the "new process." Respondent's expert also counts as three errors, under the categories of the "new process," the heating capacity of the heat pump, and the conclusion of the adequacy of the heat pump for winter use, the single error of the conclusion of the adequacy of the heat pump for winter use. In general, Respondent's expert, who is not a qualified NCEES
scorer, did not restrict his analysis to the scoring criteria contained in the Exam.
Question 147 asks two questions of the examinee. First, Question 147 asks the examinee to determine the overall heat transfer coefficient in a clean shell-and-tube heat exchanger, assuming no heat loss. Second, Question 147 asks the examinee to determine the heat exchange (Q) and exit temperature of the heated water after the exchanger has been in service sufficiently long to build up a specified resistance from fouling.
The scoring criteria for Question 147 are different from those for Question 146. To earn four points, Petitioner would have to show that the total number of cited errors added up to only 60 percent.
Petitioner's basic problem on Question 147 is that he incorrectly treated the heat exchanger as a parallel flow exchanger, rather than the more common counter flow exchanger, which should have been evident from the text and diagram constituting Question 147. Again, Petitioner's explanation of the stressful testing conditions is rejected as irrelevant.
Petitioner's challenge of Question 147 is groundless. His answer contained the following errors valued at 20 percent: a units error, an incorrect evaluation of LMTD, and an incorrect evaluation of q. His answer contained the following errors valued at 10 percent: the use of a heat exchanger correction
factor of F = 1.0, an incorrect evaluation of NTU or effectiveness, and a failure to evaluate the final temperature of the heated water. His answer contained the following error valued at 40 percent: a failure to realize that fouling causes a change in LMTD. These errors total 130 percent.
However, the NCEES scorer properly awarded Petitioner two points for his answer to Question 147. An answer is entitled to two points if the total value of the listed errors does not exceed 80 percent or if the answer displays "[v]ery little progress toward a solution," as opposed to the standard of "[n]o significant work," for which an answer earns zero points. Revealing more than "no significant work," Petitioner's answer achieved "very little progress toward a solution."
Awarding Petitioner two more points for his answer to Question 146 means that Petitioner has passed the Exam.
CONCLUSIONS OF LAW
The Division of Administrative Hearings has jurisdiction over the subject matter. Section 120.57(1), Florida Statutes. (All references to Sections are to Florida Statutes.)
Petitioner has the burden of proving that the scoring of the challenged questions was arbitrary, such as through a deviation from the stated scoring criteria. Harac v. Department of Professional Regulation, Board of Architecture, 484 So. 2d 1333, 1338 (Fla. 3d DCA 1986).
Petitioner has proved that the award of only four points for his answer to Question 146 was arbitrary because it deviated from the stated scoring criteria. Petitioner is entitled to six points for Question 146, and the additional two points entitle him to a passing score on the Exam.
Petitioner has failed to prove that the award of two points for his answer to Question 147 was arbitrary.
RECOMMENDED that the Florida Engineers Management Corporation enter a final order declaring that Respondent has passed the Exam.
DONE AND ENTERED this 21st day of April, 2000, in Tallahassee, Leon County, Florida.
ROBERT E. MEALE
Filed with the Clerk of the Division of Administrative Hearings this 21st day of April, 2000.
William H. Hollimon Ausley & McMullen
Barbara D. Auger, General Counsel Department of Business and
Dennis Barton, Executive Director Board of Professional Engineers Department of Business and
Natalie A. Lowe, Esquire
All parties have the right to submit written exceptions within 15 days from the date of this recommended order. Any exceptions to this recommended order must be filed with the agency that will issue the final order in this case.
