1.  INTRODUCTION

This is a numerical analysis report for IA-32 commodity platforms when applied to the Air Quality Model (AQM) AERMOD. New results are presented for AERMOD in two version: the executable model released by the U.S. EPA (hereafter AERMOD-EPA) and the High Performance Computing (HPC) version developed by HiCLAS1 (AERMOD-HPCS). Both version are designed to execute the U.S. EPA's regulatory AERMOD model on a single processor CPU (or core). The purpose of this report is to display numerical differences observed for both versions of AERMOD on commodity architectures. A companion report (HPC-2007-1) continues the Quality Assurance process with a discussion of runtime performance of AERMOD-EPA and AERMOD-HPCS.

2.0 CHOICE OF HARDWARE AND OPERATING SYSTEM

The hardware used for the results reported here is the Intel Pentium 4 Xeon processor identified as Machine B in Table 1 of the previous report. This used the Microsoft Windows™ operating system for 32-bit architectures. The results found are typical of such platforms and may be considered as representative.

3.0 CHOICE OF COMPILERS

The compiler used for AERMOD-EPA executable distributed by the U.S. EPA is not known, but is assumed to the the Compaq Visual Fortran compiler (CVF). The executable distributed by the U.S. EPA was obtained from the distribution center at http://www.epa.gov/scram001 and applied in all the results designated here as AERMOD-EPA. Other results designated here as AERMOD-HPCS were obtained from a compilation of AERMOD-HPCS source code that was modified away from the U.S. EPA source distribution available at the above named U.S. EPA SCRAM Web portal. The compiler used for AERMOD-HPCS in this analysis (and distribution) is un-named but has been chosen after testing of the most popular compilers currently available. Considerable effort has been invested in exhaustively testing multiple compiler options to enable the best performance consistent with the code structure changes employed at HiCLAS1. In addition a variety of compiler switches that affect floating point precision were also tested and their effect on AERMOD-HPCS numerical results were found to be negligible compared to the other numerical differences discussed here. For the purposed of clarity, Table 1 summarizes the notation used here, the compilers, and the arithmetic precision used in the compilation.

Table 1: Definition of terms used in HiCLAS1 Quality Assurance (QA) tests of AERMOD-HPCS
Mnemonic	Meaning	Compiler used and comments
AERMOD-EPA	The AERMOD executable distributed by the U.S. EPA	Not known
AERMOD-EPA(sp)	The U.S. EPA version of AERMOD source code compiled at HiCLAS1 using the compilation script in the distribution.	CVF with floating point variables and  constants declared in single precision
AERMOD-EPA(dp)	The U.S. EPA version of AERMOD source code compiled at HiCLAS1 using the compilation script in the distribution but with double precision added.	CVF with floating point variables and  constants declared in double precision
AERMOD-HPCS	The executable distributed by HiCLAS1 as v1.7 (based on performance enhanced HPC source code for AERMOD)	Compiled with an unnamed compiler in single precision

 

4.0 CHOICE OF BENCHMARKS

The AERMOD model describes pollutant dispersion and deposition and is now an approved regulatory model for new source reviews and other permitting applications. It is available in the AERMOD-EPA version at the U.S. EPA’s Support Center for Regulatory Air Models at the URL portal named above. The version used here is AERMOD 07026 and is designated as described in Table 1. To create the High Performance Computing (HPC) version of AERMOD the source code for the U.S. EPA distribution was progressively modified to enhance performance. The resulting code is designated AERMOD-HPCS, and at v1.7 (the current release) it was deemed to be a sufficient improvement over AERMOD-EPA to warrant exhaustive Quality Assurance (QA) testing. For QA testing the four Cases listed in Table 2 were used as benchmarks. These benchmarks are considered to be representative of actual applications for AERMOD and input and output files for Case 2 are included in the distribution for the purpose of testing the installation after download of the AERMOD-HPCS executable model.

Table 2: Benchmark Cases used in HiCLAS1 Quality Assurance (QA) tests of AERMOD-HPCS
Case	Source type and number				Number of  Receptors	Met Hours
	Point	Area	Volume	Openpit
1	10	0	0	0	916	8760
2	10	5	0	0	916	2160
4	120	0	0	0	771	8760
5	963	38	0	0	916	8760

5.0 BENCHMARK RESULTS

For machine B described in the previous report and cases listed in Table 2, respectively, a varying number of PLT output files were produced, as is summarized in Table 3. Since these PLT files included the computed concentrations, the values produced were compared pair-wise, as summarized in Table 4. Both the absolute error and relative error was inspected for a total of 3519 concentration values in the results of this QA performance analysis.

Table 3: Comparisons used in HiCLAS1 Quality Assurance (QA) tests of AERMOD-HPCS
Case	Number of PLT data sets compared	Number of values in each  PLT data set
1	4	916
2	4	916
4	2	771
5	6	916

Table 4 summarizes the three comparison types that were performed for the output types listed in Table 3. The first entry, Delta-EPA(sp), was used to confirm that the local compilation with CVF (in single precision) did reproduce the numerical values produced by the U.S. EPA distribution of the the AERMOD executable. All numerical differences observed in the Delta-EPA(sp) comparison were observed to be zero, confirming the possibility that the U.S. EPA executable may have been compiled with the CVF compiler in single precision. However, significant differences were observed in the Delta-EPA(dp) QA test, and these far exceeded the differences observed in the Delta-HPCS comparison. In other words, because the U.S. EPA executable appears to have been compiled in single precision, the expected accuracy of the U.S. EPA model is limited by an error threshold that exceeds the differences observed between results of the AERMOD-HPCS v1.7 distribution and those produced by the U.S. EPA executable.

Table 4: Numerical comparison used in HiCLAS1 Quality Assurance (QA) tests of AERMOD-HPCS
Mnemonic	Comparison of numerical results	Comments
Delta-EPA(sp)	AERMOD-EPA versus AERMOD-EPA(sp)	Used to validate the local CVF compilation
Delta-EPA(dp)	AERMOD-EPA(sp) versus AERMOD-EPA(dp)	Used to check precision portability of the EPA distribution
Delta-HPCS	AERMOD-HPCS versus AERMOD-EPA	QA test of AERMOD-HPCS distribution

The following sections show graphical representations of the numerical differences observed in the QA discussed here. In comparing the absolute value of the observed numerical difference a reasonable threshold of 2.e-05 was chosen. With this choice the frequency (number of occurrences) that it is exceeded in the comparison of two standard U.S. EPA distributions is surprisingly large, as is shown in Fig. 1. This suggests that the AERMOD-EPA distribution in single precision is the leading cause of reduced accuracy, and not the AERMOD-HPCS release. Fig.2 shows the corresponding maximum in difference of concentrations between the respective model runs. The largest difference observed between AERMOD-EPA and AERMOD-HPCS is less than 0.01 whereas in the difference in the two U.S. EPA runs is (as much as) an order of magnitude larger. Inspection of the differences between AERMOD-EPA and AERMOD-EPA(dp) concentration results, for all instances of 3519 data points in the 16 data sets, showed a threshold as large as 0.36% in the relative difference in concentration values.

5.1 Frequency of numerical differences

 Fig. 1: For each of the PLT files (data sets) produced in Cases 1, 2, 4, and 5 listed in Table 3, this shows the number of times (or frequency as a percent of each sample size) that the maximum of the concentration difference:  AERMOD-EPA – AERMOD-EPA(dp) and  AERMOD-EPA - AERMOD-HPCS, respectively, exceeds the threshold of 2.0e-05. The total number of all concentration values compared for all data sets from these PLT files is 3519.

5.2 Maximum value of numerical differences

 Fig. 2: For each of the PLT files (data sets) produced in Cases 1, 2, 4, and 5 listed in Table 3, this shows the maximum of the concentration difference:  AERMOD-EPA – AERMOD-EPA(dp) and AERMOD-EPA - AERMOD-HPCS, respectively. Note the logarithmic scale. The total number of all concentration values compared for all data sets from these PLT files is 3519.

6.0   CONCLUSIONS

This performance analysis of AERMOD-HPCS on a Microsoft Windows™ platform shows that it delivers a solution with small numerical differences when compared with the results of the U.S. EPA's distribution of the AERMOD executable. These differences are deemed "small" when compared with the differences observed between the U.S. EPA distribution compiled in single and double precision.