1.  INTRODUCTION

This is a numerical analysis report for commodity platforms when applied to the Air Quality Model (AQM) AERMOD on a Microsoft Windows™ operating system. New results are presented for AERMOD in two version: the executable model released by the U.S. EPA (hereafter AERMOD-EPA) and the High Performance Computing (HPC) version developed by HiCLAS1 (AERMOD-HPCS) released as Build 2 of v1.8 (hereafter v1.8.2) with the previous release designated as Build 1 (hereafter v1.8.1). Both versions are designed to execute the U.S. EPA's regulatory AERMOD model on a single processor CPU (or core) - no parallel version is studied in this report. The purpose of this report is to explore the numerical differences between the two AERMOD-HPCS builds and the U.S. EPA's executable AERMOD.EXE as well as a local compilation of the U.S. EPA's source code. This report is for the commodity platforms with 32-bit Windows™ listed in Table 1 of the previous report (HPC-2009-1).  Subsequent reports present results of AERMOD-HPCS on Linux™ operating systems for both 32-bit and 64-bit platforms (HPC-2009-3, HPC-2009-4). The purpose of this report is to display numerical differences observed for these versions of AERMOD on commodity architectures to address the requirement of a Model Equivalence Demonstration (MED).

2.0 CHOICE OF HARDWARE AND OPERATING SYSTEM

The hardware used for the results reported here includes three Intel® Pentium 4 Xeon™ (Intel) and two Advanced Micro Devices (AMD) processors. The previous report identifies the platforms used in this analysis together with their attributes and for further information visit the Web addresses given  in the References Section. In this survey older and newer generations of CPUs are included: from laptops to quad core servers. The goal was to survey a variety of processors in common use for AERMOD simulations and make an assessment of their suitability. All platforms discussed in this report used a 32-bit Microsoft Windows™ operating system on these architectures. As a consequence of the scope of this study, extensive tabulations of results are not reproduced here but are available as downloadable PDF files. The following tables show only global information or totals to facilitate the comparisons made in the following discussion.

3.0 CHOICE OF COMPILERS

The compiler used for AERMOD-EPA executable distributed by the U.S. EPA is not known, but is assumed to the the Compaq Visual Fortran compiler (CVF). The executable distributed by the U.S. EPA was obtained from the distribution center at http://www.epa.gov/scram001 and applied in all the results designated here as AERMOD-EPA. Other results designated here as AERMOD-HPCS were obtained from a compilation of AERMOD-HPCS source code that was modified away from the U.S. EPA source distribution available at the above named U.S. EPA SCRAM Web portal. The compiler used for AERMOD-HPCS in this analysis (and distribution) is un-named but has been chosen after testing of the most popular compilers currently available. Considerable effort has been invested in exhaustively testing multiple compiler options to enable the best performance consistent with the code structure changes employed at HiCLAS1.

4.0 CHOICE OF BENCHMARKS

The AERMOD model describes pollutant dispersion and deposition and is now an approved regulatory model for new source reviews and other permitting applications. It is available in the AERMOD-EPA version at the U.S. EPA’s Support Center for Regulatory Air Models at the URL portal named above. The version used here is AERMOD 07026 and to create the High Performance Computing (HPC) version the source code for the U.S. EPA distribution was progressively modified to enhance performance. The resulting code is designated AERMOD-HPCS, and at v1.8 (the current release) it was deemed to be a sufficient improvement over AERMOD-EPA to warrant exhaustive Quality Assurance (QA) testing. For QA testing the four Cases listed in Table 2 of report HPC-2001-1 were used as benchmarks. These benchmarks are considered to be representative of actual applications for AERMOD and input and output files for Case 2 are included in the distribution for the purpose of testing the installation after download of the AERMOD-HPCS executable model.

5.0 BENCHMARK RESULTS

5.1 Definition of nomenclature

Individual concentration results produced by AERMOD-HPCS in both Build 1 (previous release) and Build 2 (current release) were compared against concentration results from AERMOD-EPA (the U.S. EPA executable model) and a local compilation of the unmodified U.S. EPA source code. For each platform identified in the previous report (HPC-2009-1) there is a four-way comparison grouped as A, B, C, and D and Table 1 summarizes the notation used here for these four tabular groups on the five Windows™ platforms.

Table 1: Index to tables for Quality Assurance (QA) and MED tests of AERMOD-HPCS on Windows™ platforms
Table number	Platforms as node numbers	Platform Results compared
A-#	# = 10 Intel Pentium 4 Xeon # = 13 Intel Pentium 4 Xeon # = 16 AMD Turion # = D2 AMD 64x2 # = 17 Intel quad core	AERMOD-HPCv1.8.1 versus U.S. EPA's source code for release 07026
B-#		AERMOD-HPCv1.8.2 versus U.S. EPA's source code for release 07026
C-#		AERMOD-HPCv1.8.1 versus U.S. EPA's AERMOD.EXE for release 07026
D-#		AERMOD-HPCv1.8.2 versus U.S. EPA's AERMOD.EXE for release 07026

Each group A-D compared 14,366 individual concentration values for all four cases used in the benchmarks and tabulated absolute and relative errors. When the absolute error of the comparison exceeded a tolerance of 2.0e-05 a counter was incremented. This choice of tolerance is dictated by the use of single precision and the precision used in numerous constants throughout the AERMOD source code (as distributed by the U.S. EPA). A higher tolerance criterion is not warranted in our judgement. In each group a global maximum absolute error and corresponding relative error was tabulated separately and these results are summarized below. Full details may be found in the 20 Tables for all groups and platforms as a downloadable PDF file. These tables use the nomenclature defined in Table 1 above .

5.2 Global results

For each platform and group Table 1a and 1b below summarize global results of the individual comparisons. Table 1a is for groups A and B corresponding, respectively, to the comparison of the two AERMOD-HPCS builds against the local compilation of the U.S. EPA source code. For group A (AERMOD-HPCS Build 1) all differences are negligible, and the number of occurrences is small. However, for group B (AERMOD-HPCS Build 2), all Intel platforms do show larger differences compared to group A, with maximum absolute errors of the order of 8.2e-03 compared to 3.1e-05. However, for the AMD platforms this increases by a factor of 25.6 to 2.1e-01 (groups B and D). Also, the number of times this comparison exceeds the tolerance of 2.0e-05 jumps by an order of magnitude from 215 (B-13, B-17)  to 2304 (B-16, B-D2) out of 14366 values compared.

Table 1a: Global results of HiCLAS1 MED for AERMOD-HPCS concentration values when compared against those produced by U.S. EPA source code compiled with the same compiler and similar options on Windows™ platforms.
Table	Total Number of concentration values compared	Totals and Global results				Platform Results compared
		Number. exceeding difference of 2.0E-05	% exceeding difference of 2.0E-05	Maximum absolute error	Maximum relative error (%)
A-10	14366	9	0.06	3.1E-05	2.7E-04	AERMOD-HPCv1.8.1 versus U.S. EPA's source code
A-13	14366	9	0.06	3.1E-05	2.7E-04
A-16	14366	10	0.07	3.1E-05	2.7E-04
A-D2	14366	10	0.07	3.1E-05	2.7E-04
A-17	14366	9	0.06	3.1E-05	2.7E-04
B-10	14366	218	1.52	8.2E-03	9.3E-02	AERMOD-HPCv1.8.2 versus U.S. EPA's source code
B-13	14366	215	1.50	8.2E-03	9.3E-02
B-16	14366	2304	16.04	2.1E-01	2.0E+00
B-D2	14366	2304	16.04	2.1E-01	2.0E+00
B-17	14366	215	1.50	8.2E-03	9.3E-02

Table 1b is for groups C and D corresponding, respectively, to the comparison of the two AERMOD-HPCS builds against the U.S. EPA executable model (AERMOD.EXE). Here the difference between AERMOD-HPCS Build 1, and Build 2, is submerged by the comparison with concentration values produced by AERMOD.EXE.

Table 1b: Global results of HiCLAS1 MED for AERMOD-HPCS concentration values when compared against those produced by U.S. EPA executable AERMOD.EXE model on Windows™ platforms.
Table	Total Number of concentration values compared	Totals and Global results				Platform Results compared
		Number. exceeding difference of 2.0E-05	% exceeding difference of 2.0E-05	Maximum absolute error	Maximum relative error (%)
C-10	14366	917	6.38	8.2E-03	9.3E-02	AERMOD-HPCv1.8.1 versus U.S. EPA's executable AERMOD.EXE
C-13	14366	918	6.39	8.2E-03	9.3E-02
C-16	14366	919	6.40	8.2E-03	9.3E-02
C-D2	14366	919	6.40	8.2E-03	9.3E-02
C-17	14366	918	6.39	8.2E-03	9.3E-02
D-10	14366	866	6.03	1.0E-03	2.9E-02	AERMOD-HPCv1.8.2 versus U.S. EPA's executable AERMOD.EXE
D-13	14366	865	6.02	1.0E-03	2.9E-02
D-16	14366	1692	11.78	2.1E-01	2.0E+00
D-D2	14366	1692	11.78	2.1E-01	2.0E+00
D-17	14366	865	6.02	1.0E-03	2.9E-02

5.3 Analysis of results

Two important observations emerge from the preceding results on Windows™ systems: (a) comparison of results of groups A and C indicate clearly that the U.S. EPA executable produces larger error than does the locally compiled version of the U.S. EPA source code, and (b) the results on the AMD platforms (16 and D2) show much larger differences which originate from the use of Build 2 of AERMOD-HPCS. However, in the latter case, it should be noted that the maximum relative error does not exceed the 2% limit required by the MED.

Observation (a) has been discussed (and explained) in a previous report (HPC-2007-2) and the present results confirm that the increase in numerical differences is produced by use of the U.S. EPA executable. However, the increased differences related to observation (b) originate from Build 2 of AERMOD-HPCSv1.8.  The AERMOD-HPCSv1.8 source code is identical for both Build 1 and Build 2. However, Build 2 used a newer version of the compiler with new optimization options to enhance portable performance and, as a result, a reduction in portability of precision is evident. For this reason, AERMOD-HPCSv1.8 Build 1 should be used if this is a matter of concern on Windows™ platforms using AMD processors.

6.0   CONCLUSIONS

This performance analysis of AERMOD-HPCS on Microsoft Windows™ platforms shows that it delivers a solution with small numerical differences when compared with the results for a local compilation the U.S. EPA's distribution of the AERMOD source code. However, comparison of numerical differences across different platforms indicate the while performance optimizations may give portable performance enhancements in Build 2 of AERMOD-HPCSv1.8, these may be at the price of portability of numerical precision, and therefore the earlier version (Build 1) could be used if this is of concern. The numerical results of either build of AERMOD-HPCS are still well within the accuracy tolerance set by the Model Equivalence Demonstration requirements.