This page contains instructions for running the regression testing and validation programs for NEUGEN. These are not bundled with the package and are used primarily by developers.
To run the testing programs you will need to have the $NEUGENTEST directory defined in your area, this is where the testing programs themselves will be kept. It will need to be stocked with the necessary programs and include files. Create such a directory and do
> cp ~gallag/neugen/betatestcode/* $NEUGENTEST/.to get the necessary programs. You will also need to add /home/gallag/bin to your $PATH. I will assume that for every battery of tests one runs there will be a separate directory to contain the output. This will need to be created each time and pointed to by $OUTPUT_DIR.
These programs have been modified to work with NEUGEN versions 3.3 and greater, i.e. those created after July 2005 on the pitt branch. These require "configuration" information to make sure that the event generation is using the same cross section values as those present in the data file. In the programs below "iconfig" identifies the configuration required, where:
Goals: To check the execution characteristics of event generation. How quickly does it run, how many error messages are produced, is it making any events that are obviously wrong (because they do not conserve energy and momentum or contain NANs), etc. ICONFIG is the NEUGEN Configuration (defined above) used to produce the cross section table found in $NEUGEN3PATH.
To run: Go into the $NEUGENTEST directory. Point $NEUGEN3PATH to the version to be tested and type "make checkrun". After that
> speedtest ICONFIG >>$OUTPUT_DIR/speedtest_results.txtThis will run the program multiple times and compile statistics on each run. This output file, the execution times etc. can then be compared to previous versions (The current one is in ~gallag/neugen/betatestcode/speedtest_results.master ). I suggest you start a spreadsheet where you can keep track of the execution times, number of errors produced, etc. for each version tested so that it will be easier to spot changes.
Goals: To call many of the individual subroutines used by NEUGEN for cross section calculation to check their values and compare their output to previous versions.
To run: Since this routine will be testing two versions of neugen3 against each other, you will need to define two environmental variables: $NEUGEN3PATH_V1 and $NEUGEN3PATH_V2. Then:
> subcheck ICONFIG >& $OUTPUT_DIR/subcheck_results.txtIf things are OK, the output should look like the following (where the "version" lines at the top are replaced by the two versions you are testing. ****************************************************************
Goals: This program is identical to subcheck except for the fact that it tests a new version against the known, validated output from the "official" version.
To run: Define NEUGEN3PATH_V1 to point to the version to be tested.
> mastercheck >& $OUTPUT_DIR/mastercheck_results.txtIf things are OK the output should be the same as the previous program.
Goals: This program compares two cross section tables element-by-element.
To run: Define XSECTABLE_V1 and XSECTABLE_V2 to the two tables to be compared. It will compare the files element-by-element
> table_checker >& $OUTPUT_DIR/table_checker_results.txtThe output file will contain a list of all table elements that differ. It will also produce a file xsec_checker.output with the same information.
Goal: To compare the cross section data tables used by two neugen versions.
To Run: Set the NEUGEN3PATH_V1 and NEUGEN3PATH_V2 variables to point to the two versions to be tested. Then
> xsec_compWill create a large number of data files, these are then turned into plots by entering the "paw" program and:
PAW> lookat_xsec_fileA Large number of plots will be displayed. In each case the columns correspond to (IM=1,4) and the rows to (INT_TYPE=1,5). The first set of plots are for free nucleons, followed by scattering on iron. The predictions of the two programs will be overlayed, one in a solid black line and the other in a dotted red line. Make note of any differences (however small!) and the magnitude. If you want to make a postscript file of the plots do the following which will create the file plot.ps in the $NEUGENTEST directory: ****************************************************************
Goal: To compare two different cross section tables
To Run: Set the XSECTABLE_V1 and XSECTABLE_V2 variables to point to the two cross section table files to be tested. Then
> comp_tables typeWill create a large number of data files, these are then turned into plots by entering the "paw" program and:
PAW> lookat_xsec_fileThe plots are identical to those produced by the xsec_comp program. The "type" above must be either "Total" or "Charm".
Goal: To generate large samples of events with two versions of neugen3 and compare the output.
To Run: Have $NEUGEN3PATH_V1 and $NEUGEN3PATH_V2 set as before. The runcomp command takes five command line arguments: (number of events to generate, neutrino type, nucleus type, neutrino energy, and ICONFIG). ICONFIG is the configuration as before. Neutrino type should be "14" (muon neutrino) for the time being and nucleus type can be 1 or 2 for free nucleons or iron. The neutrino energy is entered as a decimal number. For instance if I wanted to compare generation of 10k 5 GeV muon neutrino events on iron I would enter
runcomp 10000 14 2 5. ICONFIG >& $OUTPUT_DIR/runcomp_log.txtThis will create two fairly large output files, output1.dat and output2.dat. These are then used to make plots by going into PAW and
PAW> exec allplotsThis will make a large number of plots comparing the distributions produced by the two versions of neugen3. The error bars are shown on one of the plots to give an idea of the statistical error on the distributions themselves. If there are differences in the distributions that appear to be statistically significant, note them! If you are not sure you can try running again with higher statistics. A postscript file of the plots can be created by following the same commands given in the previous example, replacing "lookat_xsec_file" with "allplots".
Goal: To generate large samples of events with two versions of neugen3 and compare the output, focussing on the hadronization and intranuclear rescattering aspects of the simulation.
To Run: Have $NEUGEN3PATH_V1 and $NEUGEN3PATH_V2 set as before. The fscomp command takes four command line arguments: (number of events to generate, neutrino type, nucleus type, and neutrino energy). ICONFIG is the configuration as before. Neutrino type should be "14" (muon neutrino) for the time being and nucleus type can be 1 or 2 for free nucleons or iron. The neutrino energy is entered as a decimal number. For instance if I wanted to compare generation of 10k 5 GeV muon neutrino events on iron I would enter
fscomp 10000 14 2 5. ICONFIG >& $OUTPUT_DIR/fscomp_log.txtThis will create two fairly large output files, output1.dat and output2.dat. These are then used to make plots by going into PAW and
PAW> exec fsplotsThis will make a large number of plots comparing the distributions produced by the two versions of neugen3. The error bars are shown on one of the plots to give an idea of the statistical error on the distributions themselves. If there are differences in the distributions that appear to be statistically significant, make note. If you are not sure you can try running again with higher statistics. A postscript file of the plots can be created by following the same commands given in the previous example, replacing "lookat_xsec_file" with "allplots".