The control_stats.log file is the primary file used to score the many runs executed during a calibration phase. By sorting on one or more of the metrics contained in the control_stats.log file, coefficient sets that performed best may be found. Using this information, the user must decide what coefficient ranges will be used for the subsequent phase of calibration.
The algorithm for narrowing these ranges is an area of continuous discussion among users, and so far no definitive "right" way has been agreed upon. Examples of approaches used thus far include: sorting on all metrics equally, weighting some metrics more heavily than others, and sorting only on one metric. Ron Matheny of the Environmental Protection Agency wrote a simple SAS script to sort the control_stats.log file that may be downloaded here. For the sake of clarity in this example, the last method, sorting on one metric, is applied. Simulations are scored on their performance for the spatial match, or lee sallee, metric.
Another disputed issue associated with this process is how many of the best scores should influence the range selection: the top 5, 10, 50? This too is an area that is ripe for more research. The pragmatic answer is enough to capture good results, while substantially reducing the values' spread.
back to Implement