6. Using the Results of the Comparison

The preceding sections have developed a framework for comparing different DEM representations of the same elevation surface. This framework was used to derive comparisons for 14 quadrangles scattered across southern California. This section reports on some significant issues raised by the findings. The first paragraphs address the relationship between DEM differences and accuracy. This turns out to be a complex issue. Assumptions about the nature of the two datasets result in different modeling paradigms. Parameters derived from the comparison techniques presented here can become quite useful in these contexts. A remaining section identifies systematic data problems which can be identified in specific DEM datasets using techniques developed in this research. Throughout, promising directions for further research are indicated.

This work has consisted of an examination of the difference between two datasets. An interesting question arises: what is the meaning of the difference? Most precisely, the difference indicates the magnitude of discrepancy between an interpolated estimation from one dataset and the elevation of another dataset for a single point in space. This answer is somewhat unsatisfactory in that it begs a further question: what is the relationship between difference and the fidelity of either dataset’s representation to the real world’s terrain? Neither DEM corresponds exactly to the real world; like most spatial data, both contain error (Thapa & Bossler, 1992; Goodchild & Gopal, 1989). Difference in the DEMs at a point is indicative of one of several possibilities, given the assumptions of this work:

1. the 3 arc second value is correct, the 30m bounding points are correct, but the terrain surface does not follow bilinear properties, causing the interpolated elevation to differ from the 3 arc second elevation.
2. the 3 arc second value is accurate, but the 30m bounding points are not, and the interpolation does not fortuitously correct the 30m error.
3. the 3 arc second value is not accurate, while the 30m dataset is able to accurately characterize the elevation at the point.
4. neither DEM is accurately representing the elevation at the point, but they differ in their (inaccurate) estimations.

Additionally, very low or no difference in the DEMs is of course no guarantee that both are accurate. Possibility one introduces the role of the interpolation method in the process. Possibilities two through four may be handled in two ways: first, by making assumptions about the relationships between the datasets and the real-world terrain they attempt to represent, and second, by identifying systematic differences that are the result of data processing. Modeling or eliminating these differences will increase the fidelity of the data to reality.

A fifth possibility is that one or both DEMs are not modeling elevation at points, but instead are in some way areal estimates. Although USGS specifications indicate that DEM elevations are point measures., in fact the data are derived from source materials with different scales, and therefore the elevations are affected by scale-related generalization. Figure 2.2, with its large contour swath, illustrates this point clearly. Isolines are 1 dimensional, but a printed contour line most certainly has an area. One can imagine sampling points at an even finer resolution in Figure 2.2, but would that make the resulting DEM more accurate? This work combines scale-related differences with other forms of difference, but it is worth noting that the underlying surface being sampled is not the same.

This research used bilinear interpolation to calculate the elevation using 30m data at each 3 arc second location. The reasons for using this method were covered in Section II. It is possible that at least some portion of the calculated difference at each point is due to the interpolation method. The choice of interpolation implicitly assumes the nature of the surface. Different interpolation methods result in different surfaces and different elevation values at the 3 arc second point locations. In the absence of information about the actual surface, the interpolation method decision is arbitrary. To explore the nature of this "error", a variety of interpolation methods could be used for any particular 30m data file; differences between the resulting interpolated values will give some indication of the range of estimations.

The problem of characterizing the relationships between both datasets and the actual terrain is more critical than the interpolation question. In the absence of perfect information about the terrain, one may consider both elevation models to be independent terrain samples of equal validity. Their characterization of terrain differs because the production method differs, and both differ from reality. Differences between the samples, in this case, indicate the degree of certainty about the actual terrain surface. Statistics of the differences could be used to develop statistical models for the uncertainty inherent in our digital representation of real world terrain.

An independent sample of two is quite small for rigorous statistical testing. In some cases, however, multiple independently derived 30m DEMs are available for the same area. The USGS has used a variety of production methods, using different source material, to generate these data sets. This work used both level 1 and level 2 DEMs, and in fact DEMs representing both levels were obtained for the Santa Barbara quadrangle. The difference statistics displayed in Table 5.2 indicate that they are not identical; a section of this quadrangle corresponding to the central portion of the city of Santa Barbara is portrayed in Figure 6.1.

Detailed comparisons of collocated portions of both show that the level 1 DEM is subject to sinks and horizontal striping, as portrayed in Figure 6.1. The level 2 DEM surface more closely meets the hydrologic surface "ideal" (Hutchinson, 1989), but its elevations may be treated as equally likely to be correct as the level 1 DEM for any given point on the surface. More research into the difference in terrain representation between level 1 and level 2 is warranted, and the methods presented in this paper will facilitate this. Under this first paradigm, however, one could use all three elevation datasets to obtain a distribution for the actual characterization of terrain in the Santa Barbara area.

One can also make a priori decisions regarding the relative quality of the two elevation models. The most clear cut example would be to specify that 30 meter quality data is adequate for the intended application. The assumption is that the higher sampling resolution data better characterizes the elevation surface, and that this characterization is close enough to correctly answer the application problem. The 30m DEM is assumed to be true (or true enough), and difference is characterized as error. 30m DEMs are not available for many areas in the United States. For analysis on terrain in these areas, using the coarser, lower quality data introduces uncertainty into the outcome of the application. A model of uncertainty in 3 arc second data can be developed to quantify this effect in areas for which the 30m data is unavailable (Ehlschlaeger & Shortridge, 1996; Ehlschlaeger et al., 1997). The research presented here was initially inspired by the problems of deriving more reliable parameters for such uncertainty models. As it now functions, the comparison techniques can be used to provide parameters about the difference distribution – mean, standard deviation – to the model. Spatial structure of the difference surface, also critical for the model, may also be calculated. Alternative approaches to error propagation modeling, such as analytical solutions derived from Taylor series approximations (Heuvelink et al., 1989), could also obtain parameters from the methods presented here.

As the research progressed, it became apparent that in many cases the 3 arc second data could be directly transformed to more closely match the 30 meter data. Findings presented earlier indicated that clear, systematic error in two of the 3 arc second DEMs was plaguing the comparison. The spikes and rising inter-spike trends in difference are almost certainly artifacts of the interpolation process. A clear research direction is the development of methods to smooth the spikes in a spatially appropriate manner. Removing the spikes would improve this data set regardless of whether the 3 arc second data was subsequently used in a comparison study with 30m data. The same section also presented a linear regression approach to model the rising trend in the inter-spike range between the two datasets. Applying the coefficients on elevations falling within the range could eliminate the trend and improve the fit between the datasets.

Figure 6.2. Potential processing steps to compare 3 arc second and 30m DEMs.

A parallel thread explored the transformation of 3 arc second coordinates to improve the fit. This method holds promise of removing positional discrepancies between DEMs. The comparison code as it is now written only processes one 30m DEM quadrangle at a time. It would be desirable to enable the processing of multiple quadrangles simultaneously for a single 3 arc second DEM. The result would be a global set of coefficients which could be applied to the entire 3 arc second dataset, with the intention of mirroring the 30m data set as closely as possible. The resulting transformed DEM could be used with the uncertainty model discussed previously. Figure 6.2 indicates a process that could be employed to generate more closely related datasets, thereby improving uncertainty models. The difference in terrain representation between different 30m DEM production methods is important, however. A clear implication of this is that, for the purposes of modeling difference across a 3 arc second DEM, researchers should avoid mixing DEMs of different level and, if possible, of different production method.

The previous paragraphs have identified alternative approaches to characterizing the meaning of the difference between elevation datasets. Sometimes, difference may directly reflect errors in individual data sets that can be removed. Error is introduced to a particular DEM from a wide variety of sources. Some of these might be particularly difficult to identify without the aid of an independent data source for the area. Elevation discrepancies within the Camarillo quadrangle may serve to illustrate the potential for this application of the methods presented here. In the comparison plot for Camarillo in Appendix A, there is a bifurcation in the point cloud for elevations ranging from 30 to 60 meters. For a portion of 3 arc second data falling in this range, the corresponding 30m points were relatively closely matched. Many of the remaining 3 arc second elevations, however, were far larger than the corresponding 30m points. Few points seemed to fall into an intermediate position between these two dominant clouds. I became interested in determining whether spatial arrangement could be a factor in explaining this distribution pattern.

In fact, the key to unraveling the problem rested with the spatial configuration of these points. Due to the limitations of paper media, Figure 6.3 uses contours to portray elevation. On the left, elevations in both 3 arc second and 30 meter DEMs from 1-60 meters are shown. In general, both data sets indicate terrain rising from the south and west to the north and east. Many of the equivalent contour lines, however, cross each other at right angles. The points of interest lie between the 30m and 60m contours in the 3 arc second data. The distinct point clouds from the nonspatial plot can be identified by separating the 30m values within this band into those falling above 24 meters and those falling below 20 meters. The result of this separation is shown in the right hand figure. The light gray region indicates those points that roughly agree about the elevation. The darker region contains those points for which the 3 arc second value was more than 10 meters higher than the 30m elevation. Portions of this region contain differences of 50 meters and more. The two regions are quite spatially distinct.

What caused this large discrepancy? The reason is a combination of terrain characteristics and production problems. The 30m data interprets the terrain for this high-difference area as being quite flat, with elevations in many cases below 10 meters above sea level. Mountains rise abruptly at the eastern verge of the area. The 3 arc second data interprets the area as a steadily rising slope to the east. From personal inspection, the 30m DEM seems much closer to the truth. This area is just to the south of Highway 101 near the Conejo Grade. The plain is extremely level, and mountains rise nearly vertically from flat agricultural land. Something about the 3 arc second data is apparently causing the difference. This "something" is the location of the source contour data for this dataset. The 60m contour occurs above the base of the mountains. For the 3 arc second interpolator, however, this contour was the first and only indication of elevation for a large portion of this quadrangle. The 30 meter contour was also in the source data (over 3,300 points in this quadrangle were either 29 or 30 meters) but it is plagued by other problems. This region abuts the naval facilities at Port Hueneme, which are located in the Oxnard quadrangle just to the west. The 3arc second data depicts the entire Hueneme area as below sea level, possibly because the information was believed to be militarily sensitive. The impact of "Hueneme Bay" on elevations to the north and east is unclear, but probably the base map 30 meter contour was adjusted, and subsequent digital conversion resulted in error across a very large, flat area. Analysis of this sort can be performed on puzzling characteristics in many quadrangles using tools and techniques from the research presented here.

This section attempted to sort out the implications of the difference between two DEMs, and how one might use it to answer questions of data accuracy. Assuming the bilinear interpolation method is not a major factor, two main approaches emerge. One is to make assumptions about the two data sets and their relation to reality. This results in the development of models to characterize uncertainty (either in the depiction of terrain generally, or resulting from the use of inappropriately poor 3 arc second data). A second is to use the comparison to identify error resulting from data collection and processing. Some of this error may be found generally, as with the systematic spiking/benching in the 3 arc second data, and other error may be identified in particular datasets, as with the Camarillo example. The comparison techniques presented in this work also promise improved methods for investigating basic research questions about error from contour interpolation and various DEM production techniques. Taken together, these applications provide a variety of approaches for increasing understanding about and improving the modeling of the nature of error in digital elevation models.