James Barringer, Julian Cone, Robert Gibb, Hamish Heke, David Medyckyj-Scott, Peter Newsome, and Janice Willoughby
In 1986, political reforms moved New Zealand into a period of "user-pays" economics. This has had significant effects on data ownership, data access and data cost for Government-held environmental data, both spatial and non-spatial. Policies encouraging "cost recovery" by Government research organisations led initially to heavy data charging, but this was found to be untenable. More recently a more open "cost-of-supply" policy has become established, and data suppliers are now ready to take advantage of the rapid developments in the INTERNET and WORLD WIDE WEB (WWW) environment. Such developments, although largely welcomed by both data providers and users, have raised new issues centred around networked database access, data interpretation and presentation standards, and links to other databases.
Landcare Research maintains New Zealand's primary land resource database, the New Zealand Land Resource Inventory. Until recently, use of the database has been hampered by data charges, limited GIS staff and facilities, and difficulty for non-expert users in interpreting the database. To resolve these issues Landcare Research has developed a WWW user interface which allows users with no GIS skills to query the spatial database interactively, and see the results on screen in GIF format, or in hardcopy. The interface structure is designed for maximum flexibility, to cope with internal database changes, as well as new links to external databases. Analyses and outputs are standardized to limit inappropriate uses of data by users.
In response to an economic crisis, major and complex political and economic reforms beginning in 1986 had far reaching effects in all elements of New Zealand's political, economic and social structure. In Government-run research organisations three key changes occurred. First, a drive to reduce Government spending led to both reductions in funding and the introduction of "users pays" policies which required that research organisations recover a proportion of their costs through consultancy and sales of services or goods.
Second, the Government restructured science in New Zealand to introduce greater accountability, and to make more efficient use of resources directed at well focussed research. This started in 1986 with a complex series of organisational amalgamations, but later involved major restructurings. Eventually the 20 divisions of the Department of Scientific and Industrial Research (DSIR), and research divisions of four large government departments (Works & Development, Forestry, Agriculture & Fisheries, and Meteorological Service), were reorganised into nine Crown Research Institutes (CRIs) which now carry out most publicly funded research in New Zealand, other than medical research.
Third, this science restructuring process also involved major changes to the funding system from bulk funding to an output oriented competitive funding system, the Public Good Science Fund (PGSF), administered by the Foundation for Research Science and Technology (FRST). CRIs, universities and research associations all bid competitively for funding through this system.
Of these three changes, the first, in particular, generated financial imperatives which had a significant effect on the perception of spatial and non-spatial databases as major assets. This, in turn, generated debate over data ownership, data access and data cost for Government held environmental databases.
This paper has two parts. First, it discusses the impact that political changes have had on collection of, and access to, environmental data over the last decade, and flow-on effects on GIS-based environmental modelling in New Zealand. Second, it looks at the future for data access in New Zealand, and also at Landcare Research's Geographic User Interface to Landcare Databases (GUILD-on-the-Web) which is leading the way to more open and accessible environmental databases in New Zealand.
In 1985, GIS was not widely used in New Zealand. The Ministry of Works and Development, Water and Soil Division (MWD/WS) had started in 1977 with its own series of GIS for storing, analysing and plotting the New Zealand Land Resource Inventory (Williams, 1985; Van Berkel & Williams, 1986), but would move to ARC/INFO in 1988. The Department of Lands and Survey had also been using CAD and GIS since 1977, first to support the development of the metric version of their cadastral database (DCDB), and later for a new metric topographic map series (DTDB). Very little other digital data, particularly spatial data, was available at that time, however much data was available in hardcopy (eg. DSIR soil and geological maps), or text based digital databases (eg. New Zealand Meteorological Service Climate Database). In this "traditional" (pre-GIS) setting, expensive data collection was well supported by public funding, there was nominal recovery of publication costs through map sales, and "data" was freely available to the public and user community with few practical limitations (Giltrap et al., 1993).
In 1986 the "user-pays" principal was applied to research organisations. Virtually overnight, data, whether spatial or non-spatial, digital or hardcopy, became a valuable asset for research organisations which were suddenly required to recover a fixed proportion of their appropriation from commercial revenues (Newell, 1992). The nature of this newly created "data market" was a series of virtual monopolies, each type of data being available from, and controlled by, one source (eg. DSIR Land Resources for soils and land resource data, and the Department of Survey and Land Information for topographic data). An immediate effect was to restrict data use, particularly for digital data, through the introduction of substantial charges for data. This created a dilemma for the data suppliers. While there was a requirement to generate income from these data assets, and considerable effort was being expended on developing digital databases to make the data more readily available, data suppliers tended to price themselves out of the market, despite the fact that in most cases there was no other alternative source from which to obtain similar data. Many data users simply could not afford the prices being asked, and did without.
So, at a time when computer technology was developing rapidly, enabling development in GIS functionality on increasingly cheaper and more powerful hardware platforms, availability of spatial data was severely restricted, primarily by cost. This and other factors combined to restrict users ability to integrate useful datasets for modelling spatial processes. GIS-based environmental modelling activities with complex data requirements were rare, and simple interpretations based on classifications of single data layers were more common.
This situation persisted throughout the late-1980s. By 1990 it had become clear that heavy charging for data was untenable, and data providers found that their data was being under-utilised. A Ministerial Science Task Group, set up to advise on the restructuring of science provided some recommendations for managing databases: the Crown should fund these databases, retain copyright to the data, but make them publicly available on the basis that the spatial information had been collected using public funds. However, data charges should remain where the costs of collection, archiving and maintenance were not covered from public good funding. The costs of actual retrieval of information from databases and collections could also be recovered (MSTG, 1991). These recommendations moved the principles of data access and charging more into line with those that had applied in the pre-GIS era of the traditional sale of hardcopy maps.
Since the formation of the CRIs in 1992, data access has steadily improved. The PGSF supports collection of data having substantial scientific merit, and has identified "nationally significant databases", for which base funding levels have been identified, to ensure that key databases are at least maintained (FRST, 1993). Current funding levels remain above this base level, but funding for some database updating can still be difficult to obtain. However, the recovering New Zealand economy, along with the role defined for Regional and District Councils by the Resource Management Act, and the removal of direct data charges, is generating an increasing demand for data from the Regional and District Councils. Collaboration between organisations to jointly fund national databases is also beginning to occur (eg. a consortium is negotiating to develop a land cover database from satellite imagery).
The PGSF focus on scientific relevance has also resulted in a move away from predominantly single layer vector format databases with classified attribute data collected by field survey. For a number of years, the limitations for modelling and problems of updating these databases (eg. NZLRI) have been well recognised, but the step to a more multi-layered single-factor database model has been difficult to initiate. However, more focused funding, scientific requirements for objectivity and mechanistic models, and the increasingly multi-disciplinary nature of much of the research being undertaken, has encouraged the move towards single factor raster format databases and greater reliance on modelling, both to derive layers from DEMs or remote sensing and to develop more defensible analyses.
These processes have now returned most data suppliers to a point where data access is limited mostly by the physical constraints of data provision. The key problem currently being addressed is how to meet the growing demand for data with the resources available. Landcare Research is no exception, and has had a project running for three years to develop tools to meet this need.
Geographic User Interface to Landcare Databases (GUILD) is a natural development from an earlier project for a geographic index of databases held by Landcare Research (GILD). In 1992, Landcare Research was formed from parts of seven previous organisations, all of which held their own spatial and non-spatial databases. Knowledge of what databases were available, and what information they held, was often poorer within Landcare Research's internal research community than for external data users who had a history of contact with one or more of the preceding organisations. Given recent increases in demand for information to solve environmental, economic, and social problems, the aim of GILD was to provide Landcare Research staff with descriptions of the data available within the organisation. GILD was implemented in ARCVIEW v1 and displayed the spatial distribution of each database record of the major databases held by Landcare Research, together with their associated meta-data on age, content, resolution, quality and accessibility. For databases associated with physical specimen collections such as herbaria and insects, the level of spatial detail in the metadata meant that the collection location of each specimen was displayed.
As database access was freed up in New Zealand during 1992 and 1993, it became clear that, while GILD had considerable merit in its own right, there was significant scope for a more ambitious project. This was also driven by the recognition that Landcare Research's limited resource of trained GIS staff would not cope with a significantly higher level of research-related database query tasks in the newly restructured company, and that a mechanism giving staff direct access to the GIS was needed. The high proportion of requests related to research applications that were being treated as one-off requests rather than standard data dumps or queries did nothing to reduce this bottleneck in staff resources. The GIS group determined to "grow" GILD from a simple geographic index to a full GIS user-interface that could deliver solutions directly to users who had varying familiarity with the data and often negligible knowledge of GIS technology. Initial plans to use ArcView 2 and AVENUE as the software platform for this interface were changed because of the hardware requirements of ARCVIEW 2, which on release were substantially greater than the staff standard of a Novell networked 486DX66-2 with 8MB of memory.