Acknowledgements Table of Contents Sections 1-3 Section 4 Sections 5-6

4. Developing a Research Agenda

4.1 Sunday small groups

4.1.1 Tool development (models and computation)

Presented by Joe Ferreira

This session started by trying to identify the classes of problems in which CSDM can be used and to map these problems to appropriate tools. The set of problems identified ranged from specific to general and included: discrete spatial choice and the need to represent values and preferences; managing uncertainty and error; generation of alternatives and the expansion of the choice set; information filtering and management; accommodating multiple value systems; providing for information demand and data browsing; interacting spatial decisions; resistance to decision-making through consensus building; education about the problem or decision-making procedures; and, optimizing service delivery. Mapping of problems to tools proved difficult and a related mapping of tools to a four-way classification of space and time (same vs. different on both dimensions) was begun. Sub-topics were identified for a number of problems and discussion then moved to a consideration of whether there were any grand themes in these problems. Several were identified, including spatial search, representation issues, generating and analyzing alternatives, process management, and the need to decompose the problem context to achieve the desired mapping to tools.

4.1.2 Human computer interaction

Presented by Rachel Jones

This group met to discuss how a possibly disparate set of users would be able to interact with software in a collaborative setting. The group felt that research should focus on the human dynamics of collaboration, rather than the technology per se, because while the technology will change rapidly, the dynamics of human behavior will not. The first issue that was presented concerned the level of intervention that would be appropriate in a particular context. By intervention, it is meant that the system would possibly provide a context-sensitive structure that enables users who are otherwise unfamiliar with a system to navigate through it. Three levels of intervention were specified. The first, and most simple, is to replicate a path - the user is guided along a deterministic sequence of steps. The second is to present alternatives to the user who selects from among them. The third is to provide a critique of the process.

A second issue addressed is related to configurability. Different individuals, as well as different groups, have different views of a problem and its representation. One way to consider configurability is to specify a set of generic operations that would encompass the types of operations, information access and user tasks that need to be supported by the system. If this list is compiled and made available it can be structured hierarchically to provide different "depths" of intervention. This presumes, however, that an analysis of the tasks required to accomplish goals has been conducted.

The group decided that task analyses would prove useful in accomplishing the goal of supporting intervention. A taxonomy of usage patterns is required that is organized around the topology of time and space (same-place and same-time to different-place and different-time). Some of the most difficult problems result from "small" changes: moving from two people working together on a map in one office to working on the same map in different offices, for example. Task analysis would also help to identify a series of primitives - generic operations that support information access and user tasks. When working on a map, for example, people often want to show or highlight some information, add or remove information, or otherwise manipulate the map's content. The group also recognized that there are user and organizational characteristics that might color the process in which tasks are attempted.

Finally, the group considered the effect of roles. Different roles, such as facilitator or mediator would need to be supported in a way that would best enable them to accomplish their tasks. One of the roles of participants, for example, is to understand a problem so that it can be defined and alternatives generated and evaluated. A key element in this process is learning that enables users to understand their problem better. One group member recounted how when a group of people were shown an aerial photograph of their town, they were amazed at the amount of green space that they saw; they had never before seen such a display and it changed their perception of the problem.

4.1.3 Problems and processes (institutional issues and use)

Presented by Mike Shiffer

This group attempted to come to terms with the institutional contexts in which CSDM systems might be used. In large part, the group was stymied by a lack of context: it was not evident what processes must be supported. Who should be involved in a decision is context-driven and cannot be discussed in a generic sense; indeed, even the set of tools that is made available for use is contextually-conditioned.

Any system has embedded within it an implicit and explicit character - its "spirit." Furthermore, a system has an embedded political structures that it supports. Discussion of this issue raised several questions:

  1. What types of conflict resolution tools are made available to users?
  2. Is a majority vote the exclusive way to resolve deadlocks, or does the system accommodate plurality or vetoes?
This group also grappled with the issue of structure. How much structure is needed to facilitate support without restricting it? What skills are assumed on the part of the user to use tools and develop appropriate decision-making strategies?

4.2 Monday a.m., small groups

4.2.1 How does the problem context constrain tool design?

Presented by Tom Pederson

The group started by acknowledging that a single problem formulation can be addressed by groups in a variety of contexts and that this might affect how the decision process is structured. Thus, a group of friends might proceed in a very different manner to one in which all the members are strangers. To support groups with varying characteristics, systems must be designed to accommodate a range of constraints. Such constraints can be placed in three classes: environmental, procedural and structural. Whilst environmental constraints define the context of the decision process, procedural constraints determine how the process evolves and structural constraints define the capabilities of a CSDM system's tools. These three types of constraints interact.

An attempt was made to try and decompose the three types of constraint into their constituent elements. It was recognized, however, that this would require picking specific examples of CSDM and decomposing them in a comparative analysis to see what is common and what is unique to each case.

Further discussion focused on using the three types of constraint as the axes of a "constraint" space. The labels of the axes were refined to reflect the group's idea that the space is better thought of as an "interaction" space that captures more of the richness of CSDM. Thus, the dimensions were relabelled as: institutional (environmental), activity (process), and physical setting (structural).

The group identified four researchable questions:

  1. How do we identify promising cells in the space for study?
  2. How do we identify paths of interest (trajectories) and match them to problem types?
  3. Can we highlight cells where "spatial" is especially important, or we are uncertain about its importance?
  4. How do we identify the technical developments that are likely to yield the biggest "bang-for-the-buck" across the full range of cells to show where tool development efforts are best directed?

4.2.2 Multiple representations

Presented by Jim Proctor

Discussion in this session began with an attempt to develop a group understanding of what was being meant by the term "multiple representations". A number of different aspects were discussed, including views versus models, internal versus external, interests versus positions. The group did not develop a definitive answer to "What is a representation?" but concluded that there are mental (including psychological, social, cultural and cognitive aspects), visual and computational perspectives. A second fundamental question, then, was "How to represent?" Should we use just maps or should we include language, databases, tables, graphics and GIS models? After setting this framework, the group quickly developed the following research questions:

  1. Given two formal models (such as GIS models), can the similarities and differences be identified independent of process? In other words: are there are measures for information equivalence, or for computational equivalence?
  2. Can we represent different interests in a common data space?
  3. What level of semantics is necessary to represent interests?
  4. What dimensions of representations are most important for comparison of stakeholder interests and positions about locational conflicts?

4.2.3 Process intervention and empowerment

Presented by Steve Carver

This group considered two issues and their interaction: process intervention and differential empowerment. The first issue addresses the problem of agency. The way in which software is written, the types of tasks supported and the level of access that individuals have to different data will all condition and shape the nature of discussion. The group considered the idea that software and the manner in which systems are used force designers to face and make complex trade-offs between simplicity and complexity, and flexibility and structure. For example, if the system is structured and participants take issue with the structure, then the system will be viewed unfavorably. If, however, a multiple-level system were available, and users could change between levels of structure, this additional flexibility would contribute to usability.

The act of participation in decision-making processes provided one motivation for the discussion of empowerment. Two aspects of access were considered: access to technology in a public decision-making context and what might be called "conceptual access" - in the sense that individuals who are unfamiliar with computer use and with geographical concepts would be disadvantaged relative to others who have more specialized training in these areas. There is a possibility (a likelihood even) that an "information underclass" could arise as a consequence. In fact, the willingness of individuals to engage in a debate would be affected if they view themselves as likely losers in a technology-supported debate.

The group identified a series of researchable questions:

  1. Can adaptive user interfaces effectively be applied to CSDM problems?
  2. Can users be profiled to structure CSDM systems in appropriate ways?
  3. What is the role of human and software agents within CSDM for intervention and empowerment?
  4. To what extent do GIS, spatial analysis, and the Internet provide appropriate tools for empowerment?

4.2.4 Metrics for evaluation

Presented by Brenda Faber

The group attempted to define approaches that could be used to evaluate the success of implemented CSDM environments. These metrics were stratified into metrics that can be used to measure the degree of participation, and the quality of the solution and the process that generated it. Discussion of metrics of participation focused on how to measure the number of participants as well as the amount of participation by each person. While metrics of solution quality measure the quantitative differences between pairs of alternatives, metrics of the quality of solution processes must include factors such as the number and type of deadlocks, user satisfaction, and the degree to which participants evaluated a range of non-trivial solutions.

The group identified the following research questions:

  1. Does CSDM create better informed stakeholders (who have a shared, consistent understanding of the issues being addressed), attract more participants, and retain more participants?
  2. Does CSDM result in greater access to and use of information by each person?
  3. Does CSDM result in decisions with an improved level of quality (outcomes)?
  4. Does the use of CSDM expand the number of non-trivial alternatives that are generated?

4.3 Monday p.m., small groups

4.3.1 Spatial data manipulation techniques

Presented by Steve Frysinger

Because this group was large, it divided into two subgroups that reconvened prior to the plenary session to synthesize their discussions. The first idea that was discussed concerned the use of existing strategies for collaboration as analogies that could be developed into computer-mediated processes. Blackboards, for example, enable people to write over the top of someone else's material and gestures, which are effective ways of communicating, are often lost in computer-supported decision-making contexts. Because it is often useful to employ graphical "gestures," such as circling an area to draw attention to it, either to indicate agreement or disagreement, the idea of a spatial markup language was advanced.

Theories of argumentation were proposed as a mechanism to frame discussion about the kinds of actions that might need to be supported in CSDM environments. The provision of different types of bargaining tools, for example, might be appropriate in different contexts. One questions that arises is "What role could agents play in negotiation?" Agents would need to be trained to help a user clearly advocate their particular approach to problem-solving.

One topic of discussion that emerged from both groups was the maintenance of audit trails that support the reconstruction of the sequence of actions and activities that led to a particular outcome. These audit trails would need to be time-stamped to determine when different actions were taken and might prove useful during discussions about why particular results are judged to be superior to others.

The group identified the following researchable questions:

  1. Can we use existing systems as analogies to support the development of CSDM software?
  2. How do we represent differences among problem representations?
  3. Can advocacy agents and bargaining tools be designed to help us advocate our own approach and understand what we are willing to give up in order to keep something else?

4.3.2 Generation of alternatives

Presented by David Bennett

A commonly-adopted strategy for addressing semi-structured problems is to generate and evaluate a number of alternative solutions, or solution processes. Thus, a computer-supported system must facilitate the generation of alternatives. The group discussed the metaphor of genetic evolution to describe the process in which certain activities, processes and solutions are judged to be "fit" given other possible paths. Given a particular starting point, a solution or solution process could be perturbed to mimic a new generation in a genetically mixed population. This perturbation could be viewed as a mutation. If this process is allowed to advance through several generations, "fitness" can be evaluated at each step. Only those elements that are judged to be most fit at each step are further perturbed and allowed to propagate. In this way, fruitful and promising paths to solutions could be generated.

This group identified four researchable questions:

  1. Can metaphor be used as a research strategy?
  2. Does a genetic algorithm mimic a collaborative agreement process?
  3. Can a collaborative agreement process elicit a merged or reconciled set of preferences?
  4. Can algorithms which enhance diversity improve the quality of the adopted solution?

4.3.3 Evaluation of representations

Presented by Seymour Mandelbaum

This session began with each participant briefly stating his or her definition of "representation." It became apparent that the resulting definitions needed a cognitive framework - finally expressed by Mike Shiffer in the following diagram. Each of the four corners of the diamond identify different aspects of representation, each aspect is linked to the adjacent one through some transformation process (indicated by the uni-directional arrows). Thus, it seems possible to evaluate representation from many different perspectives. Each aspect is individually rich in research opportunities while the links between adjacent corners also provide fruitful areas for research.

A number of research questions were identified:

  1. How do the design of IS and collaborative processes variously impact the address to multiple representations (address was defined to mean articulation, or translation)?
  2. How can stakeholders' satisfaction with the representation of their interests be measured?
  3. How can GIS and spatial data be used to enable or restrict multiple representations?
  4. Do the unique aspects of spatial information systems hinder us or help us to merge multiple individual representations?
  5. What are the aspects of maps that can be manipulated to influence a collaborative process? (We need to study the link between artifacts and effects.)
  6. To what extent do stakeholders/groups learn from each other in this process of plural representation?
  7. How do we represent collaborative processes?
  8. What is the impact of different representations on multiple users?
  9. What are the aspects of participants' interests that can be represented by different techniques?

4.4 Tuesday a.m., small groups

4.4.1 Technology and innovation barriers

Presented by Rene Reitsma

The group considered two barriers that must be overcome to improve the use of CSDM software. The first barrier is latency that can be divided into two types. The first concerns system performance: if response times increase as complex models are developed and used, then the number of alternatives that can be considered in a same-time, same-place context is reduced. This could lead to decreased user satisfaction with the system. The second type of latency considered centers on the issue of tool preparation. The group discussed the idea of successive refinement of models: it may be possible to use "quick-and-dirty" models in the earliest stages of a continuous process of decision-making but later on, as the decision-process unfolds, effort can be focused on the development of those models that show the most promise. Marginal cost was suggested as one mechanism for determining the relative suitability of tool preparation and use.

The second barrier discussed by the group was distance. Participants raised the issue of asynchronicity of use and discussed a possible environment in which individuals could enter and leave the decision-making process that would take place in a shared environment. The metaphor of a MUD (multi-user dungeons) game was discussed in this vein.

4.4.2 Theory

Presented by Thomas Gordon

This group examined the ways in which alternative theoretical frameworks could be brought to bear on CSDM problems. Three main theoretical stances were discussed: economics and decision theory; argumentation theory and dialectics; and adaptive structuration theory. The group also considered the environment in which these theoretical frameworks would be used and noted that there are two key dimensions: the availability or otherwise of resources; and the degree to which the goals of a CSDM problem are well-defined. The location of any given problem in the space defined by these two axes will help to indicate the suitability of the different theoretical approaches for that problem. The group then returned to a discussion of the attributes of argumentation theory and suggested that it is well-suited to a broad range of CSDM problem types.

4.4.3 Joe's cube

Presented by David Coleman

The group began by examining the initial formulation of "Joe's Cube." The cube arose from a desire to provide a means to map tools to problem contexts. The cube has three axes: physical setting, environmental setting and procedural setting. The group discussed at some length what each of these axes represented. The physical setting was the most clearly defined axis since the four elements are clearly distinguished: same-time, same-place; same-time, different-place; different-time, same-place; and different-time, different- place. The environmental setting axis was expressed in the context of a coupling index that ranges from "tightly coupled," representing a small group of people with similar goals working on a clearly defined project, to "loosely coupled," where there is a large group with dissimilar goals working on a problem which is multi-faceted. The procedural axis is the most problematic to refine. Beginning with a simple idea of the axis representing the progression of decision-making from preparation, to review, analysis, evaluation and decision, the group visualized a possibility that this axis might in fact be a loop or cylinder. After acknowledging that a decision-making process could move through various levels on the physical and environmental axes, the group visualized a spiral moving through the space within the cube that depicts the decision-making process as it cycles through a number of similar procedural stages.

The cube gradually evolved into a conceptual framework within which it would be possible to examine a number of different problem domains. Thus defined, the cube allows CSDM problem contexts to be decomposed in such a way that similarities and differences between them can be compared once the cells have been filled with appropriate tools or techniques. It may be necessary to refine the definitions of the axes differently for different domains. At the end of the session an effort was made to use the cube to examine the very simple, and pertinent, spatial collaborative problem of a group of people trying to decide which restaurant to choose for dinner.

The group frequently returned to the question of the spatial dimension and wrestled with how it should be expressed within the cube. One suggestion was to impose a fourth dimension to represent the spatial domain, but it was not possible to conceive how that would prove useful. No conclusive spatial aspect of the cube could be identified. The group concluded that although the cube may be useful in many collaborative decision-making studies, it would nevertheless be useful if constrained solely to CSDM problems.

A set of relevant research questions were posed:

  1. Can this model (or some other model) of an interaction space be used to help us map the problem context to the tools?
  2. What techniques can be employed to define the elements along the axes, especially the procedural axis? Could ethnographic studies help?
  3. Can we study, compare, and contrast the patterns of cells filled in the cube when the same model is applied in different problem domains?
  4. How can we define the unique spatial components of the cube and its cells? Is it simply problem specific?
  5. What do the cells contain?

4.4.4 Invention decomposition

Presented by Doug Johnston

This group discussed several themes that are essential to the development of systems that are well-received by users. The first, function, assumes that individuals are using a system in a same-time, different-place mode. In such cases, communication bandwidth plays an important role. Users may require concurrent access to spatial objects and they may need to annotate and highlight salient aspects of these objects. In such environments a process of spatial argumentation must be supported. This may take place in either geographical space or attribute space. Certain bookkeeping activities were also considered to be essential to the successful implementation of systems: an archival storage and access mechanism, for example. Finally, the group considered the potential impact of information overload on participants and suggested that filtering mechanisms be developed.

4.4.5 Breakdown, failure and disaster

Presented by Mike Shiffer

This group examined the nature of adverse outcomes on system use. They first considered technical problems that can erode confidence in a system. Clearly, an experience such as a system crash might lead users to view a system as "tainted." More subtle impacts, such as the effect of extreme latency on system use, were also considered to be important technical problems that must be treated. The second class of problems considered centered on the idea of process and the development of trust that is fostered among users and with the use of the system to address problems. The group also considered issues such as anonymity and the role(s) that the facilitator should play during the decision-making process. Finally, user interface and system complexity were considered once again because overly complex software would discourage use and lead to failure.

4.5 Tuesday p.m.: Toward a synthesis

During the final lunch break, five groups met to individually consider the synthesis of the meeting discussions and to formulate a set of relevant research questions. These questions have been grouped below under two headings: research into tool development and research into tool use. Participants also made some suggestions about the role of NCGIA in fostering and supporting CSDM-related research.

4.5.1 Research into tool development

  1. How do we best take advantage of what has been done in cognate fields to improve CSDM environments?
  2. What is the role of user needs and requirements analyses in CSDM research?
  3. What spatially-related functions are required to support CSDM?
  4. How can we tailor a system to an individual's needs? Are information filters, advocacy agents and other features required to support a range of users?
  5. How can we tailor a system to a particular group to take into account the effects of location, group makeup, and other factors?
  6. How can we use hypermedia to aggregate opinion and plans and best present commonalties and differences amongst alternatives?

4.5.2 Research into tool use

  1. What is the nature and meaning of representation in CSDM?
  2. What dimensions can best be used in comparative studies to assess the effectiveness of CSDM software?
  3. How does a particular problem formalization and CSDM system implementation affect the decision-making process?
  4. Which forms of intervention are appropriate in different contexts?
  5. What are the distinct roles of the participants within a CSDM framework?

4.5.3 Research infrastructure

Participants expressed a need for an NCGIA-supported World Wide Web site that will act as a repository for information on the Initiative's research program and that will maintain links to sites that host descriptions of CSDM-related research conducted by others.

   Go forward to Section 5

   Go back to Table of Contents

Posted March 28, 1996

Comments to Karen Kemp