Uncertainty-aware visualization and proximity monitoring in urban excavation: a geospatial augmented reality approach

Related studies

This section reviews related studies in modeling the geospatial underground utilities and the associated positional uncertainties, the methods of locating invisible underground utilities, and emerging Augmented Reality-based approaches in visualizing buried, invisible utilities in the context of their spatial context. The current practice of the 811 One-Call procedure and its limitation are also reviewed.

Geographic information systems (GIS) for utility data

Geographic information science (GISci) is the discipline that focuses on understanding the world by describing, analyzing, and explaining human relationships with the earth (Huxhold 1991). GIS is an information system built upon GISci to manage, analyze, and report spatial data, describing phenomena above, on, and underneath the earth’s surface. A GIS is both a database system to manage spatial and non-spatial data and a set of spatial operators for working on spatial relations (Poku and Arditi 2006). Integration of spatial and non-spatial information on a database platform, registration of locations to the real world coordinates, and spatial analytical capabilities are the distinguishing merits of GIS, leading to the proliferation of its applications in civil engineering (Poku and Arditi 2006;Miles and Ho 1999).

GIS allows a utility owner to have a complete utility inventory stored in a single repository that is easy to update and extract (Corbley 2007). Since its emergence, GIS has been steadily replacing paper-based as-built drawings and digital Computer Aided Design and Drafting (CADD) drawings to inventory and manage utility data. It has become the de facto tool of choice for utility owners for creating, organizing and managing geospatial utility information (Sipes 2007). Many utility owners have gone through the transition from paper maps to CADD files or GIS databases via digitization (Cypas et al. 2006).

GIS has been historical two-dimensional (2D), modeling the geometries of objects into georeferenced points, polylines, and polygons. Buried utilities are predominantly modeled as 2D polylines in GIS, missing vertical location information. Some utilities might have their buried depths (e.g. “depth of cover”) stored as attributes and their vertical location might be derived by consulting the reference surface. However, utility depths are rarely referenced to a recognized elevation datum (Federal Highway Administration (FHWA) 1999;Anspach 1995) and any changes of the reference surface make the buried depth a very unreliable source for deriving vertical utility locations.

Locating utilities in the field

When utilities are first installed, their locations are captured in as-built drawings that vary in terms of information richness, positional accuracy, and storage format (e.g. paper-based versus digital). The advancement in tracking technologies such as Global Positioning Systems (GPS) and Radio Frequency Identification (RFID) has greatly facilitated the collection of accurate utility locations for new installations in both horizontal and vertical dimensions (Dziadak et al. 2008;North 2010).

After utilities are installed and covered, the procedure of locating them in the field typically starts with the as-built drawings. Geophysical surveys based on a variety of sensing and locating techniques might be performed to locate buried utilities with good accuracies at the levels of A and B, based on the generally accepted definitions of quality levels in Subsurface Utility Engineering (SUE) (Stevens and Anspach 1993;Lew 1996; American Society of Civil Engineers (ASCE) 2002). Locating techniques include radio frequency (RF) detection techniques, electromagnetic techniques, magnetic methods, vacuum extraction, ground penetrating radar (GPR), and terrain conductivity (Anspach 1995). A number of studies have been conducted to apply the GPR technique in detecting buried utilities (Hereth et al. 2006; National Research Council (NRC) 2000;Butler 2001;Lanka et al. 2001). GPS, though not a detecting technique, has been frequently combined with locating techniques to register the location of detected utilities to real-world spatial referencing system and thus, forms the foundation of integrating with GIS to automate the inventory and update of utility locations in GIS and guide future field location of utilities (Common Ground Alliance (CGA) 2010;Ellis et al. 2009;Manacorda et al. 2007;Bakhtar 2006;Ishikawa et al. 2006;Goldstein 1997).

Uncertainty in geospatial data

Uncertainty has been a major issue in GIS for many years (Heuvelink and Burrough 2002;Goodchild 1998), and is one of the top ten research priorities in GISci (Cobb et al. 2000). Uncertainty can be generally defined as the discrepancy between what a database indicates and what actually exists in the real world (Goodchild 1998) and be described in aspects of positional inaccuracy, errors, vagueness, ambiguity, fuzziness, scale, and sampling (Goodchild 1998;Cobb et al. 2000;Fisher 1999). The literature on the topic makes use of a range of terms related to uncertainty (Crosetto and Tarantola 2001;Duckham et al. 2001), including quality, accuracy, reliability, error, ignorance, precision, clearness, distinctiveness, etc. (Foody 2003). A large amount of research articles have been included in the publications of two international conferences of Symposia on Spatial Accuracy Assessment and Symposia on Spatial Data Quality.

Current practice uses metadata (e.g. data about data) standards such as ISO 19113 (ISO 2001), ISO 19115 (ISO 2003), and Federal Geographic Data Committee (FGDC) (Federal Geographic Data Committee (FGDC) 1998) to measure and record the quality of spatial data upon a common platform (Fisher et al. 2010). Spatial data quality is generally measured in aspects of lineage (e.g. data collection methods and data sources), accuracy, consistency, and completeness as the assessment of results (Fisher et al. 2010). The concept of “fitness for use,” e.g. how well a certain data set meets the needs of an application, has also been proposed as the overall measure of the quality/certainty of geospatial data (Fisher et al. 2010;Devillers et al. 2010).

While metadata provides a common platform for recording and communicating data uncertainty information, it does not provide methods to handle and mitigate the inherent uncertainty in geospatial data. The current practice of GIS is deterministic. All visualization and analyses are being performed as if the underlying GIS data were correct without any incompleteness, errors, and inaccuracy. To cope with this limitation, Burrough (Burrough 1992) discussed an “intelligent GIS” to benefit from available metadata to support the use of uncertain data. Unwin (Unwin 1995) introduced the concept of “error-sensitive GIS” for error management such as data verification and validation, visualization of errors/uncertainties, and simulation and sensitivity analysis to obtain a range of potential results as well as associating a sense of credibility to each scenario. Duchham and McCreadie (Duckhma and McCreadie 20021999) proposed the concept of “error-aware GIS” as an extension to “error-sensitive GIS” by adding techniques to understand errors and integrate errors into decision-making. Devillers et al. (Devillers et al. 2005) designed a prototype multidimensional database system to assist users in assessing the fitness for use of geospatial data, considering errors and error propagation issues.

Error modeling for linear objects

For linear, geospatial utility data, uncertainty is mostly concerned with the positional discrepancy between the records-indicated object locations and their real world locations and thus, uncertainty might be interchangeable with positional error/inaccuracy. Accuracy assessment and error modeling of linear objects have been active research topics. A number of studies have proposed and tested several error models for geospatial linear objects such as roads, utilities, and streams (Mozas and Ariza 2011;Shi and Liu 2000;Goodchild and Hunter 1997;Caspary and Scheuring 1993;Perkal 1956).

Utility lines in GIS are typically modeled as straight line segments that connect two end points. In 2D, the uncertainty of a straight line can be captured as an uncertainty epsilon band that encloses the “true” location of the utility centerline (Mozas and Ariza 2011). This concept was initially proposed by Perkal (Perkal 1956) who used an epsilon band, the 2D space enclosed by two parallel lines that are also tangents to circular errors at the ending points as the probability range for a line. The epsilon band has been discussed frequently in the literature (Blakemore 1984;Aspinall and Pearson 1995) and has been implemented in 2D GIS in various algorithms in the form of a tolerance (Goodchild and Hunter 1997). Caspary and Sheuring (Caspary and Scheuring 1993) and Shi and Liu (Shi and Liu 2000) suggested that intermediate points on a line have smaller errors than the end points and thus, the error band will be “slim” in the middle, leading to a genetic band, or a G-band. Probabilities can be determined for G-bands with various sizes to model the uncertainty in lines (Heuvelink et al. 2007;Wu and Liu 2008). Goodchild and Hunter (Goodchild and Hunter 1997) pointed out that epsilon was often interpreted in a deterministic sense as the minimum buffer width that enclosed the true location of the objects under testing/assessing and was very sensitive to outliers. They proposed a simple buffering approach to evaluate the positional accuracy of linear objects by simultaneously referring to the buffer width and the percentage of lines within this buffer (Goodchild and Hunter 1997). The main limitations of current error modeling for linear objects are (1) being 2D, (2) being deterministic (e.g. the epsilon band is determined to enclose the true location and given the existence of outliers, the band width is unreasonably large), and (3) the lack of a method to estimate the most probable location of linear objects given their recorded location and associated positional uncertainty, the reverse procedure of accuracy assessment.

Augmented reality (AR) for utility visualization

A relatively new technological advancement in AR has enabled the visualization of invisible, buried utilities in the context of their real-world surroundings (Kamat 2003;Behzadan 2008;Roberts et al. 2002a). In an AR environment, buried utilities can be visualized as floating lines on the correct locations relative to background images and photos. If three-dimensional (3D) reference data is available, the buried utilities can be offset downward to compose an interactive 3D display (Talmaki and Kamat 2012). The positional uncertainties associated with geospatial utility data might be visualized as 3D buffers/”halos” to provide an uncertainty-aware visualization and proximity analysis (Talmaki et al. 2012).

Current practice of the One-call system

Recognizing excavation damage as the largest single cause of pipeline accidents and associated deaths and injuries, the National Transportation Safety Board (NTSB) initiated the development of the One-Call (811) notification system in 1970s (National Transportation Safety Board (NTSB) 1998). Before digging, it is required by Federal Law to have the location of all buried utilities in the vicinity of the excavation area to be pre-marked. Excavation contractors are required to contact state-wide one-call agencies 48 to 72 hours prior to the start of the operations. One-call agencies, in turn contact their member companies with the location of the excavation site. If the member companies determine an overlap between the job site and their utility lines’ location, they mark the location of the utility lines using spray paint, flags, stakes or any combination of these.

The markings are typically made referring to data from as-built drawings. An obvious limitation of this one-call procedure is that the markings are typically the very first things being removed when excavation starts. The excavation operators then have to rely on their memory to estimate the utility location and their imagination to compose mental images of the proximity of the digging implement to utilities and how the excavation operation might interact with utilities. This is an error-prone procedure that in turn, is the main reason of the large number of utility strikes even after the one-call procedure is followed.

The overarching goal of this study is to prevent unintended collisions between a digging implement and buried utilities via uncertainty-aware visualization and proximity analysis of excavation operations in a virtual environment. The premises for the research study presented in this paper are that (1) increased spatial awareness of excavator operators (e.g. being able to “see” the buried utilities and the movement of the excavator bucket and judge the proximity between utilities and the bucket) leads to improved excavation safety, and (2) such an increased in spatial awareness can be achieved through the combination of visual perception and analytical proximity monitoring of excavation operations. Considering the pivotal role of data quality and the tremendous uncertainty in utility locations, the research hypothesis is that uncertainty, particularly spatial uncertainty, can be brought into the decision-making process in an easy-to-understand manner for both visualization and analytical proximity monitoring to prevent unintended collisions and increase excavation safety.

Uncertainty modeling of geospatial utility data

The authors modeled utility lines as 3D straight line segments connected at turning points (Talmaki et al. 2012). In practice, the 3D locations of turning points are first obtained and 3D straight lines can be constructed by connecting these 3D turning points. Of all positional accuracy/error models, the most related are those that apply to points and lines. Uncertainty, when expressed as probabilities associated with geospatial extents that contain the “true” location of that object, can be derived from the recorded utility location and its positional accuracy.

Figure 2 illustrates the 3D uncertainty model that was designed specifically for geospatial utilities points Probabilities were introduced to represent uncertainties. The assumptions for this model include: (1) the positional error consists of a systematic error and a random error, (2) the random error is normally distributed in all dimensions, (3) random errors in X, Y, and Z directions are independent, and (4) horizontal errors are not different in X and Y dimensions. The systematic error can be remedied via a translation in a 3D coordinate system. The random error in the Z dimension follows a normal distribution and probability ranges can be determined as linear ranges. The random errors in X and Y dimensions are combined into a horizontal radial error calculated as square root of (e_x² + e_y²), where e_x and e_y are the random errors in the X and Y dimensions, respectively. The horizontal radial error can be treated as a normal distribution (Greenwalt and Shultz 1968) or a χ² distribution with two degrees of freedom (Caspary and Scheuring 1993). Probability circles can then be determined as pairs of a radius in the unit of standard deviation (SD) and a probability expressed as a percentage.

The horizontal probability circles and the vertical linear probability ranges can be combined into 3D probability ellipses/spheres (Greenwalt and Shultz 1968) or probability cylinders. A probability cylinder is constructed by taking a probability circle and extruding it vertically. The resulting probability is the product of the corresponding circle probability and the linear probability. For instance, the horizontal 50% circle is extruded to reach the vertical 90% linear probability, resulting in a 45% probability cylinder. Unlike probability ellipse/circle, many cylinders can have the same probability, making implementation impractical. A random simulation was conducted, of which the results confirmed that 3D ellipsoids closely follow the point clouds and thus, were chosen to model the uncertainty of utility turning points.

In this study, utility lines were modeled as straight line segments that connect two end points. In 2D, the uncertainty of a straight line is captured as an uncertainty epsilon band (Mozas and Ariza 2011). The epsilon band took a shape that is “slim” (Shi and Liu 2000;Caspary and Scheuring 1993) in the middle to indicate the positional error at turning points is larger than that at intermediate points on the line. Such a band model in 2D is commonly referred to as a genetic band, or G-band. Since this study extended the 2D G-band into 3D and introduced probabilities to represent uncertainties, the resulting uncertainty model for utility lines was named 3D Probability G-band. Figure 3 illustrates the uncertainty model for utility lines by extending the epsilon concept from 2D into 3D and incorporating probabilities to represent uncertainty. Only one probability band (the outermost surface) is shown in Figure 3. The 3D space enclosed by this boundary represents the space that encloses the true location of the utility line at a particular probability. The shape and the size of the band can be described via a mathematical function (Equation (1)):

${\mathrm{r}}_{\mathrm{x},\mathrm{p}\%}=f\left(\mathrm{x},\mathrm{p}\%,\mathrm{data}\mathrm{lineage}\right),$

(1)

where r_{x, p%} represents the radius of the p% probability band at location x along the centerline.

The function f indicates that radius is dependent on the location along the centerline, the probability of interest, and the data lineage. The determining effect of the data lineage on uncertainty models will be described detail in the section of Construction of Uncertainty Models for Geospatial Utility Data. When the locations of utility lines are collected directly such as in GPR, the 3D Probability G-band can be simplified into 3D Probability Bands that can be derived via the 3D buffering approach. Rather than being “slim” in the middle, a probability band is of uniform size along the line.

Validation of uncertainty models

Monte Carlo simulations were performed to validate both the point and line error models. Figure 4 illustrates the distributions for end points and the centerline. The Monte Carlo simulations generated random 3D locations for end points. Corresponding end points were connected to form centerlines. Corresponding 3D ranges of the 50% and 90% probabilities are highlighted in Figure 4, where the white line indicates the 3D extent that is associated with a probability of 50% to enclose the true utility location and the black line indicates the 3D extent that is associated with a probability of 90% to enclose the true utility location. Figure 4 clearly illustrates that the point distribution follows a sphere shape and the 3D Probability G-bands for the line are “slim” in the middle. Such 3D ranges (e.g. 3D Probability G-bands) compose the 3D uncertainty models for utility centerlines in this study.

Construction of uncertainty models for geospatial utility data

The authors constructed uncertainty models by quantifying the 3D Probability bands in pairs of probability and band size. The probability expressed in percentage refers to the probability of a specific 3D geospatial volume determined by a particular band size enclosing the “true” location. The band size was further detailed via a mathematical function that describes the shape and extent of the 3D G-band. The function might be simplified into a cylinder function when describing a 3D Probability band that does not “slim” in the middle along the line.

In implementation, the probability bands for centerlines must be transformed into utility circumferences. This is because GIS models 3D utility pipes/lines as lines that correspond to utility centerlines. Figure 5 illustrates this transformation needs for circular shape utilities. The size for the transformed band is the sum of the pipe radius and the model band size. This transformation works with cross-sectional dimension and is straightforward. The size of a probability range that is linear (1D), radial (2D), or 3D G-band, is quantified in the unit of standard deviation (SD), a commonly used statistical index for positional accuracy. Such a mechanism establishes a direct link between positional accuracy models (typically described via the deviation of true location from observed location) and uncertainty models (typically expressed as the probability of being true, e.g. the probability of enclosing the true utility location). Creating uncertainty models via 3D probability eclipses/spheres and probability G-bands for utility points and lines facilitated the adoption of proximity (buffer) process of GIS and formed the foundation for visualizing and analyzing positional uncertainties in utility lines and points.

The dynamic nature of the uncertainty model construction approach is highlighted in Figure 6 that also illustrates the system architecture as well as the flow chart in managing the uncertainties of geospatial utilities data in a synergetic geospatial AR environment. The system is composed of three modules. The Uncertainty Modeling Module contains accuracy models, uncertainty models, the link between data lineage and accuracy, and the link between accuracy and uncertainty. The Open Source <UML> & <XML> Module includes data models employed to facilitate the management and sharing of geospatial utility data with downstream engineering applications. The Visualization and Proximity Analysis Module focuses on visualizing invisible utilities and the excavator movement, and analyzing the proximity between a digging implement (e.g. the excavator) and buried utilities, given inherent uncertainties in the utility location and excavator movement. The workflow (indicated by arrows in Figure 6) is such that: (1) utility data is enhanced with data lineage information (based on an expanded version of Table 1), (2) based on the links between data lineage and accuracy, and between accuracy and uncertainty, utility data is enhanced with uncertainty information, e.g. probability bands, and (3) uncertainty-aware utility data feeds into the geospatial AR for visualization and proximity analysis. Throughout the workflow, probability bands are derived rather than stored as properties of the corresponding utilities. This dynamic nature allowed easy update of the uncertainty model as well as extension and expansion. Any changes/updates in the uncertainty model and/or data lineage are automatically updated during future data utilization as utility data is extracted. For instance, the data lineage of a utility might change from Quality Group B to A. Simply updating the data lineage information automatically takes care of the uncertainty bands. Similarly, when the uncertainty model of a specific type of data lineage is changed, the uncertainty bands for all utilities that have that particular type of data lineage are automatically updated. Such a design is similar to the CASCADE rules in databases to ensure data integrity and consistency (Date 2000).

Uncertainty-aware geospatial data model for utilities

To facilitate the sharing of uncertainty-aware geospatial utility data in downstream applications, the authors created an uncertainty-aware utility data model illustrated in Figure 7. A solid box represents a class. A line represents an association between two classes. Special symbols such as diamonds and triangles are added to lines to indicate specific associations. For instances, the inclusion of a solid diamond represents composition – the relationship of “owns”; the inclusion of a hollow diamond represents aggregation – the relationship of “has”; and the inclusion of a hollow triangle represents generalization/inheritance - the relationship of “is a”. In this model, the utility network is composed of utility lines, straight line segments that are spatially constructed by connecting ending points, or utility vertices. The uncertainty model is specialized into linear uncertainty model for utility lines and ellipsoid model for utility vertices. The linear uncertainty model is further specialized into 3D G-band model and 3D cylinder model to suit different application needs. All uncertainty models are quantitatively (e.g. probabilities and band sizes derived) associated with their corresponding utility objects (e.g. utility lines and vertices) by referring to the linkage model. Such a model echoes the dynamic nature highlighted in Figure 6.

The authors explored XML based data schemes such as the Geography Markup Language (GML) (Burggraf 2006) and CityGML, an extension to GML for the built environment (Kolbe et al. 2008); and created an XML-based model (Figure 8) as an implementation of the uncertainty-aware utilities data model in Figure 7. The newly created XML schema is an open source data model that is flexible and extensible to facilitate modeling, sharing, exchange, and integration of geospatial data. In the newly created model, each utility line is composed of straight line segments that are fixed in the space via their two ending points. A vertex is only needed when the utility changes direction, e.g. a bend. All spatial and non-spatial lineage information is inherent in utility vertices and lines to dynamically construct probability 3D-bands for uncertainty-aware visualization and proximity monitoring in excavation, and the sharing of uncertainty-aware data in downstream applications.

Geospatial augmented reality environment for visualization and proximity analysis

In order to visualize uncertainty-aware geospatial data in Augmented Reality, the data must first be converted to a format suitable for 3D graphical visualization. More importantly, the geodata accuracy and its uncertainty must be graphically characterized and displayed for it to be useful in excavator operation and control. This research developed methods to characterize utility geodata in terms of its lineage and accuracy. Together, the accuracy and lineage help characterize the errors of a geodata set and the reliability that can be associated with its source. This information can be usefully exploited during excavation by displaying not only the expected locations of utilities to an operator, but also the degree of uncertainty (or “buffer”) associated with the expected locations in the form of a “halo”. In a 2D projection, the buffer calculated by interpreting the geodata’s lineage and accuracy is represented as a “band” whose width represents the uncertainty associated with the utility’s location. In a 3D projection, the buffer is identified by increasing the diameter of the cylindrical geometry representing the utility line.

The proximity analysis included three scenarios illustrated in Figure 9, with increased capabilities of handling positional uncertainty. Figure 9(a) illustrates a deterministic proximity measure that is the closest distance between an underground utility and the digging implement, e.g. the excavator bucket, assuming both the utility pipe and the excavator bucket are at their “true” locations with no positional uncertainty. Figure 9(b) introduces the positional uncertainty in the geospatial utility data, but the excavator bucket is still deterministic. Figure 9(c) introduces the positional uncertainty in the excavator bucket in addition to the positional uncertainty in the utility location. For the illustration purpose, both Figure 9(b) and (c) are presented in 2D.

When the uncertainty in the bucket location is also introduced as in Figure 9(c), similar analyses can be carried out with corresponding interpretations. The difference is that instead of a single 3D geometry representing the bucket, the analysis applies to a series of 3D geometries representing the probability bands of the bucket. Also, the probability to be used is the product of the probabilities in the corresponding utility and excavator bucket bands. For instance, assuming in Figure 10, the bucket band has a 60% probability and one interpretation is that there is at most a 6% ((100%-85%)*(100%-60%)) probability that the excavator WILL hit the utility. Compared to underground utilities, the uncertainty associated with the bucket is significantly smaller. This is because most of the time, the bucket is above ground or has a line of sight. Thought it could be outside the operators view (e.g. blocked by the terrain or cabin), sensing technologies such as GPS, video cameras, and accelerometers can accurately track its location and movement. Thus, the uncertainties associated with the bucket might be ignored in visualization and proximity analysis, as illustrated in Figure 9(a) and Figure 9(b). When the utility data is acquired via highly accurate sensing technologies such as RTK GPS with an uncertainty at the same magnitude to the uncertainty of bucket, then the uncertainty associated with the bucket matters and shall be incorporated in the analysis, as illustrated in Figure 9(c).