Skip to main content

Advertisement

Visualization in Engineering
Visualization in Engineering Cover Image

Data analysis and visualization for the bridge deck inspection and evaluation robotic system

Visualization in Engineering volume 3, Article number: 6 (2015) Cite this article

Article metrics

  • 4032 Accesses

  • 12 Citations

Abstract

Background

Bridge deck inspection is essential task to monitor the health of the bridges. Condition monitoring and timely implementation of maintenance and rehabilitation procedures are needed to reduce future costs associated with bridge management. A number of Nondestructive Evaluation (NDE) technologies are currently used in bridge deck inspection and evaluation, including impact-echo (IE), ground penetrating radar (GPR), electrical resistivity (ER), ultrasonic surface waves (USW) testing, and visual inspection. However, current NDE data collection is manually conducted and thus faces with several problems such as prone to human errors, safety risks due to open traffic, and high cost process.

Methods

This paper reports the automated data collection and analysis for bridge decks based on our novel robotic system which can autonomously and accurately navigate on the bridge. The developed robotic system can lessen the cost and time of the bridge deck data collection and risks of human inspections. The advanced software is developed to allow the robot to collect visual images and conduct NDE measurements. The image stitching algorithm to build a whole bridge deck image from individual images is presented in detail. The ER, IE and USW data collected by the robot are analyzed to generate the corrosion, delamination and concrete elastic modulus maps of the deck, respectively. These condition maps provide detail information of the bridge deck quality.

Conclusions

The automated bridge deck data collection and analysis is developed. The image stitching algorithm allowed to generate a very high resolution image of the whole bridge deck, and the bridge viewer software allows to calibrate the stitched image to the bridge coordinate. The corrosion, delamination and elastic modulus maps were built based on ER, IE and USW data collected by the robot to provide easy evaluation and condition monitoring of bridge decks.

Background

The condition of bridges is critical for the safety of the traveling public and economic vitality of the country. There are many bridges through the U.S. that are structurally deficient or functionally obsolete (ASCE 2009). Condition monitoring and timely implementation of maintenance and rehabilitation procedures are needed to reduce future costs associated with bridge management. Application of nondestructive evaluation (NDE) technologies is one of the effective ways to monitor and predict bridge deterioration. A number of NDE technologies are currently used in bridge deck evaluation, including impact-echo (IE), ground penetrating radar (GPR), electrical resistivity (ER), ultrasonic surface waves (USW) testing, visual inspection, etc. (Gucunski et al. 2010; Wang et al. 2011). For a comprehensive and accurate condition assessment, data fusion of simultaneous multiple NDE techniques and sensory measurements is desirable. Automated multi-sensor NDE techniques have been proposed to meet the increasing demands for highly-efficient, cost-effective and safety-guaranteed inspection and evaluation (Huston et al. 2011).

Automated technologies have gained much attention for bridge inspection, maintenance, and rehabilitation. Mobile robotic inspection and maintenance systems are developed for vision based crack detection and maintenance of highways and tunnels (Velinsky 1993; Lorenc et al. 2000; Yu et al. 2007a). A robotic system for underwater inspection of bridge piers is reported in (DeVault 2000). An adaptive control algorithm for a bridge-climbing robot is developed (Liu et al. 2013). Additionally, robotic systems for steel structured bridges are developed (Wang and Xu 2007; Mazumdar and Asada 2009; Cho et al. 2013). In one case, a mobile manipulator is used for bridge crack inspection (Tung et al. 2002). A bridge inspection system that includes a specially designed car with a robotic mechanism and a control system for automatic crack detection is reported in (Lee et al. 2008a; Lee et al. 2008b; Oh et al. 2009). Similar systems are reported in (Lim et al. 2011; Lim et al. 2014; Liu et al. 2013; Prasanna et al. 2014) for vision-based automatic crack detection and mapping and (Yu et al. 2007b) to detect cracks on the bridge deck and tunnel. Edge/crack detection algorithms such as Sobel and Laplacian operators are used. Robotic rehabilitation systems for concrete repair and automatically filling the delamination inside bridge decks have also been reported in (Chanberlain and Gambao 2002).

Difference to all of the above mentioned works, our paper focus on the bridge deck data analysis which is collected by our novel robotic system integrated with advanced NDE technologies. The developed data analysis algorithms allows the robot to build the entire bridge deck image and the global mapping of corrosion, delamination and elastic modulus of the bridge decks. These advanced data analysis algorithms take into account the advantages of the accurate robotic localization and navigation to provide the high-efficient assessments of the bridge decks.

The paper is organized as follows. In the next section, we describe the robotic data collection system and coordinate transformation. In Section “Methods” we present the image stitching algorithm and bridge deck viewer/monitoring software, and then we present NDE methods including the ER, IE and USW methods and analysis. In Section “Results and discussion” we present and discuss results of the condition maps of some bridge decks. Finally, we provide conclusions from the current work and discuss the future work in Section “Conclusions”.

The bridge robotic inspection and evaluation system

Robot navigation on the Bridge

Figure 1 illustrates the robot navigation scheme during the bridge deck inspection. For a straightline bridge, the bridge deck area is of a rectangular shape. To cover the desired deck area as shown in Figure 1, three GPS points are first obtained at the rectangle corners such as points A, E, and F. Using the GPS points of these three corners, the zigzag-shape robot motion trajectories (with interpolated waypoints B, C, and D) are computed by the trapezoidal decomposition algorithm (LaValle 2006), as the arrows indicate in the figure. The robot motion to cover the inspection area consists of linear and omni trajectories. The linear motion control algorithm (La et al. 2013a) allows the robot to follow the straight-line precisely to collect the image and NDE data. At the end of each straightline, the omni-motion control algorithm (La et al. 2013a) is used to navigate the robot safely and to turn around sharply.

Figure 1
figure1

Schematic of the robot motion planning on the bridge deck.

Full size image

We demonstrated the robotic system for the inspection of the highway bridges in ten different states in USA such as Illinois, Virginia and New Jersey in 2013 and 2014 as shown in Figure 2. Figure 3 shows the robot trajectory on the bridge deck during the NDE data collection. To cover a half of the bridge width, the robot needs to conduct three scans where each scan covers the width of 2 m on the bridge. The bottom figure in Figure 3 shows the robot trajectory overlaid with the bridge deck image. The comparison of the extended Kalman filter (EKF)-based localization (La et al. 2013a) and the odometry-only trajectory clearly demonstrates that the EKF-based localization outperforms the odometry-only trajectory. For motion control performance, the virtual robot trajectory is plotted in the figure, and we can see that the robot follows the virtual robot closely.

Figure 2
figure2

Robot deployment for inspection of bridges in Illinois (Figure-Top-Left), Virginia (Figure-Top-Right) and New Jersey (Figure-Bottom), USA in 2013 and 2014.

Full size image
Figure 3
figure3

Top: Autonomous robot trajectory profile on the Haymarket highway bridge, Haymarket, Virgina. Bottom: Trajectories overlaid the bridge.

Full size image

Data collection

The robotic system is integrated with several nondestructive evaluation (NDE) sensors including Ground Penetrating Radar (GPR), acoustic array consisting of Impact Echo (IE) and Ultrasonic Surface Waves (USW), Electrical Resistivity (ER), and high resolution cameras as shown in Figure 2. The robot autonomously maneuvers on the bridge based on the advanced localization and navigation algorithm reported in the previous works (La et al. 2013b; Gucunski et al. 2013; La et al. 2013a).

The data (GPR, IE, USW, ER and images) collection is fully autonomous. It can be done in either the full data collection mode, or the scanning mode. In the full data collection mode, the robot moves and stops at prescribed increments, typically 30 to 60 cm, and deploys the sensor arrays to collect the data. In the scanning mode, the system moves continuously and collects data using only the GPR arrays and digital surface imaging. The robot can collect data on approximately 300 m 2 of a bridge deck area per hour. In the continuous mode, the production rate is more than 1,000 m 2 per hour.

The NDE data collection system is run on two Windows operating computers and communicate with the robot Linux operating computer through the serial communication protocols. The NDE software is developed by utilizing Qt development kit and Cpp to enable the robot to collect and monitor the data simultaneously. The software architecture is designed based on multi-thread programming. The software consists of five slave threads and one master thread. The master thread controls the entire user interface. The slave threads are:

Robot thread which communicates with LinuxWindowsSerial program in the robot computer (Linux/ROS) using RS-232 protocol and sends position information of robot to the user interface;

Acoustic thread which controls the data acquisition of the acoustic device consisting of IW and USW using USB protocol and logs the time series data;

GPR thread which communicates with IDS vendor software using TCP/IP protocol. GPR thread is able to start, stop, and receive stream data from the GPR acquisition device;

Camera thread which uses the Canon SDK protocol to control the camera such as triggering to shoot, changing lighting parameters, and downloading collected images;

Electrical Resistivity (ER) thread which communicates with Resipod sensor using RS-232 protocol and logs the resipod data.

Overall, the robot thread controls the other threads to trigger and sync the data collection system. During the operation, the robot thread waits for a serial message from robot Linux computer. When the serial message is received, it will be used to command the other NDE thread to perform the data collection. The data flow of the NDE GUI is shown in Figure 4. The serial message also consists of the robot position and orientation, and number of line inspections and their indices.

Figure 4
figure4

The GUI for NDE data collection and monitoring of a bridge near Chicago, Illinois, USA.

Full size image

NDE coordinate transformations

This subsection presents coordinate transformations in the robotic system which allows the NDE data analysis and mapping process. Since the relationship between the GPR, Acoustic, ER coordinates and the robot coordinate are fixed, we just present the transformation from camera frame to robot frame which allows the image stitching and crack mapping process to map from the local image coordinate to the world coordinate.

The system involves four coordinate systems as shown in Figure 5. They are: image coordinate system (F I ), camera coordinate system (F C ), robot coordinate system (F R ) and world coordinate system (F W ). To transform the image coordinate system (F I ) to the world coordinate system (F W ), we need to implement the sequential transformations: $(X_{im}, Y_{im}) \stackrel {{{~}^{I}T_{C}}}{\rightarrow } (X_{c}, Y_{c}) \stackrel {{{~}^{C}T_{R}}}{\rightarrow } (X_{r}, Y_{r}) \stackrel {{{~}^{R}T_{W}}}{\rightarrow } (X_{w}, Y_{w})$ .

Figure 5
figure5

Coordinate systems in the robotic bridge deck inspection system.

Full size image

The intrinsic and the extrinsic matrices are obtained once the calibration is finished. The intrinsic matrix consisting of focal length (f), skew value (s) and the origin of image coordinate system (x im (0),y im (0)) is described in Equ. (1).

$$ P=\left[ \begin{array}{rrrr} sf&0&x_{im}(0)&0\\ 0&f&y_{im}(0)&0\\ 0&0&1&0 \end{array} \right] $$
((1))

The extrinsic matrix consists of rotation and translation parameters as in Equ. (2).

$$ M=\left[ \begin{array}{rr} R&T\\ \textbf{0}&1 \end{array} \right]_{4 \times 4} $$
((2))

here R is a 3×3 rotation matrix which can be defined by the three Euler angles (Heikkila 2000), and T= [ t x ,t y ,t z ]T is the translation between two frames.

We have the following transformation from the image coordinate to the camera coordinate.

$$ \underbrace{\left[ \begin{array}{c} x_{im} \\ y_{im} \\ 1 \end{array} \right]}_{I} = {^{I}T_{C}} \times \underbrace{\left[ \begin{array}{c} x_{c} \\ y_{c} \\ z_{c} \\ 1 \end{array} \right]}_{C}, $$
((3))

here I T C is the transformation matrix from the image coordinate to the camera coordinate, and I T C =P M. Now we can find the camera coordinate corresponding to the image coordinate using pseudo-inverse as

$$ C = ({^{I}T^{'}_{C}} {^{I}T_{C}})^{-1} {^{I}T^{'}_{C}} I. $$
((4))

here $^{I}T^{'}_{C}$ is the transpose of I T C .

To find the transformation from (F C ) to (F R ), we use the static relationship between these two coordinates. Namely, the relationship between the camera and robot coordinate systems is fixed because the camera orientation is fixed (see Figure 5). Therefore the transformation from (F C ) to (F R ) can be obtained by measuring the physical offset distances between the robot center the camera pose. This transformation can be described as: $[x_{r}^{tran} \ y_{r}^{tran}]^{T} = [x_{c} \ y_{c}]^{T} - [\Delta x_{\textit {cr}} \ \Delta y_{\textit {cr}}]^{T}$ , here Δ x cr and Δ y cr are the offset distances between the camera coordinate and the robot coordinate along x and y, respectively.

Finally, to find the transformation from (F R ) to (F W ) we use the following relationship:

$$ \left[\begin{array}{c} x_{w} \\ y_{w} \\ 1 \end{array} \right] = {^{R}T_{W}} \times \left[ \begin{array}{c} x_{r}^{tran} \\ y_{r}^{tran} \\ 1 \end{array} \right], $$
((5))

here the transformation matrix R T W is defined as

$$ {^{R}}T{_{W}}= \begin{bmatrix} cos(\theta_{r}) & -sin(\theta_{r}) & x_{r} \\ sin(\theta_{r}) & cos(\theta_{r}) & y_{r} \\ 0 & 0 & 1 \end{bmatrix} $$
((6))

where (x r ,y r ,θ r ) are the position and heading of the robot obtained by the Extended Kalman Filter (EKF) (La et al. 2013a).

Methods

Bridge deck image stitching and monitoring

Bridge deck image stitching

For the ease of bridge deck inspection and monitoring, we combine taken photos into a single large image as shown in Figure 6. This is a specific case of the general image stitching problem. In image stitching problem, camera motion is unknown and not constrained and intrinsic camera parameters can change between the given images. In our specific problem of bridge deck surface image stitching, we benefit from constraints we know to exist due to the nature of the problem and the setup of the hardware system. We have two identical cameras that simultaneously take images of different but overlapping areas of the bridge. Also the robot’s estimated position each time a photo is taken is known with the help of onboard sensor fusion based EKF (La et al. 2013a).

Figure 6
figure6

The result of image stitching from 5 images.

Full size image

The two facing-down surface cameras (Canon EOS Rebel T3i, 16 MPixel) are mounted on two computer-controlled pneumatic rods (Figure 5). The resolution of the cameras is up to 5184 ×3456 pixels. These two surface cameras are extended out of the robot footprint area when the robot starts the data collection. Each of the cameras covers an area of a size of 1.83m×0.6m. The images simultaneously collected by these two cameras have a about 30 % overlap area that is used for image stitching as shown in Figure 7. Use of flash can be necessary to obtain shadow free and well-exposed photos and in our system cameras are set to auto-exposure and auto-flash modes. Intrinsic calibration of the cameras is made separately and the camera parameters are used to undistort the acquired images. Extrinsic calibration of the camera pair consists of finding the relative location of left camera with respect to right camera.

Figure 7
figure7

Projection of mobile robot coordinate and two camera field of views (FoVs) to an XY plane.

Full size image

Motion estimation

Based on the constraints imposed by the setup, we estimate the motion as a 2D rigid motion model; translation on the x−y plane and rotation around z axis. Robot and image coordinate systems can be mapped to each other by -90 degrees rotation. Robot x−a x i s corresponds to negative y−a x i s of image coordinates, robot y−a x i s corresponds to negative x−a x i s of image coordinates (7), and the constant factor resolution is the pixels per meter ratio (R im ).

$$ \left\{ \begin{array}{l} x_{im} = -y_{r} R_{im} \\ y_{im} = -x_{r} R_{im} \end{array} \right. $$
((7))

Sparse feature-matching and image-to-image matching procedures (Forsyth and Ponce 2003; Brown and Lowe 2007) are used to estimate the camera motion incrementally. We pose the problem as a template-matching problem that tries to find the location of the overlapping area of the images inside the other image. This way we perform left-to-right and frame-to-frame matching. Robot motion estimate gives us the rough location of overlapping area for consecutive frames. Rough overlapping area for left-to-right images matching is fixed since the camera locations on the platform are fixed. Knowing the overlapping area, appearance-based template matching can give finer estimation of the camera motion. If the robot motion estimation is not accessible or not accurate enough, overlapping area can be searched over the whole image, which is a more time consuming process.

To reduce the tremendous amount of data to be processed, we resort to multi-resolution pyramidal search method (Forsyth and Ponce 2003), where we search for a larger motion range in lower resolution image and reduce the possible motion range for higher resolution image. Due to possible large illumination and reflection changes between different frames, we use image comparison method Normalized Correlation Coefficient (8) that is less illumination independent. In Equ. (8) correlation coefficient for each location x,y is denoted by R(x,y), where search image region is I, template image that is searched is T and normalized versions of them are I ′ and T ′ respectively. We compare the grayscale versions of the images to get rid of any white-balance effects in different images.

$$ \left\{ \begin{array}{l} R(x,y) = \frac{\sum_{x^{\prime}, y^{\prime}}[T^{\prime}(x^{\prime}, y^{\prime})I^{\prime}(x+x^{\prime}, y+y^{\prime})]}{\sqrt{\sum_{x^{\prime}, y^{\prime}}T^{\prime}(x^{\prime}, y^{\prime})^{2}\sum_{x^{\prime}, y^{\prime}}I^{\prime}(x+x^{\prime}, y+y^{\prime})^{2}}}\\ I^{\prime}(x+x^{\prime}, y+y^{\prime}) = I(x+x^{\prime}, y+y^{\prime}) - \\ \frac{\sum_{x^{\prime\prime}, y^{\prime\prime}}I(x+ x^{\prime\prime}, y+ y^{\prime\prime})}{w.h}\\ T^{\prime}(x^{\prime}, y^{\prime}) = T(x^{\prime}, y^{\prime}) - \frac{\sum_{x^{\prime\prime}, y^{\prime\prime}}T(x^{\prime\prime}, y^{\prime\prime})}{w.h}. \end{array} \right. $$
((8))

here, w and h are the width and height of the image I, respectively.

Exposure compensation and blending

Exposure compensation step obtains the most blending exposures for each image by selecting the suitable brightness ratio of overlapping area between images. Then when combining existing image and the new arrived image, we are performing an image-blending step to remove the shadows in the image (9). If the new arriving pixel is considerably brighter than the existing pixel in the same location, we replace the pixel with the new one. A threshold value of 0.7 is used for th to indicate being considerable is brighter than corresponding pixel. Gaps in the region formed by the pixels to be used from new image are filled using a 2D median filter of size 7×7 pixels. This ensures the completeness of the shadow removal region.

$$ I(x,y)=f(x)\left\{ \begin{array}{l} I_{2}(x,y), I_{2}(x,y)*th > I_{1}(x,y) \\ I_{1}(x,y), else. \end{array} \right. $$
((9))

Bridge deck monitor

The bridge deck viewer (BDV) software is developed using Java language to support the bridge engineer to monitor the bridge decks in an efficient way. The stitched images are first loaded and then calibrated to map to the bridge coordinate as Figure 8. The BDV software can find the crack locations on the surface of bridge in the viewing image and allows to mark them for the next view or any purpose by left mouse click on that locations. The details of the crack detection algorithm is reported in (La et al.). The BDV also shows the notification about the position of the cracks. As can be seen in Figure 9, the flags appear at the crack locations corresponding with coordinates (x, y) on the bridge deck.

Figure 8
figure8

One of New Jersey bridges is loaded and calibrated by the BDV software.

Full size image
Figure 9
figure9

Zoom-in at some crack locations of a bridge in New Jersey as shown in Figure 8 .

Full size image

Additionally, the BDV software allows to measure the distance of crack on the deck by right mouse click on the starting point and drag the hold right mouse to the last point. A line that connects the starting point and ending point appears to show the length of the crack as shown in Figure 9. Figure 10 shows the image stitching results of two bridges in Virginia and Illinois states, respectively. The stitched image is calibrated to the bridge deck coordinate to allow the ease of condition assessments. Each stitched image has very high resolution of more than 3 Gigapixel. This allows the bridge engineer to zoom in at every specific locations to monitor the cracks even with millimeter size on the deck.

Figure 10
figure10

The result of image stitching results of two bridges: (a) Haymarket bridge in Virginia state, stitched image from 200 individual images; (b,c) Chicago avenue bridge in Illinois state, stitched image from 720 individual images.

Full size image

NDE methods and analysis

This section presents NDE methods including electrical resistivity (ER), impact-echo (IE) and ultrasonic surface waves (USW). The robot is equipped with four ER probes (Figure 11) and two acoustic arrays, and each array can produce 8 IE and 6 USW data set as shown in Figure 12. These raw data sets are collected by the robot at every two feet (60 cm) on the bridge deck.

Figure 11
figure11

Principle of electrical resistivity (ER) measurement using Wenner probe.

Full size image
Figure 12
figure12

Acoustic/seismic array sensor is developed and integrated with the robot to collect IE and USW data.

Full size image

Electrical resistivity (ER) data analysis

The corrosive environment of concrete and thus potential for corrosion of reinforcing steel can be well evaluated through measurement of ER of concrete. Dry concrete will pose a high resistance to the passage of current, and thus will be unable to support ionic flow. On the other hand, presence of water and chlorides in concrete, and increased porosity due to damage and cracks, will increase ion flow, and thus reduce resistivity. It has been observed that a resistivity less than 5 k Ω can support very rapid corrosion of steel. In contrast, dry concrete may have resistivity above 100 k Ω. Research has shown in a number of cases that ER of concrete can be related to the corrosion rates of reinforcing steel. The ER surveys are commonly conducted using a four-electrode Wenner probe, as illustrated in Figure 11-Left. Electrical current is applied through two outer electrodes, while the potential of the generated electrical field is measured using two inner electrodes. From the two, ER is calculated as indicated in Figure 11-Left. The robot carries four electrode Wenner probes and collects data at every two feet (60 cm) on the deck. To create a conducted environment between the ER probe and the concrete deck, the robot is integrated with the water tank and to spray water on the target locations before deploying the ER probes for measurements as shown in Figure 11-Right.

Impact-echo (IE) data analysis

Impact-Echo (IE) is a widely used NDT method that has demonstrated to be effective in identifying and characterizing delaminations in concrete structures. Impact-Echo (IE) is an elastic-wave based method to identify and characterize delaminations in concrete structures. This method uses the transient vibration response of a plate-like structure subjected to a mechanical impact. The mechanical impact generates body waves (P-waves or longitudinal waves and S-waves or transverse waves), and surface-guided waves (e.g. Lamb and Rayleigh surface waves) that propagate in the plate. The multiple-reflected and mode-converted body waves eventually construct infinite sets of vibration resonance modes within the plate. In practice, the transient time response of the solid structure is commonly measured with a contact sensor (e.g., a displacement sensor or accelerometer) coupled to the surface close to the impact source. The fast Fourier transform (amplitude spectrum) of the measured transient time-signal will show maxima (peaks) at certain frequencies, which represent particular resonance modes as show in Figure 13.

Figure 13
figure13

(Top) IE raw data of one channel, C4, in the acoustic array in time domain, and (Bottom) After using Fast Fourier Transform (FFT) in frequency domain.

Full size image

There are different ways of interpreting the severity of the delamination in a concrete deck with the IE method. One of the ways used in this study is shown in Figure 14. A test point is described as solid if the dominant frequency corresponds to the thickness stretch modes (Lamb waves) family. In that case, the frequency of the fundamental thickness stretch mode (the zero-group-velocity frequency of the first symmetric (S 1) Lamb mode, or also called the IE frequency (f IE ). The frequency can be related to the thickness of a plate H for a known P-wave velocity C p of concrete by

$$ H = \frac{\beta_{1} C_{p}}{f_{IE}} $$
((10))
Figure 14
figure14

Grades for various degrees of deck delamination for IE method.

Full size image

where β 1 is a correction factor that depends on Poisson’s ratio of concrete, ranging from 0.945 to 0.957 for the normal range of concrete. A delaminated point in the deck will theoretically demonstrate a shift in the thickness stretch mode toward higher values because the wave reflections occur at shallower depths. Depending on the extent and continuity of the delamination, the partitioning of the wave energy reflected from the bottom of the deck and the delamination may vary. The initial or incipient delamination, described as occasional separation within the depth of the slab, can be identified through the presence of dominant frequencies associated with the thickness stretch modes from both the bottom of the deck and the delamination. Progressed delamination is characterized by a single peak at a frequency corresponding to the depth of the delamination. Finally, in case of wide or shallow delaminations, the dominant response of the deck to an impact is characterized by a low frequency response of flexural-mode oscillations of the upper delaminated portion of the deck.

Ultrasonic surface waves (USW) data analysis

The ultrasonic surface waves (USW) technique is an offshoot of the spectral analysis of surface waves (SASW) method used to evaluate material properties (elastic moduli) in the near-surface zone. The SASW uses the phenomenon of surface wave dispersion (i.e., velocity of propagation as a function of frequency and wave length, in layered systems to obtain the information about layer thickness and elastic moduli) as shown in Figure 15. A SASW test consists of recording the response of the deck, at two receiver locations, to an impact on the surface of the deck (Figure 16). The surface wave velocity can be obtained by measuring the phase difference Δ ϕ between two different sensors (sensor 1 and sensor 2) as follows,

$$ C = 2\pi f\frac{d}{\Delta \phi} $$
((11))
Figure 15
figure15

Schematic of evaluation of a layer modulus by SASW (USW) method.

Full size image
Figure 16
figure16

USW raw data of some channels in the acoustic array collected by the robot at a concrete bridge deck.

Full size image

where f is frequency, d is distance between two sensors. The USW test is identical to the SASW, except that the frequency range of interest is limited to a narrow high-frequency range in which the surface wave penetration depth does not exceed the thickness of the tested object. In cases of relatively homogeneous materials, the velocity of the surface waves does not vary significantly with frequency. The surface wave velocity can be precisely related to the material modulus, or concrete modulus in the case of bridge decks, using either the measured or assumed mass density, or Poisson’s ratio of the material. In the case of a sound and homogenous deck, the velocity of the surface waves will show little variability. An average velocity is used to correlate it to the concrete modulus. Significant variation in the phase velocity will be an indication of the presence of a delamination or other anomaly.

Ground penetrating radar

Ground penetrating Radar (GPR) is a geophysical method that uses radar pulses to image the subsurface. GPR can provide a qualitative condition assessment of bridge decks and is used as a diagnostic tool to detect apparent or suspected deterioration in an existing deck (e.g., delamination or corrosive environment), or quality assurance tool for new construction or rehabilitation. The GPR system uses high-frequency (varying from 10 MHz to 2.5 GHz) radio waves and transmits into the ground. When the wave hits a buried object such as rebars in the bridge, the receiving antenna records variations in the reflected return signal. GPR data usually consist of changes of reflection strength and arrival time of specific reflections, source wave distortion, and signal attenuation. The rebar detections are marked by hyperbolic shapes in the GPR image (see Figure 4). These hyperbolas are obtained due to the reason that the antenna transmits energy in a spatially varying pattern which can be approximated to a cone. Then, the antenna receives the reflections from the rebars.

Two IDS Hi-Bright GPR arrays are attached on the rear side of the robot (Figure 2). Each of the arrays has sixteen antennas, or two sets of eight antennas with dual polarization to obtain high spatial resolution signals. For high-efficiency inspection, the GPR system can omit the wave and collect reflective signals even the robot is in motion. It is required that the GPR antenna has to be close (1-2 cm) to the deck surface to obtain good signals. The GPR attenuation map of a bridge deck is presented in Figure 17. The color plots in this condition map demonstrates deterioration grades of the bridge deck. The GPR condition map indicates a large cluster of serious deterioration around locations at longitudinal/lateral positions (45, 10) ft and (90, 10) ft.

Figure 17
figure17

GPR attenuation mapping result of a bridge in New Jersey, USA.

Full size image

Results and discussion

To present test data and results in an effective and intuitive manner, the results of the NDE surveys using the above mentioned three technologies are presented by two-dimensional contour maps (Figure 18). On each contour map, the location of joints are marked as white lines at both ends in order to better identify dimensions of the bridges and locations of deterioration. It is important to note that unlike IE and USW, ER is sensitive to environmental conditions, such as low temperature, moisture, and the surface condition of bridge deck. Condition ratings with respect to delamination (IE) and corrosion (ER) for the section of the bridge surveyed were calculated on a scale 0 (worst) to 100 (best). Three different weight factors (100, 70, 40) are assigned to the area in different conditions and delamination and corrosion rates for each bridge which are calculated by delamination rating (D r ) and corrosion rating (C r ) in (12, 13).

$$ D_{r} = \frac{100*A_{good}+ 70*A_{fair}+40*A_{poor}}{A_{total}} $$
((12))
Figure 18
figure18

The impact-echo (IE) and ultrasonic surface waves (USW) condition maps of a bridge deck in Illinois, USA, based on data collected by the developed robot system. The robot covers a half of the bridge with three scans (6ft width/scan) and 190ft along within 65 minutes.

Full size image

where A good ,A fair ,A poor are the areas in “Good”, “Fair”, and “Poor” conditions, respectively, which are described as follows:

- Good condition is established by measuring strong reflections from the bottom of the deck; - Fair condition is characterized by shallow depth delamination induced reflections, causing a shift in the response spectrum towards higher frequencies; - Poor condition is characterized by a single peak at a frequency corresponding to the depth of the delamination; - Serious condition is evidenced by a low frequency response of flexural mode oscillations of the upper, delaminated, portion of the deck.

$$ C_{r} = \frac{100*A_{low}+ 70*A_{moderate}+40*A_{high}}{A_{total}} $$
((13))

where A low ,A moderate ,A high are the areas with ER in their ranges of: 0-25 k Ω, 25-40 k Ω, and greater than 40 k Ω, respectively, and A total is the total surveyed area.

The condition with respect to delamination is illustrated in Figure 18-Top. Hot colors (reds and yellows) are an indicator of delamination, while cold colors (greens and blues) are an indicator of likely fair or good conditions. The grades follow the previously provided descriptions and illustration in Figure 14. As can be observed in Figure 18-Top, the deck is generally in a sound condition (less delamination), with some signs of incipient/initial delamination indicated by green, and very few signs of progressed delamination indicated by yellow colors. There were only several points where the delamination was identified as fully developed (red colors). The distribution of areas in the good, fair/poor condition, and severe/serious is shown in Figure 19-Top. Also, a condition rating with respect to delamination, on a scale 0 to 100, was calculated as 92. The rating is calculated as a weighted average of the areas in different states of delamination. In the condition rating formula, the percent of the area in a good/sound condition is multiplied by 100, the percent of area in fair by 70, poor by 40, and the area in severe/serious condition by 0. The concrete quality of the deck, described by its concrete modulus map, is also shown in Figure 18-Bottom. The modulus varies, in general between about 3500 and 6000 ksi, with an average value of 4860 ksi and the standard deviation of 800 ksi. However, there are also areas where the modulus drops to or below 3000 ksi, which is often associated with the formation of delamination. The modulus distribution does not vary much between the surveyed sections. The statistics of the distribution of concrete modulus is presented in Figure 19-Bottom.

Figure 19
figure19

Top: delamination assessment of the surveyed sections of a bridge deck in Figure 18 (Top). Percentage of deck area in various states of delamination. Bottom: concrete quality assessment of the surveyed sections of a bridge deck in Figure 18 (Bottom). Average modulus and modulus variability.

Full size image

The result of ER for a bridge in Illinois state are presented as contour maps in Figure 20 with corrosion ratings. We can see that the deck is in a good condition with respect to corrosion since there are only a few spots in poor or serious conditions.

Figure 20
figure20

Electrical resistivity (ER) condition maps for a bridge deck in Illinois state in April 2014, USA. The robot covers a half of the bridge with three scans (6ft width/scan) and 175ft along within 50 minutes.

Full size image

Conclusions

The bridge deck data collection and analysis has been reported in this paper. Several challenging problems of data collection software, image stitching, ER, IE and USW analysis have been tackled. The image stitching algorithm allowed to generate a very high resolution image of the whole bridge deck, and the bridge viewer software allows to calibrate the stitched image to the bridge coordinate. The corrosion, delamination and elastic modulus maps were built based on ER, IE and USW data collected by the robot to provide easy evaluation and condition monitoring of bridge decks. Extensive testings and deployments of the proposed system on a number of bridges proved the efficiency of the new approach for bridge deck inspection and evaluation.

In the future work we will include development of a fusion algorithm for the NDE sensor and camera data for a more comprehensive and intuitive bridge deck condition assessment data presentation.

References

  1. ASCE (2009). 2009 Report Card for Americas Infrastructure. Technical report, American Society of Civil Engineers http://www.infrastructurereportcard.org.

  2. Brown, M, & Lowe, DG (2007). Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 74(1), 59–73.

  3. Chanberlain, DA, & Gambao, E (2002). A robotic system for concrete repair preparation. IEEE, Robotics & Automation Magazine, 9(1), 36–44.

  4. Cho, KH, Kim, HM, Jin, YH, Liu, F, Moon, H, Koo, JC, Choi, HR (2013). Inspection robot for hanger cable of suspension bridge: Mechanism design and analysis. IEEE/ASME Transactions on Mechatronics, 18(6), 1665–1674.

  5. DeVault, JE (2000). Robotic system for underwater inspection of bridge piers. IEEE Instrumentation Measurement Magazine, 3(3), 32–37.

  6. Forsyth, DA, & Ponce, J. (2003). Computer Vision: A Modern Approach. Upper Saddle River, NJ: Prentice Hall.

  7. Gucunski, N, Romero, F, Kruschwitz, S, Feldmann, R, Abu-Hawash, A, Dunn, M (2010). Multiple complementary nondestructive evaluation technologies for condition assessment of concrete bridge decks. Transportation Research Record, 2201, 34–44.

  8. Gucunski, N, Maher, A, Basily, BB, La, HM, Lim, RS, Parvardeh, H, Kee, SH (2013). Robotic platform rabit for condition assessment of concrete bridge decks using multiple nde technologies. Journal of Croatian Society for Non Destructive Testing, 12, 5–12.

  9. Heikkila, J (2000). Geometric camera calibration using circular control points. IEEE Transactions on Pattern Analysis and Machine, 22(10), 1066–1077.

  10. Huston, D, Cui, J, Burns, D, Hurley, D (2011). Concrete bridge deck condition assessment with automated multisensor techniques. Structure and Infrastructure Engineering, 7(7–8), 613–623.

  11. La, HM, Lim, RS, Basily, BB, Gucunski, N, Yi, J, Maher, A, Romero, FA, Parvardeh, H (2013a). Mechatronic systems design for an autonomous robotic system for high-efficiency bridge deck inspection and evaluation. IEEE/ASME Transactions on Mechatronics, 18(6), 1655–1664.

  12. La, HM, Lim, R, Basily, B, Gucunski, N, Yi, J, Maher, A, Romero, F, Parvardeh, H (2013b). Autonomous robotic system for high-efficiency non-destructive bridge deck inspection and evaluation. In Proc. IEEE Conf. Automat. Sci. Eng, Madison, WI, (pp. 1065–1070).

  13. La, HM, Gucunski, N, Kee, SH, Yi, J, Senlet, T, Nguyen, L, 2014. Autonomous robotic system for bridge deck data collection and analysis. In Intelligent Robots and Systems (IROS 2014), 2014 IEEE/RSJ International Conference on. doi:10.1109/IROS.2014.6.942821, (pp. 1950–1955).

  14. LaValle, SM. (2006). Planning Algorithms. New York, NY: Cambridge University Press. Also available at http://planning.cs.uiuc.edu/.

  15. Lee, JH, Lee, JM, Park, JW, Moon, YS (2008a). Efficient algorithms for automatic detection of cracks on a concrete bridge. In Proc. 23rd Int. Tech. Conf. Circ./Syst., Comp. Communicat, Yamaguchi, Japan, (pp. 1213–1216).

  16. Lee, JH, Lee, JM, Kim, HJ, Moon, YS (2008b). Machine vision system for automatic inspection of bridges. In Cong. Image Sig. Proc. vol. 3, Sanya, China, (pp. 363–366).

  17. Lim, RS, La, HM, Shan, Z, Sheng, W (2011). Developing a crack inspection robot for bridge maintenance. In Proc. IEEE Int. Conf. Robot. Autom, Shanghai, China, (pp. 6288–6293).

  18. Lim, RS, La, HM, Sheng, W (2014). A robotic crack inspection and mapping system for bridge deck maintenance. Automation Science and Engineering, IEEE Transactions on, 11(2), 367–378.

  19. Liu, Y, Dai, Q, Liu, Q (2013). Adhesion-adaptive control of a novel bridge-climbing robot. In Cyber Technology in Automation, Control and Intelligent Systems (CYBER), 2013 IEEE 3rd Annual international conference on. doi:10.1109/CYBER.2013.6.705428, (pp. 102–107).

  20. Lorenc, SJ, Handlon, BE, Bernold, LE (2000). Development of a robotic bridge maintenance system. Automation in Construction, 9, 251–258.

  21. Mazumdar, A, & Asada, HH (2009). Mag-foot: A steel bridge inspection robot. In Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference on. doi:10.1109/IROS.2009.5354599, (pp. 1691–1696).

  22. Oh, JK, Jang, G, Oh, S, Lee, JH, Yi, BJ, Moon, YS, Lee, JS, Choi, Y (2009). Bridge inspection robot system with machine vision. Automation in Construction, 18, 929–941.

  23. Prasanna, P, Dana, KJ, Gucunski, N, Basily, BB, La, HM, Lim, RS, Parvardeh, H (2014). Automated crack detection on concrete bridges. Automation Science and Engineering, IEEE Transactions on, PP(99), 1–9. doi:10.1109/TASE.2014.2354314.

  24. Tung, PC, Hwang, YR, Wu, MC (2002). The development of a mobile manipulator imaging system for bridge crack inspection. Automation in Construction, 11, 717–729.

  25. Velinsky, SA (1993). Heavy vehicle system for automated pavement crack sealing. International Journal of Vehicle Design, 1(1), 114–128.

  26. Wang, X, & Xu, F (2007). Conceptual design and initial experiments on cable inspection robotic system. In Mechatronics and Automation, 2007. ICMA 2007. International conference on. doi:10.1109/ICMA.2007.4304149, (pp. 3628–3633).

  27. Wang, ZW, Zhou, M, Slabaugh, GG, Zhai, J, Fang, T (2011). Automatic detection of bridge deck condition from ground penetrating radar images. IEEE, Transactions on, Automation Science and Engineering, 8(3), 633–640.

  28. Yu, SN, Jang, JH, Han, CS (2007a). Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel. Automation in Construction, 16, 255–261.

  29. Yu, S-N, Jang, J-H, Han, C-S (2007b). Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel. Automation in Construction, 16, 255–261.

Download references

Acknowledgments

This work was partially supported by the Federal Highway Administration’s Long Term Bridge Performance (LTBP) Program. The authors would like to thank Profs. Basily Basily, Kristin Dana and Ali Maher of Rutgers University for their support for the project development. The authors are also grateful to Ronny Lim, Turgay Senlet, Hooman Parvardeh, Kenneth Lee, Prateek Prasanna of Rutgers University for their help during the system development and field testing.

Author information

Affiliations

  1. Department of Computer Science and Engineering, University of Nevada, Reno, NV 89557, USA

    • Hung Manh La
    •  & Luan Van Nguyen

  2. Department of Civil and Environmental Engineering, Rutgers University, Piscataway, NJ 08854, USA

    • Nenad Gucunski

  3. Department of Architectural Engineering, Dong-A University, Busan, Korea

    • Seong-Hoon Kee

Authors

  1. Search for Hung Manh La in:

  2. Search for Nenad Gucunski in:

  3. Search for Seong-Hoon Kee in:

  4. Search for Luan Van Nguyen in:

Corresponding author

Correspondence to Hung Manh La.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

HL developed navigation, data collection and image stitching algorithms for the robot, analyzed the results, and drafted the manuscript. NG developed ER analysis and offered suggestion and guidance to the research, and assisted overall data analysis. SK developed IE and USW evaluation and analysis for bridge deck inspection. LN developed bridge deck monitor software and assisted the robot data collection on the fields. All authors read and approved the final manuscript.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

La, H.M., Gucunski, N., Kee, S. et al. Data analysis and visualization for the bridge deck inspection and evaluation robotic system. Vis. in Eng. 3, 6 (2015) doi:10.1186/s40327-015-0017-3

Download citation

Keywords

  • Mobile robotic systems
  • Bridge deck inspection
  • Image stitching
  • Nondestructive evaluation