Combining photogrammetry and robotic total stations to obtain dimensional measurements of temporary facilities in construction field

Photogrammetric surveying has been widely applied in medical surgery, 3D modeling, engineering, manufacturing and map production. Virtual reality (VR) and augmented reality (AR) researches have resorted to photos as the most straightforward and cost-effective means for field data collection in the construction management domain (Kamat et al. 2011). Due to the computational complexity of photogrammetric surveying, construction engineering and management researchers attempted to reduce the number of photos used by imposing geometric constraints and automating the modeling process based on pattern recognition and feature detection (Golparvar-Fard et al. 2009). (El-Omari and Moselhi 2008) integrated both photogrammetry and laser scanning to reduce time required for collecting data and modeling. (Golparvar-Fard et al. 2011) used site photos to generate the point cloud, then match and pair the images to generate the as-planned and as-built models enabled by the technique of structure from motion (SfM). (Golparvar-Fard et al. 2011) further reduced the modeling time and cost by generating the point cloud from both photos and SfM. Successful point cloud applications were demonstrated in (Golparvar-Fard et al. 2011) and (Bhatla et al. 2012). The point cloud based application is aimed to reduce the effort in 3D as-built modeling. However, the model object must be stationary during the laser scanning process while removing the noise data (redundant or irrelevant information) requires considerable time and expertise. Thus, “point-cloud” based techniques are not suitable for modeling a particular moving object on a near real-time basis in the field.

Research has extended photogrammetry into videogrammetry; for instance, (Fathi and Brilakis 2012) measured dimensions of a roof using two video cameras based on SfM. However, extensive video post-processing effort is necessary to match the time stamp of each video frame recorded by each camera. More than three stationary site control points are required in each photo frame for 3D modeling. To reduce the minimal quantity of the required control points from three to two, a simplified photogrammetry-enabled AR approach (Photo-AR) was applied to assist in crane allocation and bored pile construction sequencing on a congested building site (Siu and Lu 2009,2010,2011). In short, considering a life-size moving object found in the construction field (like the rigging system used in the case study of the present research), the inclusion of multiple fixed control points in each photo frame is infeasible. The above limitations identified in current 3D as-built methods have inspired us to develop an alternative solution which directly tracks and surveys target points by use of two synchronized RTS units, thus automatically providing the geo-referenced inputs as needed for photo-based 3D modeling.

At present, using total station is the common practice to determine positions in the site space, instead of applying traverse and leveling methods in traditional surveying (Kavanagh 2009). Mainstream surveying research focuses on improving the accuracy of collected data by computing, for instance, through the least square adjustment algorithm (King 1997). On the other hand, the state-of-the-art robotic total station (RTS) adds tracking and automation capabilities to enhance positioning and surveying applications in the field, including building settlement monitoring, bridge deflection and tunnel boring machine guidance (Shen et al. 2011). Currently, the drawbacks of robotic total stations include the high investment and application costs and its limited capability to track only one point at a particular time.

Physical dimensions of static building products that are not safely assessable can be surveyed through use of photogrammetry for quality control purposes (Dai and Lu 2010). It takes significant effort to process images to build 3D models of a static object, while dynamic changes of the object’s geometric configurations over time are ignored. A Photo-AR scene, consisting of existing plant facilities plus virtual models of temporary facilities, can be linked with a scheduled event on an industrial construction project. The AR scene is instrumental in revealing and analyzing potential workface space conflicts and refining the estimate of productivity for construction scheduling. In previous research, the Photo-AR requires at least two control points with known coordinates fixed on the site explicitly visible in each photo. The scale of the AR scene can be fixed by using the two stationary control points (Siu and Lu 2009).

This paper reports our development and application of a modeling methodology for measuring physical dimensions of a moving object and checking any changes in those dimensions during a dynamic process in the construction field. On site, the stationary control points are usually located on the ground. It is difficult to include at least two ground-fixed points in each photo frame, especially when the large object being tracked is being lifted and moved from a source location to a destination location in the field. Therefore, this research is intended to model a moving object based on tracking a minimum quantity of dynamic control points on the object. As such, reliable dimensional measurements at one particular moment can be obtained with the least cost of equipment purchase and use. Photo-AR has been further enhanced through synchronizing cameras and RTS to track dynamic points on a moving object. The proposed time-dependent 3D modeling methodology is cost effective specifically for dynamic applications: physical dimensions of the moving object being modeled can be determined simply by processing photos from multiple cameras supplemented by point tracking results by two robotic total station units at a particular time. This would better cater to the application needs in industrial construction in terms of modeling dynamic temporary facilities which are critical to construction safety and productivity performances. The application background for our field experiments is given as follows.

As Canada’s leading producer of oil, gas, and petrochemicals, Alberta is home to four ethane-cracking plants, including two of the world’s largest, with combined annual capacity to produce 8.6 billion pounds of ethylene. In the foreseeable future, new refining capacity will be added to produce ethane from bitumen upgrading, which will directly source feedstock from downstream oil sands mining (Government of Alberta 2010). New construction and turnaround activities at industrial process plants consume substantial resources and involve diverse stakeholders who work closely towards delivering a project under a finite time window and a tight cost budget. In general, work items such as a pipe spool, a valve or a storage tank undergo a sequence of tasks which take place in a fabrication shop, at a module yard and on an industrial site. Each task is conducted by a specialist crew in a confined work space with the assistance of temporary facilities and equipment such as scaffolding, rigging system and cranes. A newly engineered rigging system was designed by a major industrial contractor to handle super modules with a maximum 160-ton lift capacity. The rigging frame system is made of steel and subjected to bending under loadings (Westover et al. 2012). Length-adjustable slings connect the rigging frame and an overhead plate to form a rigging system. The sling length measurements are critical to balance the frame. This ensures the load is evenly spread and carried by each sling. However, direct measurement of sling lengths such as using measurement tape is not feasible due to safety hazards and the dynamic movement of the rigging system.

In the remainder of this paper, the computing foundation of photogrammetry is briefly addressed. Then we describe the modeling procedure, the system design integrating the use of multiple cameras and two robotic total stations, and the field implementation to check sling lengths in modeling a rigging system in a module assembly yard. Field testing results are deliberated and analyzed. Conclusions are drawn based on discussions of the experimental findings and future research.

Computing foundation of photogrammetry

Three-point absolute modeling approach

This present research focuses on the implementation of close range photogrammetry in a time-dependent, dynamically-changing context. The collinearity equations, given in Eqs. (1) and (2), constitute the mathematical foundations of photogrammetry to determine (1) internal parameters of camera, (2) image and object coordinates, and (3) error adjustment. The direct linear transform algorithm was formalized by (Abdel-Aziz and Karara 1971) based on the collinearity equations, and has been further developed to simplify the transformation between the image pixel frame and the object space coordinates in digital photogrammetry (Bhatla et al. 2012; Mikhail et al. 2001; McGlone et al. (2004)). Basically, the camera’s position and orientation parameters with respect to an object space coordinate system is determined by solving six unknowns, namely: three perspective center coordinates and three orientation parameters: (X _C , Y _C , Z _C , ω, φ, κ). As two collinearity equations can be written for one particular point, relating its imaging point (x, y) in the photo frame to its three coordinates (X _P , Y _P , Z _P ) in the space frame, three different points with known (x, y) and (X _P , Y _P , Z _P ) can define six equations, thus resulting in unique solutions of six unknowns (X _C , Y _C , Z _C , ω, φ, κ) (Figure 1). After the camera’s position and orientation parameters are determined, any new point whose (x, y) in the photo frame are known while (X _P , Y _P , Z _P ) in the object space are unknown, (X _P , Y _P , Z _P ) can be expressed as functions of (x, y) by transforming Eqs. (1) and (2); if two photos taken from different perspectives both capture the same point, the position and orientation of the two camera stations are all determined, then four equations having three unknowns can be solved by least square adjustment techniques in order to derive the most likely values of the object space coordinates for the new point (Figure 2).

$\mathit{x}+\mathit{\delta x}-{\mathit{x}}_0=-{\mathit{c}}_{\mathit{x}}\frac{{\mathit{m}}_{11}\left({\mathit{X}}_{\mathit{P}}-{\mathit{X}}_{\mathit{C}}\right)+{\mathit{m}}_{12}\left({\mathit{Y}}_{\mathit{P}}-{\mathit{Y}}_{\mathit{C}}\right)+{\mathit{m}}_{13}\left({\mathit{Z}}_{\mathit{P}}-{\mathit{Z}}_{\mathit{C}}\right)}{{\mathit{m}}_{31}\left({\mathit{X}}_{\mathit{P}}-{\mathit{X}}_{\mathit{C}}\right)+{\mathit{m}}_{32}\left({\mathit{Y}}_{\mathit{P}}-{\mathit{Y}}_{\mathit{C}}\right)+{\mathit{m}}_{33}\left({\mathit{Z}}_{\mathit{P}}-{\mathit{Z}}_{\mathit{C}}\right)}$

(1)

$y+\mathit{\delta y}-y_0=-c_x\frac{m_{11}\left(X_P-X_C\right)+m_{12}\left(Y_P-Y_C\right)+m_{13}\left(Z_P-Z_C\right)}{m_{31}\left(X_P-X_C\right)+m_{32}\left(Y_P-Y_C\right)+m_{33}\left(Z_P-Z_C\right)}$

(2)

Where:

$\left[\begin{array}{ccc}\hfill {\mathit{m}}_{11}\hfill & \hfill {\mathit{m}}_{12}\hfill & \hfill {\mathit{m}}_{13}\hfill \\ \hfill {\mathit{m}}_{21}\hfill & \hfill {\mathit{m}}_{22}\hfill & \hfill {\mathit{m}}_{23}\hfill \\ \hfill {\mathit{m}}_{31}\hfill & \hfill {\mathit{m}}_{32}\hfill & \hfill {\mathit{m}}_{33}\hfill \end{array}\right]=\left[\begin{array}{ccc}\hfill cos\phi cos\kappa \hfill & \hfill cos\omega sin\kappa +sin\kappa sin\phi cos\kappa \hfill & \hfill sin\omega sin\kappa -cos\omega sin\phi cos\kappa \hfill \\ \hfill -cos\phi sin\kappa \hfill & \hfill cos\omega cos\kappa -sin\omega sin\phi sin\kappa \hfill & \hfill sin\omega cos\kappa +cos\omega sin\phi sin\kappa \hfill \\ \hfill sin\phi \hfill & \hfill -sin\omega cos\phi \hfill & \hfill cos\kappa cos\phi \hfill \end{array}\right]$

x, y, is image coordinate

x ₀ , y ₀ , is principal point coordinate

c _x , c _y , is principal distance scaled by λ in x and y directions (c _x  = cλ _x and c _y  = cλ _y )

c, is principal distance

λ, is scale factor

X _P , Y_P,Z _P , is object space coordinate of the point

X _C , Y _C , Z _C , is object space coordinate of the perspective center

ω, φ, κ, is rotated angles with respect to x, y and z axis

δx, δy, is total lens distortion in x and y directions determined from camera calibration.

Five-point relative modeling approach

When the only objective of a particular field application is to take measurements and build a three-dimensional (3D) model of an object, instead of fixing the exact position of the object in space, then the three points with known coordinates in the field space are not needed. Instead, the relative orientation parameters can be determined from the (x, y) coordinates of a minimum of five points on the object in a minimum of two photos based on Eqs. (1) and (2), without requiring the coordinates of any point in the field. The five control points give five sets of collinearity equation with five degrees of freedom. The coordinates and orientations of two cameras (X _C , Y _C , Z _C , ω, φ, κ) are determined with respect to the model coordinate system (Figure 3), whose origin o aligns with the principal point of one of the cameras. As such, the coordinates (X _P , Y _P , Z _P ) of the object in the model frame can be determined by pairing imaging points (x, y) on two photos. The 3D model in relative measurements can be built (Figure 4). A scale bar, which can be the absolute measurement of a line section on the object, is used to scale the relative measurement of a dimension of the object in an absolute unit of measurement. Elaboration of the complete mathematical algorithms for the above five-point-two-photo approach can be found in (Dai and Lu 2013).

The proposed dynamic modeling approach essentially follows the “five-point relative modeling approach” with the assistance of two synchronized RTS units, at a particular moment, a scale bar is automatically fixed on the object by tracking two points. In other words, the length of the scale bar is subject to change over time. In the long run, the proposed methodology and system design can be readily extended to implement a time-dependent “three point absolute modeling” approach by synchronizing three RTS units to track the absolute coordinates of three points on the moving object.

This research study proves the potential and the feasibility of the proposed time-dependent dynamic modeling methodology based on synchronization of multiple cameras and two robotic total station units for checking the physical dimensions and changes in dimension of a moving object in the construction field. By building the photo-based 3D model at one particular time point, multiple dimensions of a dynamic object can be measured in absolute units of measure in a safe, cost-effective way. The methodology was successfully implemented to monitor the sling lengths of a rigging system engineered for lifting heavy modules in industrial construction. The data quality was further analyzed in terms of residual error. The measurement error is discussed based on a rigging system prototype of a reduced scale. The time-dependent dynamic modeling methodology provides a cost-effective solution to keep track of the physical dimensions and changes of a moving object over time. The proposed methodology would better cater to the application needs of modeling, visualization and measurement of dynamic objects critical to industrial construction.

Further research will be conducted in regard to (1) fully integrating and automating the modeling methodology using robust, reliable image processing methods such as pattern recognition and feature detection, so the manual operations of picking and matching points in photogrammetry modeling can be eliminated while the quality of modeling, which is quantified and evaluated by the residual error, is guaranteed; (2) extending synchronization of RTS units from two units to three units to track three control points on a dynamic object. The proposed dynamic modeling approach essentially follows the “five-point relative modeling approach” with the assistance of two synchronized RTS units at a particular moment, a scale bar is automatically fixed on the object by tracking two points. In other words, the length of the scale bar is subject to change over time. In the long run, the proposed methodology and system design can be readily extended to implement a time-dependent “three point absolute modeling” approach by synchronizing three RTS units to track the absolute coordinates of the three points on the moving object. As such, the coordinates of any point of the object in the field coordinate system can be fixed in real-time. This would be valuable to materialize AR on site level and lend timely, relevant decision support to construction engineers to improve quality, safety and productivity as field operations unfold.

The major barrier of implementing three RTS units lies in the cost of RTS. The price of a robotic total station is around 80–100 times the price of a camera. Therefore, this research is intended to model a moving object based on tracking a minimum quantity of dynamic control points on the object. As such, the reliable dimensional measurements at one particular moment can be obtained with the lowest equipment cost. In the current site testing, we had only two RTS units available; the problem definition is to measure sling lengths and their changes for a rigging system in order to ensure quality and safety. Hence implementing the “five point relative modeling” by use of two RTS was considered sufficient and cost effective in comparison with the “three point absolute modeling” using three RTS units; (3) analyzing dynamic structural loading effects based on real-time measurements of sling sagging magnitude, which can be made possible through photogrammetry modeling. As the sagging effect of a loose sling is ignored in the current study, the end-to-end length measurement gives only an approximation of the sling length. In fact, the photo-based approach allows direct modeling of the sagging effect on the sling. The determination of the exact sling length based on the end-to-end distance and the sagging effect will be investigated in the follow-up research.