Although a large bulk of literature is implying on the benefits of BIM (Azhar 2011; McGraw-Hill Construction 2010; Becerik-Gerber and Rice 2010; Gilligan and Kunz 2007), the over-all measurement of BIM-related benefits for planning networks and practice is still difficult to justify, due to the high level of complexity of employed tools and of the process, but also due to the lack of a standardized measuring methodology (Jung and Joo 2011; Barlish and Sullivan 2012). The issue of how to measure BIM benefits is especially important in the emerging markets, such as Austrian is, in order to enhance the adoption of the technology and more over the process in the industry. For the adoption in the AEC market the closer research of interrelations within the triangle: technology (operability) – people - process is necessary, in order to create a guideline for BIM adoption, assessment, usability, risks, and evaluation (Gu and Kerry London 2010).
In order to evaluate BIM-performance within an integrated planning process in relation to the technology-people-process triangle, we conducted an experiment with students, simulating a multi-disciplinary planning process for sustainable building within a design-studio class in the winter semester of 2012/13. The experiment is a part of an on-going research project “BIM-Sustain: Process Optimisation for BIM-supported Sustainable Design” involving cooperation of university research and BIM-software vendors and developers. This interdisciplinary collaboration of academy and industry enables development of customised strategic concepts for the individual BIM-settings within multi-disciplinary planning context. The final aim of the project is compilation of guidelines for BIM-supported design and planning. The guidelines will include the conventions for efficient data-exchange and a road-map for the standardization process at Austrian Standardisation Institute (standardization body), recommendations for the planners for the inter- and intra-firm organization of BIM-supported design process, and finally proposals for improvement of interoperability for the software-vendors; based on experiment-findings. Similar guidelines were compiled by the Penn State within the Computer Integrated Construction Research Program (2012); or the Integrated Project Delivery For Public and Private Owners (2010).
The empirical research by experiment has often been employed to test of BIM-performance and capabilities. Plume and Mitchell (2007) conducted in 2004 an experiment with 23 students in a design studio setting, testing the IFC-model performance in multi-disciplinary collaboration (architecture, landscape architecture, MEP, statutory planning, sustainability and construction management.) They focused primarily on operational issues, such as building model (representation of a building model in different tools) and IFC –server data sharing issues. They conclude that the original architectural model needs significant adaptation for the use of other disciplines or their tools. Further issue needing closer attention is model management – tracing of the changes and updates carried out on the common model. Sacks et al. (2010) carried out the “Rosewood experiment”, comparing the BIM-supported versus the traditional 2D CAD the planning and fabrication process of the pre-cast façade. BIM proved to be more efficient by 57%, however IFC proved not mature enough causing data inconsistency in transfer between architectural and engineering system. Losses in translation can be assigned to object-semantic, a similar problem addressed by the Plume and Mitchell (2007).
Sturts Dossick and Neff (2011) observed the collaboration of several teams on three real projects using a BIM-technology supported design process, focusing on people and process issues. They concluded that technology can even hinder the innovation of the design process through a too rigid corset of work-flow and knowledge exchange, hindering the exchange of tacit, informal knowledge. Their concept of “messy talk” – the informal, unstructured information exchange as often practiced in architecture and construction engineering is tested within student experiment, where geographically distributed teams work using BIM on a project in a virtual environment. They conclude that “…messy talk requires both the flexible, active, and informal setting described in the 2011 study as well as mutual discovery, critical engagement, knowledge exchange, and synthesis.” (Dossick et al. 2012).
Peterson et al. (2011) conducted a simulation of integrated project management within two classes at Stanford and TU Twente, using pre-made BIM models in Revit, Tekla and AutoCAD 2006 and 2007 (which in our understanding cannot be considered as BIM models due to the lacking of parametric characteristics), the models were imported in the various cost and scheduling software such as Primavera or Vico.
The formerly mentioned experiments and research of BIM-supported planning practice focus on evaluation of singular issues - some primarily focus on the technology performance (interoperability, building model semantics), such as Plume and Mitchell (2007) experiment and Sacks et al. (2010). Sturts Dossick and Neff (2011) on the other hand focus mainly on the process issues. The student classes carried out at Stanford and TU Twente apply the holistic evaluation, however examine the BIM-supported project management, which in terms of data exchange displays lower complexity than multi-disciplinary design, involving structural and thermal simulation, which both are based on exact transfer of geometry.
In our research, we have addressed the triangulation of the technology, people and process parameters, in order to identify how they are correlated. Therefore, through the experiment the data on a) BIM-performance in terms of data-transferability in different software-constellations will be collected through protocols and revision of delivered models and b) the team performance using different BIM-tools will be assessed through protocols and recorded feedback workshop. The executed experiment is the first one to have a holistic approach on the evaluation of people-process-technology triangle, testing a large number of software tools (all together thirteen) and software combinations on the transfer of complex geometrical data, but also on usability.
Through exploratory research – an experiment within an interdisciplinary design class involving 40 students, the collaborative, multi-disciplinary BIM-supported planning for an energy-efficient office building is simulated. The multi-disciplinary teams consisting of: architect, structural engineer, building physicist (BS) were formed by the means of a pre-questionnaire, which questioned skill-level, experience and preference of the software. Upon the results of the questionnaire a matrix of software-combinations used by each team was compiled (Figure 1).
In the course of the experiment (design class) basically two work-flow models can be identified: One-Platform BIM (proprietary) and Open-Platform BIM (using IFC exchange format). The experiment began in September 2012, the latest available software versions were used. The Open-Platform BIM teams (Figure 1, Teams 3–13) use different, for each discipline typical (custom) software, and work with central architectural model, exchanging the data using the IFC. By central architectural model we mean the physical, architectural model as the point of origin for the further transfer into a) structural, analytical software or b) into the thermal analysis software. The One-Platform BIM teams (Figure 1, Teams 1 und 2) work with one software family Nemetschek Allplan (2012) or Autodesk Revit (2012) using proprietary standards, again starting with architectural model.
The teams are assigned with compilation of the architectural (in Allplan 2012, Revit Architecture 2012 or ArchiCAD16 2012), structural (Simulation in Dlubal REFM 2012; Sofistik 2012 or Scia 2012, drawings in Tekla Structures 2012; Revit Structure 2012 or Allplan 2012), and ventilation (in Plancal 2102 or Revit MEP 2012) models, as well as the light simulation and energy certificate (Figure 2). For the thermal simulation TAS 9.2 (2012) is used, for light simulation Dialux 4.9 and for energy certificate Archiphysik 10. Planning documentation was handed out, consisting of a functional programme, site-plan with orientation and set origin, layer-structure and colour scheme for latter room-stamps.
The time-schedule of the design-class is strictly organized; the experiment is taking place in the period of one semester. We have organised three presentations, where in the first one the architectural model is presented, in the second presentation the structural and thermal and in the final presentation the optimised, full model containing all the information (Figure 3). Between the presentations the reviews with teachers as well as tutorials provided by software vendors are taking place.
Figure 4 presents the final model as delivered by one of the student-teams (Team 3) at the final presentation, including architectural model with visualization, model of loadbearing structure and maximal slab deformation under load, model of ventilation and energy and HVAC concept (Figure 5).
On the level of technology, the experiment is examining the fitness of various software constellations for data transfer, import and export, documenting the data loss and needed rework if data-loss has occurred. In terms of process, the efficiency and efficacy of multi-disciplinary teams working with BIM: efficiency of the employed BIM methods for data-exchange, communication effort, and work-allocation (work-flows); and on people-level satisfaction and conflict levels are assessed. Through the mandatory protocols and time-sheets the problems related to the technology (data-transfer inconsistencies or losses, semantics) but also to the process-people related problems (conflicts, communicational difficulties, lack of work-flow definitions or responsibilities etc.) can be tracked (Figure 4). Additionally, an e-learning platform has been set up, with a forum for tutor feedback as well as for student-communication, scheduling and posting of tasks is taking place.