Evaluation Method for Bidirectional Interoperability between Heterogeneous BIM Software

Background

My career at Vertex started in the summer of 2023. As a second-year computer science student at Tampere University, I was hired for a summer job as a developer for our BIM application, Vertex BD. Since then, I worked every summer as a BD developer, and some days as a part-timer amid my studies. I was also able to tackle real-world topics in my bachelor’s and master’s theses, as they were commissioned by Vertex. Now, after graduating, I work full-time as a software engineer for Vertex BD.

The topic of my master’s started off as investigating a BD–Revit integration, which is, somehow, a very narrow and yet very wide subject simultaneously. The problem was that we really didn’t know much about this topic; how to “integrate” Vertex BD and Autodesk Revit better, what it could require, why we would even want to. I picked this topic, because I found it fascinating due to prior knowledge (the bachelor’s thesis by our colleague Valtteri Tiitinen broached a similar subject), I felt I had good opportunity to refine what I wanted to research, and perhaps most importantly, this topic could cover a research gap when it comes to building information modelling in academia.

After a decent bit of planning, I had determined what my thesis should target. A general method for evaluating technological solutions that facilitate interoperability between BIM software applications. My goal was to create such an evaluation method by discovering and defining an understanding of the BIM methodology, interoperability in computer science and in BIM, the requirements of a hypothetical system of interoperability between BD and Revit, and the four technologies we picked for evaluation: The IFC SDK and the BimRv SDK by the Open Design Alliance, the 3D InterOp SDK by Spatial Corporation, and the currently-in-development Data Exchange system by Autodesk. I and my thesis advisors considered this thesis topic as potentially risky due to scope creep, as completing the work in target time would introduce a notable time constraint. I resolved this by determining that there was to be no proof-of-concept code implementation in this study.

Formulating the method

I won’t go deep into the background on BIM or interoperability in this blogpost as those are covered in decent depth in my thesis, which can be found online, due to the extensive literature review I conducted as part of my research. In short, it is clear that the benefits of BIM that are promised in literature are not consistently realized by industry actors. The reasons for this are not known without doubt, as empirical research into it is found in short supply. However, it is essentially a matter of lacking BIM maturity; of collaborative practices and workflows between stakeholders, across project phases. Undoubtedly, the less-than-optimal interoperability capabilities of BIM software applications is a notable contributor to the shortcomings of real-world BIM implementation. Indeed, interoperability in BIM is known to be problematic – it has been estimated that problems in software interoperability account for around three percent of total project budgets. Problems in BIM interoperability are largely syntactic and semantic. Specifically, the exchange of BIM models between heterogeneous software often leads to loss of data, lacking representation of object geometry, missing object properties, missing relations and differing or missing semantics of objects. It has been argued in prior research that the fundamental causes to these problems are twofold:
1. Software tools are unable to accurately interpret objects from other disciplines due to differences in domain knowledge.
2. Software tools have different ways of representing the same geometry, properties and relations in their native data schemata, leading to inconsistency.

In other words, the root cause of these problems is semantic heterogeneity.

I undertook a lightweight requirements engineering (RE) process as part of my thesis to discover requirements of a hypothetical system between Revit and BD. In this process I identified the key stakeholder classes of such a system, held elicitation sessions over Microsoft Teams with some of our Vertex professionals, and finally analysed the gathered information and created a requirements specification in the form of a list of requirements, similar to that our colleague Aleksi Sirviö did in his 2024 thesis. I identified what I think to be the main business goal of such a system, and the main workflow that such a system would seek to improve.

I wrote brief overviews of the four selected technologies and created a summarizing table for concise result presentation. At this point in my thesis, I was in quite the hurry to bring the thesis to its finalization, so the fact that relatively little public information is available of them and their documentation cannot be used as-is for a publicly available study meant that the overview had to remain brief, especially when it comes to technical analysis. My knowledge of the domain was also a limiting factor, as it was difficult for me to recognize what parts or aspects of the interfaces these technologies offer are the most significant in this case.

Finally, I combined all the knowledge gained in the study with the ISO/IEC 25000 “SQuaRE” series of standards on software quality requirements and evaluation and formed the evaluation method by utilizing an evaluation process defined in ISO/IEC 25040. Essentially, I defined a list of evaluation criteria and a weighted scoring scheme with which each selected criteria for a given technology could be assigned a score and a weight.

Validating the method

The only available means of validation in our case was using the method to evaluate the selected technologies. Because I could not make implementations of the techs within the constraints of the thesis, I had to exclude those criteria from evaluation that would rely on quantifiable results, i.e. Correctness of conversion and Efficiency of exchange, which I had already abstracted as well. This meant that I could not determine the efficacy of complete evaluation, where working software is used. The results of my evaluation are pictured below.

As shown, each evaluated technology performed reasonably in this case. The intermediate-format-based ODA IFC SDK and Autodesk Data Exchange received slightly better scores than the technologies handling Revit files directly, BimRv SDK and 3D InterOp SDK. This was expected to a certain extent, as we know at least that our implementation of IFC support in Vertex BD with the ODA IFC SDK does not perform especially well in a real workflow in which a model from another BIM application is imported and converted to our data model. In other words, the effects of semantic heterogeneity are not well reflected in this evaluation. Still, I considered this process a success, as we were able to create the method in a generalizable and modifiable manner and to demonstrate it being used. Further discussion on the created method and implication of the results can be found in my thesis.

Conclusions

In my thesis, an evaluation method for bidirectional interoperability in BIM has been developed. First, literature review was conducted of BIM and interoperability to understand the problem, and possible technological solutions were selected and studied. Then, requirements engineering was conducted to discover and specify attributes for a possible system of interoperability between BIM applications Autodesk Revit and Vertex BD. Finally, the method was created by applying the ISO/IEC SQuaRE series of standards and connecting it to discovered requirements.

The thesis asked three research questions:

1. How can technologies be compared in a meaningful way for BIM interoperability?
   2. What are the requirements of a system of interoperability between Autodesk Revit and Vertex BD?
   3. Which of the selected technologies suits the BD–Revit case the best according to the evaluation method?

The first question was answered by creating a method for the evaluation of technologies that facilitate bidirectional interoperability between heterogeneous BIM software. The evaluation method was created in accordance with the ISO/IEC SQuaRE series of standards for software quality evaluation. The created method stands to be well applicable in general.

The second question was answered by conducting lightweight requirements engineering. The RE process was successful in identifying key goals and requirements of a BD–Revit integration, and it was necessary in order to answer the first question. A list of requirements was specified, which should be applicable beyond the case of the thesis. Advantages and disadvantages of the BD–Revit case were discussed in relation to the requirements and the selected technologies. Although the process focused on these two applications, we think the resulting requirements specification is also applicable to other, comparable software applications.

The third question was answered by evaluating ODA IFC SDK, ODA BimRv SDK, Autodesk Data Exchange and Spatial 3D InterOp SDK with the created evaluation method. No clear candidate emerged as result of the evaluation, as doing proof of concept implementations was outside the scope of this study, and consequently some important criteria could not be evaluated reliably. According to the evaluation criteria and weighting we applied, the IFC toolkit by ODA, which Vertex already uses for IFC support, is the strongest solution.