Object Detection in Floor Plans: Lessons From Designing a Human-in-the-Loop System

In the last few years, many companies in the Architecture, Engineering, and Construction (AEC) industry have started exploring how artificial intelligence can be utilized to improve work processes. This trend is fueled by several technological advances, such as the increasing adoption of Building Information Modeling (BIM). However, so far, only a few large- and medium enterprises have successfully transformed their data into actual business value. As in many other industries, employees are uncertain about the impact AI will have on their work and, therefore, are hesitant to support the upcoming transformation.

In the last few years, the concept of human-centered artificial intelligence has received attention from both scholars and practitioners. The approach is based on the idea that AI systems should not replace employees but augment and empower them. Especially, in architectural design, where many work processes are characterized by a multitude of repetitive tasks with low complexity, identifying and allocating these tasks to computers is a promising endeavor. Working with several architects in the last few years, we have realized that many employees are interested in systems that can relieve them of these “tedious tasks” while still giving them the ability to monitor these systems and contributing their domain expertise where needed. ... mehrWe predict that design tools will increasingly integrate functionality that relies on combining human and artificial intelligence in the next few years.

Our research is situated at the intersection of computer science, information systems research, and architectural design. We utilize state-of-the-art techniques from the fields of computer vision and explainable artificial intelligence to develop strategies for collaboration between humans and computers and evaluate them through field experiments. We have developed a prototype for a German architecture firm that addresses the following tedious working task:

The process of mass determination usually requires the manual counting of relevant components in plans. In the field of building operation, there is the challenge of identifying objects from floor plans, which are usually only available as rasterized images or printouts. To address this issue and foster quality control, our system detects symbols—which are relevant to the user—in scanned floor plans to support the planning process and simplify comparison with building requirements. The system “collaborates” with its users by providing them the possibility to apply their domain knowledge to adjust recommendations of the AI system where it is uncertain. To do so, the system quantifies the uncertainty of each recommendation to guide the collaboration.