Harbingers of NeRF-to-BIM: a case study of semantic segmentation on building structure with neural radiance fields

Abstract: Scan-to-BIM applications rely on point clouds obtained by laser scan, which require expensive hardware and laborious tasks. To address this issue, we introduce a NeRF-to-BIM approach, exploiting recent advancements in computer vision with Neural Radiance Fields (NeRF). NeRF is a state-of-the-art (SOTA) for 3D scene reconstruction from 2D images but lacks specific applications in the architecture, engineering, and construction (AEC) domain. We propose a 3-step approach: (1) 3D reconstruction of buildings using NeRF. (2) Semantic segmentation by fine-tuning pre-trained deep learning (DL) algorithm. (3) Conversion from the semantic segmentation point cloud to BIM. Finally, qualitative and quantitative analyses are performed.