Text-To-AR for Home Remodeling: An End-To-End Pipeline from Natural Language to AR-Compatible 3D Assets

Abstract: Home remodeling has been on the rise post pandemic due to work-from-home culture. Homeowners resort to ad-hoc methods (e.g., hand-drawn sketches) which often result in miscommunication, delays, rework, and increased costs. Although numerous augmented reality (AR) based frameworks address some of these challenges, most remain proprietary, constrained to predefined 3D asset libraries, and incapable of translating homeowners’ requirements. This study serves as a first step towards addressing this critical knowledge gap by developing and validating an end-to-end automated pipeline that can convert a homeowner’s free form textual description into an immersive AR visualization. Case study results evaluated across image resolution, input type, and quality condition based on the structural similarity index measure (SSIM), suggest visually coherent 3D assets suitable for visual communication. Ongoing work aims to advance this towards a scalable, environment-aware AR communication tool that supports real remodeling decision-making across diverse devices and platforms.