Development of a Digital Twin-Based Simulation System and A Novel Synthetic Video Dataset for Enhancing Computer Vision in Construction Site Safety

Abstract: In this work, we introduce a novel construction simulation system and the first photo-realistic synthetic video dataset for the construction industry, named ConSynth. The system simulates vehicle dynamics, worker behaviours, and various environmental conditions, generating a dataset of 200 video sequences and 24,000 images with automated labels. Our experiments have shown that models trained on ConSynth achieve robust detection with less reliance on real-world data, indicating enhanced diversity, improved performance in underrepresented scenarios, and substantial generalisability. Our work promises significant advancements in construction safety monitoring through computer vision.