NeRF and AI for 3D modeling
Programming skill is mandatory (preferably Python)
Neural Radiance Field (NeRF) stands as a category of neural networks adept at represent and render realistic 3D scenes environments using a set of 2D images. NeRFs are trained on an collection of images captured from diverse perspectives of a particular scene. Through training, the neural network establishes connections between individual pixels within the images and corresponding points within a 3D space. Consequently, this empowers the neural network to fabricate novel viewpoints of the scene from virtually any viewing point.
NeRFs possess several benefits compared to conventional techniques for presenting 3D scenes. They excel at presenting complex scenes with an exceptional degree of detail, and they possess the capability to generate further perspectives of the scene instantly. These attributes render NeRFs remarkably fitting for an array of uses, including virtual reality, augmented reality, and 3D modeling.
This study explores the prospective capabilities NeRF within the domain of 3D modeling for heritage architecture. The exploration encompasses imagery derived from Unmanned Aerial Vehicles (UAVs) and photogrammetric sources. The focal point is the practical implementation of NeRF, illustrated through a case study involving an ancient castle with a history spanning over a millennium. In particular, the advantages and the limitations that NeRF offers versus alternative modeling techniques like Structure-from-Motion (SfM) and traditional photogrammetry will be considered.
- Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2021). Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1), 99-106.
- Tancik, M., Weber, E., Ng, E., Li, R., Yi, B., Wang, T., ... & Kanazawa, A. (2023, July). Nerfstudio: A modular framework for neural radiance field development. In ACM SIGGRAPH 2023 Conference Proceedings (pp. 1-12).
- Schob, M., & Rekittke, J. (2023). Neural Radiance Fields for Landscape Architecture. Journal of Digital Landscape Architecture, 428-442.
- Samadzadegan, F., Dadrass Javan, F., & Zeynalpoor Asl, M. (2023). Architectural Heritage 3d Modelling Using Unmanned Aerial Vehicles Multi-View Imaging. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 48, 1395-1402.