Open and reproducible AI-supported multi-modal geolocation

GIMA
M-GEO
STAMP
Staff Involved
M-SE Core knowledge areas
Spatial Information Science (SIS)
Topic description

Advances in AI enable new approaches to solve existing problems, but they also create new challenges especially concerning the veracity and validity of digital content. Especially in the context and hot and cold conflicts, one can already observe increasing amounts of fabricated ("fake") content, generated with a variety of freely available and affordable generative, multi-modal AI tools. 

At the same time, using multiple geospatial data sources and AI/ML-supported methods can help in verifying information. For example, indepedent journalists and NGO use such approach to verify information from conflict and war zones: To confirm losses on the battlefield, to identify locations of war crimes, and others. One particular application is Belleingcat's OpenStreetMap search tool (Finding Geolocation Leads with Bellingcat's OpenStreetMap Search Tool - bellingcat) This tool requires signficant interaction and input from the human investigator. 

With the source code of this tool available, the opportunity exits to integrate AI into the workflow. The main challenges and constraints of this MSc topic are:

1. Ensure open, reproducible, transparent integration of AI into the workflow. 

2. Develop a framework to map multi-modal input to geospatial features. 

3. Find those geospatial features in open geospatial data sets. 

4. Provide the output to human users in an interpretable way. 

Clearly, the entire pipeline is beyond the scope of a single MSc topic. The first task of any prospective MSc researcher will therefore be to shape the topic into a manageable yet coherent research objective. The student's disciplinary background and technical skills can guide this reshaping. 

 

References for further reading

As a starting point with many relevant references:

Tahmasebzadeh, G., Hakimov, S., Ewerth, R., Müller-Budack, E. (2023). Multimodal Geolocation Estimation of News Photos. In: Kamps, J., et al. Advances in Information Retrieval. ECIR 2023. Lecture Notes in Computer Science, vol 13981. Springer, Cham. https://doi.org/10.1007/978-3-031-28238-6_14