SpatialBenchRAG: A Multimodal, Multilingual Benchmark for KG-grounded RAG over Diverse Geographies

Many every tasks and decisions rely on spatial data and Retrieval-Augmented Generation (RAG). For example, if you ask a conversational assistant to give you directions from Wasserturm to Luisenpark with a stop at a highly rated but cozy café, they are often used in the background to generate the response. Knowledge Graphs (KGs) are a type of data model that is used to search for and link information. They constitute a rich source of spatial information whose graph-based data format enables advanced spatial queries and supports spatio-temporal reasoning.

The aim of the SpatialBenchRAG project is to develop the first multimodal, multilingual benchmark for evaluating RAG systems grounded in spatial KGs. Addressing the lack of standardized resources for spatial reasoning, the benchmark combines diverse geographic datasets, including OpenStreetMap and Wikidata, to support realistic, context-aware evaluation across tasks such as spatial and visual question answering, or multi-hop reasoning. By enabling transparent and reproducible research, this initiative aims to accelerate the development of geographically grounded AI systems for applications like disaster response, geo-aware assistants, and spatial fact-checking.

The project is funded by a 2025 Open Science Grant awarded to Andreea Iana.