An Integrated Semantic Framework for Surveillance Video Information Retrieval

Modern surveillance systems continuously produce
large volumes of video, although much of the recorded footage
contains only a static background or very limited activity. Storing
such data can therefore become expensive, while finding a particular event in a long video sequence is often slow and difficult
to perform manually. To address this issue, we developed a
knowledge-based framework that transforms surveillance footage
into a structured and searchable representation. The framework
first applies a motion-based filtering step to identify frames
containing meaningful activity, which helps reduce the amount
of video processed by the object detector. The detected objects
are then tracked over time, and their movement patterns are
interpreted using an ontology to identify events such as walking,
running, falling, and intrusion. Information about the detected
events is represented as RDF triples and stored in a knowledge
graph, where it can be queried through SPARQL. We also
provide a natural-language interface that converts user questions
into SPARQL queries, making the event database accessible
without requiring users to have knowledge of the query language.
An evaluation using a subset of the VIRAT dataset shows that
the framework can detect and represent surveillance events
while reducing the amount of data that needs to be processed
and stored. The results also indicate that the natural-language
interface can retrieve the relevant events with good accuracy.