Main results of first period included user requirements, real and synthetic video datasets, ethical and legal specifications, and Video Analytics Platform (VAP) data protection by design guidelines and impact assessment. Software and hardware architecture of VAP and video analytics plug-in framework were defined. First version of all algorithms integrated into VAP V1 and implementation of interfacing between VAP and all analytics components were accomplished as well as automatic scene reconstruction, manual correction, and motion reconstruction from camera networks regardless of calibration level and visualization of 4D scene reconstruction with metadata for straightforward user interaction.
Other accomplishments included backbone implementation for VAP V1, GUI implementation, training methodology and curriculum, dissemination strategy, and the VAFI community. Also, VICTORIA officially came up with a new work item proposal to upgrade and expand replacement of ISO 22311.
For the second period, the tools for synthetic video generation were improved based on their use by VICTORIA partners, and their feedback. Improved technical and organisational guidelines for the VAP and its plug-ins as well as ethical and a legal management toolkit were completed, including data protection.
A second version of the VAP were developed, integrating improved (robustness, accuracy, speed…) analytics modules from technical partners.
Technical progress included extension of Common Objects in Context (COCO) dataset, the large-scale object detection, segmentation, and captioning dataset, latest variant re-training and evaluation of YOLO – “You Only Look Once”, (YOLOv3 detector), and further development and evaluation of tracking approaches Kernelized Correlation Filter (KCF) tracker (Deep-KCF) and Deep Learning, long short-term memory (DeepSort-LSTM). Four patents were filed, three on methods for detection and tracking and one on deep learning models and training for textual description.
Audio Event Detection module for the VAP V2 was improved and new audio event detection tool was developed for footsteps detection, with different detectable audio events implemented in Python: Speech, Footstep, Car horn and Motors/engines detection. All these detection tools are based on the same principles: No need for training step and based on physical modelling of acoustic events.
Progress for audio similarity retrieval module audio synchronisation module between VAP1 and VAP2 was obtained as well as improvements on the semi-automatic scene reconstruction including, the 3D registration tool “CACTUS-3D” with gained online and local map server support with compass, with textured mesh with high texture resolution details for the static scene reconstruction, geo-registration improvements, and dynamic reconstruction of humans in videos.
Improvements were also achieved in the 4D crime scene investigation allowing Virtual Reality with full control over scene, collaborative exploration with VR, distance measuring, annotating scene with real distances, temporal multi-dimensional horizon charts and major performance improvements for larger, drone-captured scenes as well as dynamic point clouds.
Final implementation of all Graphical User Interface (GUI) components, with revision of workflows, implementation, and integration of the components from all technical partners into the VAP V1 and V2 and delivery of the VAP V2 with manuals to consortium end user sites was done.
Gathering of manuals of the VAP and analysis tools and their usage to produce training contents following defined curriculum, migration of the training content from PoliformaT to Moodle and organisation of two training workshops was carried out.
LEA’s carried out trial of VAP using data typical to that found in police investigations and in a couple of instances using the VAP on a live investigation, with LEAs demonstrating VAP to their colleagues and peers.
Post-trial workshop in Bochum allowed LEAs and technical partners to openly discuss their experience with the VAP and form a roadmap for future development.
The project produced a video trailer, which was broadcast on YouTube, participated in numerous conferences and produced a plan showing how partners envisage exploitation of their results after the end of the project.
