Final Report Summary - GRAISEARCH (Use of Graphics Rendering and Artificial Intelligence for Improved Mobile Search Capabilities) GRAISearch is an Industry-Academia Partnerships and Pathways project (IAPP) aimed at transferring knowledge from Academia to Industry. It is also a support for training and career development of Marie Curie Fellows. The principal scientific aim of the project is to develop and apply revolutionary graphics rendering and artificial intelligence (AI) methods to an existing social media search engine platform thereby creating ground breaking mobile search capabilities with significant online commercial potential. Technologies from research in two academic institutions (INSA de Lyon (LIRIS UMR CNRS 5205) and Trinity College Dublin (TCD)), were explored in conjunction with the industry partner Tapastreet with a view to commercialising functionality which would improve the social media search capability of the company. The GRAISearch project is at the forefront of modern data processing and analysis techniques. Social media traffic and personal mobile usage is growing at accelerating rates with every new generation of personal portable device while the amount of publicly shared media - in particular visual content - is also growing with advances in mobile transmission technology (Wifi, 4G and 5G). This increasing amount of media content is being uploaded from a wide variety of locations, transforming all users into independent amateur broadcasters and providing a wealth of new information about what is happening over time at particular locations on the planet. There are potential benefits of the research in this project for many application areas including 'smart city' planning, tourism, rescue efforts in emergency situations and many other situations where up-to-date, localised information can be processed to reveal knowledge which can in turn be used to inform the public.GRAISearch has focused on the development of tools for smart mobile browsing which would allow data harvested from official APIs of social media streams to be merged, analysed, presented and visualised in a compact, comprehensive and user-friendly way. This geolocated, time-stamped data from social media is noisy, multi-modal and heterogeneous by nature. It is important therefore to have visualisation tools available to be able to extract insights about the collected information. Technologies for video summarisation of events reported on social media combined with geo-location information and 3D mapping were investigated in Work Packages 1 & 2. Prototypes were developed and demonstrated with accompanying publications. Event detection is one of the most important research topics in social media analysis. Work Packages 3 & 4 focused specifically on detection of local trends in real time data from social media streams which is a type of information not easily covered by main stream media and which would prove particularly useful in many social situations. Partners at INSA, in collaboration with industry partner Tapastreet, developed 'Gazouille' which is a system for discovering local events in geo-localized social media streams. The system is based on three core modules: (i) data acquisition from social networks in several urban areas, (ii) event detection through time series analysis, and (iii) web interface for presentation of events discovered in real-time in a city, associated to a gallery of social media that characterize the event. The methods employed in the project were shown to outperform the only state-of-the-art method designed to discover geo-located events. Work Package 5 looked at recommender systems as applied to the Tapastreet application. And Work Package 6 was concerned with the integration of the research topics into the commercial platform.All recruitments and the majority of secondments between the academic and industry partners took place in Period 1 with one long term and serveral short term secondments during 2016 and early 2017. Research output has been significant in each of the areas of research with publications in major journals and conferences, as outlined in the attached work package deliverables. In terms of Research Fellow mobility and career developments, two of the Research Fellows recruited into the project have subsequently secured permenant academic positions.The output of the project has continued to impact follow on research and collaborations with Industry including a privately funded research with an industry partner in the telecoms sector in the area of object detection and mapping (https://arxiv.org/abs/1708.08417) as well with the Intel company Movidius who carried out a drone survey of Trinity College campus in July 2017 which has been published as a dataset of aerial imagery on the TCD open access archive TARA: http://www.tara.tcd.ie/handle/2262/81836.The possibility to geo-locate objects from publicly available imagery has been further investigated in a privately funded research study which is currently under consideration for commercialisation by way of a feasibility study funded by the national enterprise support organisation in Ireland.http://graisearch.euTwitter: @graisearchYouTube: fp7graisearch
