Visual and textual content re-purposing FOR(4) architecture, Design and video virtual reality games

Project Information

V4Design

Grant agreement ID: 779962

DOI

10.3030/779962

Project closed

EC signature date 21 November 2017

Start date 1 January 2018

End date 31 March 2021

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 3 937 850,00

EU contribution

€ 3 937 850,00

Coordinated by

ETHNIKO KENTRO EREVNAS KAI TECHNOLOGIKIS ANAPTYXIS
Greece

Periodic Reporting for period 2 - V4Design (Visual and textual content re-purposing FOR(4) architecture, Design and video virtual reality games)

Reporting period: 2019-07-01 to 2021-03-31

Summary of the context and overall objectives of the project

Nowadays large amounts of visual and textual data are generated, which are of great interest to architects and video game designers, such as paintings, archival footage, documentaries, movies, reviews or catalogues, and artwork. However, in their current form, it is difficult to be reused and repurposed for game creation, architecture, and design. To bridge this gap, V4Design develops technologies and tools that allow for automatic content analysis and seamless transformation to assist the creative industries in sharing content and maximize its exploitation.

V4Design aims to enable the re-use and re-purpose of such multimedia content by proposing an innovative solution for its effective integration, so as to extract 3D representations and VR game environments. This will allow architects, designers and video game creators to re-use heterogeneous archives of already available and retrieved digital content and re-purpose it by making the wealth of 3D, VR, aesthetic and textual information easily accessible and providing resources and tools to design and model outdoors and indoors environments of architecture and VR video game projects.

In a nutshell, V4Design goal is to: (1) re-use textual and visual content by enabling its efficient collection from content providers and crawling from public web resources; (2) re-purpose of content by developing novel approaches for 3D reconstruction and modelling, buildings and objects localization, aesthetics and style extraction, generation of 3D objects enhanced with semantics and explanatory text descriptions and finally (3) deploy innovative architecture, design and VR game authoring applications.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Overall, V4Design has been a very active, successful and fruitful project with many interesting results emerging from the final evaluation, important lessons learnt and remarkable research progress. The management team and the consortium as a whole, took all the necessary corrective and proactively delivered action plans to ensure the smooth running of the project that has ended up with a successful track.

One of the key technological achievements of V4Design is the implementation of two authoring tools that exploit and integrate novel content analysis techniques to generate 3D models, extract aesthetic and stylistic information from paintings and videos, localise buildings and objects of interest within visual content, and integrate it with textual information. The first one has been developed as a plugin on top of the well known Rhinoceros 3D application and the second one is a game authoring tool that can be launched either through the Unity game engine or directly via the Virtual Reality environment. Both tools can be applied in various real-life business cases such as: a) the architectural design related to existing or historical buildings and sites , b) the creation of a time-travel VR experience where a user immersed into a place of interest in the virtual world is able to move back or forward in time.

In respect to dissemination and exploitation, dissemination and collaboration activities have further progressed, expanding the user group and participation in relevant events. Increased social media activities of the project helped us reach more people in our dissemination activities and showcase the project’s results to relevant audiences. A thorough market analysis has been conducted during the project lifecycle. The market analysis consists of the analysis of each exploitable component as well as the project as a whole. The market analysis included SWOT Analysis, PESTLE Analysis and competitor analysis. This helped us provide a better market overview to position ourselves in the market. Finally, we worked on reaching out the potential customers of the project results and working on identifying pain points and how they will be willing to integrate V4Design in their current pipelines. The third version of the platform was developed in a manner to enable and facilitate its validation by early adopters, in scenarios where actual users from the user-group work in contexts related to the envisioned market applications of the V4Design platform.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

V4Design developed a platform that provides the ability to leverage existing visual and textual content from content providers and freely available web resources. Towards this direction, several data collection mechanisms (e.g. crawling, search) have been implemented to accommodate for different types of sources. The innovative tools that compose the V4Design platform allow architects, game developers and designers to reuse and re-purpose existing heterogeneous multimedia content, inspiring and eventually significantly reinforcing the design process.

The V4Design platform includes two authoring tools, one for architects and one authoring tool for virtual-reality game designers. The architecture authoring tool is based on Rhino 3D, and works as a complete editor for architecture applications. Similarly, the game tool is developed on top of Unity, and allows users to create virtual environments, using the assets generated by the V4Design services.

To develop the tools of V4Design, we used some innovative methods in the existing technologies. Specifically, V4Design multimodal components information used the W3C recommendation Web Annotation Data Model to promote interoperability and reusability. Moreover, V4Design Crawler is a fully-fledged solution that is implemented to support all the required operations for collecting content from web and social media in a single module. Lastly, the main innovation aspect of the Architecture Authoring tool (consisting of the V4D4Rhino plugin and the Compute server) rests on its ability to integrate innovative services from other partners to add more value to the Rhino users, services such as 3D reconstruction and BIM extraction, among others.

Therefore, all these used technologies contributed to achieve the following impacts: (a) greatly enriches and diversifies the architecture and video game markets by providing an innovative SoA solution that facilitates access and re-use of previously unavailable digital content (b) helps to reduce the time required to search for relevant content and use it in their work (c) it combines leading edge ICT technologies for 3D-model extraction, coupled with BIM, and the semi-automatic localisation of buildings, objects and various architectural elements from visual content (e.g. movies, documentaries).

The main impact of the V4Design innovations is towards 3D content designers, architects and professional/amateur game developers. Even though advanced platforms like V4Design are less labour intensive, they generate a lot of employment in fields like architecture, movies, gaming and employ people right from designers, architects, movie editors, gamer developers, software engineers to marketeers. The industries targeted by V4Design are seeing a heavy surge in investments from private enterprises and foreign companies especially in the gaming industry, which further brings employment and growth across Europe. Tools such as V4Design bring creativity, innovation, and employment with its development and these are the innovations that are required to fuel the economy.

V4Design Concept and Activities

Overview of V4Design