5G Applications and Devices Benchmarking

Projektinformationen

TRIANGLE

ID Finanzhilfevereinbarung: 688712

DOI

10.3030/688712

Projekt abgeschlossen

EK-Unterschriftsdatum 10 November 2015

Startdatum 1 Januar 2016

Enddatum 31 Dezember 2018

Finanziert unter

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Gesamtkosten

€ 3 541 981,25

EU-Beitrag

€ 2 903 690,00

Koordiniert durch

KEYSIGHT TECHNOLOGIES BELGIUM
Belgium

Periodic Reporting for period 3 - TRIANGLE (5G Applications and Devices Benchmarking)

Berichtszeitraum: 2018-03-01 bis 2018-12-31

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

The future 5G system is getting day by day more defined with 3GPP releasing the specification of the system and operator starting to move towards trials and deployment. This very sharp cut has then been stressing how the initial introduction of new services will be happening in the so-called 4.5/4.9G system (LTE-A Pro Evolution, LTE-M, LTE NB-IoT).
On the other side of the coin, the mobile App and services market is experiencing a constant growth in both size and revenue, even though a common ground for verifying the App performances in a standardized fashion is still lacking. Only traditional software testing, which considers the network a reliable data pipe, is performed on the code itself. While it has been an acceptable void so far, the requirements foreseen for the 5G Apps and services will make it impossible to develop software without considering its performance in varying network conditions.
This gap brings along the lack of a homogeneous testing and certification framework that encompasses a wide range of use cases, network scenarios, and Key Performance Indicators (KPIs) that are able to cover not only the different traditional telco needs, but also the End-to-End (E2E) side, given the incredibly large variety of available and foreseen Apps and services. A new and more holistic set of tools, covering also the mobile device behavior at Operating System level and the Evolved Packet Core (EPC) network would be need to support this work.
The TRIANGLE project is funded under the FIRE/FIRE+ objective, it started in January 2016, and had a duration of three years. The primary objective of the project is to promote the testing and benchmarking of mobile applications and devices in Europe as the industry moves towards 5G and to provide a pathway towards certification in order to support qualified apps and mobile developments using FIRE testbeds as testing framework available at the University of Màlaga.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

One of the key achievements of the project was to develop a strategy around the creation of an automated measured QoE, which is crucial in the acceptance of the application by the end user. In TRIANGLE, the QoE is reported in function of the application or device performance against a set of pre-defined domains (power consumption, reliability…). These domains are then aggregated in a synthetic QoE number which is the TRIANGLE mark. To validate this approach, multiple measurement campaigns have been run including a calibration of the synthetic QoE value against a real set of users. Results obtained in TRIANGLE demonstrate that the direction taken is a good prediction of the effective QoE.

The concept around QoE evaluated with an End-to-End testing approach also gained traction within the industry. The approach was presented at the Mobile World Congress 2017 and Mobile World Congress 2018. In January 2018, NGMN published a position paper “DEFINITION OF THE TESTING FRAMEWORK FOR THE NGMN 5G PRE-COMMERCIAL NETWORKS TRIALS”, which endorses the TRIANGLE approach, the TRIANGLE service definitions, and KPI for E2E QoE measurements. The concept and approach were also presented different occasions in leading industrial bodies such as the Global Certification forum (GCF) and the International Wireless Industry Consortium (IWPC).

Over the project duration, 4 testbed releases where created expanding capabilities and tests libraries, 42 companies and universities applied to use the testbed to run experiments and 13 applied for extensions. The test approach has been so interesting and valuable that some of the applicants not selected in the Open Call have requested access to the testbed services without funding. Open Call 5 reused this non-funded approach to ensure applicants were truly interested in the testbed and not just receiving funding. Towards the end of the project, an extra open call 6 (OC6) was launched to attract further users with low amount of funding but with a high 5G content.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

The creation of a clearly identifiable “label” can help not only the business actors in the 5G ecosystem in creating a more aligned and stable E2E technology, but could impact the end users of the system providing them with a tool for simply and effectively helping in the selection of the device, app, or service to purchase, exactly as the EU energy label helps purchasing everyday objects, from cars to fridges. Finally, the creation of the certification scheme would open market possibilities for the entire testing equipment and services value chain. The consortium is working with the respective standardization bodies to gain traction on the approach.

Usage flow illustrated