Periodic Reporting for period 2 - TEXTAROSSA (Towards EXtreme scale Technologies and Accelerators for euROhpc hw/Sw Supercomputing Applications for exascale)
Reporting period: 2022-10-01 to 2024-03-31
The project is developing core technologies for the emerging generations of high-end heterogeneous computing architectures towards exascale-class systems, extensively applying the top-down co-design process from the applications to the prototypes of HW architectures and SW systems. The ambition is leveraging and further extending the expertise of the core partners involved in the European Processor Initiative (EPI) to realize the EuroHPC roadmap for energy-efficiency, high-performance and secure services by enabling new computation paradigms for HPC, AI and HPDA applications. Several technical goals are pursued: energy efficiency, sustained application performance, fine-tuned thermal policies integrated with an innovative cooling technology, seamless integration of reconfigurable accelerators, development of new IPs.
TEXTAROSSA overall objectives are strongly innovative and address 5 pillars, from HW innovation to user application: 1) integrated heterogeneous architecture at node level; 2) high-efficiency cooling system at node and system level; 3) innovative tools for seamless integration of reconfigurable accelerators; 4) open integrated development vehicle platforms; and 5) new applications computing domains.
TEXTAROSSA overall objectives are strongly innovative and address 5 pillars, from HW innovation to user application: 1) integrated heterogeneous architecture at node level; 2) high-efficiency cooling system at node and system level; 3) innovative tools for seamless integration of reconfigurable accelerators; 4) open integrated development vehicle platforms; and 5) new applications computing domains.
TEXTAROSSA goals are pursued by working at the various levels of the computing scenario.
1. At the architectural level: two innovative heterogeneous nodes have been set up, one based on the OpenSequana architecture from Atos (x86 processors accelerated by NVIDIA GPUs) and the other, developed by E4, based on ARM processors and Xilinx FPGAs. To maximize energy efficiency, an innovative two-phase liquid cooling system has been developed and tested on both the demonstrator nodes.
2. At the accelerator level: due to the high performance/watt figures of FPGAs, some IPs have been developed: an IP to support variable precision arithmetic (thus reducing memory bandwidth requirements and hardware resources), a cryptographic IP to be used in Homomorphic Encryption (HE) and in Post-Quantum Cryptography (PQC), a communication IP to allow fast intra and inter FPGA communication, an IP to support the scheduling for task-based computational models.
3. At the programming tool level: High-Level Synthesis flows are being widely addressed within the project to allow seamless access to FPGA accelerators. HLS tools are being used in conjunction with stream-based (FastFlow, APEIRON) and task-based (OmpSs, StarPU) tools, and are targeted by tools developed in the project (TAFFO) to support mixed precision arithmetic.
4. At the application level: many applications from different relevant domains are being implemented on top of the target architectures to demonstrate the effectiveness of all the solutions being developed within the project, where some of them are released as open source.
During the entire project duration, we focused on widening our audience and ensuring the visibility of the TEXTAROSSA project in the international community. Dissemination materials was created, and all activities have been supported by a public website as well as a landing page on three social media: Facebook, Twitter, and LinkedIn, which will be active for the next five years. Several international conferences including the collaboration with companion projects were pursued to maximise the spread of TEXTAROSSA achievements. A book is planned for fall 2024.
The plan to continue working on the exploitation of the project's achievements from a commercial perspective includes strengthening the established value chain between ENEA, InQuattro, E4, and POLIMI. This collaboration has already led to the release of a commercial installation at the University of Turin. Additional plans to enhance scientific cooperation among project partners are already underway. Many of these partners are involved in other ongoing EuroHPC projects, such as EU-Pilot, EUPEX, EPI-2, as well as Chips JUs like ISOLDE and Tristan. Moreover, there are ongoing proposals focusing on both platforms and applications for HPC, involving partners such as CNR, FhG, INFN, and ENEA. This interconnected network of collaborations fosters knowledge sharing, synergy, and the advancement of HPC-related research and development efforts.
1. At the architectural level: two innovative heterogeneous nodes have been set up, one based on the OpenSequana architecture from Atos (x86 processors accelerated by NVIDIA GPUs) and the other, developed by E4, based on ARM processors and Xilinx FPGAs. To maximize energy efficiency, an innovative two-phase liquid cooling system has been developed and tested on both the demonstrator nodes.
2. At the accelerator level: due to the high performance/watt figures of FPGAs, some IPs have been developed: an IP to support variable precision arithmetic (thus reducing memory bandwidth requirements and hardware resources), a cryptographic IP to be used in Homomorphic Encryption (HE) and in Post-Quantum Cryptography (PQC), a communication IP to allow fast intra and inter FPGA communication, an IP to support the scheduling for task-based computational models.
3. At the programming tool level: High-Level Synthesis flows are being widely addressed within the project to allow seamless access to FPGA accelerators. HLS tools are being used in conjunction with stream-based (FastFlow, APEIRON) and task-based (OmpSs, StarPU) tools, and are targeted by tools developed in the project (TAFFO) to support mixed precision arithmetic.
4. At the application level: many applications from different relevant domains are being implemented on top of the target architectures to demonstrate the effectiveness of all the solutions being developed within the project, where some of them are released as open source.
During the entire project duration, we focused on widening our audience and ensuring the visibility of the TEXTAROSSA project in the international community. Dissemination materials was created, and all activities have been supported by a public website as well as a landing page on three social media: Facebook, Twitter, and LinkedIn, which will be active for the next five years. Several international conferences including the collaboration with companion projects were pursued to maximise the spread of TEXTAROSSA achievements. A book is planned for fall 2024.
The plan to continue working on the exploitation of the project's achievements from a commercial perspective includes strengthening the established value chain between ENEA, InQuattro, E4, and POLIMI. This collaboration has already led to the release of a commercial installation at the University of Turin. Additional plans to enhance scientific cooperation among project partners are already underway. Many of these partners are involved in other ongoing EuroHPC projects, such as EU-Pilot, EUPEX, EPI-2, as well as Chips JUs like ISOLDE and Tristan. Moreover, there are ongoing proposals focusing on both platforms and applications for HPC, involving partners such as CNR, FhG, INFN, and ENEA. This interconnected network of collaborations fosters knowledge sharing, synergy, and the advancement of HPC-related research and development efforts.
Energy efficiency: the project achieved an increase the energy efficiency by 2-5x compared with current available HPC solutions based on CPU or CPU+GPU thanks to: i) new energy-efficient processors and accelerators, ii) integrated use of FPGAs, iii) bi-phase cooling, iv) mixed-precision and data compression techniques, v) direct communication between FPGAs.
Sustained application performance: increase in performance by 2-10x has been achieved compared with current solutions based on CPU or CPU+GPU thanks to new AI accelerators and high-performance GPUs and FPGAs, HW fast schedulers, low-latency inter-/intra-node links, better cooling allowing higher frequencies, fast flow enabled at programming mode and run time levels
Fine-tuned thermal policies integrated with an innovative cooling technology: a detailed thermal model for the evaporative cooling system has been developed and a hierarchical thermal control policy was demonstrated in real systems. Such control approach encompassed the management of DVFS actuators in the inner loop and of the control of the flow of coolant in the outer loop, so that the joint control can minimize the performance penalty while achieving an accuracy in the operating temperature of the computing cores (CPUs and GPUs) in the order of 1-2 degrees.
Seamless integration of reconfigurable accelerators. Several accelerators have been developed, same samples are the following: APEIRON as an extension of the Xilinx Vitis HLS framework able to support a network of FPGA devices interconnected with a low-latency direct network; OmpSs@FPGA supports seamless integration of reconfigurable accelerators by submitting C/C++ programs directly to CPU+FPGA heterogeneous systems only by using simple pragmas; FastFlow is a structured parallel programming environment targeting shared and distributed memory (COW) architectures, which has been extended to allow the seamless integration of FPGA kernels, developed through the Vitis HLS flow.
Development of new IPs. During the project, 18 Intellectual Properties (IPs) have been developed and most of them are released as open source. Some of them achieved a high level of exploitation during the project timeframe, such as the evaporative cooling technology developed by InQuattro, which has progressed to the stage of commercialization, since it has been installed on a computing platform commercialized by E4 now part of the Green Computing center at UniTO. Similary, the Urban Air application by PSNC, will be utilized by a municipality as the basis for developing a monitoring system for analysing air quality.
Integrated Development Platforms (IDVs). Two integrated development platforms (IDV) have been designed and realized during the project to support both existing and emerging High-Performance Computing (HPC) applications. Their architecture is heterogeneous, comprising CPUs, GPUs, and FPGAs and will be hosted and maintained by ENEA as a showcase of the TEXTAROSSA technologies to foster further scientific cooperation and exploitation activities.
Sustained application performance: increase in performance by 2-10x has been achieved compared with current solutions based on CPU or CPU+GPU thanks to new AI accelerators and high-performance GPUs and FPGAs, HW fast schedulers, low-latency inter-/intra-node links, better cooling allowing higher frequencies, fast flow enabled at programming mode and run time levels
Fine-tuned thermal policies integrated with an innovative cooling technology: a detailed thermal model for the evaporative cooling system has been developed and a hierarchical thermal control policy was demonstrated in real systems. Such control approach encompassed the management of DVFS actuators in the inner loop and of the control of the flow of coolant in the outer loop, so that the joint control can minimize the performance penalty while achieving an accuracy in the operating temperature of the computing cores (CPUs and GPUs) in the order of 1-2 degrees.
Seamless integration of reconfigurable accelerators. Several accelerators have been developed, same samples are the following: APEIRON as an extension of the Xilinx Vitis HLS framework able to support a network of FPGA devices interconnected with a low-latency direct network; OmpSs@FPGA supports seamless integration of reconfigurable accelerators by submitting C/C++ programs directly to CPU+FPGA heterogeneous systems only by using simple pragmas; FastFlow is a structured parallel programming environment targeting shared and distributed memory (COW) architectures, which has been extended to allow the seamless integration of FPGA kernels, developed through the Vitis HLS flow.
Development of new IPs. During the project, 18 Intellectual Properties (IPs) have been developed and most of them are released as open source. Some of them achieved a high level of exploitation during the project timeframe, such as the evaporative cooling technology developed by InQuattro, which has progressed to the stage of commercialization, since it has been installed on a computing platform commercialized by E4 now part of the Green Computing center at UniTO. Similary, the Urban Air application by PSNC, will be utilized by a municipality as the basis for developing a monitoring system for analysing air quality.
Integrated Development Platforms (IDVs). Two integrated development platforms (IDV) have been designed and realized during the project to support both existing and emerging High-Performance Computing (HPC) applications. Their architecture is heterogeneous, comprising CPUs, GPUs, and FPGAs and will be hosted and maintained by ENEA as a showcase of the TEXTAROSSA technologies to foster further scientific cooperation and exploitation activities.