NEuromorphic Reconfigurable Integrated photonic Circuits as artificial image processor

Project Information

NEoteRIC

Grant agreement ID: 871330

DOI

10.3030/871330

Project closed

EC signature date 5 December 2019

Start date 1 January 2020

End date 30 April 2024

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 3 965 394,63

EU contribution

€ 3 965 394,63

Coordinated by

UNIVERSITAT POLITECNICA DE VALENCIA
Spain

Periodic Reporting for period 2 - NEoteRIC (NEuromorphic Reconfigurable Integrated photonic Circuits as artificial image processor)

Reporting period: 2021-07-01 to 2024-04-30

Summary of the context and overall objectives of the project

NEoteRIC’s primary objective is the generation of holistic photonic machine learning paradigms that will address demanding imaging applications in an unconventional approach providing paramount frame rate increase, classification performance enhancement and orders of magnitude lower power consumption compared to the state-of-the-art machine learning approaches.

NEoteRIC’s implementation stratagem incorporates multiple innovations spanning from the photonic “transistor” level and extending up to the system architectural level, thus paving new, unconventional routes to neuromorphic performance enhancement.

The technological cornerstone of NEoteRIC relies on the development and upscaling of a highspeed reconfigurable photonic FPGA-like circuit that will incorporate highly dense and fully reconfigurable key silicon photonic components (ring resonators, MZIs, etc.). High-speed reconfigurability will unlock the ability to restructure the photonic components and rewire inter-component connections.

Through NEoteRIC the integrated photonic FPGAs will be strengthened by the incorporation of novel marginal-power consuming non-volatile high-speed phase shifters that will push the boundaries of energy consumption. NEoteRIC’s “unconventional” chips will be utilized as a proliferating neuromorphic computational platform that will merge the merits of photonic and electronic technology and will allow the all-optical implementation of powerful non-von Neumann architectures such as Reservoir Computing, Recurrent Neural Networks, Deep Neural Networks and Convolutional Neural Networks simultaneously by the same photonic chip.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

In the second period the project was focused on

The completion of the design for 2nd generation devices targetting low loss, high fan-in/fan-out capabilities and high density of tunable basic units. This activity also includes the design of BTO phase shifters and their co-integration with silicon photonic waveguides.

The fabrication of the 1st generation and 2nd generation chips that include high density programmable engines based on thermo-optic and BTO phase shifters.

The high performance optical and electronic packaging of the chips in order to provide functional prototypes to the evaluation work-packages.

The development of flow cytometry testbeds relying on single pixel microscopy-imaging based on STEM technique and on event-based sensing offered by PSE cameras.

The characterization of the functional prototypes as devices in terms of losses, power consumption, reconfiguration speed, supported bandwidths, etc.

The evaluation and assessment of the prototypes in artificial intelligence tasks and in flow cytometry experiments.

The dissemination of NEoteRIC's outcomes to high impact journals and conferences, the organization of specialized workshops and the participation in all significant events such as OFC, ECOC, SPIE, CLEO, IEEE conferences.

The preparation of specific exploitation plans for all partners with an emphasis on the industrial take-up of NEoteRIC results.

In summary, NEoteRIC has managed to demonstrate programmable engines that surpass the current state of the art in specs such as insertion losses (< 3 dB), number of supported basic units (>140), controllability using sophisticated electronics and power consumption, reconfiguration speed with the use of BTO phase shifters (< 1 uW/pi and > 30 MHz respectively). Moreover, NEoteRIC has technologically demonstrated unique implementations such as micro-bumps for minimizing insertion losses, BTO on silicon integration, non-volatile optical weighting. Finally, NEoteRIC has managed to propose innovative and unconventional computing methods based on photonic processing for image processing at very high speeds and mitigation of transmission impariments in high baud rate optical communication systems, whilst it paved the way for the inclusion of event-based sensing in high robustness and performance cytometers. The industrial partners have already identified routes for exploiting the outcomes of the project in different sectors (datacom, co-packaged optics, flow cytometry, telecom applications, programmable photonics of next generation, analog computing, etc). A joint patent between UoA-UGent is also another achievement of this project which will most probably lead to a new spin-off company. The project also constituted the basis for new EU funded projects that will move towards the maturing and expansion of the technology.

Not all the objectives of the project were accomplished due to COVID-19 implications and issues related to fabrication infrastructures of CEA and LUM. However, the final outcomes show that all initially specified objectives are tangible and were met at a satisfactory level in the context of the project thanks to the extension provided by the EC.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The in-project excellence has been tested through demanding high impact application such as high frame-rate image analysis and in particular single-pixel time-stretch modalities thus pushing the boundaries of state-of-the-art; exhibiting simultaneous high spatial resolution and Gframe/sec processing rate.

NEoteRIC’s concept related to utilising analog neuromorphic processing for high-speed cytometry has already attracted the interest of key players in the market of medical imaging. Moreover, specific architectures of reconfigurable photonic chips suited for machine learning have already generated intellectual property in the form of a patent in the context of the project.

The ever increasing needs for processing power and energy consumption due to artificial intelligence take-up has created a severe deadlock in its further evolution. The introduction of more efficient processors that will perform in-memory computing in an analogue and unconventional manner is a must and creates the conditions for a new geopolitical game related to the access to high performance chips suited for artificial intelligence. EU must have leading role in this and projects as NEoteRIC can contribute to its digital sovereignty. The achievements of NEoteRIC showcase alternatives in image classification that provide high accuracy at lower power consumption as the convolutional part is undertaken by photonic accelerators of minimal consumption. Moreover, the technological developments (BTO on silicon, non-volatile weighting, low-loss coupling, superb electronic controllability, etc) make the core objective for high performance programmable photonic chips a robust and promising route towards new generation processors based on the power of light.

Schematic depicting basic NEoteRIC operation

1st generation prototype of NEoteRIC's photonic processor

Schematic of the cytometry experimental testbeds