Autonomous systems process sensor data using methods of artificial intelligence (AI) in order to act adequately even in complex situations. As a basic requirement such as for the safe operation of unmanned drones or air taxis in urban areas and at elevated velocities, data – typically from several complementary sensors – must be detected, processed, and used for navigation with low latency and energy consumption. Onboard processing of the very large amounts of data, including images from high-resolution cameras, in real time, and simultaneously with high accuracy and energy efficiency, still poses a challenge.

Bauhaus Luftfahrt analysed emerging technology solutions with high potential for a range of Edge AI applications. In particular, bio-inspired, synergistically usable approaches of data acquisition and processing show up to several orders of magnitude lower latency and energy demand than conventional technology, often with comparable performance. These include event-based vision sensors that provide significantly less but more relevant data for the interpretation of dynamic scenes. They show their advantages particularly in poor light conditions and at high speeds. In addition, neuromorphic computing, which is modelled after the brain in terms of architecture and dynamics, allows to avoid time- and energy-consuming data transfer between the processor and memory. The first neuromorphic chips are based on digital or analogue electronic circuits. In the long term, the young field of neuromorphic photonics could significantly increase computing power and efficiency. The inherent benefit of high bandwidth and low propagation losses in pho- tonic integrated circuits is already evident for approaches that use light for calculations in machine-learning processes.

 

Event­based (neuromorphic) versus standard frame­based approach

Pixel-based recording of intensity change events instead of brightness images at a fixed rate excludes redundant (background) information and yields spike trains (see (b)) directly processible by neuromorphic chips.

Energy demand and computing power per area

From a physical point of view, electronic technologies are rapidly approaching their limit. For dedicated, data-intensive AI applications, optical hardware alternatives show great potential.