Eyes of the world
Computer vision, otherwise known as artificial vision, is rapidly beaming into fields other than academic research and uses in factory automation processes such as quality inspection. With the appropriate platforms and tools, the emerging possibilities now seem endless in terms of wearable applications, augmented reality, surveillance and ambient assisted living, among other applications. Vision, our richest sensor, involves mining Big Data from reality. The amount of data generated worldwide by image sensors dwarfs that created by all other sensors combined. “For the first time, this challenge posed by the sheer processing power required by artificial vision has been met head-on by our project EoT,” explains EoT project coordinator Prof. Oscar Deniz Suarez. “The aim was to build a power-size-cost-programmability optimised core vision platform that can work independently and also embedded into all types of artefacts,” he continues. As a spin-off, the capability to perform deep learning inference was not among the project’s original goals but was added later as a very desirable outcome according to the consortium. Efficient embedded computer vision with deep learning The result is a platform for incredibly efficient embedded computer vision made possible by the key hardware element, an ultralow-power Myriad 2 processor by Movidius. “Notable features include deep learning inference and low-power wifi with both lightweight messaging and video streaming, able to send alarms to devices, internal battery charger and audio connector,” lists Prof. Suarez. The board can currently interface 3 different cameras and the physical board dimensions are small: 48x56 mm. The hardware board was developed in the first half of the project on the principle of component removal to shrink the end product. Software development could then continue in parallel with existing units without major impact. Three demonstrators, a multitude of applications Project partners developed three demonstrators to illustrate the potential functions of the technology: a doll where the board is embedded into the body and head, a headset and an illegal littering detection system. “It’s worth noting that all the systems comply with privacy-first design and no images are recorded or therefore sent out,” emphasises Prof. Suarez. The doll illustrates the deep learning function in that it recognises one of six possible facial expressions. Moreover, EoT estimates that this could be done continuously for up to 13 hours on one charge from a flat 4 000 mAh battery. A headset built for museums automatically recognises a painting and provides the visitor with relevant audio information. The EoT board in the headset also connects with a smartphone app for multimedia/interactive experiences. The system was piloted at the worldwide-renowned Albertina museum in Vienna and will be the basis for a proposal for a new project, the last steps towards productisation. Another demonstrator is the ‘Litterbug’, an illegal littering EoT device that prevents litter dumping by first detecting the act with a camera and then issuing audio warnings as the offence occurs. Commercialisation – the future for computer vision A start-up, Ubotica Technologies was borne from partners’ ex-employees with not only a licence to develop a wide range of products based on EoT but also the necessary experience. “The Myriad 2 chip at the centre of EoT is a complex device to master, integrate with other electronic components and deploy,” Prof. Suarez stresses. This includes the complex software associated with the processor, sensors, communications and optimised vision as well as the deep learning inference. “Ubotica already has the expertise to develop EoT variants in the shortest time,” Prof. Suarez concludes.
Keywords
EoT, deep learning, computer vision, sensor, artificial vision, Myriad 2
