INESC TEC and Bosch Car Multimedia collaborate to ensure safety and well-being inside self-driving vehicles

Within the scope of the Easy Ride Project, promoted by Bosch Car Multimedia and co-promoted by the University of Minho, INESC TEC collaborated in the design of models for the processing of audio-visual data, which allow characterising the activity of occupants of shared self-driving vehicles, while detecting unwanted behaviours.

Pedro Miguel Carvalho, researcher at INESC TEC, explained that “the technological solutions stemming from this collaboration contribute to the development and application of mechanisms that ensure safety and well-being inside the vehicles, crucial to the success of self-driving vehicles”. In addition to the image processing technology, it was also necessary to make sure that “the developed solution could be executed on hardware installed in the vehicle itself”, he added.

The team also developed techniques for algorithms and models’ compression and acceleration, enabling the processing of video data in real-time. Through cameras installed inside vehicles, it was possible to characterise the behaviour of occupants and timely detect unwanted actions such as violence, harassment or sickness.

Niklas Hammerschmidt, Product Owner at Bosch Car Multimedia states that “INESC TEC and Bosch have not only analysed and considered highly complex problems, such as violence detection in vehicles, but have also developed an application that allows the ride-hailing and car-sharing market to be made safer in the long term. We at Bosch are pleased to be able to look back on such a successful and cooperative collaboration and look forward to future joint projects.”

A public demonstration took place at the end of 2021 to present the potential of the solution developed. The event took place at Altice Forum Braga, with António Costa, Prime Minister, and Pedro Siza Vieira, Minister of State, Economy and Digital Transition, as guests.

The Easy Ride Project aimed to generate knowledge and technology in the fields of smart sensors, human-machine interfaces and V2X (vehicle-to-everything) communication infrastructures, in order to find innovative solutions for the future of smart mobility. INESC TEC collaborated in sub-programme 5, focused on the emotional monitoring of occupants.

The INESC TEC team in charge of developing the technology included Pedro Miguel Carvalho, João Pinto, Leonardo Capozzi, and Jaime Cardoso from the Centre for Telecommunications and Multimedia (CTM).

Following this EasyRide, two new projects are starting in collaboration with Bosch Car Multimedia, in the field of smart mobility: the THEIA project, which aims to improve the sensory capabilities of self-driving vehicles, through the implementation and validation of perception algorithms supported by data collected from sensors, building an accurate, robust and safe view of the vehicle’s surroundings; and the AURORA project, which aims to improve the data cycle and the development and processing of models to characterise the activity and emotions of passengers in said vehicles.

The INESC TEC researchers mentioned in this news piece are associated with INESC TEC and UP-FEUP.