Saving Lives: Technology at the Helm

From the wheel that allowed humans to spread around the world, to the robot vacuum cleaners that entertain (and scare) cats; technology has been impacting humankind for many years. But to what extent do we benefit from it?

The oldest known human technology was created about two million years ago. The first stone tool appeared during the Palaeolithic. An oval rock, chipped on one side to obtain a sharper tip. It was used in several tasks – namely to prepare meat and cut plants. Then came fire, the wheel, irrigation systems, ships, iron, engines and the list goes on to this day. Where curiosity went, technology followed suit. Humans have been inventing since they existed, but during the second half of the 21^st century, technological expansion was accompanied by another boom – that of life expectancy.

In 1850, people from Europe lived until their early 30s. One hundred years later, the average life expectancy almost doubled. For example, inventions like fertilisers are estimated to have saved a billion lives, while sensors and wearable devices save two million every year. Technology has contributed to extend human life and continues to do so.

However, it would be unwise – and perhaps a little naïve – to disregard the other side to this story.

Technology has been widely criticised for several reasons: it claims lives in accidents, it deeply affects the environment, and it already has the power to interfere with the democratic process through advertisements. Even if we can estimate how many lives technology has saved, the question remains if we will ever be able to calculate how many lives technology has claimed (some believe that, one day, it will claim them all). Even so, the human being can’t seem to “escape” technology, because we can’t stop creating it. At INESC TEC, there are many people who are fully focused on the research and development of new tools that save lives, in many ways – e.g., marine rescue.

Saving lives on water autonomously – ICARUS

The ICARUS project (Integrated Components for Assisted Rescue and Unmanned Search Operations) came to an end in 2016, and it demonstrated that it is possible to carry out marine rescue operations autonomously. According to José Carlos Alves, researcher at INESC TEC’s Centre for Robotics and Autonomous Systems (CRAS), the project focused on developing sensor technologies that enabled the detection and rescue of lives on water. INESC TEC’s team developed a platform capable of autonomously providing help to people on water, using data collected by the sensors.

The researcher drew a simple scenario, with a devastating outcome: imagine an incident on water, like a boat capsizing on the Tagus River. It leads to dozens, or hundreds of people scattered and floating on the water. In this type of situation, the victims’ tendency is to form groups. The weather is bad and foggy, and it’s night-time; the rescue boats crew have a hard time rescuing people in these conditions. This is where the rescue system demonstrated during ICARUS comes in.

There are four major elements in this rescue system: a control centre, drones, a large autonomous surface vehicle (called ROAZ), and smaller autonomous vessels, carrying self-inflatable life rafts.

First, the drone equipped with thermal sensors detects and locates the people on the water, from the heat emitted by their bodies. This information is sent to a control centre that maps and locates the groups. This was the first part of the project: to automatise the detection victims.

Then, the autonomous vessels come into play. The data is transmitted to ROAZ, a vehicle that carries small boats with life rafts. Upon receiving the information, ROAZ automatically defines a route that minimises the time needed to get help. Ideally, the entire control system defines the route considering the sea conditions and the weather. The small boats follow the defined route, communicating with the ROAZ through antennas; after arriving near the victims, they deploy the inflatable life rafts. Although the entire mission is based on autonomous systems, there is a fifth agent during the process: people who control the entire mission on land.

“There are several solutions available in the market, like drones that patrol beaches and throw inflatable life jackets to help people at risk. But they were never integrated into an autonomous search and rescue system,” added José Carlos Alves.

The demonstration of an autonomous rescue system is the first step in a long road, in which technology protects human lives. But technology can do more than saving lives – it can also instruct those who protect us.

Training without that burning smell – PERFECT

Firefighters go through several training actions before they are ready to go to the field. Some of these activities involve live fire training. This can lead to a significant loss of money and resources – and, in some cases, to casualties. The annual report of American firefighter fatalities identified five deaths in training sessions during 2019. The PERFECT project aims to change this scenario, using virtual reality.

The project’s technologies are currently being developed at INESC TEC virtual reality laboratory MASSIVE. The main goal is to create highly immersive training sessions where recruits can simulate missions, using immersive glasses, headsets, and controllers. Although this description sounds like a Playstation game, there is one critical factor that makes this initiative unique: it appeals to almost all senses.

“We don’t focus exclusively on audio-visual stimuli. We have been using technology based on heat generating devices and thermoelectric stimulation, and we also resort to smell synthesisers. This way, we provide more stimuli, and make people process more information, which favours training in decision-making ,” explained Miguel Correia Melo, researcher at INESC TEC’s Centre for Information Systems and Computer Graphics (CSIG).

The thermoelectric stimulation device is crucial to a very specific training that involves the detection of the neutral plane in the doors inside a burning building. The neutral plane is essentially a line in a door: above that line, the temperature is extremely high (it can reach 800 degrees) and below, it is safe to pass. In rescue missions, the firefighter places the back of his/her hand close to the bottom of the door and slowly ascends until he/she feels a sudden difference in temperature. This is the neutral plane, and it is only possible to enter rooms with temperatures below this margin. According to Miguel Correia Melo, who has collaborated with volunteer firefighters from Vila Real, it is very difficult to simulate this problem in a real training environment.

“The thermoelectric stimulation device we developed is placed on the back of the firefighters’ hands, allowing them to do the entire procedure in virtual reality,” he added.

The researchers from at MASSIVE are in charge of designing what happens in immersive reality. Everything starts from conversations between firefighters and researchers, to identify the most viable and relevant exercises to prepare in virtual reality. After that, some members of the team focus on multimedia and the creation of 3D models (the most visual part of the process), while others use engineering and programming tools to develop the software itself.

The set-up components are like markers that allow the algorithms to perceive the person’s extremities during the session: the glasses, the controllers in both hands, two devices in the feet and one in the waist. “The algorithm collects the data from the markers and turns it into a ‘skeleton’. Then this ‘skeleton’ is associated with the diagonal mesh (the 3D object of a character) and we are able to update, in real time, the movement of the 3D character – in this case, a recruit,” explained the CSIG researcher.

Apart from the base set-up that includes the glasses, headphones and controllers, there are other elements that can add even more realism to the session. Recruits put on their uniform and a warm water circulation vest, which increases their body temperature. Then they use the handheld device mentioned above, actual axes to simulate actions and even oxygen bottles to train fatigue management. These sessions have already revolutionised the training of future firefighters, but the goal is not to completely replace the training in real scenario.

“In some exercises, it may not make sense to replace training in real contexts, but rather to complement it with virtual reality,” added Miguel Correia Melo. “In the future, we aim to increase the realism of simulation environments, while providing authoring tools that allow trainers to create new complementary activities”. Even though PERFECT ended in June 2021, the researchers involved wish to keep working on the project. “The firefighters we work with are very interested in continuing the work”, he concluded.

Susana Rodrigues, a psychologist and researcher at INESC TEC’s Centre for Biomedical Engineering Research (C-BER), also supported the project; she believes that training in a virtual reality environment is useful and potentially effective. “Virtual reality is a promising tool in many different areas. Naturally, these virtual environments must be realistic enough, so that users can actually benefit from a feeling of presence and immersion.”

But what if, in real-life situations post-training, technology continued to provide support by monitoring the health of these firefighters? In 2019, the solution to this issue was created at INESC TEC and in 2021, researchers will smooth the edges.

Detect Life to Preserve It – VR2MARKET

If you ask yourself whether your job is stressful, you’ll most likely say “yes”. And you wouldn’t be the only one saying that. According to a survey by the American Institute of Stress, 40% of respondents said that their job is very/extremely stressful. And in stressful environments, we are less efficient, and we make more mistakes. But while in certain jobs, a mistake leads to a wrong e-mail message, for air traffic controllers or neurosurgeons, the scenario is quite different. More than 10 years ago, an idea came emerged at INESC TEC: to help monitor these workers’ vital signs. The goal? Support the human resources management and avoid mistakes.

The research line VR (Vital Responder) comprehended three consecutive projects: VR1, VR2 and VR2Market. From their launch until 2019, the technologies have been tested successfully. Duarte Dias, C-BER researcher, explained that there is a strong cooperation between researchers and professionals such as firefighters, but that the technology can be used among other professionals, like staff in oil companies, neurosurgeons, and air traffic controllers. “We aim to explore stressful and dangerous environments, which can put the health of professionals at risk. For example, there are firefighters who develop lung cancer because they are exposed to gases for too long. We even have a partner in the United States – a retired firefighter – who is very supportive of this initiative because he believes that this type of disease is avoidable if the vital signs of first responders are better managed,” he concluded.

But how are vital signs detected? Researchers have developed a wearable sensor, a technology now patented, called the Vital Sticker. It is a small, wireless, flexible device with cardiac electrodes to perform the user’s electrocardiogram. It is applied to the chest of the person under monitoring, the information is sent via Bluetooth to patented algorithms that quantify stress based on cardiac data.

“Regarding air traffic controllers, these technologies could be useful for a team supervisor to automatically and effectively understand if any member feels sick or unable to keep up with their duties”, exemplified the C-BER researcher.

Although it sounds like an objective work, combining absolute cardiac data with subjective stress scales was not that simple. This required support from an expert in another area of knowledge – psychology.

“Since stress is a complex and multidimensional process, the isolated analysis of physiological variables by itself does not allow us to make a realistic assessment. Therefore, it was important to combine psychophysiological variables, using a multidisciplinary approach,” explained Susana Rodrigues.

Psychophysiology is the science that studies biological structures that regulate human behaviour. In VR2MARKET, in order to create an algorithm that detected stress levels based on cardiac rate, it became important to understand how stress manifested itself in users. And for that, the team used a popular test in Psychology, the TSST (Trier Social Stress Test). In this procedure, participants are asked to give a speech before an audience for three to five minutes, and perform a complex arithmetic task (for example, counting down from 1022 by subtracting 13 to each number, for two minutes), and whenever they make a mistake, they must come back to the beginning. The audience does not respond emotionally during the test, which makes the situation stressful for the participants.

“During the data collection process of VR2Market, and besides collecting data in real context, we used this procedure [TSST] to obtain a quantitative measure of stress, in a controlled environment. Participants filled out stress surveys before and after the test, while being monitored continuously with our devices,” explained the psychologist. Afterwards, engineering professionals resorted to machine learning techniques to develop the algorithm that VitalSticker uses (the technology was renamed Beat Stress).

The VR2Market project was a success, but the story doesn’t stop there. A European project is expected to start in December, following up on this initiative; the goal is to further improve these technologies before introducing them in the market.

At INESC TEC, there are more technologies to explore. Not all of them focus literally on saving lives, but the researchers’ goal is to make innovation one of our top allies. From the chipped rock of the Palaeolithic, we came to ICARUS, PERFECT and VR2Market. And even though an instrument used millions of years ago is so different from autonomous boats, virtual reality and sensors, the goal has always been the same: to serve humankind.

The researchers mentioned in this news piece are associated with UP-FEUP and INESC TEC.