Immersive learning: it’s not fiction, it’s the future.

We are all closer, yet more physically distant from one another. The social challenges posed by the 2020 pandemic boosted digital transformation. We started to meet, work, and attend training actions online, while becoming more technologically autonomous and influenced by innovation. Immersive environments (like virtual reality, but not exclusively) are no longer the subject of literary or cinema works. But there seems to be a certain degree of fear[1]. Are we oblivious to reality? Are we going to lose our emotional connections?

Let’s slow down a bit! Let’s take a trip back in time; in the 19^th century, several illustrated posters were published, showing electrocuted people in the tangle of wires of a new technology called electricity. People envisioned a dark future, in which thousands (if not millions) would die, caught in a high-voltage web. It didn’t happen! In 1950, writer Isaac Asimov released I, Robot, a collection of 10 short stories that, combined, depict a 21^st century dominated by intelligent robots (but without any trace of morality), which threaten the human species. But this has not happened (yet). As to the movies, the Wachowski brothers have created a dystopian future in which everyone lives in the Matrix, an AI simulated reality. Almost 25 years later, and after several advances and setbacks in AI, we are far from being overwhelmed by self-aware machines. In 2011, the book Ready Player One imagined a world collapsing in the year 2044, where virtual reality is more than an escape: it’s the only way for people to interact. “Video killed the Radio Star“? Not really! Have we really stopped reading books because of TV and mobile phones? The statistics say that thanks to the BookTok (community of readers who share video recommendations on TikTok) there are more and more readers (and, you’re one of them!).

Training choreography: the future of immersive environments

Virtual Reality is a fantastic technology to provide effective, safe, and controlled training tools in realistic scenarios. However, the implementation of such training programs requires a significant effort – from the costs associated with knowledge transfer, to the acceptance of technology by employees of certain companies. The VRTrainingIndustry aims to address this issue, as a solution to the limitations of immersive environments in training contexts.

“Whenever I seek a 3D training simulator, I hire a team of designers and computer scientists to (re)create a space where you can use tools and perform concrete tasks. But after a few months, this training becomes outdated, since the process has changed; hence, it becomes necessary to call the computer team to reprogram the platform. This is what usually makes it impossible to use immersive environments in training; it is an overly large upgrade investment”, explained Leonel Morgado, researcher at INESC TEC Centre for Human-Centered Computing and Information Science (HumanISE) and head of the project.

This tool, which makes it possible to monetise immersive technology, was initially applied to training on wind turbine maintenance procedures at a Danish multinational company. In the virtual environment, there is descriptive information and actions to perform, which are structured as procedures, as explained by the HumanISE researcher. Information about the course structure, environments, tools, and models is stored locally, and all actions and procedures are converted into virtual choreographies (and no, we are not talking about dancing, but behavioural patterns in time and space – meaning that coordination is still an active element).

“In an interactive way, the trainer creates and edits courses and their structure, through the immersive interface, configuring the environment, selecting or limiting the available interactions, or adding technical documentation. This also means that we can perform the same simulation using another technology – for example, by simply redoing the mapping. If you have defined 800 tasks, I can reapply them to another platform. In a virtual choreography, there are no X, Y or Z coordinates, no K-second timestamps, but more objective actions like removing a lid, closing a door, etc. In addition, these actions can then be analysed more intelligently and meaningfully. This greatly increases the usefulness of 3D simulation”, he added.

This concept of choreography is also behind the FronTowns project, which aims to reconstitute and study the evolution of the urban space of Castelo de Vide and Cáceres. The same historical narrative can be told in different spaces and platforms, without the risk of losing its usefulness with the evolution of technologies.

“We make the stories in a storyboard tool. They can then be viewed in a 3D space or via smartphones or tablets, in the actual physical space. We use the narrative of the stories to provide a context to what we see, and we have the option of creating interactive stories to define what happens next, while moving around the environment”, said Leonel Morgado.

This opens the possibility of learning about history(s), experiencing it/them. “Perceiving environments as immersive allows us to direct the intervention of educators and creators of environments to more than just visual content. It also allows us to consider the narrative that grants meaning and agency opportunities” he concluded.

A marketplace of technological solutions for training

The main goal of Redvile is to promote the use of technologies like Virtual Reality (VR) or Augmented Reality (AR) in training actions. The project started in 2021, thanks to researchers António Almeida, from TEC4Industry and Ana Simões, from the Centre for Enterprise Systems Engineering (CESE) – who explained that the main idea was the development of a platform with commercial features, capitalising on immersive learning technologies.

“INESC TEC joined the project after an invitation by EIT Manufacturing, with the objective of creating a kind of marketplace of immersive technologies that could be used in training actions. Through a survey, it’d be possible to understand the ideal teaching methodologies and associated equipment, based on information like the available budget, target-audience, objectives, and duration. This would lead to a “turnkey” solution”, she claimed.

With the methodology created, António Coelho took the reins of the project to move forward with a recommendation platform. “In this second phase, we are operationalising what has been designed, presenting the most appropriate immersive solutions, considering three main axes: technology, narrative and challenge. The idea is providing recommendations based on previously studied examples”, mentioned the researcher at INESC TEC HumanISE.

According to Ana Simões, Redvile could play a key-role in terms of improving the potential of immersive environments in training. “What happens is that trainers and teachers have ideas, but they don’t know what equipment to use; or they have the required equipment, and they don’t know how to apply it. This platform can translate teaching and training objectives and needs into concrete suggestions for the adoption of AR and VR technologies. People need to see it to believe it”, she said.

And if you don’t believe that immersive environments can even contribute to a more innovative training experience, just look at the examples developed in partnership with BIBA (Bremer Institut für Produktion und Logistik) and TU Delft (Delft University of Technology): an AR pilot that simulates sound, thermal sensation and even sparks in the welding of parts. It is a very complete immersive experience that does not require the presence of a technician, reducing waste in terms of materials and consumables, while enabling the evaluation of performance in real-time.

Any company can now access the platform and use this recommendation system to plan their immersive training. “We have been working on creating immersive environments tailored to each company’s platforms. Ideally, we seek agnostic platforms that can be adapted and integrated into any learning system, of any company”, mentioned António Coelho.

Is this the future of education?

Immersive environments allow faster and more complete learning, training in extreme situations (e.g., aviation), and remote collaboration protocols, reducing costs in terms of experts’ travelling. The associated hardware has lower associated costs, which translates into faster investment returns.

The benefits are in plain sight and, according to Ana Simões, the future of the educational system may even involve the use of immersive technologies. “Today, kids have access to smartphones, tablets, computers. Technology is normal for them, and if we can use it to educate, we must do so – especially when learning in a real environment is not possible. We are still far from being a standard, but it’s crucial to adopt drastic changes in teaching methods, particularly due to the current crisis”.

Leonel Morgado goes even further: “through immersive solutions, we can better perceive the concepts of geometric progression, rather than just repeating geometric progression formulas a hundred times, while expecting each person to intuit the pattern ‘even small differences in rhythm make a difference’.” This does not require less effort; it may even require more, because a hundred calculations are mechanised – even performed without much need to think after some time. Rather, it allows us to acquire a deeper understanding of our efforts”.

The three researchers agree that total alienation, by virtue of the use of VR and AR, is a fantasy. Normally, changes are never that radical nor the results so invariable. Hence, it’s up to us to keep crossing this bridge that takes us from where we are to where we want to go[2].

The researchers mentioned in this edition of Spotlight are associated with INESC TEC, UP-FEUP and UAb.

[1] Virtual reality headsets could put children’s health at risk, The Guardian
The very real health dangers of virtual reality, CNN

[2]Tom Furness, Virtual Reality pioneer and Professor of Industrial Science and Engineering at the University of Washington