INESC TEC’s Centre for Information Systems and Computer Graphics (CSIG) developed a 360 video editing and viewing tool, which enables the creation of interactive experiences that explore not only the 360º video aspect, with interactive visual elements, but also the interactive sound component.
This solution was developed within the scope of the Augmented Video 360 project and tested in partnership with the Público newspaper, at Serralves (Porto), during a harp concert by Spanish artist Angélica Salvi.
“The idea of creating a 360 video editing and viewing tool, with dynamic annotations to improve narratives, came about in 2017. The technological solution was developed by INESC TEC and the Público newspaper, joining as a media partner to test the technology. The video developed at Serralves is a sort of first outcome, accessible to the general audience, illustrating the tool’s potential – not only in terms of journalism, but also considering cultural initiatives, explained Rui Rodrigues, researcher at CSIG.
The video was first recorded with a 360 camera and later edited with a tool that allows content creators or editors to add annotations, 3D oriented subtitles or other type of interactive visual effects, as well as sound elements – depending on the viewer’s orientation, which are relevant to complete or improve the narrative and experience.
In the video – already available on the Público newspaper website, one can watch and hear the harpist playing in different spaces at Serralves, in 360. The video includes dynamic annotations, which add information about the harp, the new record and the artist’s history.
The project is carrying out other tests, with emphasis on the exploration of interactive audio, which allows, for instance, “to create an experience in which certain elements (instruments) only emit sound when spectators look at them, or even notification sounds when the spectator is not watching”, explained Rui Rodrigues.
The editing tool is currently available to project partners, but other interested parties can request access, by getting in touch via the project website: https://av360.inesctec.pt/.
Augmented Video 360 was a project associated with digital journalism, funded by Google’s Digital News Initiative innovation fund – two rounds of funding (in 2017 and 2019). The goal was to bridge the gap between content creators and the technology needed to add dynamic annotations to 360º videos, one of the great trends in digital journalism.
The INESC TEC researcher mentioned in this news piece is associated with UP-FEUP.
News, current topics, curiosities and so much more about INESC TEC and its community!
