Watching videos can be immersive, but unless it's a game, you never have any control over the characters in them. The viewer cannot make them sit, turn around, or sneeze, if they want to – they are prisoners to what has been produced.

But now, a team of researchers, part of MIT's Computer Science and Artificial Intelligence Laboratory, is trying to change just that. It has created a way to allow people to use the cursor to interact with objects in a video, just as you would in real life: by touching, poking and prodding.

Advertisement

This technology can be applied to create special effects, especially low budget ones, where animated characters can be made to interact with their surroundings. Apart from that, the load that a bridge can carry or the safety of the structure of an old building can be tested.

Normal cameras are used but an algorithm analyses vibrations in the video to imagine the directions different objects in the video would take. Here's an example of how the technology could make Pokémon GO better, by allowing the floating creatures to actually react to the surroundings they magically appear in. It's a small improvement, but it shows the way.