<p>You can use photogrammetry if you had enough different viewpoints to build a 3d representation of the scene and apply all necessary texturing and lighting.  The video would then play as an animated 3d scene rather than a traditional movie.  </p>

<p>The amount of processing involved is high but it is possible.  Microsoft's Hyperlapse video stabilization actually does some of this by determining a point cloud of the scene from some basic photogrammetry.</p>

<p>Software like Destinations on the Vive already use photogrammetry for reproducing landmarks in a fully explore-able 3D world.  There are also several animated projects which are composed of scenes that you can fully explore.  8i also has done some early work in video like you describe.  The quality level still leaves a lot to be desired, but work is being done in trying to accomplish what you are talking about.  </p>

<p>As an early Vive owner who has tried all these things first hand, I'd say "volumetric video" is at the interesting experimental phase, but not really that great yet.  I'm sure we'll see significant improvements in the future though as the technology is refined, particularly since it has implications outside of just VR and AR for things like computer vision.</p>

CC BY-SA 3.0
2017-07-01T04:07:47.630Z
I found out that these videos are called "light field videos" or "holographic videos".  I guess "volumetric videos" is yet another word for them.  In any case, I have an Oculus Rift, not an HTC vive.  Do you know any apps or experiences for Oculus Rift that show such videos?
Keshav Srinivasan
CC BY-SA 3.0
2017-07-01T04:25:32.913Z
Light field is something different.  That's what the Lytro shoots and is still single direction but contain focusing and depth information as it records the direction of light rather than just the intensity.
AJ Henderson
CC BY-SA 3.0
2017-07-01T04:37:14.423Z
I think some people are using the term "light field video" as a synonym for "360 light field video", i.e. the sort of thing that the [Lytro Immerge](https://www.lytro.com/immerge) shoots, which is synonymous with "volumetric".  In any case, it looks like that "8i" app you were referring to is available for Oculus Rift: http://store.steampowered.com/app/462530/8i__Make_VR_Human/ Do you know any other Oculus Rift apps or experiences that showcase volumetric video?
Keshav Srinivasan
CC BY-SA 3.0
2017-07-01T04:52:45.747Z
Volumetric means it has volume, ie geometry, light field is significantly different in how it works since light field is still relatively fixed perspective.
AJ Henderson
CC BY-SA 3.0
2017-07-01T05:05:49.123Z
I'm seeing people use light field to describe what 8i is doing, and volumetric to describe Lytro Immerge (and vice versa).  Are 8i and Lytro Immerge (as opposed to regular Lytro) doing fundamentally different things?
Keshav Srinivasan
CC BY-SA 3.0
2017-07-01T05:13:25.023Z
Yes, they are different from my understanding.  A traditional camera captures the amount of light that hits a particular spot.  This capture doesn't understand the direction of light and therefore can't adjust the focus of the image.  A light field actually captures direction of light hitting a particular spot so it can change focus and reproduce what light conditions were actually like for that spot, but it is limited to just that spot.  True full volumetric has multiple angles for complete geometry and texture reproduction.
AJ Henderson
CC BY-SA 3.0
2017-07-01T05:15:07.790Z
It is possible 8i is using multiple light field cameras to produce the volumetric content more easily though.  The depth information you can determine from a light field would make volumetric reconstruction much simpler.
AJ Henderson
CC BY-SA 3.0
2017-07-01T03:50:53.037Z
2017-07-01T03:50:53.037Z
AJ Henderson
<p>For this you are going to have to build a 3d-world. This is because if you want to move position you are going to have to be moving in 3d. The easiest way to build up the 3d scene is with photogrammetry like Aj Henderson said. </p>

<p>I would load it into a game engine because a game engine is designed to render 3d scenes read time and already lots of them have vr features built in. For example Unreal Engine or Unity. </p>

CC BY-SA 3.0
2017-07-01T19:09:44.303Z
But then how would you show such a video to someone else?  Are there any apps for Oculus Rift besides [8i](http://store.steampowered.com/app/462530/8i__Make_VR_Human/) which show such videos?
Keshav Srinivasan
CC BY-SA 3.0
2017-07-01T19:17:47.597Z
To show the video you would have to have a full game set up in the game engine which you can export. In 8i case they used the unity game engine as the base for their 3d environments.
Tim
CC BY-SA 3.0
2017-07-01T18:51:25.780Z
2017-07-01T18:51:25.780Z
Tim
<p>To my understanding there are basically 4 types of video:</p>

<ol>
<li>Ordinary 2D videos - these can be viewed by pretty much any device that has a screen</li>
<li>360 videos - these can be viewed by any smartphone that has a gyroscope</li>
<li>3D videos - these can be viewed by 3D glasses along with the right kind of screen, or by using a VR headset</li>
<li>VR videos - these are a combination of types 2 and 3, and can be viewed by any VR headset, like a Samsung Gear VR headset or even a Google Cardboard headset</li>
</ol>

<p>But I'm interested in whether there's a 5th type of video.  My question is, is it possible to make a VR video that isn't just 360 + 3D, but also makes use of positional tracking?  Such a video, if it existed, would only be viewable on a high-end VR headset like the Oculus Rift and the HTC Vive, which utilize both rotational tracking and positional tracking.  They wouldn't be viewable on low-end headsets like Google Cardboard, or mid-tier headsets like Samsung Gear VR, since they only utilize rotational tracking.</p>

<p>If it is possible, how would you go about creating such a video?  It's the equivalent of shooting an ordinary (360+3D) VR video not just from one point in a room, but from every point in the room.  I imagine that it would be useful for creating virtual tours. In any case, even if you managed to create such a video, how exactly would you make that video by viewable by others?  Are there any apps for Oculus Rift or HTC Vive that would allow you to play a positional-tracking VR video?</p>

<p>EDIT: I found out that there is such a type of video.  It's called a "light field video" or "holographic video", and it can be captured with a special <a href="https://en.wikipedia.org/wiki/Light-field_camera" rel="nofollow noreferrer">light field camera</a>.  But I still don't know how you would show that video to someone.</p>

CC BY-SA 3.0
2017-07-03T17:20:37.037Z
There is a lot of ongoing basic and applied research into light field capture, display, and the technology to get from one to the other. There are similarities to what you're describing, but nothing commercially viable yet.
Michael Liebman
<video-recording><3d><360video>
Can you create a VR video that makes use of positional tracking?
Keshav Srinivasan
# Can you create a VR video that makes use of positional tracking?

(Asked by: username_0 on 2017-07-01)

To my understanding there are basically 4 types of video:

1. Ordinary 2D videos - these can be viewed by pretty much any device that has a screen
2. 360 videos - these can be viewed by any smartphone that has a gyroscope
3. 3D videos - these can be viewed by 3D glasses along with the right kind of screen, or by using a VR headset
4. VR videos - these are a combination of types 2 and 3, and can be viewed by any VR headset, like a Samsung Gear VR headset or even a Google Cardboard headset

But I'm interested in whether there's a 5th type of video. My question is, is it possible to make a VR video that isn't just 360 + 3D, but also makes use of positional tracking? Such a video, if it existed, would only be viewable on a high-end VR headset like the Oculus Rift and the HTC Vive, which utilize both rotational tracking and positional tracking. They wouldn't be viewable on low-end headsets like Google Cardboard, or mid-tier headsets like Samsung Gear VR, since they only utilize rotational tracking.

If it is possible, how would you go about creating such a video? It's the equivalent of shooting an ordinary (360+3D) VR video not just from one point in a room, but from every point in the room. I imagine that it would be useful for creating virtual tours. In any case, even if you managed to create such a video, how exactly would you make that video by viewable by others? Are there any apps for Oculus Rift or HTC Vive that would allow you to play a positional-tracking VR video?

EDIT: I found out that there is such a type of video. It's called a "light field video" or "holographic video", and it can be captured with a special [light field camera](https://en.wikipedia.org/wiki/Light-field_camera). But I still don't know how you would show that video to someone.

**Tags:** video-recording, 3d, 360video

> **Comments:**
> **username_1**: There is a lot of ongoing basic and applied research into light field capture, display, and the technology to get from one to the other. There are similarities to what you're describing, but nothing commercially viable yet.  

---

## 2 answers

### Answer by username_2 on 2017-07-01. Score: 1

You can use photogrammetry if you had enough different viewpoints to build a 3d representation of the scene and apply all necessary texturing and lighting. The video would then play as an animated 3d scene rather than a traditional movie.

The amount of processing involved is high but it is possible. Microsoft's Hyperlapse video stabilization actually does some of this by determining a point cloud of the scene from some basic photogrammetry.

Software like Destinations on the Vive already use photogrammetry for reproducing landmarks in a fully explore-able 3D world. There are also several animated projects which are composed of scenes that you can fully explore. 8i also has done some early work in video like you describe. The quality level still leaves a lot to be desired, but work is being done in trying to accomplish what you are talking about.

As an early Vive owner who has tried all these things first hand, I'd say "volumetric video" is at the interesting experimental phase, but not really that great yet. I'm sure we'll see significant improvements in the future though as the technology is refined, particularly since it has implications outside of just VR and AR for things like computer vision.

> **Comments:**
> **username_0**: I found out that these videos are called "light field videos" or "holographic videos".  I guess "volumetric videos" is yet another word for them.  In any case, I have an Oculus Rift, not an HTC vive.  Do you know any apps or experiences for Oculus Rift that show such videos?  
> **username_2**: Light field is something different.  That's what the Lytro shoots and is still single direction but contain focusing and depth information as it records the direction of light rather than just the intensity.  
> **username_0**: I think some people are using the term "light field video" as a synonym for "360 light field video", i.e. the sort of thing that the [Lytro Immerge](https://www.lytro.com/immerge) shoots, which is synonymous with "volumetric".  In any case, it looks like that "8i" app you were referring to is available for Oculus Rift: http://store.steampowered.com/app/462530/8i__Make_VR_Human/ Do you know any other Oculus Rift apps or experiences that showcase volumetric video?  
> **username_2**: Volumetric means it has volume, ie geometry, light field is significantly different in how it works since light field is still relatively fixed perspective.  
> **username_0**: I'm seeing people use light field to describe what 8i is doing, and volumetric to describe Lytro Immerge (and vice versa).  Are 8i and Lytro Immerge (as opposed to regular Lytro) doing fundamentally different things?  
> **username_2**: Yes, they are different from my understanding.  A traditional camera captures the amount of light that hits a particular spot.  This capture doesn't understand the direction of light and therefore can't adjust the focus of the image.  A light field actually captures direction of light hitting a particular spot so it can change focus and reproduce what light conditions were actually like for that spot, but it is limited to just that spot.  True full volumetric has multiple angles for complete geometry and texture reproduction.  
> **username_2**: It is possible 8i is using multiple light field cameras to produce the volumetric content more easily though.  The depth information you can determine from a light field would make volumetric reconstruction much simpler.  

---

### Answer by username_3 on 2017-07-01. Score: 1

For this you are going to have to build a 3d-world. This is because if you want to move position you are going to have to be moving in 3d. The easiest way to build up the 3d scene is with photogrammetry like Aj Henderson said.

I would load it into a game engine because a game engine is designed to render 3d scenes read time and already lots of them have vr features built in. For example Unreal Engine or Unity.

> **Comments:**
> **username_0**: But then how would you show such a video to someone else?  Are there any apps for Oculus Rift besides [8i](http://store.steampowered.com/app/462530/8i__Make_VR_Human/) which show such videos?  
> **username_3**: To show the video you would have to have a full game set up in the game engine which you can export. In 8i case they used the unity game engine as the base for their 3d environments.  

---

