Free Viewpoint Television - Wikipedia

This article includes a list of general references, but it lacks sufficient corresponding inline citations. Please help to improve this article by introducing more precise citations. (July 2020) (Learn how and when to remove this message)

Free viewpoint television (FTV) is a system for viewing natural video, allowing the user to interactively control the viewpoint and generate new views of a dynamic scene from any 3D position.[1] The equivalent system for computer-simulated video is known as virtual reality. With FTV, the focus of attention can be controlled by the viewers rather than a director, meaning that each viewer may be observing a unique viewpoint. It remains to be seen how FTV will affect television watching as a group activity.

History

[edit]

Systems for rendering arbitrary views of natural scenes have been well known in the computer vision community for a long time but only in recent years[when?] has the speed and quality reached levels that are suitable for serious consideration as an end-user system.[citation needed]

Professor Masayuki Tanimoto from Nagoya University (Japan) has done much to promote the use of the term "free viewpoint television" and has published many papers on the ray space representation,[citation needed][clarification needed] although other techniques can be, and are used for FTV.

QuickTime VR might be considered a predecessor to FTV.[citation needed]

Capture and display

[edit]

In order to acquire the views necessary to allow a high-quality rendering of the scene from any angle, several cameras are placed around the scene; either in a studio environment or an outdoor venue, such as a sporting arena for example. The output Multiview Video (MVV) must then be packaged suitably so that the data may be compressed and also so that the users' viewing device may easily access the relevant views to interpolate new views.[citation needed]

It is not enough to simply place cameras around the scene to be captured. The geometry of the camera set up must be measured by a process known in computer vision as "camera calibration."[citation needed] Manual alignment would be too cumbersome so typically a "best effort" alignment is performed prior to capturing a test pattern that is used to generate calibration parameters.

Restricted free viewpoint television views for large environments can be captured from a single location camera system mounted on a moving platform.[citation needed] Depth data must also be captured, which is necessary to generate the free viewpoint. The Google Street View capture system is an example with limited functionality. The first full commercial implementation, iFlex, was delivered in 2009 by Real Time Race.[2]

Multiview video capture varies from partial (usually about 30 degrees) to complete (360 degrees) coverage of the scene. Therefore, it is possible to output stereoscopic views suitable for viewing with a 3D display or other 3D methods. Systems with more physical cameras can capture images with more coverage of the viewable scene, however, it is likely that certain regions will always be occluded from any viewpoint. A larger number of cameras should make it possible to obtain high quality output because less interpolation is needed.

More cameras mean that efficient coding of the Multiview Video is required. This may not be such a big disadvantage as there are representations that can remove the redundancy in MVV; such as inter view coding using MPEG-4 or Multiview Video Coding, the ray space representation, geometry videos,[3][clarification needed] etc.

In terms of hardware, the user requires a viewing device that can decode MVV and synthesize new viewpoints, and a 2D or 3D display.

Standardization

[edit]

The Moving Picture Experts Group (MPEG) has normalized Annex H of MPEG-4 AVC in March 2009 called Multiview Video Coding after the work of a group called '3DAV' (3D Audio and Visual) headed by Aljoscha Smolic[4] at the Heinrich-Hertz Institute.

See also

[edit]

• 3D reconstruction
• Rendering (computer graphics)
• Volumetric video

References

[edit]

1. ^ Tanimoto, Masayuki. "FTV (free-viewpoint television)." APSIPA Transactions on Signal and Information Processing 1 (2012).
2. ^ "Real Time Racing | Automotive News". Diseno-art.com. 2009-11-02. Archived from the original on 2011-04-17. Retrieved 2010-09-13.
3. ^ "Geometry videos: A new representation for 3D animations". Retrieved 2016-12-08.
4. ^ "Joschi's Home (Aljoscha Smolic)". Iphome.hhi.de. Archived from the original on 2009-11-26. Retrieved 2010-09-13.

Bibliography

[edit]

• Carranza, Joel; Theobalt, Christian; Magnor, Marcus A.; Seidel, Hans-Peter (July 2003). "Free-viewpoint video of human actors" (PDF). ACM Transactions on Graphics. 22 (3). ACM: 569–577. doi:10.1145/882262.882309.

External links

[edit]

• Canon announced development of a Free Viewpoint TV system on 21 September 2017, to be showcased at Inter BEE 2017.
• iview is a British DTI project between BBC, Snell & Wilcox and University of Surrey to develop an FTV system.
• Eye Vision is a system developed by Professor Takeo Kanade at CMU for CBS's coverage of Super Bowl XXXV. The user is not able to change viewpoint but the camera operator is able to choose any virtual viewpoint by synthesizing images from an active vision system.
• Replay Technologies created the first-ever live on-air 3D reconstruction during the London 2012 Olympic Games; their website now seems to point to Intel freeD 360 Replay

v t e Extended reality/XR
From least to most virtual: Augmented reality (AR) Mixed reality (MR) Virtual reality (VR) Virtuality
Concepts	Main Cinematic virtual reality Computer-mediated reality Immersion Metaverse Avatar Projection augmented model "Room-scale" Six degrees of freedom (6DoF) Spatial computing Telepresence Virtual reality applications Virtual reality sickness Virtual world Other Camera filter Quantified self Screen-door effect Simulation hypothesis Transhumanism Vergence-accommodation conflict VTuber
Technologies	Display EyeTap Head-mounted display optical Head-up display Pancake lens Smartglasses Virtual reality headset list Virtual retinal display 3D interaction Brain–computer interface Eye tracking Facial motion capture Finger/hand tracking Pose tracking Simultaneous localization and mapping Software Asynchronous reprojection Foveated rendering Image-based modeling and rendering Spatial audio Photography 360-degree video Free viewpoint television Omnidirectional camera VR photography Other Haptic suit Omnidirectional treadmill Wearable computer
Peripherals	Cyberith Virtualizer Leap Motion Oculus Touch PlayStation Move Razer Hydra Virtuix Omni Wired glove Wizdish ROVR
Companies	Apple Inc. Brilliant Labs ByteDance Collabora Google HTC Khronos Group Liquid Image Magic Leap Meta Platforms Reality Labs Microsoft Niantic, Inc. Niantic Spatial Pimax Rokoko Samsung Electronics Valve Corporation Varjo Vuzix
Devices	Current Apple Vision Pro Bigscreen Beyond Golden-i headsets HTC Vive Magic Leap Meta Quest 3 3S Open Source Virtual Reality PICO 4 Ultra Pimax PlayStation VR2 Samsung Galaxy XR Vuzix Former AntVR castAR EyePhone Google Cardboard Google Daydream Google Glass Meta Quest Pro Microsoft HoloLens 2 Oculus Go Oculus Quest Oculus/Meta Quest 2 Oculus Rift CV1 S PICO 4 PlayStation VR Samsung Gear VR Sensorama SixthSense The Sword of Damocles VFX1 Headgear Virtual Boy Virtual fixture Virtuality VR-1 Valve Index Unreleased Project Iris Sega VR Upcoming Steam Frame
Software	General Interactive art Virtual graffiti Metaverse Pervasive game Software related to augmented reality Virtual reality game Operating systems anddesktop environments Android XR Meta Horizon OS version history visionOS Windows Mixed Reality SteamOS Development tools andgame engines A-Frame ARCore ARKit ARToolKit Godot Engine Jetpack Compose XR Meta Spatial SDK OpenVR OpenXR RealityKit S&box Source 2 TabletopKit Unity Universal Scene Description Unreal Engine Vuforia Augmented Reality SDK WebXR Games List of HTC Vive games List of Meta Quest games List of Oculus Rift games List of PlayStation VR games VR2 Communities andsocial networks AltspaceVR FaceTime for visionOS Horizon Worlds NeosVR Rec Room Resonite Sansar Sensorium Sinespace VRChat VTime XR
Portals Technology Category Augmented reality Mixed reality Virtual reality Metaverse Spatial computing Commons Augmented reality Mixed reality Virtual reality Metaverse

v t e Computer vision
Categories	Datasets Digital geometry Commercial systems Feature detection Geometry Image sensor technology Learning Morphology Motion analysis Noise reduction techniques Recognition and categorization Research infrastructure Researchers Segmentation Software
Technologies	Computer stereo vision Motion capture Object recognition 3D object recognition
Applications	3D reconstruction 3D reconstruction from multiple images 2D to 3D conversion Gaussian splatting Neural radiance field Shape from focus Simultaneous localization and mapping Structure from motion View synthesis Visual hull 4D reconstruction Free viewpoint television Volumetric capture 3D pose estimation Activity recognition Audio-visual speech recognition Automatic image annotation Automatic number-plate recognition Automated species identification Augmented reality Bioimage informatics Blob detection Computer-aided diagnosis Content-based image retrieval Reverse image search Eye tracking Face recognition Foreground detection Gesture recognition Image denoising Image restoration Landmark detection Medical image computing Object detection Moving object detection Small object detection Optical character recognition Pose tracking Remote sensing Robotic mapping Autonomous vehicles Video content analysis Video motion analysis Video surveillance Video tracking
Main category