Virtual camera system
 |
| Part of a series on |
| Video game graphics |
|---|
In 3D video games, a virtual camera system aims at controlling a camera or a set of cameras to display a view of a 3D virtual world. Camera systems are used in video games where their purpose is to show the action at the best possible angle; more generally, they are used in 3D virtual worlds when a third-person view is required.
As opposed to filmmakers, virtual camera system creators have to deal with a world that is interactive and unpredictable. It is not possible to know where the player character is going to be in the next few seconds; therefore, it is not possible to plan the shots as a filmmaker would do. To solve this issue, the system relies on certain rules or artificial intelligence to select the most appropriate shots.
There are mainly three types of camera systems. In fixed camera systems, the camera does not move at all, and the system displays the player's character in a succession of still shots. Tracking cameras, on the other hand, follow the character's movements. Finally, interactive camera systems are partially automated and allow the player to directly change the view. To implement camera systems, video game developers use techniques such as constraint solvers, artificial intelligence scripts, or autonomous agents.
Implementation
There is a large body of research on how to implement a camera system.[1] The role of a constraint solver software is to generate the best possible shot given a set of visual constraints. In other words, the constraint solver is given a requested shot composition such as "show this character and ensure that he covers at least 30 percent of the screen space". The solver will then use various methods to try to create a shot that would satisfy this request. Once a suitable shot is found, the solver outputs the coordinates and rotation of the camera, which can then be used by the graphic engine renderer to display the view.[2]
In some camera systems, if no solution can be found, constraints are relaxed. For example, if the solver cannot generate a shot where the character occupies 30 percent of the screen space, it might ignore the screen space constraint and simply ensure that the character is visible at all.[3] Such methods include zooming out.
Some camera systems use predefined scripts to decide how to select the current shot for commonly seen shot scenarios called film idioms. Typically, the script is going to be triggered as a result of an action. For instance, when the player's character initiates a conversation with another character, the "conversation" script is going to be triggered. This script will contain instructions on how to "shoot" a two-character conversation. Thus the shots will be a combination of, for instance, over the shoulder shots and close-up shots. Such script-based approaches may switch the camera between a set of predefined cameras or rely on a constraint solver to generate the camera coordinates to account for variability in scene layout. This scripted approach and the use of a constraint solver to compute virtual cameras was first proposed by Drucker.[4] Subsequent research demonstrated how a script-based system could automatically switch cameras to view conversations between avatars in a realtime chat application.[5]
Bill Tomlinson used a more original approach to the problem. He devised a system in which the camera is an autonomous agent with its own personality. The style of the shots and their rhythm will be affected by their mood. Thus a happy camera will "cut more frequently, spend more time in close-up shots, move with a bouncy, swooping motion, and brightly illuminate the scene".[6]
While much of the prior work in automated virtual camera control systems has been directed towards reducing the need for a human to manually control the camera, the Director's Lens solution computes and proposes a palette of suggested virtual camera shots leaving the human operator to make the creative shot selection. In computing subsequent suggested virtual camera shots, the system analyzes the visual compositions and editing patterns of prior recorded shots to compute suggested camera shots that conform to continuity conventions such as not crossing the line of action, matching placement of virtual characters so they appear to look at one another across cuts, and favors those shots which the human operator had previously used in sequence.[7]
In mixed-reality applications
In 2010, the Kinect was released by Microsoft as a 3D scanner/webcam hybrid peripheral device which provides full-body detection of Xbox 360 players and hands-free control of the user interfaces of video games and other software on the console. This was later modified by Oliver Kreylos[8] of University of California, Davis in a series of YouTube videos which showed him combining the Kinect with a PC-based virtual camera.[9] Because the Kinect is capable of detecting a full range of depth (through computer stereo vision and Structured light) within a captured scene, Kreylos demonstrated the capacity of the Kinect and the virtual camera to allow free-viewpoint navigation of the range of depth, although the camera could only allow video capture of the scene as shown to the front of the Kinect, resulting in fields of black, empty space where the camera was unable to capture video within the field of depth. Later, Kreylos demonstrated a further elaboration on the modification by combining the video streams of two Kinects in order to further enhance the video capture within the view of the virtual camera.[10] Kreylos' developments using the Kinect were covered among the works of others in the Kinect hacking and homebrew community in a New York Times article.[11]
Real-time recording and motion tracking
Virtual cameras have been developed which allow a director to film motion capture and view the digital character's movements in real time[12] in a pre-constructed digital environment, such as a house or spaceship.[13] Resident Evil 5 was the first video game to use the technology,[14] which was developed for the 2009 film Avatar.[13][15] The use of motion capture to control the position and orientation of a virtual camera enables the operator to intuitively move and aim the virtual camera by simply walking about and turning the virtual camera rig. A virtual camera rig consists of a portable monitor or tablet device, motion sensors, an optional support framework, and optional joystick or button controls that are commonly used to start or stop recording and adjust lens properties.[16] In 1992, Michael McKenna of MIT's Media Lab demonstrated the earliest documented virtual camera rig when he fixed a Polhemus magnetic motion sensor and a 3.2 inch portable LCD TV to a wooden ruler.[17] The Walkthrough Project at the University of North Carolina at Chapel Hill produced a number of physical input devices for virtual camera view control including dual three-axis joysticks and a billiard-ball shaped prop known as the UNC Eyeball that featured an embedded six-degree of freedom motion tracker and a digital button.[18]
See also
References
- ^ "Cameracontrol.org: The virtual camera control bibliography". Archived from the original on 13 August 2011. Retrieved 6 May 2011.
- ^ Bares, William; Scott McDermott; Christina Boudreaux; Somying Thainimit (2000). "Virtual 3D camera composition from frame constraints" (PDF). International Multimedia Conference. California, United States: Marina del Rey: 177–186. Archived from the original (PDF) on 10 July 2010. Retrieved 22 March 2009.
- ^ Drucker, Steven M.; David Zeltzer (1995). CamDroid: A System for Implementing Intelligent Camera Control (PDF). ISBN 978-0-89791-736-0. Archived from the original (PDF) on 5 June 2011. Retrieved 22 March 2009.
{{cite book}}:|journal=ignored (help) - ^ Drucker, Steven M.; David Zeltzer (1995). CamDroid: A System for Implementing Intelligent Camera Control (PDF). ISBN 978-0-89791-736-0. Archived from the original (PDF) on 5 June 2011. Retrieved 15 March 2015.
{{cite book}}:|journal=ignored (help) - ^ He, Li-wei; Michael F. Cohen; David H. Salesin (1996). "The Virtual Cinematographer: A Paradigm for Automatic Real-Time Camera Control and Directing" (PDF). International Conference on Computer Graphics and Interactive Techniques. 23rd. New York: 217–224. Archived from the original (PDF) on 28 August 2008. Retrieved 22 March 2009.
- ^ Tomlinson, Bill; Bruce Blumberg; Delphine Nain (2000). "Expressive autonomous cinematography for interactive virtual environments". Proceedings of the fourth international conference on Autonomous agents (PDF). Vol. 4th. Barcelona, Spain. pp. 317–324. CiteSeerX 10.1.1.19.7502. doi:10.1145/336595.337513. ISBN 978-1-58113-230-4. S2CID 5532829. Archived (PDF) from the original on 29 March 2005. Retrieved 22 March 2009.
{{cite book}}: CS1 maint: location missing publisher (link) - ^ Lino, Christophe; Marc Christie; Roberto Ranon; William Bares (1 December 2011). "The director's lens". Proceedings of the 19th ACM international conference on Multimedia. ACM. pp. 323–332. doi:10.1145/2072298.2072341. ISBN 9781450306164. S2CID 14079689.
- ^ "Oliver Krelos' Homepage". Archived from the original on 23 December 2010. Retrieved 8 March 2011.
- ^ Kevin Parrish (17 November 2010). "Kinect Used As 3D Video Capture Tool". Tom's Hardware.
- ^ Tim Stevens (29 November 2010). "Two Kinects join forces to create better 3D video, blow our minds (video)". Engadget.
- ^ Jenna Wortham (21 November 2010). "With Kinect Controller, Hackers Take Liberties". The New York Times.
- ^ Hsu, Jeremy (27 February 2009). ""Virtual Camera" Captures Actors' Movements for Resident Evil 5". Popular Science. Archived from the original on 2 March 2009.
- ^ a b Lewinski, John Scott (27 February 2009). "Resident Evil 5 Offers Sneak Peek at Avatar's 'Virtual Camera'". Wired. Retrieved 25 February 2015.
- ^ Lowe, Scott (27 February 2009). "The Tech Behind RE5". IGN. Retrieved 24 February 2015.
- ^ Thompson, Anne (1 January 2010). "How James Cameron's Innovative New 3D Tech Created Avatar". Popular Mechanics. Retrieved 25 February 2015.
- ^ "Optitrack InsightVCS". Retrieved 15 March 2015.
- ^ Michael McKenna (March 1992). "Interactive viewpoint control and three-dimensional operations". Proceedings of the 1992 symposium on Interactive 3D graphics - SI3D '92. ACM. pp. 53–56. CiteSeerX 10.1.1.132.8599. doi:10.1145/147156.147163. ISBN 978-0897914673. S2CID 17308648.
- ^ Frederick Brooks Jr. (June 1992). "Final Technical Report – Walkthrough Project" (PDF). Tr92-026. University of North Carolina at Chapel Hill. Archived (PDF) from the original on 23 September 2015. Retrieved 23 March 2015.