DeCS Media


Depth Capture Systems for Media

1. What is DeCS Media?

DeCS Media is a EXIST-Forschungstransfer funded project at the University Kiel. Funding is granted by the Bundesministerium für Wirtschaft und Technologie (BMWi) and the Europäischer Sozialfonds der Europäischen Union (ESF). In this project a innovative recording system for 3D film and television is developed. First approaches this project is based on have been developed in the EU-project 3D4YOU in our department.

The main problem in recording content for 3D television and 3D cinema is the necessity to use 2 cameras which have to be calibrated exactly and to configure the camera setting which determines the 3D effect. The desired 3D effect has to be planned and it is not possible to change this effects after the content has been recorded. Furthermore, it is very difficult to transfer the recorded content between displays of different size as the 3D effect changes with the display dimension.

But if instead of a second color image a depth or disparity image is available, in which the distances of the scene to the camera is included the images for 3D display can be generated from this information for different display sizes. Furthermore the 3D effect is scalable and can be altered dependent on scene content or viewing preferences.

To record such a depth image we use a combination of active and passive cameras with Computer Vision algorithms and code the information in a layered depth-based video format. This enables us to generate almost arbitrary views of the scene which makes it possible to bring the content to today's stereoscopic displays as well as to tomorrows autostereoscopic displays which do not require special glasses to view 3D.
Furthermore occlusion information can be recorded, coded and reproduced enabling the spectator to view behind the objects to some extent. This delivers a far more realistic impression of the things that are happening on the screen and the spectator has the impression to be part of the plot.

We develop recording and post-processing systems for professional and semi-professional film makers
which are eager to record their content for the current and next generation of 3D displays. The recorded content is also ready to be used for 3D television and 3D cinema as the 3D effect is scalable and therefore easy to adapt for different screen dimensions.

To sum up we go further than the classical stereo approach and we make the production of 3D content easier because the traditional recording process with just one main camera is maintained.

2. Our Competences and Main Research Topics

Our main competence is the recording and production of depth information with the usage of special cameras. We realize this as special add-on modules which are combined with standard professional cameras and do not affect the film making procedure. For every color pixel we provide additional depth information.

Camera-Rig for the recording of depth and color footage. This example shows a digital camera with 2 DeCS modules.
Camera-Rig for the recording of depth and color footage. This example shows a digital camera with 2 DeCS modules.

Example of a recording situation with the DeCS system and live 3D preview on a Phillips WOW-Display.
Example of a recording situation with the DeCS system and live 3D preview on a Phillips WOW-Display.

2.1 Recording of 3D content including live-preview and auto-stereoscopic display

By using our real-time capable algorithms we offer a 3D live preview during the shooting, for example on a auto-stereoscopic display such as the Phillips WOW-Display. This enables the persons in authority to check the 3D result of the shooting while recording it.
The real-time processing also makes the live-broadcasting of 3D-content in LDV- (Layered Depth Video) format or in a stereo-format generated from the LDV possible. This guarantees full flexibility during recording, reproduction and delivering.

2.2 Calibration approaches

For the recording of scenes in 3D more than one camera is needed. We extend any standard camera by our camera modules and determine the calibration between camera and DeCS module using our intuitive calibration software automatically.
But we also develop calibration software for the classic stereo production in which two HD or 4K cameras are used to minimize calibration errors.
Today the calibration of such stereo camera rigs is not perfect in many cases which negatively influences the viewing comfort. We offer approaches to correct the effects of wrong calibration.

2.3 Recording and Coding of Occlusion Information

By the usage of additional cameras occlusion information can be recorded which is essential if the content shall be reproduced on displays which are capable to show more than 2 views. This is important to allow a reproduction and viewing of the content in an pleasant way.

Layered Depth Video (LDV) Format, consisting of color, depth and occlusion information.
Layered Depth Video (LDV) Format, consisting of color, depth and occlusion information.

Flash player not available.

This movie shows the different views of a scene, an auto-stereoscopic display generates of the recorded footage in the LDV-format. 2 of these view are offered to the viewer, generating the 3D effect.

2.4 Post-Processing of 3D Content

Despite great care errors can occur during the estimation of depth information, which have to be corrected. Or ofter shooting special 3D effects shall be added to the footage or reduced. We are working on automatic and semi-automatic approaches which allow to edit the 3D effect and propagate it over sequences. This provides a fast and economic way to edit the 3D effect to ensure viewing comfort and freedom of artistic expression.

3. How you can profit from us

By the usage of our modules and software you will obtain:

  • Depth and occlusion, matching the footage of your main camera, which saves costs and time
  • Scalable 3D effect and freedom of artistic expression
  • The possibility to display the movie on any kind of 3D display
  • You see the result during shooting it, live and in 3D without glasses.
  • Our technology is designed to be future-proof

You are interested in our technology or our approaches? Contact us!

Project management: Dr.-Ing. Ingo Schiller
Project development Anatol Frick & Falko Kellner

DeCS Media in the News

Press release of the University of Kiel on 2011-12-14: 3D per Knopfdruck (german)

Image Image Image

Created by ischiller. Last Modification: Wednesday 14 of December, 2011 14:56:31 CET by ischiller.