Loading...
 
Page 1 Page 2 Download

MIP Multi-Camera Calibration


At the Multimedia Information Processing Group a powerful calibration software has been developed, which is capable to calibrate multiple cameras which are rigidly coupled in one single program (e.g. a stereo rig or a rig with many rigidly coupled cameras). This software is able to calibrate the intrinsic parameters and the extrinsic parameters of all cameras altogether. Additionally 3D-cameras can be calibrated together with the standard camera(s). The 3D-/ToF-/PMD-cameras suffer from systematic depth measurement errors which are modeled by a higher order function and are also estimated during the calibration process.


Kinect Support

Recently the Kinect cameras developed for Microsoft by PrimeSense have gained a lot of importance in the Computer Vision community. As the Kinect cameras also deliver depth images along with color images, Kinect cameras can be calibrated with the MIP - MultiCameraCalibration, which is therefore the first software which is capable to calibrate Kinect cameras together with other cameras.

To calibrate Kinect cameras depth and intensity images are needed. While taking the images for calibration a depth and intensity image has to be made for every checkerboard pose. The Kinect cameras cannot deliver both at the same time but it is possible to change the mode and take the two images one after the other. Please do not change the position of checkerboard or camera between taking depth and intensity image. The Kinect cameras deliver z-depth instead of ray lengths in the depth images. The MIP-MultiCameraCalibration internally uses ray lengths. So for a proper depth calibration the images have to be transformed to what we call "polar"-coordinate depth images. The function ProjectionParametersPerspective::TransformCartesianToPolarCoordinates() from the BIAS Software is used for this purpose. This is not absolutely necessary as the depth delivered by the Kinect cameras is quite reliable and a linear depth error model is sufficient.

The BIAS Software also supports the Kinect cameras. It can be used to directly capture images using the biasShowCamWx application. Note that the official and the open source drivers for the Kinect cameras are supported in BIAS. If the BIAS software is used to capture images the BIAS image format (.mip) can be used to store the depth images, otherwise the .pmd image format described below should be used.


License

This Software is an extension to the BIAS Software Library. This Software and
BIAS are distributed in the hope that they will be useful, but please note:

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
AUTHORS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES;LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
OF THE POSSIBILITY OF SUCH DAMAGE.

Permission is granted to any individual or institution to use, copy and distribute this software, provided that this complete copyright and permission notice is maintained intact, in all copies and supporting documentation.


Image Formats

  • For the intensity images most common file formats are supported. To name a few: mip, ppm, png, pgm, jpg.
  • For the ToF images only two formats are supported. Our own mip-file format where the distance values are stored in mm. and a xml-file format which includes distance-values (in m!), greyscale values and amplitude values. (See "pmd0000.pmd" for an example. This is a zipped file containig a ".pmd" file.)

Note that if the xml-file format is used, the files must have the suffix ".pmd" to be loaded properly. Eventually you will have to give the image list with the pmd images two times in the properties GUI at the beginning. (As pmd-Intensity and pmd-Depth images.)


Download

Go to the Download Page.


General Notes

  • What we calibrate: Intrinsic and extrinsic camera parameters (focal length, principle point, skew/shear, radial/tangential lens distortion, rotation and translation)
  • The calibration approach is suitable for perspective and spherical (fisheye) cameras. The perspective case works very well and reliable, while the spherical case still has some known weaknesses. We use the method proposed by D. Scaramuzza to approximate the parameters and try to refine them afterwards. In this refinement unfrequent crashes occur.
  • The calibration algorithm is based on a planar checkerboard pattern. So if you want to calibrate your camera(s) you have to make pictures of a planar checkerboard pattern. The more pictures you make the more precise your result will be. It is also important that you cover all areas of your images with the calibration pattern. Especially the border regions of the images are mandatory if you want to have a precise result for radial and tangential lens distortion parameters. We typically use between 20 and 80 pictures per camera for calibration from different angles/ directions and distances.
  • If you want to calibrate more than one camera you have to take the pictures of the checkerboard with all cameras at the same moment. Remember the cameras should be rigidly coupled! Again many images help to produce better results.
  • If you want to calibrate a PMD-/ToF-/3D-camera and are interested in the depth error correction you have to take images of the calibration pattern at different distances. The distances should cover the whole operation area of the camera. Typical values are between 2.0m and 7.5m.
  • The checkerboard has to be visible and detectable in your images, so ensure that it is big enough to be identified even in some distance to the cameras.
  • The algorithm uses lists of images. The images have to be in the same order in the image lists. Images that are in the same position in the lists have to be taken simultaneously. The lists must have the same length. You can invalidate some images if, for example the checkerboard is not visible in that image. However you can't invalidate an image which belongs to the first camera in your rig, because the position and rotation of the other cameras is calculated relative to that first camera. (If you only want to calibrate one camera it is possible to invalidate pictures in this camera.)

Depth Error Model

In the current version of the Software, the depth error is modeled as a polynomial:

Image

where λ* is the corrected depth, (x,y) are the image coordinates, λ is the measured depth and d0, ..., d5 are the parameters to estimate.

Besides the polynomial depth error model a spline model and linear error model are also implemented. The linear model consists only of an offset and an inclination parameter. The spline error model follows the proposed spline model introduced by Marvin Lindner and Andreas Kolb.

Marvin Lindner, Andreas Kolb:
Lateral and Depth Calibration of PMD-Distance Sensors
Advances in Visual Computing, Springer, 2, 2006, pp. 524-533.


You can choose which model to use in the project definition window.


Reference


If this software is used to produce scientific publications please add a reference to it in the reference section:

Ingo Schiller: MIP - Multi-Camera Calibration.
http://www.mip.informatik.uni-kiel.de/tiki-index.php?page=Calibration


Further details can be found in the papers:

Marvin Lindner, Ingo Schiller, Andreas Kolb, Reinhard Koch:
Time-of-Flight Sensor Calibration for Accurate Range Sensing
Journal of Computer Vision and Image Understanding (CVIU) 114 (12) (2010), pp. 1318–1328. Bibtex
Ingo Schiller, Christian Beder, Reinhard Koch:
Calibration of a PMD-Camera using a Planar Calibration Pattern together with a Multi-Camera Setup
Proceedings of the XXI ISPRS Congress, Beijing July 3-11 2008 BibTeX

Configuration:

Starting the program will present a properties GUI where you can add the image lists and set some parameters.

Image
  • Add image lists to the calibration project. Image lists are just textfiles containing the paths of the image files. You can use the button on the bottom right corner to make image lists.
  • When you add an image list you are asked if this is a spherical camera. This means if the FoV is ~180 degrees. In this case the software tries to estimate spline parameters after the model of Scaramuzza. If you use a standard perspective camera just press "No".
  • After that you are asked if you want to fix the intrinsic parameters. This mean you can load the parameters such as principal point, focal length and lens distortion from file. Therefor the parameters have to be given in our file format. (Tip: Calibrate one camera without fixed parameters then you have a point to start)
  • The depth distortion parameters are handled in a special way. If you load a file which includes depth parameters they are taken as fixed. If you load a file withour depth parameters or with all set to zero they are estimated. (Of course if you don't load anything they are estimated as well)
  • Select the number of the inner corners of your checkerboard. For example this checkerboard:
    Image

has xCorners=7 and yCorners=4

  • Select the size of one square of your checkerboard. That is for example the width of one single black rectangle in the image above.
  • Select whether to use Software rendering or GPU rendering. The GPU rendering only works in NVIDIA cards Geforce 7950+. You can try it, but I recommend to use the software rendering, although it is slower.
  • Choose which radial distortion model you wish to estimate, only the radial distortion parameters (Bouguet), radial and tangential distortion parameters or distortion parameters as Brown defined them (still experimental).
  • Select which depth error model you wish to use. Polynomial, Linear or Spline (still experimental).
  • Finally select whether your checkerboard is black or white in the top left corner.



Save you project before pressing ok or you will have to configure it again!

--> Go to next page

Created by ischiller. Last Modification: Monday 09 of May, 2016 12:53:34 CEST by sandro.