## Vanishing Point Estimation and Line Classification:

The problem of estimating vanishing points for visual scenes under the Manhattan world assumption has been addressed for more than a decade. Surprisingly, the special characteristic of the Manhattan world that lines should be orthogonal or parallel to each other is seldom well utilized. In this work, we present an algorithm that accurately and efficiently estimates vanishing points and classifies lines by thoroughly taking advantage of this simple fact in the Manhattan world for images grabbed by a camera with a single effective viewpoint (e.g. perspective camera or central catadioptric camera). The algorithm is also extended to estimate the focal length of the camera when it is uncalibrated. The key novelty is to estimate three orthogonal line directions in the camera frame simultaneously instead of estimating vanishing points in the image plane directly. The performance of the proposed algorithm is demonstrated on four publicly available databases. Compared to the state-of-the-art methods, the experiments show its superiority in terms of both accuracy and efficiency.

Here are the matlab code and the C++ code of the proposed algorithm, the link of image datasets are YUD and ECD.

## Robust and Efficient Pose Estimation from Line Correspondences:

Determining the pose of a calibrated camera from n correspondences between 3D reference features and their 2D projections has numerous applications. In this work, we propose a non-iterative solution for the Perspective-n-Line (PnL) problem, which can efficiently and accurately estimate the camera pose for both small number and large number of line correspondences. By selecting a rotation axis in the camera framework, the reference lines are divided into triplets to form a sixteenth order cost function, and then the optimum is retrieved from the roots of the derivative of the cost function by evaluating the orthogonal errors and the reprojected errors of the local minima. The final pose estimation is normalized by a 3D alignment approach. The advantages of the proposed method are as follows: (1) it stably retrieves the optimum of the solution with very little computational complexity and high accuracy; (2) small line sets can be robustly handled to achieve highly accurate results and; (3) large line sets can be efficiently handled because it is O(n).

Here are the matlab code of the RPnL algorithm.

## Line Matching:

Line segment matching plays an important role in image processing and computer vision, while it remains a challenging task for images under various transformations. In this work, we present a line matching algorithm which considers both the local appearance of lines and their geometric attributes. To overcome the problem of segment fragmentation and geometric variation, we extract lines in the scale space. To depict the local appearance of lines, we design a novel line descriptor called as Line Band Descriptor (LBD) based on the band representation of the Line Support Region (LSR). To evaluate the pairwise geometric consistencies, we define the pairwise geometric attributes between line pairs. Then we built a relational graph for candidate line matches and employ a spectral technique to solve this matching problem efficiently. The advantages of the proposed algorithm are as follows: (1) it's robust to image transformations because of the multi-scale line detection strategy; (2) it's efficient because the designed LBD descritpor is fast to compute and the appearance similarities reduce the dimension of the graph matching problem; (3) it's accurate even for low-texture images because of the pairwise geometric consistency evaluation.

Here are the source code of the proposed line matching algorithm, the first image set ( a,b,c,d,e,f,g,h,Homographies) for evaluating descriptor performance and the second image set for comparing the line matching algorithms. The illustration of matching results can be downloaded here ( ResultsOnImageset1 and ResultsOnImageset2).

## Scene reconstruction from line features:

Line features are prominent in most man-made environments, both indoor and urban scenes, and a map of line segments gives higher level of relevant information on the structure of the environment than point features. In this work we address the problem of structure and motion from line correspondences, which ranges from the representation of lines, their projections and the initialization procedure to the final adjustment.The Cayley representation of spatial lines is developed, which is a nonlinear minimal parametrization circumventing the tiresome Pluecker constraint. The relationships between different line representations are given. Based on these relationships, we derive a novel line projection function which is consistent with previous results. After building the line observation model, we employ a closed-form solution for the first image triplet, then develop an incremental initialization approach to initialize the motion and structure parameters. Finally, the Sparse Bundle Adjustment (SBA) is applied to refine the parameters, which updates the spatial lines by using the Cayley representation with an unconstrained optimization engine.

## 2013

Lilian Zhang and Reinhard Koch: An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency JVCI 24(7), pp:794-805, 2013, DIO: 10.1016/j.jvcir.2013.05.006. |

## 2012

Lilian Zhang and Reinhard Koch: Vanishing Points Estimation and Line Classification in a Manhattan World Proceedings of the ACCV 2012, November 2012, Daejeon, Korea. |

Lilian Zhang and Chi Xu and Kok-Meng Lee and Reinhard Koch: Robust and Efficient Pose Estimation from Line Correspondences Proceedings of the ACCV 2012, November 2012, Daejeon, Korea. |

Lilian Zhang and Reinhard Koch: Line Matching Using Appearance Similarities and Geometric Constraints DAGM-OAGM2012, Graz, Lecture Notes in Computer Science Volume 7476, 2012, pp 236-245 |

## 2011

Lilian Zhang and Reinhard Koch: Hand-held Monocular SLAM Based on Line Segments Proceedings of IMVIP2011, September 2011, Dublin, Ireland, pages 8-15. (Oral presentation) |