Vision is a very important sense and is essential to humans and animals. It permits toperceive, manipulate, model the environment and also navigate into it. The vision ability ismimicked for robots using cameras. However conventional cameras suffer from a small fieldof view, which might lead to important limitations and difficulties. This PhD thesis is ratherdedicated to omnidirectional vision systems, especially catadioptric cameras, that permit toovercome these difficulties. An omnidirectional camera can see in every direction or at leasthas a very wide field of view. A catadioptric camera is a special kind of omnidirectionalsystem: it is composed of a camera and a convex mirror, which permits to greatly enlargethe field of view and thus acquire much more information from the environment. Theobjective of this thesis is to study the role of omnidirectional vision for robotic applicationssuch as lowlevel feature extraction (lines, skyline, object) but also motion estimation and 3Dreconstruction in urban environment, while facing the fundamental issues of omnidirectionalvision. Our work has also been motivated by the \megacity modeling" project, performedin collaboration with Ikeuchi`s laboratory at the University of Tokyo, Japan.
Several algorithms have been proposed in the literature to estimate the motion of acatadioptric camera. Combined with existing quasidense matching technique and texturemapping methods, some existing methods can obtain impressive results for urban 3D reconstruction. It is important to note that most of these methods are simply based ontextbook materials and do not really take into account the inherent aspects of omnidirectional vision. On the contrary, our work aims to face some fundamental issues associated toomnidirectional vision, especially image distortion and repetitive textures/interest points.These issues are often related to each others. For example, because of distortions, it becomes difficult to match corresponding points in a pair of images and thus many wrongcorrespondences (outliers) are built. These outliers lead to a higher complexity of robustestimator like RANSACand thus increase the execution time. Another example is the vanishing pointbased rotation that must detect and extract lines despite distortion. Moreoverthe catadioptric image is often converted into a more traditional format like a panoramicview. This process requires color interpolation which is very time consuming. The contribution of our work is the development of methods to overcome these fundamental issues ofomnidirectional vision. Our approach is detailed in this thesis and provides several concreteadvantages. For example, no timeconsuming interpolation is required: we work directlyon the distorted catadioptric image. Moreover the complexity of the robust estimators isstrongly decreased thanks to the proposed socalled 2point algorithm. It also permits todetect the outliers more efficiently by a rotation constraint.
This PhD thesis presents a complete omnidirectional visionbased framework for roboticapplications while facing the fundamental difficulties inherent to omnidirectional vision.It is composed of four main parts: lowlevel feature extraction (lines, skyline and objects),rotation/motion estimation, a general and globally optimal framework for motion estimation,and ffnally, 3D reconstruction. The ffrst part is dedicated to the extraction of lowlevelfeatures. These features constitute key inputs for higherlevel methods. Firstly, we presenta new and efficient method for line detection, which will be applied for realtime rotationestimation. Secondly, we extend a method of skyline extraction and the concept of dynamicprogramming, that could be useful in diverse applications (e.g. matching with a DigitalElevation Map). Thirdly, we extend the neighborhood histogram representation and theconcept of particle fflter towards omnidirectional vision. This has been successfully appliedto object tracking despite the strong distortions.
The second part of our system aims to estimate six degrees of freedom camera motionsfrom central catadioptric images in manmade environments. Our proposed system providestwo strong advantages over the existing methods: ffrstly, it can implicitly handle the difficulty of planar/nonplanar scenes, and secondly, it decreases the complexity of the robustestimators (RANSAC). The only assumption deals with the presence of parallel straight lineswhich is legitime in manmade environments. More precisely, we estimate the motion bydecoupling the rotation and the translation. The rotation is computed by an efficient algorithm based on the detection of dominant bundles of parallel catadioptric lines and thetranslation is calculated from a robust 2point algorithm. We also show that the linebasedapproach allows to estimate the absolute attitude (roll and pitch angles) at each frame,without error accumulation.
The third part introduces an original and alternative approach for motion estimation.The proposed framework is general, in the sense that it builds the feature correspondences,detects the inliers/outliers and estimates the underlying model, all in a single uniffed formulation. Moreover, we show that the mathematical formulation can be solved in a globallyoptimal manner based on branchandbound technique and convex envelopes.
Finally, the fourth part studies 3D reconstruction using a pure visionbased approach andalso an hybrid approach which combines omnidirectional vision and a laser range ffnder.We present a new method to calibrate the camera and the laser, that is to say estimatethe relative motion between these two sensors. Contrary to other existing methods, ourproposed framework does not require to know the distance to a certain calibration target,simply needs to observe some lines, implicitly enforces the orthonormality of the rotationmatrix and guarantees the global optimality of the solution.
본 박사학위 논문은 전방향 비전 시스템 중 반사 굴절 비전을 중심으로 한 연구를 논한다. 일반 카메라와 달리 반사 굴절 비전 카메라는 주위 환경의 정보를 훨씬 더 많이 가져올 수 있다. 반사 굴절 비전 카메라는 전방향 비전 시스템의 한 종류이며 카메라와 굴절 거울을 이용해 보다 넓은 시야를 확보한다. 이 논문은 로봇 분야에 응용할 수 있는 전방향 비전 시스템을 다루며 전방향 비전 시스템이 갖는 근본적인 문제를 풀어나간다.
본 논문은 로봇 분야 내에서 필요한 여러 작업을 다룬다. 이미 반사 굴절 비전을 이용한 로봇 작업을 위한 알고리즘이 나와있으나 많은 부분들이 간단한 교과물의 구현으로 그쳤을 뿐 실제 핵심적인 문제를 다루고 있지 않는 게 현실이다. 또한 대부분 반사 굴절 비전으로 맺힌 상을 파노라마 영상으로 되돌리는 불필요한 작업을 수행하고 있다. 본 연구는 이러한 전방향 비전 시스템이 갖는 근본적이 문제를 다루며 해결 방안을 좀 더 구체적으로 묘사하고 있다.
본 연구는 가장 먼저 하위 레벨 특징점을 뽑는 단계에서 시작된다. 이 특징점들은 상위 레벨 기법들을 위한 매우 중요한 요소가 된다. 우리는 선 검출, 하늘과 맞닿는 윤곽선 검출 및 particle filter의 기술 연장을 이용한 전방향 비전에 사용 될 수 있는 새로운 기법을 제시한다.
두 번째 단계에서는 인위로 구성된 환경에서 취득한 영상들을 이용한 카메라 모션 추정을 다룬다. 소실점을 이용하여 카메라의 회전 값을 찾고 2point 알고리즘을 이용하여 translation을 찾는다. 우리가 제시하는 시스템은 기존의 방법들보다 더 강력한 이점을 가지고 있다. 첫째로 우리의 시스템은 평면/비평면 영상이 주는 어려움을 내부적으로 다루고 있으며 두번째로 RANSAC과 같은 robust estimator들의 복잡도를 줄일 수 있다.
세 번째 단계에서는 모션 추정을 위한 새로운 방안과 대안 방안을 제시한다. 제시하는 시스템은 일반적으로 사용될 수 있으며 단일 공식을 통해 대칭점들을 찾는 과정에서 inlier와 outlier를 검출하여 근본적인 모델을 형성한다. Branchandbound 기법과 convex envelopes를 이용하여 제시하는 수학적 모델의 최적 해답을 찾는다.
마지막으로 네 번째 단계에서는 3차원 복원을 위한 순수 비전 방법 및 전방향 비전과 레이저 데이터를 접목시킨 방법을 다룬다. 또한 카메라와 레이저의 관계를 보정하는 새로운 기법을 소개한다. 우리가 제시하는 기법은 센서간의 실제거리를 요구하지 않으며 간단한 선 검출로 rotation matrix간의 직교 규격화 관계를 내부적으로 계산하여 해의 최적화를 보장한다.
