=black>
Help -
Help for Webmasters « back to results for ""
Below is a cache of http://www.wisdom.weizmann.ac.il/~irani/PAPERS/eccv98.pdf. It's a snapshot of the page taken as our search engine crawled the Web.
The web site itself may have changed. You can check the current page or check for previous versions at the Internet Archive. Yahoo! is not affiliated with the authors of this page or responsible for its content.
From Reference Frames to Reference Planes: Multi-View Parallax Geometry and Applications
From Reference Frames to Reference Planes:
Multi-View Parallax Geometry and Applications
M. Irani
1
, P. Anandan
2
, and D. Weinshall
3
1
Dept. of Applied Math and CS, The Weizmann Inst. of Science, Rehovot, Israel,
irani@wisdom.weizmann.ac.il
2
Microsoft Research, One Microsoft Way, Redmond, WA 98052, USA,
anandan@microsoft.com,
3
Institute of Computer Science Hebrew University 91904 Jerusalem, Israel
daphna@cs.huji.ac.il
Abstract. This paper presents a new framework for analyzing the ge-
ometry of multiple 3D scene points from multiple uncalibrated images,
based on decomposing the projection of these points on the images into
two stages: (i) the projection of the scene points onto a (real or vir-
tual) physical reference planar surface in the scene; this creates a virtual
image on the reference plane, and (ii) the re-projection of the virtual
image onto the actual image plane of the camera. The positions of the
virtual image points are directly related to the 3D locations of the scene
points and the camera centers relative to the reference plane alone. All
dependency on the internal camera calibration parameters and the ori-
entation of the camera are folded into homographies relating each image
plane to the reference plane.
Bi-linear and tri-linear constraints involving multiple points and views
are given a concrete physical interpretation in terms of geometric rela-
tions on the physical reference plane. In particular, the possible duali-
ties in the relations between scene points and camera centers are shown
to have simple and symmetric mathematical forms. In contrast to the
plane+parallax (p+p) representation, which also uses a reference plane,
the approach described here removes the dependency on a reference im-
age plane and extends the analysis to multiple views. This leads to sim-
pler geometric relations and complete symmetry in multi-point multi-
view duality.
The simple and intuitive expressions derived in the reference-plane based
formulation lead to useful applications in 3D scene analysis. In particular,
simpler tri-focal constraints are derived that lead to simple methods for
New View Synthesis. Moreover, the separation and compact packing of
the unknown camera calibration and orientation into the 2D projection
transformation (a homography) allows also partial reconstruction using
partial calibration information.
Keywords: Multi-point multi-view geometry, uncalibrated images, new view
synthesis, duality of cameras and scene points, plane+parallax, trilinearity.
M. Irani is supported in part by DARPA through ARL Contract DAAL01-97-K-0101 1
Introduction
The analysis of 3D scenes from multiple perspective images has been a topic of
considerable interest in the vision literature. Given two calibrated cameras, their
relative orientations can be determined by applying the epipolar constraint to
the observed image points, and the 3D structure of the scene can be recovered
relative to the coordinate frame of a reference camera (referred to here as the
reference framee.g., see [13, 6]). This is done by using the epipolar constraint
and recovering the Essential Matrix E which depends on the rotation
R and
translation
T between the two cameras. Constraints directly involving the image
positions of a point in three calibrated views of a point have also been derived
[19].
If the calibration of the cameras is unavailable, then it is known that re-
construction is still possible from two views, but only up to a 3D projective
transformation [4]. In this case the epipolar constraint still holds, but the Essen-
tial Matrix is replaced by the Fundamental Matrix, which also incorporates
the unknown camera calibration information. The 3D scene points, the camera
centers and their image positions are represented in 3D and 2D projective spaces
(using homogeneous projective coordinates). In this case, the reference frame
reconstruction may either be a reference camera coordinate frame [8], or as de-
ned by a set of 5 basis points in the 3D world [14]. A complete set of constraints
relating the image positions of multiple points in multiple views have been de-
rived [5, 15]. Alternatively, given a projective coordinate system specied by 5
basis points, the set of constraints directly relating the projective coordinates of
the camera centers to the image measurements (in 2D projective coordinates)
and their dual constraints relating to the projective coordinates of the 3D scene
points have also been derived [2, 20].
Alternatively, multiple uncalibrated images can be handled using the plane
+ parallax (P+P) approach, which analyzes the parallax displacements of a
point between two views relative to a (real or virtual) physical planar surface in the scene [16, 12, 11]. The magnitude of the parallax displacement is called the
relative-ane structure in [16]. [12] shows that this quantity depends both on
the Height
H of P from and its depth Z relative to the reference camera.
Since the relative-ane-structure measure is relative to both the reference frame
(through
Z) and the reference plane (through H), we refer to the P+P frame-
work also as the reference-frame + reference-plane formulation. The P+P
has the practical advantage that it avoids the inherent ambiguities associated
with estimating the relative orientation (rotation + translation) between the
cameras; this is because it requires only estimating the homography induced by
the reference plane between the two views, which folds together the rotation and
translation. Also, when the scene is at, the
F matrix estimation is unstable,
whereas the planar homography can be reliably recovered [18].
In this paper, we remove the dependency on the reference frame of the anal-
ysis of multi-point multi-view geometry. We break down the projection from 3D
to 2D into 2 operations: the projection of the 3D world onto the 2D reference
plane
, followed by a 2D projective transformation (homography) which maps the reference plane to the image plane. Given the virtual images formed by
the projection onto the reference plane, we derive algebraic and geometric rela-
tions involving the image locations of multiple points in multiple views in these
virtual images. The positions of virtual image points are directly related to the
3D locations of the scene points and the camera centers relative to the reference
plane alone. All dependency on the internal camera calibration parameters and
the orientation of the camera are folded into homographies relating each image
plane to the reference plane. We obtain a structure measure that depends only
on the heights of the scene points relative to the reference plane
In this paper, we derive a complete set dual relationships involving 2 and
3 points in 2 and 3 views. On the reference plane the multi-point multi-view
geometry is simple and intuitive. These relations are directly related to phys-
ical points on the reference plane such as the epipole and the dual-epipole[9].
We identify these points, and also two new entities called the tri-focal line and
the dual trifocal-line which are analogous to the epipole and the dual-epipole
when considering three-view and three-point geometries on the reference plane.
Structures such as the fundamental matrix and the trilinear tensor have a rather
simple form and depend only on the epipoles, and nothing else. The symmetry
between points and cameras is complete, and they can be simply switched around
to get from the epipolar geometry to the dual-epipolar geometry.
The simple and intuitive expressions derived in the reference-plane based
formulation in this paper lead to useful applications in 3D scene analysis. In
particular, simpler tri-focal constraints are derived, and these lead to simple
methods for New View Synthesis. Also, the separation and compact packing of
the unknown camera calibration and orientation into the 2D projection transfor-
mation (a homography) that relates the image plane to the reference plane, leads
to potentially powerful reconstruction and calibration algorithms. For instance,
based on minimal partial domain information, partial calibration and partial
reconstruction can be achieved. This is also briey discussed in this paper.
The remainder of this paper is organized as follows: Section 2 introduces our
notations, and describes the two-view geometric and algebraic constraints (bi-
linearity and parallax) in the r