Face Reconstruction from a Small Number of Feature Points
color=ccccff>
« back to results for ""
Below is a cache of http://gravis.cs.unibas.ch/publications/ICPR00.pdf. It's a snapshot of the page taken as our search engine crawled the Web.
The web site itself may have changed. You can check the current page or check for previous versions at the Internet Archive.
Yahoo! is not affiliated with the authors of this page or responsible for its content.
Face Reconstruction from a Small Number of Feature Points
Face Reconstruction from a Small Number of Feature Points
Bon-Woo Hwang
, Volker Blanz
, Thomas Vetter
and Seong-Whan Lee
Center for Articial Vision Research, Korea University
Anam-dong, Seongbuk-ku, Seoul 136-701, Korea
bwhwang, hhsong, swlee
@image.korea.ac.kr
Max-Planck-Institute for Biological Cybernetics
Spemannstr. 38, 72076 Tuebingen, Germany
volker.blanz, thomas.vetter
@tuebingen.mpg.de
Abstract
This paper proposes a method for face reconstruction
that makes use of only a small set of feature points. Faces
can be modeled by forming linear combinations of proto-
types of shape and texture information. With the shape
and texture information at the feature points alone, we can
achieve only an approximation to the deformation required.
In such an under-determined condition, we nd an optimal
solution using a simple least square minimization method.
As experimental results, we show well-reconstructed 2D
faces even from a small number of feature points.
1. Introduction
It is difcult for traditional bottom-up, generic ap-
proaches to reconstruct the whole image of an object from
parts of its image or to restore the missing space informa-
tion due to noise, occlusion by other objects, or shadow
caused by illumination effects.
In contrast to such ap-
proaches, top-down, object-class-specic and model-based
approaches are highly tolerant to sensor noise and incom-
pleteness of input image information[2]. Hence, the top-
down approaches to the interpretation of images of vari-
able objects are now attracting considerable interest[1-4, 6].
Kruger et al. proposed a system for the automatic determi-
nation of the position, size and pose of a human head, which
is modeled by a labeled graph[3]. The nodes of the graph
refer to feature points of a head. However, the location and
texture information are not utilized for face reconstruction.
Lanitis et al. implemented a face reconstruction using a
Flexible Model[4]. About 150 feature points inside a face
This research was supported by Creative Research Initiatives of the
Ministry of Science and Technology, Korea.
and its contour were used for reconstruction. The system
performs well, but it requires more than 100 feature points
for shape reconstruction, and it also requires full texture in-
formation.
In this paper, we propose an efcient face reconstruction
method from a small set of feature points. Our approach is
based on the 2D shapes and textures of a data set of faces.
Shape is coded by a dense deformation eld between a ref-
erence face image and each individual face image. Given
only a small number of feature points on a novel face, we
use the database to reconstruct its full shape and texture.
It is a combination of stored shapes and textures that best
matches the positions and gray values of the feature points.
We anticipate that the proposed method will play an im-
portant role in reconstructing the whole information of an
object out of information reduced for compressing or par-
tially damaged due to occlusion or noise. In Section 2 and
3, we describe a 2D face model where shape and texture
are treated separately, and a method for nding coefcients
for face reconstruction, respectively. Experimental results
for face reconstruction are given in Section 4. Finally, in
Section 5, we present conclusive remarks and discuss some
future work.
2. 2D face model
On the assumption that the correspondence on the face
images has already been established[1], the 2D shape of a
face is coded as the deformation eld from a reference im-
age that serves as the origin of our space. The texture is
coded as the intensity map of the image which results from
mapping the face onto the reference face[6].
Let
Θ
be the displacement of point
, or the posi-
tion of the point in the face that corresponds to point
in the reference face. Let
be the gray value of the
point in the face that corresponds to point x in the reference
face. With shape and texture information separated from the
face image, we t a multivariate normal distribution to our
data set of faces according to the average of shape
and
that of texture
and covariance matrices
。
and
ˉ
com-
puted over shape and texture differences
ě
and
. By Principal Component Analysis(PCA), a ba-
sis transformation is performed to an orthogonal coordinate
system formed by eigenvectors
and
of the covariance
matrices on our data set of
faces.
!
"$#&%
'
)(
%10
2354
6
7
"$#&%
'
)(
%$8
954
(1)
where
@
0
4
@
8BADC
"$#E%
. The probability for coefcients
@
0
is
dened as
F
G@
0
IHP
FRQ
S
T
"$#&%
'
U(
%
0
V
WYX`4
(2)
with
V
W
being the eigenvalues of the shape covariance ma-
trix
. Likewise, the probability
F
@
8
can be computed.
3. Face reconstruction
In this section, we describe a method for nding coef-
cients for face reconstruction. First, we dene an energy
function as the sum of normalized coefcients and set a con-
dition for minimizing the energy function. Then, we solve
this problem by a least square minimization.
3.1. Problem denition
Since there are shape and texture elements only for fea-
ture points, we achieve an approximation to the deformation
required. Our goal is to nd an optimal solution in such an
underdetermined condition. We dene an energy function
as the sum of the normalized coefcients. We also set a
condition that the given shape or texture information at the
feature points must be reconstructed perfectly. The energy
function,
a
0
, describes the degree of deformation from
the the reference face. The problem(Equation 3) is to nd
0
which minimizes the energy function,
a
0
, which is given
as:
0cb
dgfih`qUr
s
a
0
Y4
(3)
with the energy function,
a
0
I
"$#&%
'
)(
%
0
V
Wut
(4)
under the condition,
﹚v
x
"$#E%
'
U(
%
0
yi
v
Y4
U亗
S
4I3儎34唴
Y4
(5)
where
%
4們剝3僘4
垏
are the selected feature points. Since
we select only a small number of feature points,
is much
smaller than
墿
S
.
3.2. Solution by least square minimization
According to Equation 3
H
5, we can solve this problem
using general quadratic programming. In order to make this
problem simpler, we reduce it to a least square problem.
Equation 5 is equivalent to the following:
%
%
3儎儞
"$#E%
%
.
.
.
. ..
.
.
.
%
攦3儎僣
"#E%
杽
0
%
.
.
.
0
"$#E%
杽
%
.
.
.
Θ
杽
t
(6)
To exploit the inherent orthogonal nature of the problem,
we rewrite Equation 6 as:
d
0唀
Жf
d
4
(7)
where
d
V
%
%
%
搩剝3
V
"$#&%
"$#&%
%
.
.
.
. ..
.
.
.
V
%
%
c儎3
V
"$#&%
"$#E%
﹫
4
0ge
0ch
V
h
43儎儎儎4
0gi
#
h
V
i
#
h
jI4
d
%
k43儎儎34E
4
(8)
and the row vectors of
d
are assumed to be linearly inde-
pendent.
0
e
can be computed by
0ge
dEl
f
d
4
(9)
where
d
l
is the pseudoinverse of the matrix
d
, and can
be obtained easily using a singular value decomposition as
follows[5].
Supposing the singular value decomposition of
d
is
d
6menpo
4
(10)
the pseudoinverse of
d
is
d
l
oqn
l
m
t
(11)
The columns of
m
are eigenvectors of
dgd
, and the
columns of
o
are eigenvectors of
d
d
. The main diago-
nals of
n
are lled with the square roots of the nonzero
eigenvalues of both. In
n
l
, all nonzero elements of
n
are
replaced by their reciprocals.
Figure 1 represents the process described above in 2D-
0
e
space. This is the case that, as we previously assumed, the
row vectors of
d
are linearly independent, and the number
of the bases m-1 and the feature point n are 2 and 1, respec-
tively. Circular contour lines designate two-dimensional
probability plots of Gaussian,
0
. The solid line repre-
sents the condition given in Equation 5.
The point at which the energy function is minimized can
be obtained by nding the point which has the minimum
distance from the origin.
Using Equations 1, 8 and we obtain
B
"$#E%
'
U(
%
0
e
V
t
(12)
By using Equation 12, we can get correspondence of all
points. Similarly, we can construct full texture information
.
Figure 1. The Example of least square mini-
mization in 2D-
0
e
space
We previously made the assumption that the row vec-
tors of
d
in Equation 8 are linearly independent. Other-
wise, Equation 5 may not be satised. In other words, the
correspondence obtained from Equation 12 may be inaccu