=right>

ECE 513: Vector Space Signal Processing
ECE 513: Vector Space Signal Processing
Spring 2008
http://courses.ece.uiuc.edu/ece513
Lectures
: Mondays and Wednesdays, 11-12:20 p.m.; 57 Everitt Lab.
Instructor
: Prof. Yoram Bresler (ybresler@uiuc.edu, 112 Coordinated Science Lab, 244-9660).
Oce Hours: Monday 3:30-4:30 (+ appointments by email).
Course Overview
:
Rigorous presentation of key mathematical tools in a vector space framework, and their applica-
tions in signal processing, including: nite and innite dimensional vector spaces, Hilbert spaces,
linear operators, inverse problems (e.g. deconvolution, tomography, Fourier imaging), least-squares
methods, conditioning and regularization, matrix decompositions, subspace methods, bases and
frames for signal representation (e.g. generalized Fourier series, wavelets, splines), Hilbert space of
random variables, random processes, signal and spectral estimation.
Course Purpose
:
This course is suitable for graduate students in the areas of signal processing, communications,
control, and computational science and engineering. It is also suitable for students in other areas
who seek graduate level treatment of vector-space methods that are used in various engineering
elds. This course complements ECE 551 and ECE 534, and will be particularly useful for students
wishing to pursuit a PhD in the above mentioned areas. It provides additional mathematical tools
and rigorous foundation for research, and the latest relevant research will be incorporated to keep
the contents current.
Prerequisites
: ECE410, ECE 413, and a linear algebra course; or equivalents, or consent of
instructor.
Topics
:
Inverse problems and matrix theory (12 hours): linear inverse problems; orthogonal pro-
jections; minimum-norm least squares solutions; Moore-Penrose pseudoinverse; singular value
decomposition; matrix decomposition and approximation; conditioning and regularization.
General linear vector spaces (15 hours): nite and innite dimensional vector spaces;
Hilbert spaces; projection theorem; inverse problems in innite dimensional vector spaces;
approximation and Fourier series; pseudoinverse operators; iterative methods for optimization
and inverse problems; bases and frames for signal representation;
Hilbert space of random variables (6 hours): random processes; least-squares estimation;
Wiener ltering; Wold decomposition; discrete-time Kalman lter.
Applications in signal processing (12 hours, during the course): deconvolution, optimal
lter design, temporal and spatial spectrum estimation, tomography, harmonic retrieval, sub-
space methods, sensor array processing, extrapolation of band-limited sequences, generalized
sampling, wavelets, splines, subset selection, sparse approximation.
Grading
: Homework: 25%; Midterm exam I: 25%; Midterm exam II: 25%; Final project (presen-
tation and report): 25%.
1 Texts:
Class notes by Bresler, Basu, and Couvreur (required; available online with restricted access)
T. Moon and W. Sterling, Mathematical Methods and Algorithms for Signal Processing, Pren-
tice Hall, 2000 (recommended).
A. W. Naylor and G. R. Sell, Linear Operator theory in Engineering and Science, Springer
Verlag, 1982 (recommended). Excellent book for the Hilbert Space theory in the course.
C.L. Byrne, Signal Processing, a Mathematical Approach, A.K Peters, 2005.
P. Bremaud, Mathematical Principles of Signal Processing, Fourier and Wavelet Analaysis
Springer-Verlag, 2002.
P. Stoica and R. L. Moses, Introduction to Spectral Analysis, Prentice Hall, 1997.
T. Kailath, A. H. Sayed, and B. Hassibi, Linear Estimation, Prentice Hall, 2000.
B. Porat, Digital Processing of Random Signals: Theory and Methods, Prentice Hall, 1994.
J. W. Demmel, Applied Numerical Linear Algebra, SIAM, 1997.
L. Trefethen and D. Bau, Numerical Linear Algebra, SIAM, 1997.
C. R. Vogel, Computational Methods for Inverse Problems, SIAM, 2002.
G. Golub and C. Van Loan, Matrix Computations, Johns Hopkins University Press, 3rd ed.,
1996.
D. Luenberger, Optimization by Vector Space Methods, Wiley, 1969.
Course Schedule
:
BBC = Class notes by Bresler, Basu and Couvreur.
Each homework is assigned on Wednesday of the indicated week and due on the following Wednes-
day, in class.
Week 1 (1/14)
Introduction to vector space signal processing. Linear inverse problems in
C
n
(BBC 1.11.5)
Homework #1
Week 2,3 (1/21)
Orthogonal projections. Minimum norm least squares solutions (BBC 1.6
1.11).
No class Wed. 1/30/08 Homework #2
Week 4 (2/4)
Singular value decomposition. Matrix approximation (BBC 2.1-7).
Homework #3
Week 5 (2/11)
Conditioning and regularization. Total least-squares. Subspace tting (BBC
2.8-11).
Homework #4
Week 6 (2/18)
Temporal and spatial spectrum estimation (BBC 3).
Homework #5
Week 7 (2/25)
General linear vector spaces. Linear operators (BBC 4).
First midterm exam: Tuesday 3/4/08, 79 p.m.
2 Week 8 (3/3)
Finite and innite dimensional vector spaces (BBC 5 & 6).
Homework #6
Week 9 (3/10)
Hilbert spaces. Linear inverse problems in innite dimensional vector spaces
(BBC 7).
Homework #7
(3/173/23)
Spring Break
Week 10 (3/24)
Iterative methods for optimization and inverse problems (extra notes).
Homework #8
Week 11 (3/31)
Bases and frames for signal representations (extra notes).
Homework #9
Project proposals due on Friday, 3/28
Week 12 (4/7)
Approximation and sampling in Hilbert spaces (extra notes).
Homework #10
Week 13 (4/14)
Hilbert spaces of random variables. Linear least-squares estimation (BBC
8).
Homework #11
Week 14 (4/21)
Random processes, Wiener ltering, and Wold decomposition (BBC 9 &
extra notes).
Second midterm exam. Tuesday, 4/22, 79 p.m.
Week 15 (4/28- 4/30)
Project presentations.
Project reports due on Friday 5/5
3