Epipolar Geometry class 11 Multiple View Geometry Comp 290-089 Marc Pollefeys Multiple View Geometry course schedule (subject to change) Jan. 7, 9 Intro & motivation Projective 2D

Geometry Jan. 14, 16 (no class) Projective 2D Geometry Jan. 21, 23

Projective 3D Geometry (no class) Jan. 28, 30 Parameter Estimation Parameter Estimation Feb. 4, 6

Algorithm Evaluation Camera Models Feb. 11, 13 Camera Calibration Single View Geometry Feb. 18, 20

Epipolar Geometry 3D reconstruction Feb. 25, 27 Fund. Matrix Comp. Structure Comp. Planes & Homographies

Trifocal Tensor Three View Reconstruction Multiple View Geometry Mar. 4, 6 Mar. 18, 20 More Single-View Geometry

Projective cameras and planes, lines, conics and quadrics. P T l P T CP Q cone PQ *P T C* Camera calibration and vanishing points, calibrating conic and the IAC Two-view geometry

Epipolar geometry 3D reconstruction F-matrix comp. Structure comp. Three questions: (i) Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point x in the second image? (ii) Camera geometry (motion): Given a set of corresponding

image points {xi xi}, i=1,,n, what are the cameras P and P for the two views? (iii) Scene geometry (structure): Given corresponding image points xi xi and cameras P, P, what is the position of (their pre-image) X in space? The epipolar geometry (a) C,C,x,x and X are coplanar The epipolar geometry

b What if only C,C,x are known? The epipolar geometry a All points on project on l and l The epipolar geometry b Family of planes and lines l and l Intersection in e and e

The epipolar geometry epipoles e,e = intersection of baseline with image plane = projection of projection center in other image = vanishing point of camera motion direction an epipolar plane = plane containing baseline (1-D family) an epipolar line = intersection of epipolar plane with image (always come in corresponding pairs) Example: converging

cameras Example: motion parallel with image plane The fundamental matrix F algebraic representation of epipolar geometry x l' we will see that mapping is (singular) correlation (i.e. projective mapping from points to lines) represented by the fundamental matrix F The fundamental matrix F

geometric derivation x' H x l' e'x' e' H x Fx mapping from 2-D to 1-D family (rank 2) The fundamental matrix F algebraic derivation X C P x CC l P' C P' P x

F e' P' P (note: doesnt work for C=C F=0) P P I Px X C The fundamental matrix F correspondence condition The fundamental matrix satisfies the condition

that for any pair of corresponding points xx in the two images T T x' Fx 0 x' l' 0

The fundamental matrix F F is the unique 3x3 rank 2 matrix that satisfies xTFx=0 for all xx (i) (ii) (iii) (iv) (v) Transpose: if F is fundamental matrix for (P,P), then FT is fundamental matrix for (P,P) Epipolar lines: l=Fx & l=FTx Epipoles: on all epipolar lines, thus eTFx=0, x

eTF=0, similarly Fe=0 F has 7 d.o.f. , i.e. 3x3-1(homogeneous)-1(rank2) F is a correlation, projective mapping from a point x to a line l=Fx (not a proper correlation, i.e. not invertible) Epipolar Line Homography The epipolar line geometry l,l epipolar lines, k line not through e l=F[k]xl and symmetrically l=FT[k]xl k l

k l e Fk l e' (pick k=e, since eTe0) l' F e l l FT e' l'

Pure Translation camera motion Fundamental matrix for pure translation Forward motion Fundamental matrix for pure translation F e' H e' example: e'1,0,0

T H K 1RK 0 0 0 F 0 0 - 1 0 1 0

x'T Fx 0 y y' x PX K[I | 0]X -1 K x' P' X K[I | t] x Z ( X,Y,Z ) T K -1x/Z x' x Kt/Z

motion starts at x and moves towards e, faster depending on Z pure translation: F only 2 d.o.f., xT[e]xx=0 auto-epipolar General motion x 'T e' Hx 0 x 'T e' x 0 x' K' RK -1x K' t/Z Geometric representation of F

FS F FT / 2 xx FA F FT / 2 T

x Fx 0 x T FS x 0 Fs: Steiner conic, 5 d.o.f. Fa=[xa]x: pole of line ee w.r.t. Fs, 2 d.o.f. F FS FA x T

FA x 0 Pure planar motion Steiner conic Fs is degenerate (two lines) Projective transformation and invariance Derivation based purely on projective concepts -T x Hx, x' H' x' F H' FH

-1 F invariant to transformations of projective 3-space P, P' F F P, P' unique not unique canonical form P [I | 0]

P' [M | m] F m M Projective ambiguity of cameras given F previous slide: at least projective ambiguity this slide: not more! ~~ Show that if F is same for (P,P) and (P,P), there exists a projective transformation H so that ~ ~ P=HP and P=HP

~ ~ ~ ~ P [I | 0] P' [A | a] P [I | 0] P' [A | a ] ~ F a A ~ a A ~ lemma: ~ a ka A k 1 A av T

Canonical cameras given F F matrix corresponds to P,P iff PTFP is skew-symmetric X P' T Possible choice: P [I | 0] P' [[e' ]F | e' ]

T FPX 0, X The essential matrix ~fundamental matrix for calibrated cameras (remove K) E t R R[R T t] x'T Ex 0 x K

E K'T FK 5 d.o.f. (3 for R; 2 for t up to scale) E is essential matrix if and only if two singularvalues are equal (and third=0) E Udiag(1,1,0)V T -1 x; x' K -1x'

Four possible reconstructions from E (only one solution where points is in front of both cameras) Next class: 3D reconstruction