# Multiple View Geometry in Computer Vision Epipolar Geometry class 11 Multiple View Geometry Comp 290-089 Marc Pollefeys Multiple View Geometry course schedule (subject to change) Jan. 7, 9 Intro & motivation Projective 2D

Geometry Jan. 14, 16 (no class) Projective 2D Geometry Jan. 21, 23

Projective 3D Geometry (no class) Jan. 28, 30 Parameter Estimation Parameter Estimation Feb. 4, 6

Algorithm Evaluation Camera Models Feb. 11, 13 Camera Calibration Single View Geometry Feb. 18, 20

Epipolar Geometry 3D reconstruction Feb. 25, 27 Fund. Matrix Comp. Structure Comp. Planes & Homographies

Trifocal Tensor Three View Reconstruction Multiple View Geometry Mar. 4, 6 Mar. 18, 20 More Single-View Geometry

Projective cameras and planes, lines, conics and quadrics. P T l P T CP Q cone PQ *P T C* Camera calibration and vanishing points, calibrating conic and the IAC Two-view geometry

Epipolar geometry 3D reconstruction F-matrix comp. Structure comp. Three questions: (i) Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point x in the second image? (ii) Camera geometry (motion): Given a set of corresponding

image points {xi xi}, i=1,,n, what are the cameras P and P for the two views? (iii) Scene geometry (structure): Given corresponding image points xi xi and cameras P, P, what is the position of (their pre-image) X in space? The epipolar geometry (a) C,C,x,x and X are coplanar The epipolar geometry

b What if only C,C,x are known? The epipolar geometry a All points on project on l and l The epipolar geometry b Family of planes and lines l and l Intersection in e and e

The epipolar geometry epipoles e,e = intersection of baseline with image plane = projection of projection center in other image = vanishing point of camera motion direction an epipolar plane = plane containing baseline (1-D family) an epipolar line = intersection of epipolar plane with image (always come in corresponding pairs) Example: converging

cameras Example: motion parallel with image plane The fundamental matrix F algebraic representation of epipolar geometry x l' we will see that mapping is (singular) correlation (i.e. projective mapping from points to lines) represented by the fundamental matrix F The fundamental matrix F

geometric derivation x' H x l' e'x' e' H x Fx mapping from 2-D to 1-D family (rank 2) The fundamental matrix F algebraic derivation X C P x CC l P' C P' P x

F e' P' P (note: doesnt work for C=C F=0) P P I Px X C The fundamental matrix F correspondence condition The fundamental matrix satisfies the condition

that for any pair of corresponding points xx in the two images T T x' Fx 0 x' l' 0

The fundamental matrix F F is the unique 3x3 rank 2 matrix that satisfies xTFx=0 for all xx (i) (ii) (iii) (iv) (v) Transpose: if F is fundamental matrix for (P,P), then FT is fundamental matrix for (P,P) Epipolar lines: l=Fx & l=FTx Epipoles: on all epipolar lines, thus eTFx=0, x

eTF=0, similarly Fe=0 F has 7 d.o.f. , i.e. 3x3-1(homogeneous)-1(rank2) F is a correlation, projective mapping from a point x to a line l=Fx (not a proper correlation, i.e. not invertible) Epipolar Line Homography The epipolar line geometry l,l epipolar lines, k line not through e l=F[k]xl and symmetrically l=FT[k]xl k l

k l e Fk l e' (pick k=e, since eTe0) l' F e l l FT e' l'

Pure Translation camera motion Fundamental matrix for pure translation Forward motion Fundamental matrix for pure translation F e' H e' example: e'1,0,0

T H K 1RK 0 0 0 F 0 0 - 1 0 1 0

x'T Fx 0 y y' x PX K[I | 0]X -1 K x' P' X K[I | t] x Z ( X,Y,Z ) T K -1x/Z x' x Kt/Z

motion starts at x and moves towards e, faster depending on Z pure translation: F only 2 d.o.f., xT[e]xx=0 auto-epipolar General motion x 'T e' Hx 0 x 'T e' x 0 x' K' RK -1x K' t/Z Geometric representation of F

FS F FT / 2 xx FA F FT / 2 T

x Fx 0 x T FS x 0 Fs: Steiner conic, 5 d.o.f. Fa=[xa]x: pole of line ee w.r.t. Fs, 2 d.o.f. F FS FA x T

FA x 0 Pure planar motion Steiner conic Fs is degenerate (two lines) Projective transformation and invariance Derivation based purely on projective concepts -T x Hx, x' H' x' F H' FH

-1 F invariant to transformations of projective 3-space P, P' F F P, P' unique not unique canonical form P [I | 0]

P' [M | m] F m M Projective ambiguity of cameras given F previous slide: at least projective ambiguity this slide: not more! ~~ Show that if F is same for (P,P) and (P,P), there exists a projective transformation H so that ~ ~ P=HP and P=HP

~ ~ ~ ~ P [I | 0] P' [A | a] P [I | 0] P' [A | a ] ~ F a A ~ a A ~ lemma: ~ a ka A k 1 A av T

Canonical cameras given F F matrix corresponds to P,P iff PTFP is skew-symmetric X P' T Possible choice: P [I | 0] P' [[e' ]F | e' ]

T FPX 0, X The essential matrix ~fundamental matrix for calibrated cameras (remove K) E t R R[R T t] x'T Ex 0 x K

E K'T FK 5 d.o.f. (3 for R; 2 for t up to scale) E is essential matrix if and only if two singularvalues are equal (and third=0) E Udiag(1,1,0)V T -1 x; x' K -1x'

Four possible reconstructions from E (only one solution where points is in front of both cameras) Next class: 3D reconstruction

## Recently Viewed Presentations

• The Quest of the Golden Fleece Max, Nate, and Andrew The Ram A king named Athamas leaves his wife Nephele for a second one named Ino Ino wants her son to inherit the throne but first she has to get...
• Introductions. Kaufman Wills Fusting & Company. Consulting and services for scholarly publishers since 2000. American Society for Nutrition. Publishing timely, important findings in basic, clinical, and emerging nutrition science research, the society journals are the premier journals in the field...
• Kompetencie ako ciele vyučovacieho procesu Kľúčové kompetencie = súhrn vedomostí, zručností, schopností, ale aj postojov, hodnôt a charakterových vlastností, ktoré umožňujú efektívne plnenie životných funkcií a rol sú vymedzené v profile absolventa sú nadpredmetové Kompetencie k celoživotnému učeniu sa Kompetencie...
• Topic: Atomic structure and the periodic table 1 Define an isotope in under 20 words. An isotope is an element with the same number of protons, but different number of neutrons. Why . has the Bohr model been accepted as...
• MCO 5800.14 created a "bill of rights" for victims and witnesses to ALL crimes under the UCMJ. VWAP is the program which ensures victims and witnesses are aware of these rights and are able to exercise them. Victim Witness Assistance...
• Cheng Jin David Wei Steven Low ... frequent delays between dev and TCP layers Linux loss recovery too many acks during fast recovery high CPU overhead per SACK very long recovery times Scalable TCP and H-TCP offer enhancements TCP/AQM Optimization...
• "The X Sensor monitors temperature, relative humidity and wood moisture content. It combines environmental sensors, wireless transceiver, microcontroller, and battery into a compact enclosure designed to be embedded into structures at common points of moisture failure.
• Language play- games that emphasize rhyming and thinking about the structure of words. Sociodramatic. play: Role-playing how to treat each other. Reading Aloud: Models what language sounds like and how one reads, it helps children foster an appreciation and comprehension...