CS131( Panoramic(Image(S6tching( - Artificial...
Transcript of CS131( Panoramic(Image(S6tching( - Artificial...
![Page 1: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/1.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
CS131(Panoramic(Image(S6tching(
Ranjay(Krishna(16#Oct#15(
1(
![Page 2: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/2.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Agenda(
• Objec6ve(• Main(flow(• Skeleton(code(• Results(
2(
![Page 3: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/3.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Objec6ve(
3(
Mul6ple(images(into(one(panorama!(
![Page 4: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/4.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Main(Flow(
• Detect(key(points(
4(
![Page 5: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/5.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Detect(Key(Points(
5(
![Page 6: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/6.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Skeleton(Code(
• Detect(key(points((Done(for(you!)(– Under(KeypointDetect(
6(
[feature,(DoG(pyr,(Gaussian(pyr](=(detect_features(input(image)(
![Page 7: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/7.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Main(Flow(
• Detect(key(points(• Build(the(SIFT(descriptors(
7(
![Page 8: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/8.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Build(the(SIFT(Descriptors(
8(
This(is(just(an(illustra6on!(
![Page 9: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/9.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Skeleton(Code(
• Build(the(SIFT(descriptors(– Read(this(paper(h[p://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf(first!((
• Input(– Gaussian(pyramid(– key(point(loca6on(– key(point(scale(index(
• Output(– A(set(of(128#dim(vectors(
9(
![Page 10: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/10.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Skeleton(Code(
• Build(the(SIFT(descriptors((30(lines(of(code)(– Compute(gradient(magnitude(and(orienta6on(– For(each(key(point(
• Find(a(patch((tricky(round#off)(• Compute(orienta6on(of(the(patch(• Build(the(histogram((edge(case)(
10(
![Page 11: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/11.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Main(Flow(
• Detect(key(points(• Build(the(SIFT(descriptors(• Match(SIFT(descriptors(
11(
![Page 12: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/12.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Match(SIFT(Descriptors(
• Euclidean(distance(between(descriptors(
12(
![Page 13: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/13.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Skeleton(Code(
• Match(SIFT(descriptors((6(lines(of(code)(– Input:(D1,(D2,(thresh((default(0.7)(– Output:(match([D1’s(index,(D2’s(index](– Try(to(use(one(for(loop(– Useful(command(
• repmat(• sort(
13(
![Page 14: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/14.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Main(Flow(
• Detect(key(points(• Build(the(SIFT(descriptors(• Match(SIFT(descriptors(• Figng(the(transforma6on(
14(
![Page 15: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/15.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Figng(the(transforma6on(
• 2D(transforma6ons(
15(
![Page 16: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/16.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Skeleton(Code(
• Fit(the(transforma6on(matrix(
• Six(variables(– each(point(give(two(equa6ons(– at(least(three(points(
• Least(squares(
16(
![Page 17: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/17.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Main(Flow(
• Detect(key(points(• Build(the(SIFT(descriptors(• Match(SIFT(descriptors(• Figng(the(transforma6on(• RANSAC(
17(
![Page 18: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/18.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
RANSAC(
• A(further(refinement(of(matches(
18(
![Page 19: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/19.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Skeleton(Code(
• RANSAC(– ComputeError(
19(
![Page 20: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/20.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Main(Flow(
• Detect(key(points(• Build(the(SIFT(descriptors(• Match(SIFT(descriptors(• Figng(the(transforma6on(• RANSAC(
20(
![Page 21: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/21.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Image(S6tching(
• Almost(done(for(you(
21(
![Page 22: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/22.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Skeleton(Code(
• Mul6ple(S6tch((2(lines(of(code)(– A(simplified(case(of(real#world(scenario(– Transforma6on(is(associa6ve(and(inver6ble(– Useful(command(• pinv(
22(
![Page 23: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/23.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Skeleton(Code(
• Tester.m(– Scripts(that(help(you(to(get(started(
• Evaluate.m(– Scripts(that(tests(your(solu6on(
• Load(fixed(input(from(checkpoint(• Run(your(implementa6on(• Compare(results(with(reference(solu6on(
23(
![Page 24: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/24.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Requirement(• Due(Date:(5pm(Oct(30,(2015(• Electronic(submission(only(– [email protected](for(code(– Gradescope(for(report(
• Code(+(Report(– SIFT(invariance(and(why(it(helps(– DoG(v.s.(Dense(SIFT(– Why(RANSAC(– Your(own(s6tches(– Error(discussion(
24(
![Page 25: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/25.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Results(
25(
![Page 26: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/26.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma
Results(
26(
![Page 27: CS131( Panoramic(Image(S6tching( - Artificial Intelligencevision.stanford.edu/teaching/cs131_fall1617/lectures/lecture_panora… · Jiayuan Ma PA1 Session 16#Oct#15(CS131(Panoramic(Image(S6tching(Ranjay(Krishna(16#Oct#15(1(Leonid](https://reader033.fdocuments.us/reader033/viewer/2022060221/5f074f2d7e708231d41c58ea/html5/thumbnails/27.jpg)
PA1 Session 16#Oct#15(Jiayuan Ma 27(
Ques6ons?(