Initial word Robogames - Harvey Mudd College

Initial word... Robogames

Initial word... Robogames

The "CS graveyard"

Final words: project schedule…

This week: example talk and a bit on Turing Machines

Next week: project work during class (join in!)

Wed. 5/6: project presentations (~10 min. per project)

• Your source-code to date (showing off your library)

• One-page write-up of progress so far + plans still to-go

• 10-15 minute talk (example this week…)

Milestone: due 5/8/15

• 2-3 page write-up of results, +s, -s, and all of your source code

Final Project : due by 5/18/15 (or earlier, if you'd like!)

(final class meeting)

(final hw due 5/1)

• Include your project presentation slides!

(by email)

(by email)

Final project FSM!

it's broken

it works

stop adding

features + start

adding print

statements

comment out

print statements

+ start adding

more features

still

worksstill

broken

State-machine limits?Are there limits to what FSMs can do?

But are there any binary-string problems

that FSMs can't solve?

Page Rank

they can't

necessarily

drive safely...

Robotics

PageRank

State-machine limits?

Let's build a FSM that accepts bit strings

with the SAME NUMBER of 0s as 1s

01100

01110

rejected

0100

000

# of 0s

== # of 1s

0 1

011001

0110

accepted

10

λ this last string

is empty

FSMs are limited…

So, let's build a better machine!

Turing Machine

Shiny!

a Turing Machine rule:

the tape

R/W

head

the input

0 1 0 1 0"blanks" are

everywhere else

READ WRITE MOTION

0 ; ,1 R

if a transition is

missing, the

input FAILS!

an accepting state

always halts -- then

basks in its success!

try it in JFLAP...

Run this TM on this input:

0 0 1 1 1

Is this input accepted or rejected?

What inputs are accepted in general? How does it work?

Extra: How could you change this to accept palindromes?

Try it!

(a thought experiment – and ex. cr.)

Turing Machine machines!












(final hw due 5/1)


(by email)

(by email)

10-15 min. talk guideline (feel free to alter to suit!)

What's your

approach ?What library or

libraries did you

explore/use?

Your testing --

and results

What worked

and what didn't?

How did you adjust

to what was

feasible vs. not?

Further progress

and/or insights

Things you're

still planning to

work on… anything else…?


Title + Intro "Big Picture"

overall context +

motivation

What's the

problem?

Problem details

(narrative > bullets)to reducing the problem

Questions…

insights!

"Aside Slides"

Interlaced notes about the slides…

Suggestions and ideas…

Stuff to ignore…

This is a talk that summarizes a robotics and

vision research project we worked on…

blue!

PixelLaser: Evaluating range from texture

Nicole Lesperance ’11

Michael Leece ’11

Steve Matsumoto ’12

Max Korbel ’13

Kenny Lei

Zach Dodds

REUHarvey Mudd College – TePRA – 4/12/11

Idea: use monocular texture to

create a traditional range scan

Learning distance from texture

Zach Dodds 4/22/2015 IST338

"Title" slide…

You don't need a fancy title

I'm a fan of explanatory pictures…

I'm less a fan of using clip art…

Never hurts to have a short

"big-picture" summary…

really?

Looking at this image, we (humans) know how far

away obstacles are from the camera, relatively…

Big picture: obtain distances from an image

"Big picture" slide

What's the problem's overall goal

Overall context

Why it's interesting

You may have a lot to say, or just a little,

but include at least one such slide… I'll

help you brainstorm.

You do not need "alternative approaches"

Context: other approaches

active lightingdepth, cost, weight

Context: directly detecting distances…

Zeke and the laser…depth, cost, weight

Context: directly detecting distances…

how the Kinect works…depth, cost, weight

Alternatives to learning?

point-feature tracking

to estimate: how and

how much the camera

(or robot) moved

previous image next image

∆∆∆∆

to estimate: how and

how much the image

feature moved

δδδδ

texture vs. time

InspirationHors

will (p

olly) ’9

4Saxena

(rccar)

’05 Could we improve

resolution?

Could we handle more general indoor textures?

More recently

C. Plagemann et al., ICRA 2008

platform "omnicam" images errors...

~1m RMS error

Could we benefit from larger

patches of image texture?

The problem: there's lots of floor!

indoor robot + webcam on netbook

The data: lots of image patches!

sometimes the floor != the obstacles

The dataset: lots of patches!

sometimes not

The dataset: lots of patches!

not always interesting image patches…

Image patches

hand-segmented image

Each 20x20 pixel square

is a single "patch"

Image details…

hand-segmented image


is a single "patch"

~ 1200 numbers each!

(1) Train groundplane vs.

obstacle textures.

PixelLaser's challenge ("problem"/"workflow")

(3) Segment new images to

find traversibility hypotheses

(4) Unwarp segmentations

into range scans

(2) Classify new images using

groundplane/obstacle models

"Problem" slides

What's the computational problem?

Insights into its value…

Insights into its difficulty…

Highlight your contributions, e.g., columns

Is it labeled (how? by hand?)

Can we see some of it?

Library explored… OpenCV

… watched it form and grow …

Well documented! OpenCV

What does OpenCV do?

… it manipulates images (patches)

138 87 53

20x20 texture patches, each

represented by a few values.

For example, average Red,

Green, and Blue (out of 255)

average

blue

average

green

average

red

From images to numbers…

hand-segmented image features, in kd-trees

194 191 211

3.2 25.6 4.1 ...

138 87 53

-1.14 8.6 1.4 ...

8 filters chosen:

20x20 texture patches,

distilled into 8 values

ob

sta

cle

gro

un

d

texture + color filters

../TrainingImages/Playspacepswo13Patches/00029/randomBelow/0009.png

194.2575 191.4525 211.4775 195.0 192.0 212.0 8.6707 8.7688 7.2910 211.4775

191.4525 194.2575 212.0 192.0 195.0 7.2910 8.7688 8.6707 3113.1847 2918.6196

194.70780 -8.295e-09 -2.9999 3.4887 -0.1821 1.0586 0.7981 0.1422 1.3764 1.5171 -

3.2110 25.6053 4.0897 0.2096 3.4240 7.9828 0.6552 1.4017 1.2862 -0.0688 7.9768...

(Means, medians, stdev.,… for RGB/HSV colors and texture filters)

"features"


is a single "patch"

From images to numbers…

Nearest-neighbor classification

RGB alone did not work well…

top 10 distances to obstacle textures

top 10 distances to ground textures

Nearest-neighbors

RGB + texture produced better classifications

top 10 distances to obstacle textures

top 10 distances to ground textures

our real decision-making process…

"Library" slide(s)

what modules or libraries are you using?

explain a bit about what it can do… (!)

show off some of its basics

• after all, others may find they have a use for it!

embrace the messiness of

open-ended projects!

Evaluating the classifier Number of nearest

neighbors used:

1,5,10,25,50,&100

Training and testing

on SAME data set.

Train and test on

non-overlapping

data set.

Train and test on

overlapping data set

(same orientation)

Train and test on

overlapping data set

(novel orientation)

Hallway

overall accuracy: 95.4%

Library

overall accuracy: 94.1%

other

locations

Example predictions

classification + confidence segmentations

after training on a small part of this lab environment…

segmentation.mp4classification.mp4

"Next steps" slides

how well did initial trials work?

what are "typical" examples it gets right?

what are "typical" examples it gets wrong?

What other things are you

planning to analyze/investigate?

(note: this may just be the cross-validation answer!)

It's always nice to "drill down" to a few detailed examples.

Still to investigate…

• Go from classification to segmentation…

• How accurate are the distances?

• Enough to control the robot?

pixels

vs.

meters

Questions…?

"Still to go" slides

What is left in your project?

How could it be expanded/extended?

What would you do differently, if you

were to do it all again?

Perhaps

nothing!?

Nice to tie in to any personal plans

you might have relating to the data

or the project …

everyone can benefit

from your insights! loop at least once

more by the final

version!

Segmentation 3 approaches tested

(1) basic multiresolution

search (green)

(2) snap to strong image edges + median filter

(3) use a third texture category for on-

boundary patches (blue)

human segmentation in red

Segmentation approaches abandoned

seam carvinglargest transition probability

(classification probabilities too uncertain near the boundary)

(does not capture the inevitable discontinuities)

Results

Average RMS pixel errors

Stopping to take scans

median pixel error <2 pixels

in all environmentsunits of pixels

We didn't want our distances in pixels!

∆y

yo

θ

α

θ

α

∆y cos(θ)

f

f / cos(θ)

∆y sin(θ)

D

hFor a fixed horizon:

From segmentation to distance!

desired distance to the ground/object transition, D

c,k are known values

known height, h

known camera angle, θ

known focal length, f

Segmentation to dist (empirical)

pix

el ro

w

range

row-to-range mapscan

segmentation

Results

Average pixel errors

median metric error <3 inches

in all environments

Stopping to take scans

Average metric errors

units of pixels

units of inches

~ 30-75% drop in RMS error

compared to Plagemann et al.

Results

Average pixel errors

Average metric errors

Moving while taking scans

much worse!

as good as when stopped

units of pixels

units of inches

too shaky!

Applications Monte Carlo Localization

mcl.avi

Applications Scan matching & mapping (CoreSLAM)

+

Applications Texture-only navigation

When arriving in Atlanta for the AAAI exhibition: 10 seconds per frame


� C++ knn implementation with

variable precision levels

� Allowed for more complex real-time

segmentation algorithms

� Made autonomous wandering

feasible, using only the segmenter

� 2 orders of magnitude speedup Marius Muja: thank you!

FLANN Fast Library for Approximate

Nearest Neighbors

When arriving in Atlanta for the AAAI exhibition: 10 seconds per frame


10 hertz

4x video

AAAI exhibition wandering

Conclusions and future possibilities…

• Texture-based range is

• feasible enough

• fast enough

• accurate enough

to replace range scans

when other sensors are unavailable.

• Even so, challenges + opportunities remain!

e.g., ARDrones

What's your

approach ?What library or

libraries did you

explore/use?

Your testing --

and results

What worked

and what didn't?

How did you adjust

to what was

feasible vs. not?

Further progress

and/or insights

Things you're

still planning to

work on… anything else…?


Title + Intro "Big Picture"

overall context +

motivation

What's the

problem?

Problem details

(narrative > bullets)to reducing the problem

Questions…

insights!












(final hw due 5/1)


(by email)

(by email)

Electromechanical computation

Water-based computation

Integrated circuits

Tinkertoy computation

Parallel computers

Molecular computation

Quantum computationhttp://www.cs.virginia.edu/~robins/The_Limits_of_Quantum_Computers.pdf

http://www.arstechnica.com/reviews/2q00/dna/dna-1.html

So far, all known computational devices can

compute only what Turing Machines can...some are faster than others…

Turing machine

Alan Turing

1946

WWII

Enigma machine ~ The

axis's encryption engine

2007Bletchley Park

AI!

1912-1954

Initial word Robogames - Harvey Mudd College

Documents

Transcript of Initial word Robogames - Harvey Mudd College