Uvrgrp ml

15
David Callender • Finished in top 2% (18th out of >1300) on 3 year $3 million Machine Learning competition. • Studied disease propagation in an urban setting using probabilistic graphical models at Dartmouth College • Studied computational protein design at the University of Washington • Studied Mathematical foundations of Quantum Mechanics at Macalester College

TAGS:

description

 

Transcript of Uvrgrp ml

Page 1: Uvrgrp ml

David Callender• Finished in top 2% (18th out of >1300) on 3 year

$3 million Machine Learning competition.

• Studied disease propagation in an urban setting using probabilistic graphical models at Dartmouth College

• Studied computational protein design at the University of Washington

• Studied Mathematical foundations of Quantum Mechanics at Macalester College

Page 2: Uvrgrp ml

Machine Learning in Rcirca 2013

David Callender

Page 3: Uvrgrp ml

a.k.a. Using R on Kaggle

who will end up in the hospital

} drug effectiveness

Computer Security:Determining employee

access needs

What will the salary be for a given job advertisement

Page 4: Uvrgrp ml

Not Just Kaggle

•Movie recomendations•Popular productions

•Product recomendations•Good business oportunities

•The Entire Internet•Probably a lot more too

Page 5: Uvrgrp ml

Talk Outline

• Motivation

• Concepts

• Algorithms

• Decision Trees and Forests

• Neural networks

• Kaggle

• Interactive session with R packages

• randomForest

• gbm

• neuralnet

Page 6: Uvrgrp ml

Supervised Learning

Survived Pclass Sex Age SibSp Parch Fare Embarked0 3 male 22 1 0 7.25 S1 1 female 38 1 0 71.2833 C1 3 female 26 0 0 7.925 S1 1 female 35 1 0 53.1 S0 3 male 35 0 0 8.05 S0 3 male 33 0 0 8.4583 Q0 1 male 54 0 0 51.8625 S0 3 male 2 3 1 21.075 S1 3 female 27 0 2 11.1333 S1 2 female 14 1 0 30.0708 C

Survived Pclass Sex Age SibSp Parch Fare Embarked? 3 male 34.5 0 0 7.8292 Q? 3 female 47 1 0 7 S? 2 male 62 0 0 9.6875 Q? 3 male 27 0 0 8.6625 S? 3 female 22 1 1 12.2875 S? 3 male 14 0 0 9.225 S? 3 female 30 0 0 7.6292 Q? 2 male 26 1 1 29 S? 3 female 18 0 0 7.2292 C? 3 male 21 2 0 24.15 S

Train model with examples where

you know value of “survived”

Use model to predict value of

“survived”

Predicting survival for passengers of Titanic

binary

numeric

catagorical

Page 7: Uvrgrp ml

Overfitting

http://en.wikipedia.org/wiki/File:Overfitting_on_Training_Set_Data.pdf Tomaso Poggio

http://en.wikipedia.org/wiki/File:Overfitting_on_Training_Set_Data.pdf
http://en.wikipedia.org/wiki/File:Overfitting_on_Training_Set_Data.pdf
Page 8: Uvrgrp ml

Decision Trees

http://en.wikipedia.org/wiki/File:CART_tree_titanic_survivors.png | Stephen Milborrow | Made using R

Survived Pclass Sex Age SibSp Parch Fare Embarked? 3 male 34.5 0 0 7.8292 Q? 3 female 47 1 0 7 S? 2 male 62 0 0 9.6875 Q

? 3 male 27 0 0 8.7 S? 3 female 22 1 1 12.2875 S? 3 male 14 0 0 9.225 S? 3 female 30 0 0 7.6292 Q? 2 male 26 1 1 29 S? 3 female 18 0 0 7.2292 C? 3 male 21 2 0 24.15 S

http://en.wikipedia.org/wiki/File:CART_tree_titanic_survivors.png
http://en.wikipedia.org/wiki/File:CART_tree_titanic_survivors.png
Page 9: Uvrgrp ml

Random Forest (RF)Survived Pclass Sex Age SibSp Parch Fare Embarked

0 3 male 22 1 0 7.25 S1 1 female 38 1 0 71.2833 C1 3 female 26 0 0 7.925 S1 1 female 35 1 0 53.1 S0 3 male 35 0 0 8.05 S0 3 male 33 0 0 8.4583 Q0 1 male 54 0 0 51.8625 S0 3 male 2 3 1 21.075 S1 3 female 27 0 2 11.1333 S1 2 female 14 1 0 30.0708 C

Survived Pclass Sex Age SibSp Parch Fare Embarked0 3 male 22 1 0 7.25 S1 1 female 38 1 0 71.2833 C1 3 female 26 0 0 7.925 S1 1 female 35 1 0 53.1 S0 3 male 35 0 0 8.05 S0 3 male 33 0 0 8.4583 Q0 1 male 54 0 0 51.8625 S0 3 male 2 3 1 21.075 S1 3 female 27 0 2 11.1333 S1 2 female 14 1 0 30.0708 C

Random Sub-SpacesBagging

{

{Voting/Avg

Prediction

Training

Page 10: Uvrgrp ml

Adaboost &Gradient Boosting

• Initialize a set of weights, One for each training example, with equal value

• Train a tree with weighted training examples

• Add tree to set of trees

• Make predictions with set of trees

• Adjust weights so that the training examples you got wrong have more weight

• repeat

Page 11: Uvrgrp ml

Logistic Regressiona.k.a The Perceptron

ActivationFunction

Weighted sum

Page 12: Uvrgrp ml

Multilayer Feed-forwardNeural Network

Page 13: Uvrgrp ml

R’s Popularity

Tools mentioned in Kaggle user profiles

From blog entry by Ben Hammerhttp://blog.kaggle.com/2011/11/27/kagglers-favorite-tools/

http://blog.kaggle.com/2011/11/27/kagglers-favorite-tools/
http://blog.kaggle.com/2011/11/27/kagglers-favorite-tools/
Page 14: Uvrgrp ml

Summary of Recent Competition Winners

Position Algorithm Other Algs. Tools

AdzunaSalary

1stAdzunaSalary

2ndAdzunaSalary

3rd

Merck

1st

Merck 2ndMerck

3rd

NN* - Python GPU

NN - C++

NN NB, SVM, LR Python

NN* - Python GPU

GBM & SVM RF, PCA,KNN, SVM R & Python

RF & SVM GBM, NN R

Page 15: Uvrgrp ml

Learning More

• Pedro Domingos at University of Washington

• www.coursera.org/course/machlearning

• www.coursera.org/uw

• A Few Useful Things to Know about Machine Learning. Communications of the ACM

• homes.cs.washington.edu/~pedrod

• blog.kaggle.com

• ufldl.stanford.edu/wiki/

http://www.coursera.org/course/machlearning
http://www.coursera.org/course/machlearning
http://www.coursera.org/uw
http://www.coursera.org/uw
http://www.cs.washington.edu/homes/pedrod/papers/cacm12.pdf
http://www.cs.washington.edu/homes/pedrod/papers/cacm12.pdf
http://blog.kaggle.com
http://blog.kaggle.com

Samsung ML-2850 Series ML-2850D / ML-2851ND service manual

Samsung ML-2850 Series ML-2850D / ML-2851ND service manual

Manual de Recarga Samsung ML 2251N | ML 2251NP | ML 2251W | ML 2250

Manual de Recarga Samsung ML 2251N | ML 2251NP | ML 2251W | ML 2250

ML 320 ML 350 ML 500 ML 55 AMG

ML 320 ML 350 ML 500 ML 55 AMG

Med. Music by Oscar Levant Lyric man Ml MA Ml ieved Ml Ml ...jepa.pp.fi/nuotit/Blame It On My Youth.pdf · Med. Music by Oscar Levant Lyric man Ml MA Ml ieved Ml Ml in ev— 'ry-

Med. Music by Oscar Levant Lyric man Ml MA Ml ieved Ml Ml ...jepa.pp.fi/nuotit/Blame It On My Youth.pdf · Med. Music by Oscar Levant Lyric man Ml MA Ml ieved Ml Ml in ev— 'ry-

REPLACEMENT PARTS · 2015-03-25 · legacy mixer series hl120 ml-134296 hl120 ml-134459 hl200 ml-134289 hl200 ml-134331 hl200 ml-134455 hl200 ml-134457 hl200 ml-134458 hl200c ml-134456

REPLACEMENT PARTS · 2015-03-25 · legacy mixer series hl120 ml-134296 hl120 ml-134459 hl200 ml-134289 hl200 ml-134331 hl200 ml-134455 hl200 ml-134457 hl200 ml-134458 hl200c ml-134456

Final - English · English Breakfast 20 Camomile 20 Earl Grey 20 350 ml 350 ml 250ml 1800 ml 1800 ml 135 ml 750 ml 135 ml 300 ml 300 ml. Title: Final - English Created Date: 7/24/2018

Final - English · English Breakfast 20 Camomile 20 Earl Grey 20 350 ml 350 ml 250ml 1800 ml 1800 ml 135 ml 750 ml 135 ml 300 ml 300 ml. Title: Final - English Created Date: 7/24/2018

ML15-17 (ML-1510 ML-1710 ML-1750)

ML15-17 (ML-1510 ML-1710 ML-1750)

Customizing Contrast Injection for Body MDCT: Algorithmic … · 2013-12-09 · Contrast Volume 98 mL 140 mL 175 mL 200 mL* Contrast Volume 55 mL 75 mL 90 mL Weight Body CT Based

Customizing Contrast Injection for Body MDCT: Algorithmic … · 2013-12-09 · Contrast Volume 98 mL 140 mL 175 mL 200 mL* Contrast Volume 55 mL 75 mL 90 mL Weight Body CT Based

ML Princess, ML Solo, ML Flex Air humidification

ML Princess, ML Solo, ML Flex Air humidification

CATALOG OF REPLACEMENT PARTS€¦ · hl800 & hl1400 series legacy mixers (multi drive) ml-141074 hl800 ml-141077 hl800c ml-141078 hl1400 ml-141081 hl1400c ml-141111 hl1400n ml-141113

CATALOG OF REPLACEMENT PARTS€¦ · hl800 & hl1400 series legacy mixers (multi drive) ml-141074 hl800 ml-141077 hl800c ml-141078 hl1400 ml-141081 hl1400c ml-141111 hl1400n ml-141113

Grow Micro Bloom Honey · Week 7 Late Flowering - 6.0 mL 11.0 mL 10.0 mL 8.0 mL 8.0 mL - Week 8 Ripening - 4.0 mL 11.0 mL 5.0 mL 4.0 mL 6.0 mL - Week 9 Flush 2.0 mL 2.0 mL 2.0 mL

Grow Micro Bloom Honey · Week 7 Late Flowering - 6.0 mL 11.0 mL 10.0 mL 8.0 mL 8.0 mL - Week 8 Ripening - 4.0 mL 11.0 mL 5.0 mL 4.0 mL 6.0 mL - Week 9 Flush 2.0 mL 2.0 mL 2.0 mL

ml-1640, ml-2240, ml-2241, ml-2245 general

ml-1640, ml-2240, ml-2241, ml-2245 general

ML OVERSEAS ML PILOTS FieldOps

ML OVERSEAS ML PILOTS FieldOps

FARM ANIMALS PRODUCTS€¦ · 100 mg/ml menbutone 25 mg/ml Polymerized silicone 500.000 IU/ml Vitamin A, 75.000 IU/ml Vitamin D3, 50 mg/ml Vitamin E 100 mg/ml Butaphosphan, 0.05 mg/ml

FARM ANIMALS PRODUCTS€¦ · 100 mg/ml menbutone 25 mg/ml Polymerized silicone 500.000 IU/ml Vitamin A, 75.000 IU/ml Vitamin D3, 50 mg/ml Vitamin E 100 mg/ml Butaphosphan, 0.05 mg/ml

d163axztg8am2h.cloudfront.net...Analysis Volume 100 ml IOO ml 400 ml IOO ml 50 ml 50 ml 200 ml 50 ml 500 ml 300 ml IOO ml 25 ml 500 ml 200 ml IOO ml 200 ml EPA Method # 305 310 350

d163axztg8am2h.cloudfront.net...Analysis Volume 100 ml IOO ml 400 ml IOO ml 50 ml 50 ml 200 ml 50 ml 500 ml 300 ml IOO ml 25 ml 500 ml 200 ml IOO ml 200 ml EPA Method # 305 310 350

Syringes & Needles - B. Braun...(non-sterile) 3 ml 0.1 ml centric/luer 9200210 5 ml 0.2 ml centric 9200510 10 ml 0.5 ml centric 9201010 20 ml 1 ml centric 9202010 9 new! Blue plunger

Syringes & Needles - B. Braun...(non-sterile) 3 ml 0.1 ml centric/luer 9200210 5 ml 0.2 ml centric 9200510 10 ml 0.5 ml centric 9201010 20 ml 1 ml centric 9202010 9 new! Blue plunger

Bert Niesters - National Institute for Biological Standards and · PDF fileCopies/ml Genome equivalents/ml Mega-equivalents/ml Log10 copies/ml SuperQuant copies/ml International units/ml

Bert Niesters - National Institute for Biological Standards and · PDF fileCopies/ml Genome equivalents/ml Mega-equivalents/ml Log10 copies/ml SuperQuant copies/ml International units/ml

SPIRITS - Stoelzle Glass Group€¦ · SPIRITS STANDARD COLLECTION ART/REF 35433 35433 35433 35433 VOLUME 100 ml 200 ml 350 ml 500 ml BRIMFUL 108 ml 220 ml 365 ml 515 ml WEIGHT 255

SPIRITS - Stoelzle Glass Group€¦ · SPIRITS STANDARD COLLECTION ART/REF 35433 35433 35433 35433 VOLUME 100 ml 200 ml 350 ml 500 ml BRIMFUL 108 ml 220 ml 365 ml 515 ml WEIGHT 255

REPLACEMENT PARTS · hs series manual slicers ml-136244 hs6 ml-136245 hs6n ml-136264 hs6 (canada) ml-136265 hs6n (canada) ml-136292 hs6n-1gc ml-136303 hs6-phs ml-136250 hs8

REPLACEMENT PARTS · hs series manual slicers ml-136244 hs6 ml-136245 hs6n ml-136264 hs6 (canada) ml-136265 hs6n (canada) ml-136292 hs6n-1gc ml-136303 hs6-phs ml-136250 hs8

ML-2240 & ML-2245 & ML-1640 Series training February 2008diagramasde.com/diagramas/otros2/ml-1640, ml-2240... · ML-2240 & ML-2245 & ML-1640 Series training February 2008 ML-2240

ML-2240 & ML-2245 & ML-1640 Series training February 2008diagramasde.com/diagramas/otros2/ml-1640, ml-2240... · ML-2240 & ML-2245 & ML-1640 Series training February 2008 ML-2240