Now What?stats.research.att.com/nycseminars/slides/provost.pdfSo. ExplanaQons of individual...
Transcript of Now What?stats.research.att.com/nycseminars/slides/provost.pdfSo. ExplanaQons of individual...
FosterProvost–11/17/17
1
So You’ve Built a Machine Learning Model…
Now What?
Foster Provost
Thanks to Josh Attenburgh, Henry Chen, Brian Dalessandro, Sam Fraiberger, Thore Graepel,
Panos Ipeirotis, Michal Kosinski, David Martens, Claudia Perlich, David Stillwell
TheDataScienceProcessisausefulframeworkforthinkingthroughlotsofmodeling&managerialdecisionsaboutsolvingproblemswithAI/MachineLearning/DataScience
Formore,seeDataScienceforBusinessProvost&FawceF.O’ReillyMedia2013
FosterProvost–11/17/17
2
Justafewissues:• Misalignmentofproblem
formulaQon• Leakageinfeatures• Samplingbias• Learningbias(MLfavors
largersubpopulaQons)• Labelingbias• EvaluaQonbias
TheDataScienceProcessisausefulframeworkforthinkingthroughlotsofmodeling&managerialdecisionsaboutsolvingproblemswithAI/MachineLearning/DataScience
InReality…
FosterProvost–11/17/17
3
InthistalkI’llfocusontwocommonproblemsfacedwhendeployingmachinelearnedmodels
• Lackoftransparencyintowhymodel-drivensystemsmakethedecisionsthattheydo– importantforawholebunchofreasons
• useracceptance,managerialacceptance,debugging/improving
– ofcurrentinterest:areyourdecisionsfair?• “UnknownUnknowns”
– doyouknowwhatyourmodelismissing?Especiallywhatit’smissingand“thinks”it’sge[ngright?
6GabrielleGiffordsShooQng,Tucson,AZ,Jan2011
FosterProvost–11/17/17
4
7
WhywasMarikoshownthisPoFeryBarnad?
FosterProvost–11/17/17
5
Whywasthisdecisionmade?
evidence ? decision
data-drivenmodel
Customer Manager
DataScienceTeam
Explana5onsforwhom?
FosterProvost–11/17/17
6
!"#$%&'()"*$'$+,
TheComplexWorldofModels
(Martens&FP,“ExplainingData-drivenDocumentClassificaQon.”MISQ2014)
AnoQonofexplanaQonTheEvidenceCounterfactual
• Modelscanbeviewedasevidence-combiningsystems• Weareconsideringcaseswhereindividualpiecesofevidenceareinterpretable
• Thus,foranyspecificdecision*fromanymodelwecanask:
Whatisaminimalsetofevidencesuchthatifitwerenotpresent,
thedecision*wouldnothavebeenmade?*The“decision”canbeathresholdcrossingforaprob.esQmaQon,scoringorregressionmodel
see(Martens&FPMISQ2014);(Chen,Moakler,Fraiberger,FP,BigData2017)(Moeyersomsetal.;Chen,etal.;ICML’16WkshponHumanInterpretabilityInML)
(cf.Hume1748)
FosterProvost–11/17/17
7
WhywasMarikoshownthisPoFeryBarnad?
WhywasMarikoshownthisPoFeryBarnad?
Becauseshevisited:
• www.diningroomtableshowroom.com• www.mazeltovfurniture.com• www.realtor.com• www.recipezaar.com• www.americanidol.com
FosterProvost–11/17/17
8
Let’sfocusonthedevelopersExplanaQonsaidthedatascienceprocess
• HelptounderstandfalseposiQves–omenrevealingproblemswiththetrainingdata
• Canrevealproblemswiththemodel
FosterProvost–11/17/17
9
Withtheincreasinguseofpredic=vemodelsfrommassivefine-grainedbehaviordata…
Consumersareincreasinglyconcernedaboutthe
inferencesdrawnaboutthem.
Kosinski,M.,SQllwell,D.,&Graepel,T.(2013).ProceedingsoftheNaQonalAcademyofSciences,110(15),5802-5805.
FosterProvost–11/17/17
10
EffectofremovingselectedFacebookLikesfromconsideraQonbythepredicQvemodel
Twoguyspredictedtobegay:
Model:logisQcregressiononthetop100latentdimensionsfromanSVDoftheuser/Likematrix.
(Chen,Moakler,Fraiberger,…BigData2017)(Chen,etal.,ICMLWkshpInterpretability2016)
FosterProvost–11/17/17
11
Whywasthisguypredictedtobesmart?
Opportunityforofferinguserscontrolviaa“cloakingdevice”?
EffectofremovingselectedLikesfromconsideraQonbythepredicQvemodel
FalsePosiQves
(Chen,Moakler,Fraiberger,…BigData2017)(Chen,etal.,ICMLWkshpInterpretability2016)
FosterProvost–11/17/17
12
Butthere’satwist…
Afirmcouldpurporttogiveuserstransparencyandcontrol……butactuallymakeitcumbersomeforuserstoaffecttheinferencesdrawnaboutthem:
(Chen,Moakler,Fraiberger,…BigData2017)(Chen,etal.,ICMLWkshpInterpretability2016)
FosterProvost–11/17/17
13
So.ExplanaQonsofindividualdecisionscanhelpwithmanyissuesintheprocessofbuildingandusingmachinelearnedmodels.Butweneedmorehelpwithoneveryimportantproblem…
TheproblemofUnknownUnknowns• Whatisyourmodelmissing?Whatisitmissinganditreallythinksthatit’scorrect?
• Whywoulditbemissingthings?
FosterProvost–11/17/17
14
Weneedtothinkcarefullyaboutthedata-generaQngprocess(es)andthedatapreparaQonprocesses–especiallytheprocessofge[nglabeledtraining&tesQngdata.
TheproblemofUnknownUnknowns• Whatisyourmodelmissing?Whatisitmissinganditreallythinksthatit’scorrect?
• Whywoulditbemissingthings?– Samplingbias– Learningbias(MLfavorslargersubpopulaQons)– Labelingbias– EspeciallysevereforNon-self-revealingproblems
(AFenberg,IpeiroQs&ProvostJDIQ2015)
FosterProvost–11/17/17
15
HarnessHumanstoImproveMachineLearning
• Withnormallabeling,humansarepassivelylabelingthedatathatwegivethem
31
Instead ask humans to search and find positive instances of a rare class
Searchinginsteadoflabelinghasintriguingperformance
(AFenberg&FPKDD2010)
FosterProvost–11/17/17
16
Active learning missing disjunctive subconcepts
33
(AFenberg&FPKDD2010)
NIPS 2016
FosterProvost–11/17/17
17
35
BeFer,but…..• Classifierseemsgreat:Cross-validaQontestsshowexcellent
performance
• Alas,classifierfailson“unknownunknowns”
“Unknown unknowns” à classifier fails with high confidence
(AFenberg,IpeiroQs&ProvostJDIQ2015)
FosterProvost–11/17/17
18
37
BeattheMachine!
Askhumanstofindexamplesthat• theclassifierwillclassifyincorrectly• anotherhumanwillclassifycorrectly
Example: Find hate speech pages that the machine
will classify as benign
(AFenberg,IpeiroQs&ProvostJDIQ2015)
38
BeattheMachine!
Example: Find hate speech pages that the machine
will classify as benign
IncenQvestructure:• $1ifyou“beatthemachine”
• $0.001ifthemachinealreadyknows (AFenberg,IpeiroQs&ProvostJDIQ2015)
FosterProvost–11/17/17
19
AAAI 2017
(AFenberg,IpeiroQs&ProvostJDIQ2015)
AAAI 2017
FosterProvost–11/17/17
20
Summary
• WecanprovidetransparencyintothereasonswhyAIsystemsmakethedecisionsthattheydo
• Wecancreatemechanismstohelpfindthe“UnknownUnknowns”
• Asaresearcharea,there’ssQllalottodo
Somereading
Martens&FP,“ExplainingData-drivenDocumentClassificaQon.”MISQ2014
Moeyersomsetal.2016,ICML’16WkshponHumanInterpretabilityInML
Chen,etal.2016,ICML’16WkshponHumanInterpretabilityInMLChen,Fraiberger,Moakler,Provost.BigData5(3)2017
AFenberg,J.&Provost,F.Whylabelwhenyoucansearch?AlternaQvestoacQvelearningforapplyinghumanresourcestobuildclassificaQonmodelsunderextremeclassimbalance.InKDD2010.AFenberg,J.,IpeiroQs,P.&Provost,F.BeattheMachine:ChallengingHumanstoFindaPredicQveModel's“UnknownUnknowns”.JournalofDataandInformaQonQuality(JDIQ),6(1)2015.