@JackS'lgoe

Machinelearning,sociallearningandself-drivingcars

Outline

•  Responsibleinnova'onassociallearning•  Socialisingmachinelearning•  Self-drivingasa‘solvableproblem’•  Self-drivingasmisnomer•  Limitstoresponsiveness

Source:electrek.com

“Perfectsafetyisreallyanimpossiblegoal.”ElonMusk,Sept2016

Being“inBeta”“Becausetheprobabilityissosmall,thereisnoprac'calpossibilityofdeterminingthisfailureratedirectly-i.e.,bybuilding,letussay,1,000reactors,opera'ngthemfor10,000yearsandtabula'ngtheiropera'nghistories”.

(AlvinWeinberg,1972)“Eachnuclearpowerplantisitsowntestcase.Itmaybecategorizedasanimplicitexperiment,anditsmostrevealingcaseistheaccident”.

(KrohnandWeingart,1987)

“Emergingprac'cesdefine‘rules’,ratherthanrulescontrollingprac'ces.“

(Wynne1988)

Thewhat,thehowandthewhyofinnova'on

Products•  Whatarethelikelyrisks

andbenefits?•  Howwilltherisksand

benefitsbedistributed?•  Whatotherimpactscan

wepredict?•  Howmightthese

changeinthefuture?•  Whatdon’tweknow

about?•  Whatmightwenever

knowabout?

Processes•  Howshouldresearch

andinnova'ontakeplace?

•  Howshouldstandardsbedrawnupandapplied?

•  Howshouldrisksandbenefitsbedefinedandmeasured?

•  Whoisincontrol?•  Whowilltake

responsibilityifthingsgowrong?

•  Whatifwearewrong?

Purposes•  Whyshouldthis

researchbeundertaken?

•  Whowillbenefit?•  Whatarethe

alterna'ves?

Responsibleinnova'onis‘collec'vecareforthefuturethroughthestewardshipofinnova'oninthepresent’

(S'lgoe,OwenandMacnaghten2013)(alsoS'lgoeandGuston2016)

An'cipa'on Inclusion

Reflexivity Responsiveness

Responsibleinnova'on

Limitstoresponsiveness

•  Inflexibility,irreversibility,commitmentandotherformsoflock-in

•  Lackofreflexivity•  ‘Ins'tu'onalvoid’(Hajer)•  Presump'onsofnoveltyactagainstlearning(Rayner)

Approachestosociallearning

1.  Engineering–machineslearnaboutsociety2.  Educa'onal–learninghappenssocially3.  Ins'tu'onal– socialsystemsneedtolearn– AircrabvsNuclearpower(Perrow)

4.  Poli'cal– learningtakesplaceinpublic–  ‘Experimentalgovernment’(Deweyetal)

Socialisingmachinelearning

What’snew?

2015

GartnerHypeCycle2015

Levelsofautoma'on

“Howquicklycanwegetthisintopeople’shands?Ifyoureadthepapers,youseemaybeit’sthreeyears,maybeit’sthirtyyears.AndIamheretotellyouthathonestly,it’sabitofboth,”

ChrisUrmson(ex-Google),March2016

‘Solving’self-driving

Source:TheEconomist,2016

Ifsafetyistheques'on,areself-drivingcarstheanswer?

SeeinglikeaTesla

MobileyeandTeslapartcompany“Weneedtobethereonallaspectsofhowthetechnologyisbeingused,andnotsimplyprovidingtechnology”“Thisincidentinvolvedalaterallycrossingvehicle,whichcurrent-genera'onAEB[automa'cemergencybraking]systemsarenotdesignedtoactuateupon”

Mobileye

“AutonomousvehiclesisanAIcompu'ngproblem.It’snotadetec'onproblem”

Nvidia

Modesofautomo'velearningRule-following Rule-making

‘Autonomousvehicles’? Technicalandpoli'calentanglement

Technicalandpoli'cal‘autonomy’

Modesofmachinelearning

Formallogic End-to-endlearning

RelevantExperDse Domainexper'se,ergonomicsetc

Drivingasagame

Pedagogy Tradi'onal Autodidac'c

SocialconsDtuDon Favoursincumbents Upstartsandstart-ups

Strategy Incrementalinnova'on Disrup've,permissionlessinnova'on

Historicaltouchpoints Automatedhighways DARPAGrandChallenges

Escapingtheriskframe

Machinecontrolandsocialcontrol

Alterna'velearning

‘It'samazingtechnology…ButTeslashouldn'tmaketheircustomerstest

pilots… [Autopilot]allowsyoutothinkit'sworking.Andatahigherspeed,youcan'treact.It'salreadyhardenoughtoreact

whenyouarefocused,thatwhenyouarenot,anditerrs,it'sover.’

MarkMolthan,crashedAugust2016

Imagineddeficits

Thetyrannyofurgency

We“cannotwaitforperfect…Weshouldbedesperateforanythingwecanfindtosavepeople'slives,"

NHTSAAdministratorMarkRosekind

“Ifanybodylegislatesagainstit,theyshouldprobablygotojail”

EngineerAlainKornhauser

“Speedisoftheessencehere…We’reinaveryimportantraceagainstAsianandEuropeancountriesthataremovingforwardwiththistechnology”

SenatorGaryPeters

‘Dothemath’‘Ifanyonebotheredtodothemath(obviously,youdidnot)theywouldrealizethatoftheover1Mautodeathsperyearworldwide,approximatelyhalfamillionpeoplewouldhavebeensavediftheTeslaautopilotwasuniversallyavailable.Please,take5minsanddothebloodymathbeforeyouwriteanar'clethatmisleadsthepublic.’

ElonMusk,July2016‘If,inwri'ngsomear'clethat'snega've,you...dissuadepeoplefromusinganautonomousvehicle,you'rekillingpeople’

ElonMusk,October2016

Boundingtheexperiment?Broaderenvironment,economyandsociety

Landscape

Infrastructure

Otherroadusers

Otherdrivers

Self-drivingcaranduser

Ques'onsforsociallearning:aresearchagenda

•  Transi'ons•  An'cipa'ngtransportsystemsandcarusage– Whatmightbethepressuresonmachinereadableinfrastructure?

•  Scale•  Thegovernanceofwhat?– Sobware,hardware,system?

An'cipatorygovernance

Histories •  Pre-history(Wetmore2003;Vinsel,forthcoming)

–  Integra'ngvehiclesandinfrastructure•  Automa'onofcarsystems

–  Airbags–  ABS/Trac'oncontrol–  Cruisecontrol–  ADAS

•  2004,2005,2007:Darpagrandchallenges•  2008àMachinelearning

–  Neuralnetworksandimagerecogni'on–  GPUs–  Data

Thecomplexityoftransi'ons

•  Closed,homogenoussystemsareeasytoimagine;Mixed,opensystemsarealmostimpossible

•  Thehand-offproblem•  Smarttechcanmakepeopledumb

outline•  Teslaphoto

–  Hardwareandsobware–poli'csofnovelty–  Itisaworkinprogress-learninginpublic–andinnova'ng

•  Crash•  Emergingsociotechnicalsystem•  Responsivenessandsociallearning•  ImaginariesofAVs•  Governance

–  Stateasexperimenter,notjustlaboratory–  Safety–limitstoresponsiveness?–  Data-sharing

•  DataisthecurrencyofAVinnova'on–  Condi'onsforexperimenta'on–  Sobwarevshardware–productliability–  Democra'singimagina'on–  Whobenefits?Whatarelikelytobethetransfersofpowerandmoney?Socialcons'tu'on?

Techaslegisla'on?

PerrowontheNTSB

•  Whydoaeroplanesgetsaferwhilenuclearpowerdoesn’t?– Accidentsarenormal–notout-of-the-blue

•  Learningascrucial– “pastaccidentsalsofailaswarningsifthewarningisavailabletoonlyonepartofthesystem”–andthatwasaproblemwithTMI

Star'ngwithtransport

•  Infrastructure?•  Condi'onsforacceptability?

U.S.AirCarrierSafetyIndices,1920-1975

‘NeverNeutral’

Ethicsandtrolleyproblems

An'cipa'on•  Par'cipatory,notpredic've•  Understandingexpecta'ons,promises

Inclusion•  Publicengagement•  User-driveninnova'on•  Value-centreddesign

Reflexivity• 1stand2ndorder

Responsiveness•  Answeringandreac'ng•  Understandingthepoli'caleconomyofinnova'on

Responsible(researchand)Innova'on

Therulesoftheroad

Newlaboratoryrulesforself-drivingcars?

•  Sobwarevshardware–  Productliabilityrules

•  Ethicalrules–  GermanEthicsCommission

1.  propertydamagebeforepersonalinjury.

2.  noclassifica'onofpeoplebysize,ageetc.

3.  “Ifsomethinghappens,themanufacturerisliable”

•  AIredflags•  Black-boxes

•  Ini'alUSrules.NHTSA:“Theagencydoesnotbelievethatself-drivingvehiclesarecurrentlyreadytobedrivenonpublicroadsforpurposesotherthantes'ng.”…un'lTuesday

•  Responsibilitymovingfromstatetofederallevel– 15-pointvoluntarychecklistformanufacturers

SeeinglikeaTesla

“Ifanything,Uber’sdriversaretheR&DforUber’sdriverlessfuture.Theyarespendingtheirlaborandcapitalinvestments(cars)ontheirownfutureunemployment.”–DouglasRushkoff

•  NewCaliforniarules–talkinBerkeleyabouthowmisguidedthismoveis…•  Thiscouldevenbeaseparatepaper–withHarrySurden?•  Fornow,thepointisthatthingsareframedintermsofsafetyonly

–  Goodtostartwiththis–butaswithTeslastory,askwhatwecanlearnabouttechgoingwrongforthegovernanceoftechgoingright.–  Safetyframeisahugebarriertosociallearning–  WerefusetodemocraDsethediscussionofbenefits

•  •  •  TheCaliforniaDMVoriginallydrabedregula'onsthatdemandedsubstan'alcollabora'onbetweencompaniesandthestateonthetermsand

conductofthesocialexperiment.•  The‘lawlag’isobenactuallyaperformance–‘gexngbehind’thetechnology,beforeitisdefined•  A'ghteningofthesafetyframe•  Makingdeploymentdefinedasusebymembersofthepublic)easier•  AccidentsandDisengagements

–  (Theserequirementsares'llintheregula'ons–don’tunderstandtherepor'ng)•  Mademorepermissive–innova'onmorePermissionless

–  ‘Embracing’thetechnologyactuallymeansdistancingoneselffromit–disengaging•  Presumestechnologiesaredefinedascurrentlyimagined–level4and5,ratherthanawork-in-progress.Presumestechnologicalcompleteness

–  Interes'ngdefini'onof“opera'onaldesigndomain”,asopposedtoreal-worldcondi'ons–  Interes'ngthattrainingisrequiredforAVdrivers–  Languageof‘accidents’hasbeenreplacedbythatof‘collisions’–  Datarecorderincluded

•  •  CaliforniaannouncedinMarch2017theirinten'ontorelaxthesecontrols.Instead,theywouldputtheonusonmanufacturerstodeclarethattheir

carsweresafeandfully-insuredandtrustthatthelegalsystemwouldworkoutques'onsofliabilityandunintendedconsequences.Thethreatofbeingsuedisimaginedtobethestrongestlever.

Socialconstruc'onasperformance

Escapingtheriskframe