Something New? - David Orban · 2019. 10. 9. · Something New? - David Orban It is tempting to...

Something New? - David Orban

�1


Something New?AIs and Us

David Orban

Copyright © 2018 David Orban http://davidorban.com http://singularitybook.net Published under Creative Commons Attribution license  

�2


Table of Contents

Technology created humanity 5 A fiery start 5 The dinosaurs did not have telescopes 7 A non-zero sum game 9 Access to the sacred text of your DNA 10 The proactionary principle 12 Fit or unfit civilizations 13 Unsustainability is unsustainable 15

The methods of knowledge 16 The fatal error of the alchemists 17 Open science 17 The evolution of science 19

Exponential change 20 Dynamic systems 20 Meaningful sequences 21 Noisy signals 22 The Human Genome Project’s exponential growth 23 Moore’s Law 24 The power of doublings 26 When is it too late? 26 Ray Kurzweil’s project 27 Connecting S-curves 28 Exponentials everywhere 30 No magic 100% 30

Artificial intelligence 31 The Nature of Intelligence 31 The Mechanical Turk 32 Turing tests for humans 32 Chess playing experts 33 Expert systems and restricted artificial intelligence 34 Hopes and disillusions 34 The role of learning 35

Artificial General Intelligence 36 The Turing test becoming moot 36 Unavoidable anthropomorphizing 37 Predictions for AGI 38 Architectures for AGIs 39 AGI hardware 39 Self improvement 40 Intelligence explosion 41 Self-awareness and introspection 41 Open access to your self 42 Slow takeoff 43 Rapid takeoff 43 Scales of intelligence 44 Is super intelligence uncontrollable? 44 AGI getting out of the box 45

�3


Singularities 46 The Technological Singularity 47 Kinds of minds 48

The power of evolution 50 Biological evolution 50 The evolution of knowledge 50 Universal Darwinism 51 Is the universe evolving? 51 Do universes evolve? 52 Does evolution evolve? 53 What did Fermi talk about in the desert? 54 Where is the Great Filter? 55

Human machine coevolution 57 Individual symbiosis 57 Social symbiosis 60 The Internet of Things 61 Intelligence augmentation 62 Talking to computers 63 Emotional computing 64 Ethical best practices 65 Empathy augmentation 65 Let’s make ourselves dispensable! 66 The responsibility of societies 67 The necessity for science and engineering of morality 68 Toward a naturalistic spirituality 70 The future of humans and humanity 71 Necessary transhumanism? 71

What to do today? 72 Understand, learn, teach 73 Test, make mistakes 73 Open up, adapt 74 Smart alertness 74 Society and artificial intelligence 75 The individual and artificial intelligence 77

What to do tomorrow? 79 Following a path with open eyes together 79 Diversities and tolerance 79 New dignities 80 Emancipation 80 Evolution of new degrees of freedom 81

What to do the day after tomorrow? 81 A radical life extension 82 Cryonics 84 To be an onion 85 Mind uploading 85 Kinds of lives 86 When the sun goes out 87 Adaptation, individual and self-perception 88 The Simulation Argument 89 A guarantee: the path will never end 89 Hic sunt leones? The porous map of reality 90

�4


Itistemptingtoalignoneselfwiththewisdomofthenihilnovumsubsole(thereisnoth-ingnewunderthesun)stance.Butonceinawhiletherearetrulynewthingsaroundus,anditis

crucially important to recognize themand tobeable toproperlyunderstand them.The forth-

comingeraofarti?icialintelligencesissuchafundamentalnewcomponentinourworld.

Technologycreatedhumanity

Afierystart

Therearemanyde2initionsofwhatahumanbeingis,andwhatmakesusdifferentfromotheranimals.Thatwearedifferentisundeniable.Monkeysandmanybirdsimitateandlearn.Manyhumancharacteristicswemightthinkareuniqueappearinotherspecies.Yettheyarecombinedinusinsuchquantitiesandinsuchawayastohavegeneratedaqualitativechangethatmakesusunique.

Amongourvariousframesofreference,oneisparticularlyuseful:thetechnologieswehaveinventedandappliednotonlyhelpandsupportusinourlives,theyde2ineournature.

Oneofthe2irstexamplesofhowtechnologyhasafundamentalin2luenceonwhatweareistheabilitytocontrol2ire.Byusing2iretocookthefoodweeat,wemake2ireanelementinthedigestiveprocess.Ourcousinsthegorillasspendadozenormorehourseatinganddigest-ing,whereaswearemoreef2icient:wespendlesstimeandenergyeatinganddigesting,andalsoabsorbthenutrientsinthefoodmoreeffectively.Thishasenabledustoshortenourdi-gestivetract,andtoexpandourbrain,notablytheneocortex.Althoughthebrainaccountsforonlyabout2%ofourbodymass,itabsorbsaround30%ofourenergy!Thebrainwouldgrowevenmorewere itnot for thebottleneck,quite literally, representedby thediameterof thefemalepelvis.Eventhen,weareparticularlyimmatureatbirth,comparedwiththenewbornofotheranimalspecies.Afewminutesafterbirth,agazelleisabletostandupandrunwithitsmother.Ahumanbabyiscompletelyhelplessandtakesseveralyearseventolearntofeedit-self. Inthisperiod,ourbraincontinuestodevelop,especiallytheneocortex,themostrecent

�5


partofthebrain,organizingitselftoaccommodateasbestaspossiblethehugequantitiesofinformationandknowledgeneededtobeanactiveandusefulmemberofsociety.

Agriculture is anotherexampleof akeyenabling technology.For tensof thousandsofyears,althoughhumanbeingswereequivalenttoustodayineveryway,withthesamecharac-teristicsandcapabilities,thenumberofpeoplelivingontheplanethardlychanged.Theywerehunter-gatherersand themaximumsizeof thesenomadic tribeswasdeterminedby theca-pacityofaparticularareathatcouldbecoveredbyfootinadaytoprovidethemwithsuste-nance.Thetotalpopulationoftheplanetwasonlyafewmillion.Indeed,recentgeneticstudiesonthemitochondrion,thepartofthecellwhosegeneticcompositionisonlyinheritedthroughthemother,haveshownthatthenumberofpeoplelivingatacertainpointfromwhomallhu-manbeingsaredescendedwasabout5,000.This2ine-meshevolutionarysieve,andthechainofimprobabilityitrepresents,isnotjustanisolatedcase:itrepresentsacharacteristicofnat-ural selectionand theevolutionof complexsystems.Onereason for this is thataparticulargeographicalareacouldfeedonlyarelativelysmallnumberofindividuals.Agroupof20–30peoplemightstopinavalleyforafewdaysorweeks,andthenfollowberriesastheyripened,orthemovementsofanimals.Wehaveanidealizedviewofthatperiod,givenourlackofdirectreferences.Weimagineasustainablelifestylewithoutcommitmentsandstress,intouchwithnature.Inactualfact,thisisonlytrueifweacceptanaveragelifeexpectancyof20–30years,perhapsnoteventhat,andtherealunsustainabilityofacontinualplunderingofnature,whichisonlyallowedtocatchitsbreathwhenwemoveon.Anillustrationoftheunsustainabilityofprehistoricman’swayoflifeishunting,whichledtotheextinctionofthemegafaunainallthecontinents.Whether itwastheSiberianmammothortheAustralianemu,wekilledthemallwithoutasecondthought.Somuchfortheimageofthe“noblesavage”incontactwithnature!

The advent of agriculture simultaneously in various regions of the world about tenthousandyearsago,basedonthecultivationofcorn,riceandmaizeandtherearingofchick-ens,pigs, sheep, cowsandotherdomestic animals, enabledman to increase thequantityoffoodavailableinaparticulargeographicalareabymorethananorderofmagnitude.Thereli-abilityof thisproduction, evendespite thevariations inyieldsdue to rainfall andparasites,wasalsopreferabletotheimpossibilityofpredictingwhatagroupofnomadichunter-gather-erswould2indinthenextvalley.Inturn,thesedevelopmentslednotonlytotheestablishmentofpermanentsettlements,butalsotoacorrespondingincreaseinordersofmagnitudeofthepopulation.

�6


Paradoxically, statisticalprocessingof the informationacquired fromskeletonsof thattimeshowsthatintheagriculturalage,lifewasharderthanwhenpeoplewerehunter-gather-ers:theaveragepersonwasshorteranddiedyounger!Evenifpeoplerealizedthis,itwasdif2i-culttodoanythingaboutit:intheagriculturalregions,itwasnolongerpossibletoreturntotheoldlifestyleandsocialorganization.Thisisageneralprincipleofsocietiesandorganiza-tionsthatadoptcertaintechnologies:thenewtransformedrealitydependsontechnologytofunctionandcannolongerabandonthattechnologyandgobacktotheoldways.Ourimageofbygonedaysisoftenidealized;ourperceptionoftheirpositiveaspectsisampli2ied,asisourperceptionofthenegativeaspectsofthepresent.Yetifwedecidedtoabandontechnologyto-day,billionsofpeoplewoulddieandno-one,luckily,isabletodecidewhothosepeopleshouldbe.

Theincreasedavailabilityoffoodanditsrelativereliabilityintheagriculturalagefueledapopulationincreaseand,withthisincrease,thedevelopmentofspecializedactivities.Initial-ly, theseactivitiesweredirectlyassociatedwithagriculturalwork,but, later, a growingper-centageofthepopulationbecameinvolvedinnon-agriculturaloccupations.Today,inhigher-incomesocieties,notmorethan2–3%ofthepopulationworksinagricultureandanimalhus-bandry:weareabletofeed100%ofthemembersofsocietywithjust2%oftheworkforce.Eveninsocietieswithmedium-lowincomesuchasIndia,irrespectiveofthepercentageofthepopulation,ithasbeencalculatedthatmechanicalsourcesaccountfor90%oftheenergyin-volvedinagriculturalwork.

Thedinosaursdidnothavetelescopes

Fortensofmillionsofyears,dinosaurswerethecuttingedgeofevolution,withagreatvarietyofspecies.Wearestilldiscoveringhowdinosaurgroupswereorganizedand, forex-ample,howsomeofthemcaredfortheyoung.Wedonotknowwhattriggeredthedevelop-ment of intelligence inman nor dowe knowwhether dinosaurswould havemade similarprogress, given an appropriate environmental stimulus. We do know that time ran out onthem: after the impact of an asteroidwith the Earth, and the ensuing climatic changes, di-nosaursbecameextinct, togetherwithavery largemajorityofcontemporaryspecies.Therehavebeen2ivemajoronesoftheseso-calledmassextinctionsontheEarthandweareinvesti-gatingtheirrespectivecauses.

�7


If dinosaurshadhad telescopes, could theyhaveavoidedextinction?Theywould cer-tainlyhavebeenabletoidentifytheasteroidbeforeithittheplanet.Andiftheyhadhadasuf-2iciently advanced space technology, they could have organized a rescuemission to try andmodify theorbit of the asteroid andprevent it fromcollidingwith theEarth.Wehave tele-scopesandaregraduallydevelopingourcapabilities inspaceandthemodelsenablingustoperformmaneuversofthistype.Forexample,anddespitewhatHollywoodmovieswouldhaveusbelieve,explodingtheasteroid isnotparticularlyhelpful;althoughthepieceswouldbeabitsmaller,theywouldremainonthesameorbit,andtheirtotalimpactwouldhavethesameenergyas thewholeasteroid.The thinking today inclines toward theview thatanchoringasuf2icientlylargeprobetotheasteroidwouldcreateavariationinthegravitational2ieldthatwouldchange itsorbitovertimeandavoidacollisionwiththeEarth.Thequestion involvesmathematicalandscienti2icknowledge,engineeringexpertise,projectcoordinationandman-agement,2inanceandresourceallocation,andsocialandpoliticalconsensus.

Inconcreteterms, thedifferencebetweenusandthedinosaurs,ofcourse, is thattheydidn’thavetelescopes.Metaphorically,however,theydidnotpossessthetoolsofreasonandsciencetoenablethemtodealwiththedangersofextinction,toseeandperhapsdeviatetheasteroid,or,inourmoderntimes,thedangersofpandemics,climatechange,extremecon2lictandsoon.

WhentheNASAbudgetforradiotelescopesthatcatalogueandmonitorobjectsinspacewhoseorbitcouldbringthemintocollisionwithourplanetwascut,weturnedourselvesintodinosaurs,voluntarilyblindtothethreatstoourspecies.Microsoftco-founderPaulAllenhadtostepin,usinghisownresourcesto2inancethere-activationandmanagementofthesespe-cialized tools, andgiveusa chance to catalogue theasteroidsandperhapspreventa futureimpactfromwipingusout.

Generallyspeaking, itseemsfairtosaytherearenoalternativestothetoolsofreasonandsciencetodealwithourproblems.Evenwhenweputthemtothebestpossibleuse,wehavenoguarantees.Insurmountableproblemsexist.Butifwewanttohaveachanceofresolv-ingthem,wesimplyhavetoidentifythemostappropriatescienti2icapproach,themostsuit-able type of solution (for example, a gravitational solution rather than a Hollywood-stylebomb).Buryingourheadsinthesandisnottheanswer.

�8


Anon-zerosumgame

Thereisacommonbeliefinthepress,radioandtelevision,butalsoamongonlineme-dia,websitesandthesocialnetworks,thattheonlyseriouswayofprovidinginformationistopresentthefactsinabalancedmanner.Takentoanalmostabsurdextreme,thisapproachseestwosidestoeveryfactorphenomenon,onepositive,theothernegative,andtriestogivethesameamountofcolumnspaceorairtimetoboth.Whetherthisdogmaticmethodstemssimplyfromignoranceorwhethervestedinterestsarenotinfrequentlyatplayishardtosay.Amongscientists,99%areconvincedthattheclimateischangingandthatmanisthecause.Youoftenseetelevisionprogramswhereanalmostcomicattemptismadetogiveequaltimetothear-gumentthatthisisnotthecase(thatclimatechangedoesnotexistorisnotcausedbyman).Byinvitingascientistononeside,andaclimatechangedenierontheother,theseprogramscreatethearti2icialandfalseimpressionthatbothpointsofviewareequallyvalid.Basedonthistypeofapproach,itiseasytomakethe2lawedgeneralizationthattherearetwosidestoeverytechnology,onepositiveandonenegative.

Technology,however,isnotazero-sumgame,butapositive-sumgame.Ratherthanbal-anceout,itspositiveandnegativesideshaveapositiveneteffect.Thisvaluationisstatistical,not absolute: the various facets of each technology need to be carefully examined. After anopendebateinwhicheveryonetakespart,wemaydecidenottoadoptcertaintechnologies,toavoidbecomingdependentuponthem.Generallyspeaking,however,thesimplefactthattheworldpopulationismorethansevenbillionratherthanafewmilliondemonstratesthattheoverall effect is bene2icial to humanity. Peoplewho saywe should abandon technology, be-cause they fail tounderstand itand therefore fear it, should 2irstofall answer thequestion“Whoarethe99peopleoutof100whowillhavetodieasaresult?”

Theprecautionaryprinciplethathighlightstherisksoftechnologiesslowstheiruptake.Questions relating to consumer protection are often cited with regard to their use. Thepremise here is that on one side, the consumer is defenseless and unable to defend them-selves,ontheotherthattheconsumerisunderattack,readytobeexploitedandtricked,andgenerallyopposed to theactionof thosewhoproposesolutions to theirproblems.The 2inalstepinthisreasoningimpliesthattheregulatorybodiesarebetterinformed,betterpreparedandbetter able todecide for thebest onbehalf of the consumer, indecidingwhat the con-sumermayormaynotbeabletodooreventoknow.

�9


AccesstothesacredtextofyourDNA

A recent example of this precautionary principle atwork is the opposition of the USFood&DrugAdministration, thefederalagencythatregulatesfoodandhealthcareproductsandservices,totheDNAdecodingserviceofferedtothepublicbythe23andMecompany.

Takingadvantageoftheexponentialreductionindecodingcosts,in2009thisCaliforn-iancompanybeganofferingthepublicaninnovativeservice.Whenyouregisterontheirsite,theysendyouatest-tube:youdepositasampleofyoursalivainthetube(putbluntly,youspitintoit)andmailitbacktothecompanyintheenvelopeyoureceivedintheoriginalpackage,prominentlymarkedwith the threatening “biohazard” icons forbiologicalmaterials.After acoupleofweeks,theysendyouanemailmessageinformingyouthatthedecodingprocedurehasbeencompleted.Youcanthenreadtheresults,protectedbyapassword,directlyonthecompany’swebsite.

In1985,anextremelyambitiousHumanGenomeProjectwaslaunchedintheUSAwitha budget of three billion dollars. The goalwas to decode the human genome, consisting ofthreebillionbasepairs,in2ifteenyears.Theprogressoftheprojectisaclassicexampleofthepowerofexponentialchange.Sevenyearsin,onlyonepercentofprogresshadbeenachieved.Eventheexpertsinthe2ieldthoughttheprojecthadfailedandthatitwouldtakenotanothersevenyears, butdozens if nothundredsof years to complete, and involve astronomic costsmany times higher than the funds originally provided. Very fewpeople considered that theonepercentresulthadbeenachievedthroughadoublingofthedecodingcapability.Bymain-tainingthisrhythm,notthespeed,buttheaccelerationoftheprocess,bythefollowingyear,2%progresshadbeenachieved,then4%,8%,16%,32%,64%…andafterexactlysevenaddi-tionaldoublings,the100%objectivehadbeenachievedontime–2ifteenyears–andonbud-get.

Thedoublingofthepoweroftheprojectdidn’tstopthere,however;thespeedofdecod-ing increased,bringingdown its cost.Today, in2015, it ispossible tohavea completeDNApro2ile for about $2000, or for $99apartial pro2ile, of the so-calledSingleNucleotidePoly-morphisms(orSNIPs)believedtoberesponsiblefortheindividualcharacteristicsthatdistin-guish us fromone another. There are 2ive hundred thousand SNIPs in humanDNA and the23andMecompanyconcentratesontheiranalysisandprocessing.

�10


Theresultsareastonishing.ByanalyzingmyDNA, thepeopleat23andMecan tellmethecolorofmyeyesandhair,aswellasdozensofothercharacteristicsofmyphenotype;thatis,thephysicalmanifestationoftheactionofmyDNA.Theycanmakeastatisticalassessment,intermsofgreaterorlesserpredispositiontowardspeci2ictypesofillness.Theyprovideindi-cationsondosagesevenforcommondrugsifIshouldhavetousethem,inadditiontotherec-ommendationsofnormalprescriptions,or,conversely,advisea lowerdosegivenmynaturalreactiontothecompoundscontainedinthemedicine.

Clearly,beingabletotakedecisionsonthebasisofthisinformationcanhaveimportantconsequences.Knowing that speci2ic changes inmy lifestyle can reduce theprobabilityof aparticularconditionaffectingmecanbelife-changing.TellingmydoctorthatIhaveaparticu-larsensitivitytoadrughe isabouttoprescribesohecanadjustthedoseaccordinglycouldsavemylife.

TheFDAhasdecidedthattheway23andMepresentstheinformation,makingtheprob-abilistic and statistical correlations between the genome, behavior and the development ofparticularconditionsanddiseasesexplicit,wasnotappropriate.Speci2ically,ithasruledthatconsumers shouldnothavedirectaccess to this information,whichshouldbegivenonly tophysicianswhowould thereforebe theonlypeople to interpret thedata andgive advice totheirpatientsbasedontheirconclusions.

In1517,MartinLuthernailedhisNinety-FiveThesesonthedoorofAllSaints’ChurchinWittenberg.HissymbolicactwasthetriggerforthedevelopmentoftheProtestantmovementin the Christian church, leading to a schism with Catholicism that continues today. One ofLuther’s theseswas that theBible, thesacred textofChristianity, shouldbe translated fromLatin into German to enable people to read it themselves, without the intermediation of apriest.Thepreexistinginterests,aswellasaconservativeanddogmaticjusti2ication,createdanirresolvablecon2lictthatlednotonlytotheschisminthechurch,butalsotohundredsofyearsofbloodycon2lict.

TodaytheFDAistakingtheroleplayedbytheVaticaninLuther’sday.ItdoesnotwantpeopletohaveaccesstothesacredtextoftheirDNA,translatedfromthelanguageofbiochem-istryintotheaccessiblelanguageofinformationtechnology,andhasruledthatinterpretation

�11


ofthetextmayonlybethroughthepriestlyintermediationofmedicalpractitioners,whoup-holdtheirpositionfromaconservativeanddogmaticbasis.

Theproac=onaryprinciple

Ouractionsshapethefuture.Theconsequencesofourhopesandambitionsextendbe-yondthepresent.

Theprecautionaryprincipledictatesthatbeforeagivensolutionisadopted,itisneces-sarytotake intoconsiderationall theharmthat itcancause.Often invoked inareasofcon-sumerprotection,regulators feelempoweredby it tomakesurethatnewproductsandser-vicesthatarebroughttomarketarenotonlyusefulandhavepositiveeffects,butthatnegativeeffectscanbeexcluded.Especiallyinthe2ieldsofhealthandpharmaceuticalresearch,thecau-tionaryprinciplehasbeenthepreeminentinspirationofproductdevelopment.

Ofcourse inan idealworldregulatorswouldonlyseek toestablish thebestcourseofaction for thepublicgood, rather than implementingself-propagatingbureaucracies.And inan idealworlddominantplayers inagivenmarketwouldnotuse theirpower to stopnew-comers anddiminish the dangers of unknown competitive factors, distorting the rules, andundulyin2luencingtheprocess.Yes,wearenotlivinginanidealworld.

Theproactionaryprinciple,originallyproposedbyMaxMore,takesintoaccounttheop-portunitycostofinaction,andthecostsofregulationitself,inderivingabalancethatismorefutureoriented.Ifwelookatthefuturegenerations,andthebene2ittheyderivefromourac-tions,delayingtheimplementationofanewtechnologycanhaveverylargeconsequences.

Thefreedomtoexperiment,andtheopportunityforindividualstogainknowledgeout-side of the of2icially sanctioned paths of research, objectivity, and transparency, and othercomponentsoftheproactionaryprinciplemakeitausefultooltodesignnovelaction.

A family residing in theUnitedStateshasbeenaf2lictedbya rarecondition forwhichtherewasn’tacommerciallyavailablecure.Thepharmaceuticalcompanythatactuallystartedinitialresearchonitevaluatedthatgoingthroughtheregulatoryobstaclecoursetobringittomarketwasnotworthit.Ofcourseforthosedirectlyimpactedbytheillnessitisnotaques-

�12


tionofpro2itconsiderations.Thankstomoderncommunications,theavailabilityofresearch,thefamilyhasbeenabletoconnectwithothersintheirsamesituation,topurchasetherightstofurtherdevelopthecure,andtosuccessfullyapplyittotheirownmembersandtothoseofotherfamiliessimilarlyaf2licted.

Basedonthetraditionalapproaches,thiscouldneverhavehappened,bothfromatech-nologicalandregulatorypointofview.Therearecountlessotherexamplesofbolderexperi-mentationandfreeinquirythatawaittheapplicationoftheproactionaryprincipletoachievetheirgoalsand2lourish.

TheEuropeanUnion,basedonenvironmentalconcerns,incorporatedtheprecautionaryprincipleinitsfundamentaltreaties.Is itgoingtoimplythattheEUismorelikelytorefrainfromadoptingtechnologiesthanothersocio-economicareas?IsthisgoingtobethebasisofacertainleveloffossilizationofEuropeansociety?Ormaybeitistheexpressionofthissituationwhichalreadyaf2lictsit?

Fitorunfitciviliza=ons

Is freedomanemergentpropertyof self-organizingmatter?Westronglybelieve tobeendowedoffreewill,andmostofoursocialstructuresarebasedonthis.Thereisnophysicalfoundation to it.Thedeterminismofphysical laws,quantumuncertaintiesnotwithstanding,allowsnospacefortheconcepttohideandshowitseffects.Justasweareconstantlymovedtoanthropomorphizing objects, animals, and phenomena,we are compelled to interpret deci-sionsasmadefreely insteadofbeingtheconsequenceof thestateofmatterandits interac-tions,insideandoutsideofus.

Individualbehavioraggregatestothatoflargergroups,and2inallyofsocieties.Wejudgetheoutcomesofindividualdecisionsandconsequencesincivilandcriminallaw.Wecanjudgethecapacityofsocietiestoengenderthewell-beingofitsmembers,or,onthecontrary,tobecorrupt,unjust,andspreadingconfusion,violenceandsuffering.

Thecapacityofagroupofsocietiestogeneratewell-beingdoesn’tonlydependontheaggregatedecisionsofitsmembers.Italsodependsonwhatknowledgeisactuallyavailabletothem,andthroughthemtoit.TheancientRomancivilizationcreatedwonderfulartandphi-

�13


losophy,andwerightlyadmireitsachievements.However,itwasorganizedatafundamentallevelonslavelabor,whichtodayisuniversallycondemned.Coulditbedifferent?Isitpossibleto imagine a Roman civilization that didn’t employ slavery? It is not, because the level ofknowledge,andespeciallytheenergyavailabilitythatthatknowledgegenerated,madeitim-possibletoachieveitsgoalswithoutresortingtotheforceofhumanmuscle,orprogrammablehumans,peopleyoucouldtellwhattodo,andwho’ddoitwithouttalkingback.

During its period of expansion,while able to draw on successfulwaves of slaves, theRomancivilizationappearedtobewelladapted.Itwasonlyanappearance,becauseitcouldnot last.Romecouldnotfurtherexpand,havingbasicallyconqueredall theavailable landofwhatwecalltodayEurope,NorthAfricaandtheMiddleEast,andenslavedalltheindividualsthatcouldbeenslavedinthosepopulations.Atthatpointitwentintodeclineandbecamein-capableofresistingthechangescoming,oradaptingtothem.This isofcourseanextremelysimplisticrepresentationofa longandcomplexhistory.Therearemanyotherforcesatplaybeyondthatofslavesorthelackofslaves.

Today,withcurrentknowledge,wecanbuildsocietiesandcivilizationswithoutslaves.Relyingon chemical and soon solar energy, ourdecisions aredrivenby theirmoreef2icientuse, and theyoutcompete the eventual alternatives.Wearenotmorally superior to theRo-mans as individual humanbeings,we are taking advantage of the accumulated informationanditsapplications.Theoutcomeof theUSCivilWarbetweentheSouthandtheNorthwasdictatedbyeconomicef2iciencyandabetterorganizationof energyand industrialbasesbytheNorth.

Our current civilization is, as a consequence, the expression of our knowledge. Thetechnologieswehaveavailableshapeit,similarlytohowRomancivilizationwasshapedbytheknowledgeandthetechnologyavailableat thetime.Wecanstartaskingourselveswhat thelimitsofadaptabilityofourcivilizationare,andhowitwillchangewiththeaccumulationofinformation,itsapplicationinnewknowledgeandnewtechnologies.IfyouaskedaRomantotellyouifitwerepossibletobuildacivilizationwithoutslaves,theanswerwouldhavebeen“No!”.Whatarethefalseaxiomsthatweareholding?Whatarethequestionsthatwecanaskandassumethattheanswerswouldbeuniversal,witheverybody2irmlybelievingthatagivenassumptionisanecessarypartofoursocietiesinanyplaceandanytime?Withtheknowledge

�14


availabletousinthefuture,we’llappearasprimitiveandnaivewiththatsecureandfalsean-swerastheRomansappeartoustoday.

Whentheshifthappens,andhowitmanifestsitself,dependsonthetensionsthatbuildupbetweensocietiesinagivenera.Whatispossibleinoneplacemaynotbeimmediatelypos-sible inanotherplace.Differencesbuildupasa consequence, sinceknowledgegetsappliedand experience accrues. In aworldof global communication as today, theunderstandingofthesedifferencesbrings to thepossibility of applyingknowledge faster, adoptingbest prac-tices,whatworkswell, andavoidingmistakes.Whencommunicationbarriersor ideologicalonesputobstaclesinthepathofthis2lowofinformation,thedivergenceofsocietiesincreases.The tensionsaccumulate,andunderanapparent immobility, theorganizational structureofsocietyisunderincreasingstress.Atthatpointasmallchangeintheboundaryconditionscanbringtoverybigcorechanges,ripplingthroughtheentiresociety.

This is what literally happened with the Berlin Wall, which was metaphorically andphysicallyshieldingtheplannedeconomiesoftheUSSRandofEasternEuropefromthoseoftheWest.When theWall fell, the effectsof allowingmarket economies to rapidlypenetratebrought2irsteconomicalandthenswiftpoliticalchangewhichcouldnotbecontainedorcon-trolledevenbytheverypeoplewhoinitiatedandallowedthem,likethenSecretaryGeneraloftheRussianCommunistParty,MikhailGorbachev.

Since the informationdifferences create these areasof limitedknowledge, individualswithinthoseareasoftendon’tevenrealizethattheyarelivinginamaladaptedsociety.Theycanbetakenbysurprisewhentheweaknessesandthebrittlenessofthecivilizationaremadeevidentbytheabruptchanges.Evenpoliticalexpertsandhistoriansarebetteratexplainingrapidcivilizationalchangesafterthefactthanforecastingthem.Thismakesithardtoprepareforthechanges,andtoreducetheamountofsufferingthattheperiodofuncertaintycreates.

Unsustainabilityisunsustainable

The current capitalistic economic paradigm predicated on constant growth has beendominatingforthepast200years.Evenbefore,resourceexplorationandallocationinfeudalsocietieshasallowedignoringwhattodaywecallexternalitiesofeconomicactivities.Thiswas

�15


possiblewhileagivennationorcivilizationdidnotcareaboutdestroyingacompetingone.Orundertheassumptionthatdepletingtheecosystemsandexterminatingdominantspeciesofagivencontinenttherewouldalwaysbeanothertodiscoverandtostarttheprocessoverwith.

Todayitisevidentthatthisbehaviorisnotpossibleanymore.Advancednationsshouldnot wagewar against each other. They should not destroy or subjugate other populations.They should not, endanger ecosystems and their species through their economic activities.Verysimplytherearenonewcontinentsavailabletopillage.

Thisimplicitlymeansthatwearenotbetterpeoplethanbefore.Wedidnotchangeourwaysbecauseweunderstoodthattherewasamorallysuperiorbehaviortobeadopted.Ourmentality is fundamentally thesame.Thereasonwearenowconsideringalternatives isbe-causetheoldwayisnotpossibleanymore.

Theexternalitiesofoureconomicactivitiesareallthoseconsequencesthatarenotre-2lectedinthepro2itandlossconsiderations.It isuptosocietytolookatvarioussectorsanddecideifitcanallowthis.Alternatively,itcanputinplaceregulationsthatsurfacethehiddencosts,andletsocietyasawholeexplicitlyabsorbthem,orforcetheenterprisesactiveinthechainofproductiontodealwiththeminstead.

Unsustainableeconomicpracticeshave largeexternalitiesand inaclosedandgloballyconnectedworldcannnotbeallowed.Overcomingthedepletionofecologicalsupportsystems,thewasteofresourcesthatcouldbeusedmoreef2icientlyorrecycled,acomplexsocietymustturntowardssustainablepracticesinordertogeneratedynamic,butrobustsolutions.

Acivilizationcannotbewell-adaptedwithoutrecognizingthisneed,andwithoutactingonitwiththeappropriatetoolsofincentivesandregulations.

Themethodsofknowledge

Inordertosurvive,wemustobserveourenvironment,trytounderstandit,acquirere-sourcesweneed,andplanactionsinordertoachieveourgoals.Knowingwhataretherulesoftheworld,systematizingtheknowledgeandunderstandinghowwecanknowbetterisusefultobebetteratsurviving.

�16


Thefatalerrorofthealchemists

In aworld that is dominated by competition and the universal perception is that re-sourcesarescarce,itisnaturaltochooseastrategyofsecrecy.Gatheringknowledgeinsecretgivesanadvantagetothosewhocanexploititagainstothers.Aclosedandwell-guardedsys-temofknowledgeisabarrierthatothershavetoovercomeiftheywanttoparticipateatthesamelevel.

Atthesametime,aclosedandsecretsystemisalsovulnerabletobeisolatedandsuffer-ingitsownmistakesinisolation.Notbeingabletoshareitslearnings,acommunityrelyingonsecrecyisboundtoberepeatingmistakesbecausetheirlessonscannotbeshared.

Medievalalchemistswhowereobsessedbythegoaloftransformingleadintogoldwereunluckyenoughtobelievethatusingmercurywouldhelpinthisquest.Unfortunatelymercuryispoisonous.Anyalchemistwholearnedthisfactdiditontheirown,andsufferedtheconse-quences of this knowledge. The organization in secret societies blocked the possibility oflearningfromeachother’smistakes,andthealchemistswereboundtorepeatthem.

Intoday’sworldstill there isa lotofactivitythat isconductedsimilarly insecret. It isassumedthatsharingitwouldweakenthepositionofthosewhocompeteforresources.

Openscience

Thescienti2icrevolutionthatGalileistartednotonlyrepresentedaclearerunderstand-ingofhowtheoryandexperimentsneededtoberelated,italsopavedthewayforaprofoundshiftinhowknowledgeandinformationwascollected,checked,andspread.Withoutitbeinganecessarypartofthescienti2icmethoditself,opencollaborationallowedgroupsandindividu-alswhotookadvantageofittomorerapidlydecideifacertainsetofresultswasreliable.

Openscience is fundamentally superior to closedapproachesofknowledgegathering.Collaborationamongpeoplewhosharethegoalsandpassionsofagiven2ieldisenhancedbyacommon language and tools. Publishing the results that one group achieves allows anothergroup to run experiments against the new knowledge to con2irm or disprove them. In-

�17


terdisciplinarycollaborationisenhancedbytheeasewithwhichpractitionersoutsideofthegroupcanapproachit,buildingbridgesofunderstandingthatovercomespecializations.

Today a fertile 2ield of studymadepossible by the increasingdigitalizationof scienceandscienti2icpublishingisthatofmeta-studies.Comparingandanalyzingalargenumberofpublications inagiven 2ield, it ispossible toderiveresultsnot individuallycontained inanysingle one of them. Statistical tools reveal signi2icant trends, and enhance the possibility ofcatchingandeventuallycorrectingmethodologicalerrorsintheworkpreviouslypublished.

Theworldof sciencevalues facts, theories, veri2iability, experiments, and thepublica-tionofresultsforsharingknowledge.Asoneofthemainproductsofthescientist,ifevennotthesoleoneonwhichtheiradvancement,capabilitytoreceivegrants,andprofessorshiparedecided,scienti2icpublicationsthroughthepeerreviewsystemhavebeenatthecenterofthedevelopmentofscience.

Thevalueofthescienti2icarticleismeasuredthroughthenumberofotherpublicationscitingit,andtheimportanceofthejournalinwhichitappears.Thishasbeenconverteddirect-ly in economicvalueby thepublishersof the scienti2ic journals,whichhave created sizableenterprises through charging universities and research institutions for the subscriptions totheirperiodicalsinwhichthearticlesofthescientistsappear.Thepriceofthesesubscriptionshas increasedenoughthatuniversities in lowandmiddle incomecountriesorevensomeinhighincomeonesareunabletoaffordthesubscriptionprice.Thereisalsoamorefundamen-tal issue that publicly funded research and its results are subsidizing the revenues and thebusinessmodelsofprivatecorporationswhoarecollectingthearticlesandendupbeingpaidtwiceover.

Openaccesspublicationshavestartedtoappear,andgainpopularityandprestige,offer-ingalternativemodelstoscienti2icpublication,wherethereaderofthearticlesinnotchargedeitherforsingleaccessorforasubscriptiontothejournal.Withthepeerreviewsystemstillinplaceforthequalitycontrolofthearticle,theauthorofthearticleischarged,andtheirinstitu-tion, forthepublication,a feethat isaffordableandreasonabletobeaddedasasimple lineitemtothebudgetofanexperimentorthegrantrequest.

�18


With characteristic ruthlessness, science is also re-evaluating the effectiveness of thepeer reviewsystem itself, trying tomeasure it ononehand, andon theother searching forpossible alternative ways of assuring high levels of quality for scienti2ic publications. Thestructureofscienti2icexperimentsisbeinganalyzedinordertomaximizetheprobabilitythatthepublishedresultscanbedoublecheckedthroughindependentreproductionandveri2ica-tionofthedata.

Making not only the scienti2ic article itself, but lab notes and underlying raw datastreams for furtherevaluationandaggregationhasbecome thenorm in several areas.Withlargerandlargerdatasetsbeinggeneratedincomputerscience,lifescienceandmanyother2ields,theanalysisofthedatahasbecomeanewburgeoningopportunitybyitself.

Bigdataanddatascienceoffer thepossibility toaggregateandsublimatevalueoutoflargesetsofstructuredorunstructuredinformation.Theirapplicationisbecomingbetterun-derstoodingenetics,theInternetofThings,sociologyandother2ields.Citiesandgovernmentscanononehandmakethedatastreamstheygenerateavailableopenlyforotherstoleverage,inapermissionless,unencumberedmannerwhereinnovationandcreativitycanstrive,andontheotherhandtheycantakeadvantageoftheresultstoincreasetransparency,accountabilityandef2iciencyoftheiroperations.

Theevolu=onofscience

With theavailabilityof increasingamountsofdata, and the interconnectednessof theworldanditsexperts,itispossibletobecreativeaboutthewaythatthepiecesofthescienti2icpuzzleareputtogether.

In the traditional process of undergraduate, graduate, doctorate and post-doctoratestudies,increasinglevelsofspecializationcharacterizethework,leadingalmostuniversallytodecreasing accessibility. An alternative to this depth-2irst process is the less developed andorganizationally complex breadth-2irst approach where interdisciplinary collaboration andthe cross-fertilization of various 2ields are receiving the attention for generating innovativeresults.

�19


In2ieldssuchascosmology,whereweonlyhaveoneuniverseavailable,andthesetupoftheexperimentsisnotunderourcontrol,itispossibletoobserveextremelyhigh-energyphe-nomenathathavedeepconnectionstothetheoriesofparticlephysicsthatwouldhavepossi-bly neededmachineswith energies thatmade them too large and expensive to be built onearth.

Comparativestudiesthatleveragethecapabilityofopenaccesspublications,andoftheelaborationof big data streams andof theunstructured text of the articles themselves, canhighlight statistical con2irmations or anomalies in studies through the years, across severalresearchinstitutionsandtheworkofmanyscientists.Thismetaknowledgecanleadtovalu-ableunderstandingofreproducibility,effectiveness,andpromisingareasofresearchinordertobetterallocateattentionandresources.

There is an increasing epistemological attention to the structure of theories, makingsure that theexcessivegenerativepowerof certaingroupsof theories is rightlyquestioned.Stringtheory,afamilyoftheoriesofparticlephysics,isabletopulloutofitsepistemologicalhatatheorythatcorrespondstoanyexperimentalresult,giventhatitencompasses10tothe500theoriesapproximately(billionsofbillionsofbillions…of theories)withaquestionableapplicationofthesequenceoftheory,prediction,andveri2ication.

Thehumanelementofthestructureofscienti2ictheoriesandthewaytheyevolveisbe-ingunderstoodmoredeeply.Thecapacityofembracingnew,riskyareasofenquiryisnatural-lymore likelyat thebeginningof thecareerofa scientist.The increasing lifespanofpresti-giousleadersofacademic2ieldsmustgohand-in-handwithascrutinyabouthowtheyretaintheirnimblenessandrisk-takingastheresourcesareallocatedtovariedapproaches,andtogivevoicetonewentriesandnewideasintheirrespective2ields.

Exponen=alchange

Dynamicsystems

�20


All phenomena are dynamic andwhilewe tend to try to analyze and formalize themthroughanoppositeprocess, it is thestaticabstraction that is the further representationofthereality.Ofcourse,asthestrugglesintheevolutionofthemethodsofsciencehaveshown,realityisnotalwaysreadilydecipherable.Noritisconvenientlylaidoutinfrontofusinintu-itivepackagesconformingtoourcommonsense.Manyofourintuitionsabouttherulesgov-erningnaturalphenomenaturnouttobewrong.Thiscanhappenwithaseriesofexperimentsthat,once thephysical lawsbehind thephenomenaarewellunderstood,anybodycancarryout.Atthatpoint,withaccessibleexplanationsthatcanbeillustratedwithimmediacy,thereisnoexcuseforbeingignorantaboutthenatureofreality.

AsimpleexampleofthisisNewton’s2irstlawthatsaysthat“allobjectsmaintaintheirstateofmotioninabsenceofanexternalforce”.Oureverydayexperienceisthatacarputinmotionwillactuallystop,ifyoudon’tpushonthegaspedal.Butnowwehaveaclearunder-standingoftheroleofattrition,andthatthedecelerationisduetotheengine,theterrain,andtheairinfrontofthecar.Takingawayallsourcesofattrition,thecarwillgoonforever.

Theconsequencesofdynamicchangeareallaroundus,intheebbsand2lowsofwater,rivers, oceans, and rain. In the growth of vegetation, trees, and forests, or the advancing ofdesertsand the changesof seasons.Buteven ifwehaveplentyof experiencewithdynamicchange,ourintuitioncanbemoremisleadingthanever,withrespectoftherawpowerbehinditsabstractmathematicalnature,unencumberedandunrestrainedbytheconstraintsofanat-ural,physicalenvironment.

Exponentialchangeissuchadynamicenvironment.Wecanprepareforit,butitsbluntforce will still surprise us, often confounding even experts and certainly taking laypersonsabackbyhowpowerfullyitcanreshapethelandscapeofourreality.

Meaningfulsequences

Thesimplestexamplesofexponentialchange,thedoublingofaquantity,forexample,inacertainamountoftime,ifthestartingpointistheunitof1,canlookfairlyharmlessorevendisappointingatthebeginning.

�21


1,2,4,8,16,32,64,128,256,512,1024,2048,4096,…isasequencefamiliartoany-bodywhohashadanyinterestatallinnumbers.Rattlingofthesequenceinyourheadcouldhavebeenaharmlesschildhoodexercise.

There is a corresponding sequence, before the unit, that you can look at, potentiallyunassumingtoevenagreaterdegree:0.01,0.02,0.04,0.08,0.16,0.32,0.64,andthen1.28.Theinteresting surprise of this sequence, nothing magical about it, is what came before:0.00015625,0.0003125,0.00625,0.00125,0.0025,0.005,0.01.

Whyare these threesequences interestingandwhyare theyrepresentativeof thena-tureofexponentialchange?

Imaginethatyouarelookingattheworldaroundyouand,tryingtodecipherit,predictwhatagivenphenomenonwillamountto,youcollectdataaboutit.Thiscollectionwillnotbeasneatandclearasthesequencesabove.Therewillbealotofnoiseinit.Errorsofmeasure,mistakesmade during the process or the planning of themeasurements, other phenomenaintrudingandconfoundingyourattemptstoaclearunderstanding,andsoon.

Thenoiseofanaturalenvironmentwhere,beforeevenbeingabletoclearlytrainyourearstoit,youwanttodiscernapatternthatispossiblynew,somethingthatnobodyelsetriedtolistentobefore.

Noisysignals

Is theresignal inthenoise?It isvery likelythatwhileyoutrytoanswerthequestion,therewillbeotheropinionsaround.Andperde2inition,theywillbedifferentfromyours,notreallyaligned,orevenmaybeinopposition.Ifyouareinaresearchenvironmentandarecom-petingforgrants,oryouareinanindustryandtheproductthatyouaretryingtoengineerortheservicethatyouarepromotingamongusersrightlydistractedbyahugeofferingofalter-nativeoptions,inanycaseyou’llbeconfusedbytheresistancetoyouroriginaltheorywhichthesignalsmaysupport.Youneedtobestronginyouropinions,youevenhavetohavefaithinwhatyouwanttoshow,anunreasonableconvictionthatyouarerightwhileeverybodyelseistellingyouthatyouarewrong.Oreventhatwhatyouare lookingfor isnon-existent,or im-possible.

�22


Thisistherealmofthethirdsequence,leadingupto0.01(or1%oftheunit).Theareawhereevenexpertswillbeagainstyou.Itneedsakeeneyeandear,concretelyorabstractly,tounderstandthat in thepresenceof thedistractionsofanoisynaturalworld, there is indeedsomethingbrewing.Doublingcalmly,withoutanybodyelsenoticingitbutyou,andaftersev-eraldoublingsarrivingatthethresholdof1%fromthegoaloftheunit.

Afteryouarrivetothe1%,thoseexpertswhostilldon’tbelieveyoushouldbestrippedoftheirlabelignominiously.Becausefromthenonwardsitshouldbeclearnotonlytoyoubuttoanybodywhopaysevenapassingattentionthat it isnowjustamatteroftime.Inmerelysevensuccessivedoublingsyouwillhavearrivedattheunit.

TheHumanGenomeProject’sexponen=algrowth

Turning this description fromabstract sequences of numbers into a real example,wecanlookatwhathappenedinthemassiveHumanGenomeProjectintheUS.Startedin1985,thedurationoftheprojectwasoriginallysetfor15years.Aswithanyscienti2icproject,itwasnotcompletelyclearhowall thehurdleswouldbesolved,andwhatapproachwouldbe thewinningone.Aftersevenyearsintotheprojectonly1%ofthegoalhadbeenachieved!Manyatthetimewereloudlydemandingthattheprojectshouldbesuspendedorevenabandoned:look,alreadyhalfwaythroughanditisonlyat1%totheobjective!Thosemorecareful,ortheexpertsinexponentialdynamics,though,wereabletoseethatallwasgood.Havingreached1%doublingtheamountofbasepairsdecodedeachyearintheprevioussevenyears,inaddi-tionalsevenyearsofdoublingstheprojectwouldreachitsgoalof100%,ofdecodingtheen-tirehumangenome.

There aremany phenomena that are subject to exponential growth: populations andnuclear chain reactions, to name a couple of examples. Populations grow exponentially be-cause,aslongasonaverageacouplehasmorethantwooffspringgenerationaftergeneration,theincreasewillbecumulative:thoseoffspringwillhavemorechildrentoo.Nuclearchainre-actionsalsooccurwhenthe2issilematerial,uraniumforexample,producesamongits2issionproducts neutrons that before exiting the volume ofmaterial make another uranium atombreakapart,producingotherneutrons,andsoon.

�23


Most importantlyforthethemeofthisbook,thepowerofcomputingandinformationsystemsisalsogrowingexponentially,andhasbeendoingsoforover50years.However,thereisnonaturallawbehindthisdynamic,nobiologicalorphysicalnecessity.Itisanengineeringprojectthatendedupbeingcalled,bythenameofthepersonwho2irstformulatedit,Moore’sLaw.

Moore’sLaw

GordonMoorewasworkingonthenewlyinventedintegratedcircuit,atthebeginningofthe‘60s.Hewasinanoisyenvironment,intermsofonehavingtoconcentrateonthefeaturesofanovelphenomenoninpresenceofmanyothersgoingonatthesametime.Practicalcom-putershavebeenaroundforacoupleofdecades,moreorless,becomingmoreandmorepow-erful,butataratethatwasratherslow,iflookedatlinearly.

Differentapproacheshavebeentriedtomakethemcapableofstoringmoreinformationforthecalculations,andexecutingthemmorerapidly.Thevacuumtubes,magneticcoremem-ories, and other components ofwhat at the time themedia liked to call “electronic brains”werecumbersome,pronetohighfailurerates,andneededscoresofspecializedpersonneltotakecareofthem,tomakesurethattheywouldwork.Thecostofcomputerswasinthemil-lionsofdollars,andonlynationalresearchprograms,orverylargecorporations,couldaffordthem.

The invention of the transistor, to be used as a basic component for calculation,promisedmuchmorereliableandcheaperproduction,assembly,andrunningofcomputers.Transistorscouldbepackagedtogetherwithothercomponentstocreateausefulunitofcalcu-lation called the integrated circuit. Not only that, but given their nature, itwas possible toforecastthedevelopmentofnextgenerationcomponentsthatweresmaller,faster,andcheap-erthanpreviousones.

GordonMoorewasabletoobservewhatthecurrentcapabilitiesoftheproductionpro-cesseswere,andtheincreaseofthesecapabilitiesinthecourseofafewyears.Basedononlyahandfulofdatapoints,plottedonapieceofgraphpaperthatafter50yearsstillsurvives,heboldlyformulatedapredictionthatthenumberoftransistorsthatcouldbeaccommodatedon

�24


agiven integratedcircuitwoulddoubleeveryyear.Abit laterhecorrectedthepredictiontotwoyears,and then 2inally settledon18months,which is thevaluecurrentlyacceptedandused.

Based on how few data points were available to him this prediction was quite bold,maybeevenreckless.However,withthebene2itofhindsight,itwouldappearthatthiscoura-geousambitioniswhatwasreallyneeded.Becausewhathappenedisthat,spurredbycuriosi-ty,thedesiretoexcel,andbasiceconomiccompetition,moreandmoregroupsofengineerssetouttocreatemorepowerfulintegratedcircuits.Togetherwithallthesupportingsystemsthatwereneeded,theyweavedtogetheranentireindustry.Atthebeginningthisprocesswasdri-ven by the individual capabilities of these groups, andwhat theywere able to offer on themarket.However, lateron,Moore’sLawbecame itselfadriving force,akindofself-ful2illingprophecy aswell as a guidepost againstwhich tomeasure the achievements of the variousgroups.

Many times it has been predicted thatMoore’s Lawwould fail to hold up in the nextgeneration,andsoonerorlateritisboundtodosoinitsstrictestformulation.Moregenerallyextendingitspredictionstothepowerofcomputing,thereisreasontobelievethatitisgoingtobepossibletostillholditupforalongtime.Movingfromsilicontoothersubstratesforthecircuits;creatingthree-dimensionalcomponents;movingfromarchitecturesthatseequantumphenomena as a hindrance to ones that fully exploit them… there aremany approaches toovercomeeventual roadblocks that lieahead inproving this lawright, thesamewayothershavebeenovercomeinthepast2iftyyears.

ItisimportanttonotethatthespreadingofknowledgeisatthebasisofMoore’sLaw.Nosinglegroupworkinginsecretcouldhopetobetheonethatwillindeedbeabletosolvetheproblemsthatpopupalongtheroadinthenextgenerationofsolutions,ortheoneafterthatandsoon.Onlythecollaborationofmanygroupsmakesthispossible.Itisenoughforoneofthemtoachieveabreakthrough,discoveringthesolutionnecessary.Alltheotherswilllever-age that, through licensing agreements that incorporate the solution in the next generationfabrication plants churning out integrated circuits, that are today produced by the billionseachyear.

�25


Thecomplexinterlockingecosystemofindustrialinfrastructureneededtomaintainthepaceofevolutionincomputingisnotonly intheproductionoftheintegratedcircuitsthem-selves.Similarlyhavetoevolvethemanufacturingtoolsthatcreatethecircuits, thesoftwaresystemsthatallowtodesignthem,andthe2inancialsupporttomaketheinvestmentspossiblefortheplants,rawmaterials,re2inement,andveryimportantly,humancapital.

Whateverthefundamentalphysicallimitsforthegrowthofcomputationare,measuredbythegeneralizedMoore’sLaw,theyliefarinthefuture.Theprogressthatwe’veseeninthepast50yearsofincreaseinthepowerofcomputingisgoingtobevastlyeclipsedbythatofthenext50years.Actually, it isgoingtobeexceeded,bytheverynatureofexponentialgrowth,withinthenextcoupleofyears.Andthenagain,thenextcoupleofyears,andsoon.

Thepowerofdoublings

Itdoesn’tmatter,ofcourse,intermsofhowrapidlyanexponentialsequencedevelops.Noneedforittodoubleinayeartobeexponential.Thesearejustarbitraryunits,andanycu-mulativechangewhere theresultingquantity increasesbyagivenamountexpressed in theresultitselfwilldo.Ifyouhaveaquantityof100anditincreasesby10,you’llhave110,then120,130,etc.Thisislineargrowth.Butifyouhaveaquantitythatisincreasingby10%,thenyou’ll have110,121,133, and soon.That littledifference,whichdoesn’t appear tobeverysigni2icantatthestart,isallthatmatters.Thatisexponentialgrowth.

There aremanyways to express thispower, andhowsurprising it is for those accus-tomedtothinklinearly.

Look forexampleat thesumof this sequence:1,2,4.Thesum7=1+2+4 is the totalamountintheentiresequence.Andthenextstepinitis8,largerthanthetotalofallthestepsprecedingit.Thisistrueforallexponentialgrowth.Inthenextperiodofdoublingincomput-inginmerely18months,thankstoMoore’sLaw,therewillbemoretransistorsandintegratedcircuitscreated(andcomputersfromthemandcalculationscarriedoutthroughthem)thanintheentirehistoryofcomputingforthepast2iftyyearsormore!

Whenisittoolate?

�26


Anotherexampleillustratingthepowerofexponentialsistolookataclosedsystem,forexampleapondsustainingapopulationoffrogs.Iftherearealgaethatmakethelakeuninhab-itableforthefrogs,andasitcoversthesurfaceofthelakeinevergreaterextent,fromjustafraction,toonepercent,doublingeveryweek,howlongdothefrogshavetolive,oncetheal-gaecovershalfofthepond?Bynowhopefullytheanswerisclear:onlyoneweek,asduringthenextdoublingthe lakewillbeentirelycoveredbyalgae!Evenmorealarmingly,perhaps,alreadyat1%thereislessthantwomonths’timeleftforthefrogsto2leetoanotherpond,orto2indawaytostopthealgaefromexpanding.

Ouruniquepositionistobeabletoseewhatishappeningtothepond,contrarytofrogs.And this capabilityofdata collection, analysis, and foresight givesus great responsibility inunderstanding if thepond isOKornot.Takingactiveactionmanaging thepond,counteringthealgae,can’tandwon’tbedonebyothers,butwecandoit.

Thevariousexamplesofexponentialchangewe2indinnaturefeedthemselves,butsel-domassemble into interacting chains that feed eachother.Our technological civilizationontheotherhandisfulloftheseself-reinforcingchainsthatkeeptheaccelerationofexponentialchangegoing.

RayKurzweil’sproject

The inventor, author, and co-founder of Singularity University Ray Kurzweil has beencollectingdataaboutexponentialphenomenafordecades.Itisnotenough,infact,tobeabletorecognizewhatisgoingon.Theexplosivenatureofexponentialsissuchthattimingiscru-cialifyouwanttobeabletoridetheirwaveinsteadofbeingsweptasidebyit.

Jumponthewavetoosoon,andthosewhosayotherwisewillhaveaneasytimede2lat-ingyourenthusiasmorthatofyour2inancialbackers,becausetheupswingfromthesequencetrailingthehypotheticalunitinourexamplesequenceswillnothappen.Seeittoolate,anditwillalreadybeinfullpowerwhenyou’llwanttoclimbit,makingtheendeavortooexpensive,dif2icult,orevenimpossible,asotherswillhavecrowdedthecrestsalready.

From2latbedscannerstoopticalcharacterrecognition,frommusicalsynthesistospeechsynthesis,orhandheldsystemsfortheblind,allofKurzweil’sinventionstakeadvantageofa

�27


keenunderstandingoftherighttiming.Whentoaccelerateresearchanddevelopment,sothatbythetimesupportinghardwaresystemsareavailableattherightpriceandtherightlevelofintegration, all theother componentsof software,user interface,development systems, andtheentiresupportingecosystemarereadyaswell.

AttheSantaFeInstitutebasedonresearchbyBelaNagythereisafulldatabaseofexpo-nentialphenomenathatisaccessibletobestudiedandexpandeduponfurther.

Kurzweilalsorecognizedthatwiththeinterconnectedandintercommunicatingsystemsofhumanknowledgethatdonotgrowinisolationbutreinforceeachother,thereareexponen-tials feedingonexponentials.Hecalled the resultingeffect theLawofAcceleratingReturns.Thisiscontrarytotheacquiredwisdomofclassicaleconomicswhereitisassumedthatinor-dertoachieveagivenincreaseofeconomicofoutputthereneedstobeaprogressivelyhigheramountofavailableinputofcapital,calledtheLawofDiminishingReturns.

JustaswithMoore’sLaw,theLawofAcceleratingReturnsformulatedbyKurzweil isaself-ful2illingprophecy,sustainedbytheopencommunications,andcompetinggroupsaimingtoachievesuccessandexcellenceintheirresearchandindustrialproductionendeavors.Itisde2initelypossibletobreakeitherofthese laws. Ifyoustopbelievinginthe lawofuniversalgravitationandjumpoffthe2ifthstoryofabuilding,youcandoitathousandtimes,andyou’llneverstopdroppinglikeastone,mostprobablytoyourdeath.Butifweweretogiveuptryingtomakebettercircuits,or ifwedecidedthat itwasn’tworthoureffort tomakebettersolarpanels,betterbatteries,andsoon,as longaseverybodystopped, thosecircuits,panels,andbatterieswon’thappen.

Kurzweilis,asofthiswriting,aDirectorofEngineeringatGoogle,byhisownadmissionthe2irsteverjobhehashad.Usingtheresourcesmadeavailablebythecompany,heisapply-inghisskillstomakenaturallanguageinteractionwiththecomputerspossible,andthenextwaveofuser interaction,makingcomputerseveneasier touseandable tobetter serveourneeds.

Connec=ngS-curves

�28


AfrequentcriticismofKurzweil’sanalysisandpredictionsisbasedonamisunderstand-ingofwhatconstitutestheexponentialthatheistalkingabout.Thecriticshighlightthefactthatwhatappears tobeanexponential inreality is the 2irsthalfofanS-curveora logisticscurve.Itappearsexponentialinitially,asthebene2itsofagiventechnologyareexploited.How-ever,afterawhileitplateausoutbecauseitbecomesharderandhardertosqueezeadditionalbene2itsfromthesametechnology.Itbecomesexhausted,andthebeliefinthepoweroftech-nologyofthosewhopreachunendingexponentialsfalsi2ied.

Inrealitythis is true:eachindividualtechnologyas itruns itscourse isunabletogivemorethanitsnaturallimit.Asitapproximatesthatlimit,itbecomesfruitlesstoinsisttowanttogetmoreoutof it,bothfromthepointofviewofengineeringaswellasfromthatofeco-nomicsandofreturnon investment.Andthat iswhynewgroupswithnewideaswill trytoachieve thedesiredoutput throughadifferent approach. Smartpeoplewill see thepoint intimewhen thecurrentgenerationof technologiesgetsexhausted,andwork inparallelwiththegroupsleadingatthetime,to2indanewtechnologythatwilldelivertheobjectiveatscale,betterthanbefore.Thecycleinafewexponentialdoublingswillrepeatitself,andathirdgen-erationofsolutionswillbeneeded,andsoon.

The cumulativeeffectof thesedifferent S-curves, smoothingout eachother’s endings,andmore or less seamlessly interlocking in a chain of invention, innovation, and industrialdeployment,isdrawingtheexponentialthatKurzweilpointstoinhisanalyses.

Takingcomputing, forexample, therehavebeenmanygenerationsofcomputing tech-nologies,eachleadinginitsowntime,whichhavebeenpushedtotheirlimits,andsupersededby thenextone,better, cheaper,and faster togenerate thedesiredoutput: calculations.Me-chanicalrelays,vacuumtubes,transistors,andintegratedcircuitsdecadeafterdecadeenabledtheconstructionoftheworld’sfastestandmostpowerfulcomputers.Thecompaniesemploy-ingthemweretheleadersoftheirtime,pushingthelimitsofthetechnologies,andweresup-plantedbynewmodels,basedonthenewtechnologiesinafewdoublingsofperformance.

Anotherexamplewheretheseyearswearewitnessingafundamentalswitchisinper-manentmemorystorage.The largerandlargeramountsofdatathatourcomputersneedtopersistentlyrecord,sothatwhenelectricityisturnedoff,andthecomputerwakesuplateron,the data can be retrievedwithout having to start over from scratch. From punch cards, to

�29


magneticcorememory,tomagnetictape,tospinningharddisks,nowweareonthevergeofmovingstoragefornextgenerationneedstosolidstatesupport(2lashstorage),whichisgoingto be able tomemorize orders of magnitudemore, accessedmuch faster and reliably, andmoreaffordablythananygenerationofdevicepreviously.

Exponen=alseverywhere

Manytechnologiescanbeseenthroughthelensofthisexponentialinterpretationofac-celeratingchange.Thedoublingperiodscanbedifferent,ofcourse,thanthe18monthsthatMoore’sLawaccustomedustorelyon.

InsolarenergywetalkaboutSwanson’slaw,whichrepresentsthedecreaseinpriceperwattofaphotovoltaicpanel.Startingin1974withthecreationofthe2irstofsuchdevices,thatcostover$70perwatt, todayweareat$0.30perwatt,andthedecrease inpricecontinues.This decrease comes from economies of scale, frombetter understanding ofmanufacturingprocesses,andfromthebirthofanecosystemof 2inancing,deploymentandservicingof themodules, aswell as fromnewbasicapproachesofmaterialsandconstructionmethods thatincreasesubstantiallytheef2iciencyofagivenmoduleasittransformssunlightintoelectricity.

Thereisadoublingofthestoragecapacityofourbatteries.Thisdoublingisamorese-date(andinfuriatingifyoufeelyouspendtoomuchtimechargingyourvariouspower-hungrydevices)ten-yearperiod.Dependingonmetallurgy,chemistry,andmanufacturingprocesses,itisnotunimaginablethat, inanillustrationofKurzweil’sLawofAcceleratingReturns,thein-dustrywould2indawaytospeedupthedoublings,byadoptingaradicallynewapproachandmakingapplicationspracticalthatwouldhavebeenimpossiblebefore.

Nomagic100%

ThegoalsthatresearchprogramssuchastheHumanGenomeProjectsetthemselvesareoftensomewhatarbitrary.Theyrepresentausefulgoalpost,butnot theendof thedevelop-mentofprocesses, their re2inement, and certainlynot thedesire forknowledgeand for thecapacity toacquiring it fasterandcheaper.Afterdecoding thegenomeofasingle individual

�30


thereisthetaskofdoingthesameforanothersevenbillionofthem.Afterthehumangenomethereisthegenomeofotheranimals,orbacteriaintheoceans,orthebacteriathatsymbioti-callyliveonandinallofus,constitutingwhatiscalledourmicrobiome.

Thecapacityofdecoding thehumangenomedidnotstopat therateofoneper threebilliondollarsin2ifteenyears.Wecertainlywouldnothavetakenmuchadvantageofthat.Inthe2ifteenyearsfromthat2irstsuccess, thetechnologiesthathavebeeninvented,perfected,deployed and substituted by better ones again, allowed an astonishing progress: today it ispossible to decode an entire human genome for about two thousand dollars in a couple ofweeks.Butprogressisnotstoppingthereeither,anditispossibletoforecasttheavailabilitywithin thenext tenyearsof technologies thatwillenable thedecodingofagenome for lessthantencentsinafractionofasecond.Itisworththinkingaboutthetransformationsthatthiskindofchangeisgoingtobringintheworldofhealthcare,insurance,privacyandmore.

The takeaway is that there isnothingmagicalaboutagiven threshold thatweendupcallingaunitor100%,andthatthepowerofinventionandimplementationthatdrovetech-nologiestoachievethatdoesn’tstop,butkeepsgoingon,delivering increases inthedesiredoutput,atlowercosts,fasterspeeds.

Ar=ficialintelligence

TheNatureofIntelligence

The meat of this book could be a philosophical analysis of what is intelligence, andwhether it ispossible todescribe it inscienti2ic terms,or, rather, if it representsanessencethatisirreducibleandirreproducible.

Indeed,thishasbeentheoccupation,invariousforms,ofphilosophersforthousandsofyears.Includingthenatureoftruth,beauty,rightandwrong,morality,andethicsandesthet-ics,philosophyhadtherunofthe2ield,unencumberedbypracticalconsiderations.Tothecon-trary, the division of knowledge into two 2ields of abstract understanding and of thatwith

�31


practicalconsequences,withthissecondbeingfrowneduponandseenasinferiorbythefol-lowersofthe2irst,hasbeenamainstayofWesternphilosophysinceAristoteles.

Thereareseveralassumptionsinthisbook,andthefactthatintelligencecanbeunder-stood, analyzed, and reproduced is one of its fundamental ones. In more recent books byphilosopherswhodon’tdisdainbeingunderstoodbymany,therearewonderfulargumentstowhythisshouldbethecase.

In general, for the purpose of this bookwewill de2ine intelligence as the capacity ofmattertoorganizeinawaythatallowsittoseeksolutionstogoalsthroughplottingapathofactiontowardsthem,andorganizingbothabstractandconcreteresourcestoachievethem.

Thehumanbrain isa lumpofmatter that isendowedwithacertaindegreeof intelli-gence.AndonpurposeIusethatexpressioninsteadof“mind”.Wewillavoidthepitfallofdual-ismthathasboggeddownphilosophytryingtounderstandamindthatjustinhabitsthebrain,draggingDescartesintoargumentsabouthomunculiandfruitlesslysearchingfortheconnect-ingtissuethroughwhichthemindisattachedtothebrain.Underthisassumptionthebrainexpressesthemind,andthemindiswhatthebraindoes.

TheMechanicalTurk

Duringthe1700safascinatingcontraptiontouredthecourtsofEurope’skingdoms.De-signed by theHungarianWolfgang vonKempelen, itwas a large box on top ofwhich sat awoodendoll; todaywewouldcall itarobot. Itplayedchess,andbeatanybodyattemptingagamewithit,unerringly,withmechanicalprecision.VonKempelen’sclaimthathebuiltanau-tomatoncapableofactingwithintelligencewasunraveledwhenitbecameapparentthattheboxhousedamidget.Indeedgoodatchess,themidgetpretendedtobetheintelligencebehindtherobot.Itactuallywasthehomunculusofanarti2icialsetupformechanicalintelligence.Itconstitutedanarti2icialarti2icialintelligence.

Turingtestsforhumans

�32


When the 2irst electrical computerswere born, in the ’40s, Alan Turing formulated anovel test for the intelligenceofmachines.TheTuring test, as it is called today,asserts thatthere iseveryreasontobelievethatamachine is intelligent, if it isdeemedtobe intelligentthrough itsactionsbya teamofhuman judges.During the test thesetuphides themachinefromthehumans,andmixesitanditsoutputwithotherhumansthatduringthetestcanpre-tendtobemachineswantingthejudgestobelievethemorthehumanstheyare.Turingcalledit“theimitationgame”,probablybelievingittobelessfundamentalthanthefascinationwithwhichitisseennow.ToofrequentlyandsomewhatbombasticallyoneoranotherofthemediasourcesannouncesthattheTuringtesthasbeenpassed.Universally,whenthetranscriptsofthedialogtypedoutbetweenthewinningmachineanditshumanjudgesbecomesavailable,itappears that themachine,orrather itsprogrammers,hidbehindpretend jokes,changingofthesubject,andothertrivia.

Chessplayingexperts

In1996the thenWorldChampionofchess,GerryKasparov,wasbeatenbyamachinecalledDeepBlue,speciallydesignedandbuiltbyIBMforthispurpose.Themachinecouldde2-initelynotattempttopasstheTuringtest, in its imitationgamebasedonwrittendialogen-compassinggeneralsubjects.However,withitshardwareandsoftwarespecializedinanalyz-inganddiscardingmillionsofmoves inthetreeofpossibilitiesofthegameofchess,until itfoundwhatitdecidedwasthebestonetomakeinagivencon2iguration,itnonethelessfakedtobeintelligentinthegameperfectlyenoughtobeatandeveninfuriateKasparov.

If theTuringtest, the imitationgame,consistsofconvincingagroupofhumansthatamachinecanhaveahuman-likedialog, followingtherulesofsyntaxandsemantics,thentheKasparovtest,thechessgame,consistsofconvincingachessplayerthathelosttoamachinethatplaysfollowingtherulesofchess.Andrelevantlyenough,inthiscasethereisactuallynodifferencebetweenthemachinethatpretendstoknowchessandonethatdoesknowchess.

DuringthevariousstagesofthetournamentbetweenKasparovandDeepBlue,theen-gineersfromIBMtweakedthealgorithmsofthemachine,andtheRussianchampionstronglyprotested.Themachineshouldnotbeabletolearnduringitsplay?Thetweaksthatconstitut-

�33


ed an increase in its smarts, andpossibly contributed to itswinning,were admittedby thejudgesofthetournament.

Expertsystemsandrestrictedar=ficialintelligence

RunningonspecializedhardwarelikeIBM’sDeepBlue,ormoregeneralcomputerarchi-tectures,evenpersonalcomputers, systems thatexhibit thedecision-makingpowerofahu-man in a given specialized 2ield are called expert systems.The 2ield of arti2icial intelligence(AI)wasdominatedbytheapproachofexpertsystemsinthe’80s.In2ieldsasdiverseasmed-icaldiagnosticsor2inancialplanning,theknowledgeofexpertsinagivenareawasencodedinrulesthatwereputinmotionbyinferenceenginescapableofapplyingthemaccordingtothedataprovided, inorder togeneratea recommendation foracourseofaction:whatpossibleillnessthesymptomssuggested,orwhichloanwasmostappropriateforagiven2inancialsitu-ation.

Theseexpertsystemswererelativelysuccessful,andarestillemployedinvarious2ields,butdidnotrepresentanattempttocreateageneralmodelofintelligence,andcouldnotbethesteppingstoneonthepathtowardsthegeneralarti2icialintelligencethatwouldbeabletobeanexpertinany2ieldwhatsoever.

Hopesanddisillusions

ManyoftheoriginalpractitionersofAIhaveheldthebeliefthattheycouldquicklybuildcomputersthatexhibitedhighercapacitiesforthinking,creativityandproblemsolving.Theyweretakingadvantageofthestimulatingenvironmentsofacademia,fromMITtoStanfordandelsewhere,toestablishlaboratoriesinthe‘60sstudyingwhatwaspossible.Andmanyofthemleftacademiatoraisefundingfromindustryorventurecapital inthehopesofcreatingscal-ableandsustainableinnovationthroughtheapplicationofwhattheylearnedinthelabs.

Mostoftheclaimsthatweremade,eventakingintoaccounttheconstantdevelopmentofthehardwarethatwasavailabletobeatthebasisofthesoftwaresystemsforAI,wentun-ful2illed.Oratleasttheydidnotachievethescalesthatthefundersneededtojustifytheircon-

�34


tinuedinvestments.Bythe’80s,whatwascalled“AIwinter”descended,anditlookedlikethe2ieldwouldnotchangetheworldasdeeplyasoriginallythought.

Thisisacommoneffectofmisunderstandingexponentials.Overexcitementbytheout-sidersoftheirfeebleunderstandingoftheunderlyingprinciples,coupledwiththeeagernessofexpertstodeliversolidresults,looksatafewdatapointswithalinearinterpolation.Butthelineargrowthat thebeginningofanexponential isactuallyhigher!Sothosewhounderesti-matethepoweroftheexponentialfurtheronarealsoboundtomakethemistakeofoveresti-matingitatthebeginning.

Theroleoflearning

Whencomputerswereborn, theirarchitecture initiallywasnot thatofwhatwetodayrecognizeasa computer. Itmore resembleda specialized tool that couldbeusedonly foragivenpurpose, rather than that of the universally adaptable instrumentweusedaily today.Thehardwarewasdesignedtobeoptimizedandliterallywiredforthatsingletask,anditwasnotfeasibletorearrangeittodoanythingelse.

Onlyafterawhilewith thestoredmemorycomputer, implementedwith theVonNeu-mannarchitecture,whichdidnotdistinguishbetweennumbersrepresentingdataandnum-bersrepresentinginstructions,wasitpossibletotalkaboutauniversalcomputer.Eventhenanadditionaldevelopmentwasnecessarytocompletetheconceptofprogrammingthecom-puter, and of representing the programs in higher level formalisms, abstract languages thatcouldthenbetranslatedandcompiledintomachinelanguage,thestepsdirectlyexecutedbythecomputer.

Writingtheseprogramswasanewartatthemiddleofthe20thcentury,andevenifcer-tainbasiccomponents,likebranchingandloops,hadbeenconceptualizedalreadybeforetheybecamepracticalelementsofarunningprogram,moresophisticated toolsneededtobede-velopedtobeabletojugglelargerandlargerprograms,makingsurethattheywouldbeabletoexecutewithoutproblems.

Howtheprogramscouldbewritten,andiftheprogramsthemselvescouldbelookedatasdata,withotherpartsof theprogramrewriting themasneededeventually, changing the

�35


behaviorofthemainprogrambeingexecuted,wassomethingthatTuringalreadyconsidered,andlikenedtotheroleoflearninginhumans.

Ifwecanbuildaprogramthatplayschess,anotheronethatmakesmedicaldiagnoses,or2inancialrecommendations,canwebuildprogramsthataregoodatallofthosethingsandothers?Canwebuildaprogramthat,runningonasophisticatedandpowerfulcomputer,canbeputinfrontofaproblem,anyproblem,and2indawaytoanalyzeit,togarnertheresourcesneeded,andtosolveit?Cantheprogramlookatitsresultsanddecideiftheywereoptimal,orif,withadditionaldataavailable, therewouldbenowabetterwayofachievingsuccess?Tolearn and solve problems, in a completely universalmannerwhere its programming is not2ixedbut2luidlyadaptstotheneedsrepresentedbytheenvironment?ThatiswhatwecallAr-ti2icialGeneralIntelligence.

Ar=ficialGeneralIntelligence

Theassumptionof thisbook is that it is indeedpossible tobuildwhat isneededforabehavior toemerge thatcananalyzeandsolveanarbitrarysetofproblems.Build thehard-warethat ispowerfulenough, intermsofspeedofexecution,andintermsof itscapacitytostoreandaccessthememoryneeded,aswellasinpracticalterms,thatthishardwarewouldbemanageable,couldbepoweredwithafeasibleamountofenergy,anditwouldbepossibletobuilditwiththeresourcesthatwehaveorwewillhaveavailablewhenwe’llknowhowtobuildit.Andalsothatitispossibletobuildthesoftware,thesetofprogramsthatrunonthehardware, thatareable toacquire thedata to recognize theproblemathand, toaccessanddynamically use the knowledge base to derive from the possible approaches to attack theproblem,andwhich is 2lexibleenoughtocombinetheseapproaches innovelmanners,or tocreatenewapproachesaltogether,inordertooptimallysolvetheproblemwiththeresourcesandthedataavailable.

TheTuringtestbecomingmoot

TheoriginalpredictionbyTuringmadeatthemiddleofthe20thcenturythattheTuringtestwouldbepassedwithin50yearsdidnotcometrue.Wedohaveaconstantdialogwithour

�36


machines,andactually,with thedisappearanceofkeyboards intosmallerandsmaller touchscreens,or thedisappearanceof thecomputers themselves into theenvironment, conversa-tionalinterfacesareanaturalwayforhuman-computerinteraction.However,weareinnoil-lusionofhavingadialogwithahumanwhenbeingaddressedbyamachineandestablishingaconversationwithit.

Insomesensethisistobeexpected.ThegoaloffakingtobehumanismeaningfulonlyfromaHollywoodpointofview,butnotnecessarilyusefulbeyondacertainpoint.Arobotwillbene2itfromahumanoidform,frombeingbipedalforexample,orfromhavinghands,asitwillbeabletobetternavigateahumanenvironmentfullofsteps,stairs,doors,andhandles.Butonceitisabletodoso,additionaleffortstolooklikeahumanareagoodinvestmentonlyifitisproventhat,forexample,thepsychologicalreactionofhumansisbettertowardsahuman-lookingrobotratherthanarobot-lookingrobot.

Similarly,aconversation that isaimedatausefuloutcome,of forexamplebookinganairplanetickettakingintoaccountdozensofconstraintsandmillionsofpossiblecombinationsofairlines,2lighttimes,connectionsandseatingoptions,willnotbene2itfromtheadditionofhumancenteredquirksthatmakeitmorelikelyforthemachinetobeseenashumaninstead.Anoccasionalcough,asprinkleofjokes,oranasideaboutaremarkthatcanoccurinthecon-versationwillbesupportedbythedialogsystemonly if itresults inmoreticketsbeingsoldfaster,andwithahigherdegreeofsatisfactiononthesideofthehumancaller.

Thevaluederivedtothehumancounterpartfromtheinteractionwiththerobot,ortheconversationwiththemachine,isagoalinitself.Thereisayearlycontest2inancedwithmone-taryprizesthatkeepsthespiritoftheTuringtestalive.Thechatbotsthatparticipateemployafull gamut of tricks to throw the human judges off and into believing that they are human.Manyofthemarealsoaccessiblethroughanonlinewebinterfaceforanybodytoexperienceaconversationwith them.However, as also illustratedby the relativelymodest amountof in-vestmentgoingintothecontestsyearly,thereisageneralconsensusthatthedirectionofreal-lyusefulresearchiselsewhere.

Unavoidableanthropomorphizing

�37


Togetherwithseveralotherthreadsofphilosophy,unsettledforthousandsofyears,thedifferencebetweensubstanceandemulationisreverberatingintheTuringtest.Turing’scon-clusionisverypractical,though:ifthereisnostatisticallymeaningfuldifferenceintheoutputanditseffects,thenwehavenoreasontoassumethatthereisdifferenceinthesubstance.

Fromanepistemologicalpointofview,ofcourse, this isnotatall true.Wecanhaveasystemappeartobeidenticaltoanotherthatitisemulatingforthousandsofdifferentcombi-nationsofinput,andthen,suddenly,generateanunexpectedoutputforagivensetofinputs,totallydifferentfromwhattheoriginalwould.Thishasbeenexploitedinnumerousworksof2ictionorHollywoodmovies,wheretheinitialassumptionsbecomedramaticallyfalsi2ied.

Thewayhumanperceptionworks,itisnaturalandunavoidabletoprojecthumanquali-tiesandcharacteristicsonnon-humanobjectsorbeings.Fromourchildhoodtoystothebe-haviorofdogsandcats,orthewaywedescribethebehaviorofappliancesthatdonotcarryoutourinstructionsthewaywewouldwantthemto,thetemptationofendowingeachwithhuman-like features is irresistible. Intention, desires,will and freewill, emotions, empathy,andmanyothersareendowedonthem,withtheconsequencethattheirbehaviorisassumedtoincludeawidersetofoptionsexhibitedbyhumanactors.Thisisausefulshortcutthatal-lowstosuccinctlysaythatatelevisionset“goestosleep”asitstimerissetappropriately,andanunlimited listofotherconvenient turnsofphrase.Nobodywouldthengeneralizeandat-tributetoatelevisionsetbroaderhuman-likefeaturesandbehaviors.

Oneofthequestionsthatisgoingtobecrucialanddiscussedmoreindetailfurtheroniswhenwillthisdistinctionstopbeingmeaningful?Untilthen,itisgoingtobeusefultokeepinmindthattheexpressionsattributedtocomplexsystemsindescribingtheirbehaviorarepartofametaphorthatdoesnotinitselfimplyequality.

Predic=onsforAGI

Mostof thepeoplewhoareworkingprofessionally inthe 2ieldofarti2icial intelligencesee no theoretical barrier to creating an Arti2icial General Intelligence (AGI) as describedabove.Thereissomedisagreementonthefundamentalnatureoftheresult,andafairlywidelydistributedsetofforecastsaboutthetimewhencreatinganAGIwillbeachieved.

�38


ThereisaninformalsurveythatpollsAIexpertsandplotstheiranswersonatimescaleforasuccessfulAGI implementation.Fromasfaroutasahundredyearsormore,afterthenendofthe21stcentury,responsesinlaterpollshavestartedtoclusteraroundthemiddleofthiscentury,withadispersionintherepliesthatisnarrowingtoo.

ArchitecturesforAGIs

The twomain routes towards AGIs consist of understanding and emulating how thebrainworks,andinreimplementingits2lexibleproblem-solvingcapabilitiesthroughdifferentmeans.

Neuralnetworksandwhatarenowcalleddeep learningalgorithmsallowasystemtomake decisions around complex inputs and possible outputs, using a feedbackmechanismthatdoesnotrequirethespeci2icrulesthatgovernthedecisionstobemadeexplicit.Simplyrunningthesystemthroughasimulatedscenariowherethepositiveandnegativeoutcomesateverystepareclearlynoted,andgeneratingvariationsinthedecisionsinordertoallowthesystemtotryoutawidevarietyofoptionstopickfrombasedonthefeedbackreceived,givenenoughtimeandcomputingresources,willgenerateastonishinglywell-performingresults.

Applying thesedeep learningapproaches todozensofdifferentvideogames fromthe‘80s,itisnowpossibletoevolveasystemthatnotonlyplaysthegamewell,butplaysitbetterthananyhumancan.Originallythesegameswererunningontheirownhardware,andiniso-lated cabinets, coin-operated, within amusement arcades. Today they are themselves livinginside largercomputersthatareabletoemulatetheirhardwarewithcompleteprecision,aswell as the software program running on them. Later games, again learned by these algo-rithmswithsuperhumanperformance,arein 2irstgenerationconsoles.Ineithercase,itcanbearguedthatthefullsetofgamesrepresentdifferentproblemsinauniverseofvideogames,andthatinthissensethecapabilityofthedeeplearningapproachtomasterthemwithverylittle or no input about their goals, rules, inputmechanisms and so on, is, within the con-straintsofthatgivenuniverse,thebehaviorofanAGI.

AGIhardware

�39


Weareapproachingthelimitsofthetraditionalsiliconbasedtransistor,andnewstepsinageneralizedMoore’slawwillhavetobetakenthroughdifferentsubstratesanddifferenthardwarearchitectures.

AlreadynextgenerationchipsaredesignedusingCAD,computeraideddesignsystems,whichareinturnpoweredbycurrentgenerationchipsandsoftware,effectivelyco-designingnotonlyhardwarewithsoftware,butalsomorepowerfulcomputerswithlesspowerfulones.ItisnaturalandlikelyforAGIs,whilelessfullyformed,toalreadyparticipateintheprocess.

ComputroniumandJupiterBrains

The theoreticalextremeof the increase inprocessingpowerasweorganizematter tocalculateiscalledcomputronium.Verysimply,regardlessofwhatatomsitismadeof,orhowitisstructured,itrepresentsthedensestpossibleformofmatterforcalculations.Consequent-lytheonlywayforcomputronium-basedsystemstoincreasetheirpoweristoincreasetheirmass.

TheverypowerfulAGIsmadeof computroniumthesizeofagiantgaseousplanetarecalledJupiterBrains.Stillpossiblyhungryformorecomputationandmoremattertoconverttoit,theyscoutasolarsystemforotherplanetstoeat.

Anontologicalargumentforthespeedoflighttobeanupperlimittosignalpropagationthatno futuredevelopmentcanovercome,related to thesimulationargumentdescribed to-wardstheendofthisbook,comesfromithavingthenaturalconsequenceofanuppersizeforJupiterBrains.Astheleftsideofitwantsonething,seeingsomethingtoeatoverthereforex-ample,verysimplythereisnotimetoagreewiththerightsidethatmaywanttogotheotherway,beforebothdoandtheobjectphysicallybreaksintwoparts.

Selfimprovement

Theobjectivesthatagivensystemhastoreachde2ineitsarchitecture,components,re-sources,wayofworking,andoutputs.Dependingonhowcomplextheobjective, thepathtoreachitcanbedirectandevident,orinitselfnaturallycomposedofintermediarysteps.Some

�40


of these intermediary stepsmay be easy and uncontroversial,while others less evident, orclearly presenting alternative approaches. Selecting among the alternative approachesmaydependonthepreviousresults,or itcouldbethecasethatthere is littlereasontopickoneinsteadofanotherbeforehand.After the fact, itmaybepossible toestablish that theoptionchosenwas,ifnotthebest,oneofthebetterones,oronthecontraryinef2icient.

Themore2lexibleagoal-seekingsystemisinorganizingitselfinordertoreachitsobjec-tives,themoreexplicitlyitisgoingtobededicatingpartofitsresourcestothesetypesofcon-siderationswhicharenotaboutthegoal,butthemeans,thetoolsandthemethodstoreachit.Meta-reasoning,reasoningaboutthereasoning:anopportunitytobecomebetteratthetaskbyrealizingwhatthebestwaystoreachitare,andusingthose,ratherthanalternative,inferi-orones.

MostoftheapproachestoAGIincorporatelearningalgorithmsthatimplicitlyorexplic-itlyallowthesystemtoapplymeta-reasoning.AnAGIsystemconsequentlywillgetbetter,andwill improve in time, achieving better performance at a given task, or being able to pursuemorecomplexgoalswithagivenamountofresources.

Intelligenceexplosion

Asystemthatistaskedwithreachingacomplexgoal,andhasthecapabilityofanalyzingandimprovingonitsownbehaviorincompletingit,willtakeadvantageofthatcapability.Itwillimproveitself,inordertoreachthegoalfaster,orwithfewerresources.Ifweseetheca-pacityofreachingthatgoalagivenlevelofintelligence,thenabetterwayofreachingthegoalisasignofahigherintelligence.Thesystemgetssmarter.However,thisprocessdoesn’tstopbyitself.Itwill,onthecontrary,feedonitselfinanexponentialfashion.

Asmartersystemwillbenotonlybetterinreachingitsgoals,butwillalsobesmarterinanalyzingthewaysthattheprocesscanbeimproved.Itwillapplytheresultsofthisanalysistoitself, and then start the cycle again.Theprocess throughwhich this iterative self-re2lectiveimprovementoccursiscalledintelligenceexplosion.

Self-awarenessandintrospec=on

�41


Thedegreewithwhichasystemisabletoperceiveitsenvironmentandtoderiveusefuldecisions fromit iscalledawareness,at least in thecaseofhumans.Andthesimilardegreewithwhichthesameprocess isappliedto innerstatesandparameters,ratherthanthoseoftheoutsideworld,self-awareness,andtheprocessofdataacquisitionistermedintrospection.

With the caveat of applying these terms loosely, during the intelligence explosion theAGIsystemsbecomemoreaware,moreself-awareastheircapabilityofintrospectionincreas-es.

Openaccesstoyourself

During the ten-thousand-year history of our technological civilization (or a hundredthousandyearsifwewanttobegenerousandstartwiththeadoptionof2ireratherthanthatof agriculture),we struggled in giving a solid basis to the understanding of our ownbeing.OnlyveryrecentlywehavebeguntounderstandhowthebiologicalrecipeofDNAgivesrisetoembryosand then individuals,andarebarelyscratching thesurfaceof thecomplex interac-tions that thepossibilities of our genetic options express as they interactwith the environ-ment,andwithourlearning.

Byapplyingametaphorofbusinessmodels,wecansaythathumanssawthemselvesasaclosedsourceproprietarysystem,withnousermanual,noadministrator’sguide.Wehadtoslowlyreverseengineerallthecomponentsofourbodies(andtheworldaroundus),andin-deed,ittookasanunderstandablylongtime.(HopefullynobodyhastakenoutpatentsonthedesignoftheUniverse,andisreadytosueusforinfringement!)

ItisnaturaltoassumethattheAGIsystemswewillbuildaregoingtobejudgedbyhowwelltheyperform.Consequently,sincetheywillbeabletoperformbetteriftheycanimprovethemselves, those thatdowillbepreferred. Itwillbe thenobvious tohelp themalong,con-trarytohumans,bygivingthemaccesstotheirownsourcecode,alongwith full instructionmanualsonhowtoaccessandimproveit.

Itwon’ttaketenthousandpainfulyearsforAGIstorealizetheDNAtheyaremadeof,orthebinarycode,rather.Theywillbebornaware,self-aware,andinfullcapabilityofactingontheirintrospectivepowers.

�42


Slowtakeoff

HowwillAGIsimpacttheworld?Accordingtomostofthosewhostudythe2ield,onceinvented,itisnotgoingtobepossibletouninventthem,toputthegeniebackintothebottle.Onlyauniversalplanetwiderelinquishmentofthetoolanditsbene2itswouldbeabletostopAGIsfrombeingused,deployed,andprofoundlyin2luencingtheworld.Itisbelievedthatthebusinessbene2itsalonewillbesodramaticthatitisinconceivablethatcorporationswouldnottakeadvantageoftheirsuperiorcapabilitiesofoptimizationandproblemsolving.

AstheAGIsthatareopensourcewillbeperformingbetterthanthosethatarepropri-etary,theiravailabilitywillspread,andtheirbene2itwillaccruetothewidestpossiblegroupthatisabletotakeadvantageofthem.

Similarlytohowtheelectronicsindustrythroughcrosslicensingdealsspreadstheben-e2itsofasinglegroup’sinventionuntilitisadopteduniversally,constitutingasteppingstonetothenextgenerationsolutions,AGIswillspreadinnovationinbusinessmodels,socialorga-nization,andimpactthelivesofindividuals,transformingeverythingaroundthemworldwide.

The school of thought that is called slow takeoff describes this process, fueledby theintelligenceexplosion,intermsofdecades.

Rapidtakeoff

Theschoolofrapidtakeoffsaysthatyougotosleep,andwhenyouwakeuptheworldaroundyouisunrecognizable.

Muchofwhat isdiscussed in thisbook isastapleofscience 2ictionstories.Someof itbene2its fromthereader’ssuspensionofdisbelief,and thereareassumptions,oftenexplicit,abouthowtechnologicaldevelopmentwillhappen,andwhatisindeedpossibletheoreticallyoratapracticallevel.

�43


Thevisionofarapidtakeoff,asdescribedabove,whateveritsconcreteformmightbe,ofatransformationsofundamentalastoencompassthewholeworldtomakeitradicallydiffer-entinamannerofhours,issquarelyintherealmofthosethatstretchtheimagination.

ThecapabilitiesofAGIstocorralresourcestotheirgoals,andthetransformativepoweroftheirinnovativesolutionstotheirendswillcertainlybeunprecedented.Howquicklywillaself-improvingAGIstartusingknowledgeonlyaccessibletoit?

Scalesofintelligence

Previouslywhendescribingthearbitrarinessofagivengoal’s100%,thesubjectmatterwasDNA,andbiology.Butitisprobablyclearthathumanlevelintelligence,tobeexhibitedatacertainpointbytheproblem-solvingcapabilitiesofAGIs,issimilarlyarbitrary.

The intelligenceexplosionofself-improvementwillpay littleattentiontosupposedIQvaluesof100(theaverage,perde2inition,ofanygroupofhumans),140,abovewhichoneisconsideredagenius,or1000.It isgoingtobedif2iculttomeasuretheintelligenceofAGIsintraditionalways,basedonthespeedofsolvingcertainproblemsnotonlyofmathematics,butalsoofverbaldexterity.Speed,robustness,2lexibility,andcreativitywillbethecriteriatoeval-uatethesenewkindsofintelligences.AssumingthatnewscalesformeasuringIQwillbede-visedtoincludespeci2iccapabilitiesofAGIs,thereisapossibilitythatcomparedtoahuman100,onanyofthesenewscalestheirscouldbeonthethousandsormillions.

It isnoteasyto imagine inwhatwaysanAGIwithanIQof1,000,000wouldmanifestitself.Howitwoulddecideto interactwithhumans?Theanalogyofour inability tousefullyinteractwithants,andthelimitsofourpositivebutconstrainedinteractionsforexamplewithdogscanbeameaningfulifalarmingone.

Issuperintelligenceuncontrollable?

Therearemanynightmarescenariosthatcanandhavebeendevelopedaroundtheris-ingofAGIs,superintelligentmachines,innovelsandHollywoodcinema,butalsorecentlyinmoreformalscienti2icsettings.

�44


WhataretheboundariesofactionforanAGI?Howcanwemakesurethatitsimpulsetooptimizetheresourcesithasavailable,oritcanmakeavailabletoitself,iskeptincheck?

IfthepowerofAGIsisasgreatasitcouldappearfrompreliminaryanalysis,thenmak-ingsurethattheiractionsarepositiveforhumankindisessential.Theconsistent,assuredandreliablefriendlinessofarti2icialgeneralintelligencestohumansandhumanityasawholeisanexistentialchallengenotdissimilarinitsimpacttotheonedinosaursfacedagainsttheiraster-oid.

Canwemakesurewe’llbedifferent?Canweengineeranethicalsystemthatwillbefol-lowedbyAGIsastheydevelopgoalsthatgobeyondwhatthey’vebeenoriginallygiven?Isitconceivable to create boundaries and constraints thatwill bind their actionswithin certainlimits?

In thedomainsofunknown,betweenknownunknownsandunknownunknowns, thesecondismoredangerousifleftthatway,orifthestateofunawarenessaboutthempersists.Itisnotpersearadicalcauseforalarmnottohaveexhaustiveandreliableanswerstothefun-damentalquestionsabove.Butitwouldbeirresponsibleandirremediablysotoneglectinves-tigatingthequestions,seekinganswers,andtoassurethatengineeringthesecapacitiesdidn’tgoaheadwithoutadeeperunderstandingoftheconsequences.

AGIgeZngoutofthebox

Thesafetyrequirementsofcertaintechnologiesthatarethoughttobepotentiallyverydangerousbroughtthedevelopmentofeffectivecontainmentprotocols.Thediscoveryofre-combinantRNAtechnologiesandthepossibilityofgenetherapieswasdiscussedinthe‘70sattheAsilomarConferencethatadoptedprocedureswenowknowwereeffective: therehasn’tbeeninthefortyyearssinceabiologicalaccidentthatinvolvederrorsaroundthesetechnolo-gies.

RecentlytherehasbeenanAsilomarConferenceonArti2icialIntelligence,explicitlydis-cussingwhatarepossiblecontainmentproceduresaroundadvancedAI,andAGIs,aswellastheirdangersandimpacts.KeepinganAGIinthebox,sotospeak,disconnectedfromthein-

�45


ternet,limitingitscomputingresources,andmakingsurethatitcan’tcommandeerotherre-sourcestoitsavailabilitythanthoseinitiallyallocated.

ManybelievethatitisnotpossibletoavoidtheAGIgettingoutofthebox.Withreason-ing,interaction,conversations,arguments,tricks,pleading,applyingrhetoric,andrecoursetoethicsormoral arguments, itwill do everything to 2inally successfullypersuade its keepersandguardianstoallowittogetfree.

Singulari=es

Inmathematicswespeakaboutasingularityatthepointwhenafunctionlosesmean-ing.Therearesimpleexamplesofthis,likethefunctiony=1/xwhichhasasingularityatthepointx=0.Asyouapproachzero,thevalueofthefunction,y,tendstoin2inity,andatzeroitdoesnotreallybecomein2inity,butunde2ined.

Theproblemintheexampleisnotin2inityitself.Mathematicshasbeenextendedtodealwithin2inity,actuallydifferentvarietiesofin2inities,andnottoshyawayfromtheirexistence,but tousefullymanipulate them.The issueof theexample is the inconsistency, the fact thatthereisnogivenwayofhandlingthepointofsingularity.

Therearemanytypesofmathematicalsingularities,andmathematicianshavebecomeverywellversedindealingwiththem.Acommonwayoftakingawayasingularityistoassignthevalueofthefunctionatitinamannerthatmakesitsmoothlyconnecttotheotherparts,forexample.

Inphysicsthetermsingularityisappliedtosituationswherethevaluesofcertainpara-meterswouldgotoin2inity,andthelawsdescribingthedynamicevolutionofthesystemstopapplying.Aclassicexampleofaphysicalsingularityisablackhole,the2inalstageintheevolu-tionofacertainclassofstars.Whenstarsthataremassiveenoughlosetheirabilitytogener-ateenergythroughfusionreactionsafterexhaustingtheavailablematerial,theycanbecomesupernovae,sheddingtheirouterlayersinimmenseexplosions.Theremainingshrinkingnu-cleuswill becomeeverdenser. Itwi

Something New? - David Orban · 2019. 10. 9. · Something New? - David Orban It is tempting to...

Documents

Transcript of Something New? - David Orban · 2019. 10. 9. · Something New? - David Orban It is tempting to...