Post on 16-Apr-2017
Has AI Arrived?
Big Data SpainMadrid, 2016-11-17
Paco Nathan, @pacoidDirector, Learning Group @ O’Reilly Media
1
A rhetorical question:
from:Beyond the AI Wintergoo.gl/tKug8u
Can you name ten successful tech start-ups which lack any application of Machine Learning on their roadmaps?
2
An interesting perspective:
To paraphrase Peter Norvig, Google @ AI Conference 2016:
Marc Andreessen noted famously how softwarewas disrupting so many incumbents … and now Machine Learning is disrupting many incumbents
from:Software engineering of systems that learn in uncertain domainssafaribooksonline.com/library/view/oreilly-ai-conference/9781491973912/video260721.html
3
A related perspective:
Pedro Domingos believes we’re getting closer to realizinga “universal learner”
The future belongs to those who understand ata very deep level how to combine their uniqueexpertise with what algorithms do best.
from:The Master Algorithmgoodreads.com/book/show/24612233-the-master-algorithm
4
A related perspective:
Domingos describes “five tribes” of machine learning (see especially on page 54):
• symbolists: inverse deduction, e.g., rule systems
• connectionists: what the brain does, e.g., deep learning
• evolutionaries: natural selection, e.g., genetic programming
• bayesians: uncertainty, e.g., probabilistic inference
• analogizers: similarities, e.g., support vectors
from:The Master Algorithmgoodreads.com/book/show/24612233-the-master-algorithm
5
In retrospect:
During the past few years applications of deep learning have exploded. Among those tribes, “connectionists” now prevail.
Even so, deep learning is only a portion of machine learning. Moreover machine learning does not represent the entirety of machine intelligence.
What else will be needed?
6
Where are the examples?
7
Major tech firms (just a sample):
8
“An Ecosystem of Machine Intelligence”
oreilly.com/ideas/the-current-state-of-machine-intelligence-3-0 Shivon Zilis, James Cham, Heidi Skinner
9
Reaching Human Parity:
Historic Achievement: Microsoft researchers reach human parity in conversational speech recognition blogs.microsoft.com/next/2016/10/18/historic-achievement-microsoft-researchers-reach-human-parity-conversational-speech-recognition/
10
Reaching Human Parity:
Historic Achievement: Microsoft researchers reach human parity in conversational speech recognition blogs.microsoft.com/next/2016/10/18/historic-achievement-microsoft-researchers-reach-human-parity-conversational-speech-recognition/
Shades of HAL: openreview.net/pdf?id=BkjLkSqxg
11
Realistically…
consider the control system at the heart of, say, Uber – manipulating supply chains of resources for particular outcomes
12
Some favorite examples in arts & lit:
Benjamin.ai / Sunspring youtu.be/LY7x2Ihqjmc
13
Some favorite examples in arts & lit:
Flash Forward: “The Witch Who Came From Mars” flashforwardpod.com/2016/09/05/episode-20-something-martian-witch-way-comes/
14
Artificial Intelligence conference series:
New York City (last Sep) conferences.oreilly.com/artificial-intelligence/ai-ny-2016 San Francisco (last Oct) conferences.oreilly.com/artificial-intelligence/bot-ca
New York City, Jun 26-29 2017 conferences.oreilly.com/artificial-intelligence/ai-ny (CFP open through Jan 18)
15
Artificial Intelligence conference series:
New York City (last Sep) conferences.oreilly.com/artificial-intelligence/ai-ny-2016San Francisco (last Oct) conferences.oreilly.com/artificial-intelligence/bot-ca
New York City, Jun 26-29 conferences.oreilly.com/artificial-intelligence/ai-ny(CFP open through Jan 18)
As one might imagine, the presenters discussed much deep learning – although there were other important points… let’s consider those
16
AI requires sophisticated engineering?
Software engineering of systems that learn in uncertain domains safaribooksonline.com/library/view/oreilly-ai-conference/9781491973912/video260721.html
17
Observations by Peter Norvig:
• difficult to debug, revise incrementally, verify • less transparency into algorithms • components are hard to isolate, for debugging • automated integration introduces unusual risks • tech debt accumulates more readily
Machine Learning: The High Interest Credit Card of Technical Debt research.google.com/pubs/pub43146.html
Software engineering of systems that learn in uncertain domains safaribooksonline.com/library/view/oreilly-ai-conference/9781491973912/video260721.html
AI requires sophisticated engineering?
18
Why should I trust you? Explaining the predictions of any classifier
safaribooksonline.com/library/view/strata-hadoop/9781491944660/video282744.html
kdd.org/kdd2016/subtopic/view/why-should-i-trust-you-explaining-the-predictions-of-any-classifier
Carlos Guestrin: LIME
19
Impact on Big Data, Cloud, etc.:
Overall, AI drives product features
That process in turn drives cloud consumption (look at the major players)
What’s the impact for those already immersed in Big Data, Data Science, Machine Learning, Distributed Systems, Cloud technologies, DevOps practice, etc.? In word: Good
The results will be in healthcare, manufacturing, agriculture,energy, transportation, etc.
20
Artificial intelligence: making a human connection safaribooksonline.com/library/view/oreilly-ai-conference/9781491973912/video260723.html
AI work is mostly human?
21
Observations by Genevieve Bell @ Intel:
An anthropologist would ask: “Who raised you? Who were your mummies and your daddies?” ... AI has had a lot of daddies.
If we understand the founders, we can ask what do we need to bring back into the conversation?
Artificial intelligence: making a human connection safaribooksonline.com/library/view/oreilly-ai-conference/9781491973912/video260723.html
AI work is mostly human?
22
AI work is mostly human?
The Future of AI, Oren Etzioni @ AI2 safaribooksonline.com/library/view/oreilly-ai-conference/9781491973912/video282377.html
23
Etzioni stressed the key role of humans-in-the-loop:
99% of machine learning is human work
AI work is mostly human?
24
Over-anthropomorphization may become problematic:
• does this analysis introduce unneeded bias?
• machine intelligence differs from human cognition, e.g., abductive reasoning (e.g., C.S. Peirce)
• consider examples of evolved antenna
AI work is mostly human?
25
Jobs won’t be displaced by AI?
Why we’ll never run out of jobs safaribooksonline.com/library/view/oreilly-ai-conference/9781491973912/video260722.html
26
Observations by Tim O’Reilly:
We won’t run out of work until we run out of problems
Our main advances have come when we invested in other people's children – massive investment in EU following WWII, built from something that resembles Syria today
21st c great question: “Who’s black box do you trust?”
Jobs won’t be displaced by AI?
Why we’ll never run out of jobs safaribooksonline.com/library/view/oreilly-ai-conference/9781491973912/video260722.html
27
US voting by state g.co/kgs/PSq9JS
Jobs won’t be displaced by AI?
28
US jobs by state npr.org/sections/money/2015/02/05/382664837/map-the-most-common-job-in-every-state
Jobs won’t be displaced by AI?
29
Realistically, fully self-driving trucks are a bit further away fool.com/investing/2016/10/30/despite-ubers-self-driving-truck-delivery-truck-dr.aspx
Some contend that no existing economic model addresses the accelerating pull of technological deflation
Meanwhile, social reforms regarding health care andUniversal Basic Income become urgent priorities
Jobs won’t be displaced by AI?
30
Does AI = Deep Learning?
Obstacles to progress in AI safaribooksonline.com/library/view/oreilly-ai-conference/9781491973912/video260902.html
31
Yann LeCun described some necessary components of AI:
• perception • predictive model • memory • reasoning and planning
Obstacles to progress in AI safaribooksonline.com/library/view/oreilly-ai-conference/9781491973912/video260902.html
Does AI = Deep Learning?
32
AI is much more than Deep Learning
Perception, prediction, memory – these are necessary; however, they do not address understanding
Winograd Schemas show the need for common sense and contextual understanding – replacement for Turing Test
see: The Winograd Schema Challenge Hector Levesque commonsensereasoning.org/2011/papers/Levesque.pdf
33
AI is much more than Deep Learning
Common sense and context: for example, without ample knowledge of the world, a sentence cannot be understood
⇒ embodied cognition (prevailed for a while)
⇒ ontology (more difficult, likely much more useful)
34
A lesson from history
see: Why AM and Eurisko Appear to Work Doug Lenat, John Seely Brown aaaipress.org/Papers/AAAI/1983/AAAI83-059.pdf
Eurisko, The Computer With A Mind Of Its OwnGeorge Johnson aliciapatterson.org/stories/eurisko-computer-mind-its-own
Eurisko, and a mobius strip memory cell
Learning, rules, patterns – these only go so far
Ontology and the quest for common sense
35
Some Missing Pieces
With ML, we assume there’s structure embedded in the data, then build ML models to validate those assumptions
However, which tools serve to identify structure?
see: Persistent Homology: An Introduction and a New Text Representation for Natural Language Processing Xiaojin Zhu pages.cs.wisc.edu/~jerryzhu/pub/homology.pdf
Topological Data Analysis Chad Topaz dsweb.siam.org/TheMagazine/Article/TabId/823/ArtMID/1971/ArticleID/777/Topological-Data-Analysis.aspx
36
AI transformations
Recently launched our own AI project within O’Reilly Media…
We’re not a high-tech company; even so, the value of our data gets unlocked through AI
This project makes use of cloud, Spark, Mesos, Kubernetes, Docker, etc., leveraging the tools we know, but in more complex use cases now.
37
13K lexemes: our “universe” for customer interaction
Too much cognitive load for any editor or engineer to master; however, not so difficult for a small cluster.
Curation is hard; you don’t want it full automated – related to what Norvig calls the “Inattention Valley”
AI transformations
38
Challenge: generating an implicit graph versus curating an explicit graph, then maintaining integrity between:
A
C
BE
D
ML, Big Data, etc.: computed similarity, inferred links, etc. (empiricists)
Curated ontology: graph queries, search rewrites, etc. (rationalists)
a
c
be
d
AI transformations
39
A
C
BE
D
a
c
be
d
Needs better tooling (SPARQL and triple store crowd haven’t gotten the memo yet about containers, orchestration, microservices, etc.)
AI transformations
BTW, this repo is fantastic: github.com/danielricks/penseur
40
David Beyer: Reshaping global industries
Machine intelligence in the wild: How AI will reshape global industries safaribooksonline.com/library/view/strata-hadoop/9781491944660/video282803.html
41
To paraphrase:
Consider the shift from steam to electric power: it took a generation before factory managers understood they could reconfigure the physical arrangement
AI may be quicker adoption, but faces similar extremes of cognitive embrace
Machine intelligence in the wild: How AI will reshape global industries safaribooksonline.com/library/view/strata-hadoop/9781491944660/video282803.html
David Beyer: Reshaping global industries
42
Looking ahead…
We have a need now to distinguish between what humans and computers can do well, respectively
cognitive load, speed, scale, repeatability:computers > humans
curation (captchas, as an example):computers < humans
Organizations which focus on this expertise for AI applications willhave a distinct advantage
43
presenter:
Just Enough Math O’Reilly (2014) justenoughmath.com
monthly newsletter for updates, events, conf summaries, etc.:
liber118.com/pxn/@pacoid