Artificial Intelligence for the Film Industry
-
Upload
georg-rehm -
Category
Technology
-
view
690 -
download
4
Transcript of Artificial Intelligence for the Film Industry
Artificial Intelligencefor the Film Industry
Georg RehmDFKI, Germany
Propellor FilmTech Meetup #1 – 25 July 2017 – Berlin, Germany
• Sites in Saarbrücken, Kaiserslautern, Bremen, Berlin, Osnabrück, St. Wendel
• Intelligent software systems: robotics, agents, image processing, language understanding, augemented reality, 3D, knowledge management,HMI, security, Industrie 4.0.
• 900+ staff – ca. 300 running projects• CEO: Prof. Dr. Wolfgang Wahlster
Propellor FilmTech Meetup #1 – 25 July 2017
Deutschland-GmbH
2
German Research Centre forArtificial Intelligence GmbH (founded in 1988)
Artificial Intelligence• Strong AI: hypothetical machine with a consciousness
and behaviour at least as flexible as that of a human.• Weak AI: software without consciousness, tailored to
one specific purpose and task.• Machine Learning: give “computers the ability
to learn without being explicitly programmed” (Arthur Samuel, 1959)
• Examples: pattern recognition (e.g., handwriting), predictions (stock exchange), recommendations (films!) etc.
Propellor FilmTech Meetup #1 – 25 July 2017 7
Propellor FilmTech Meetup #1 – 25 July 2017 8
Data Intelligence
Current breakthroughs with machine learning methods(Deep Learning). Also still in use: symbolic, rule-based methods
Language Technology• Language Technology makes use of theoretical results
in linguistics in marketable solutions and applications.• Uses research results from:
– Artificial Intelligence– Computer Science– Computational Linguistics
• Natural Language Processing• Natural Language Understanding
– Psychology, Psycholinguistics– Cognitive Science
• Language: Next big thing for AI!
Propellor FilmTech Meetup #1 – 25 July 2017 9
Example Applications• Spellchecker• Dictation systems• Translation systems• Search engines• Report generation• Expert systems• Dialogue systems• Text summarisation
AI and the Film Industry• AI and Language Technology:
Many breakthroughs in multiple different application areas
• Focus: Film industry
• Massive potential!
Propellor FilmTech Meetup #1 – 25 July 2017 10
FilmIndustry
Language Technology
AI andDeep
Learning
Big DataFast
machines and
networks
Internet of Things
! Editing Trailers
! Writing Scripts
! Recommenders
Propellor FilmTech Meetup #1 – 25 July 2017 13
• Simple Machine Learning• Training data: 100 trailers• Create model and apply it
(i.e., to the film “Morgan”)• Watson selected scenes• “A human editor was still
needed to patch thescenes together to tell a coherent story.”
Propellor FilmTech Meetup #1 – 25 July 2017 14
• Good example of usingtech in a curation setting
• With the machine you’re faster but you arrive at thesame result as the human
• The “AI” part is attributedto the technology by the(astonished) human who’s also been influenced by clever marketing
• Note: an “AI” is only good at one very specific task!
Propellor FilmTech Meetup #1 – 25 July 2017 16
• No, it didn’t.• This is fake news
(category: clickbait).
Propellor FilmTech Meetup #1 – 25 July 2017 19
• Simple ML again• Training data: scripts
of sci-fi movies• Neural network learns
patterns and is able to generate a new script
• Deep Learning for Natural Language Generation (NLG)
• Can also be applied to Shakespeare
Propellor FilmTech Meetup #1 – 25 July 2017 20
• Simple ML again• Training data: scripts
of sci-fi movies• Neural network learns
patterns in scripts and is able to generate new script
• Deep Learning for Natural Language Generation (NLG)
• Can also be applied to Shakespeare
To me, fair, so you never be,Each trifle, way, when bore your beauty when,Such hence your can still,O thou how much were your self the wrong chide.
Thy youth’s time and face his form shall cover?Now all fresh beauty, my love there,Will ever time to greet, forget each, like ever decease, But in a best at worship his glory die.
Stanley Xie, Ruchir Rastogi, Max Chang: “Deep Poetry: Word-Level and Character-Level Language Models for Shakespearean Sonnet Generation” (Stanford)
Propellor FilmTech Meetup #1 – 25 July 2017 21
• The automatically generated script doesn’t make any sense whatsoever.
• “Sunspring” is an interesting exercise but, essentially, unwatchable.
AI – Taking StockWhat AI is good at• Identifying patterns• Extracting structure• Data analysis• Mimicking regularities• Important: training data
(ideally structured)• Emulating smart
behaviour
What AI is really bad at• Creativity• Eloquence• Curiosity• Freshness• Originality• Poetry• Out-of-the-box’ness• Understanding of the
world that surrounds us
Propellor FilmTech Meetup #1 – 25 July 2017 22
Propellor FilmTech Meetup #1 – 25 July 2017 26
Even “automatic mockbuster generation” required a level of creativity that is way beyond
anything Artificial Intelligence can achieve today.
Propellor FilmTech Meetup #1 – 25 July 2017 27
https://medium.com/@bootstrappingme/the-german-artificial-intelligence-landscape-b3708b325124
Film AI Startups• VaultML, ScriptBook, Pilot Movies: Project ticket sales
and box office performance (script or trailer analysis)• Iris.tv: Better recommendations• Qloo: Cultural AI, predicts the tastes for any target
audience and maps relationships (music, books, films)• Valossa: Detects people, context, topics etc. in video
and audio streams (assist video content discovery)• Cinuru: Customer Relationship Management• Much more can be done …
Propellor FilmTech Meetup #1 – 25 July 2017 28
http://www.nanalyze.com/2017/07/6-startups-ai-movies-entertainment/
Data for Film AI• Current AI methods can do a lot with interesting data.• What is “interesting data” in the film industry?• Could be anything from every part of the life cycle:
– Scripts – Preferences – List of scenes– Reviews – Films watched – Credits– Emails – Categories – Rankings– Production notes – Genres – Relations– Demographics – Lexicons – Databases– Statistics – Focus groups – Marketing– Box office results – Target audience – etc.
Propellor FilmTech Meetup #1 – 25 July 2017 29
Example Use Case• Let’s have a look at a concrete use case and challenge• Deep, context-aware recommendations that fit the
viewer’s mood, time constraints, interests, focus areas• Example: you have ca. 60 minutes, you’re interested in
current politics in the US, have an upcoming trip to Vancouver, like running, AI, languages and technology
• Recommender could suggest films or documentaries that exactly fit this bill (using a deep user model)
• How? By pulling different sources of data together• Calendar (upcoming trips and meetings), browsing and
search history, to do list, social media, IMDB profile etc.
Propellor FilmTech Meetup #1 – 25 July 2017 30
Example: Details• Data sources:
– Calendar: upcoming trip to Vancouver– To do list: prepare the trip (e.g., “find running route”)– Email archive: hotel booking in Vancouver
• The smart recommender algorithm could examine these data points and help the user get a few things done
• Upcoming trip + likes running + location of hotel = videos of running routes or running clubs in Vancouver
• Upcoming trip + likes running = films about, or including, running that are set in or that were shot in Vancouver
Propellor FilmTech Meetup #1 – 25 July 2017 31
Lifelogging and IoT• Lifelogging = record your whole life• Mobile phones and activity trackers
are getting closer (quantified self)• Measuring heart-rate 24/7/365• Advanced measurements like
VO2 max through several sensors is consumer-grade technology!
• What about film-related data points?• Measuring excitement, boredom, attention, repetition,
amazement, imitation, cringe-worthiness, disgust, tenseness, eye-tracking etc.
Propellor FilmTech Meetup #1 – 25 July 2017 32
https://en.wikipedia.org/wiki/Lifelog
Film and Quantified Self• Vision: create deep user models by pulling together a
user’s heterogeneous information and data streams (calendar, contacts, to do lists, profiles, youtube etc.)
• Add lifelogging-related data by tapping into activity trackers, smart watches, mobile phone sensors
• Endless possibilities would emerge … – and will!• Measure the reactions of one viewer or a whole theater
by measuring their vital stats when watching a film• Revolutionise film development and test screenings• Adapt films dynamically (insert explosion when bored)
Propellor FilmTech Meetup #1 – 25 July 2017 33
• Propellor | Forum #1 created intriguing results
• Any Film, Anywhere – user model, watchlist, loc, reco
• Bubble Buster – user model, reco (safe & surprising)
• Super AI Brain – user model, reco
• Data of the Movie – user model, reco, biofeedback
• AI-based Storytelling – user model, audience clustering, Big Data-based storytelling
Propellor FilmTech Meetup #1 – 25 July 2017 34
http://www.propellorfilmtech.com/forum
Challenges• Integration of heterogeneous data sources (from silos!)
into a unified and homogeneous model as well as making this model available to recommender algorithms.
• Getting the data is hard, so is mapping the data.
• How do we get – on a very large scale – the data out of connected devices (smart phones, smart watches, activity trackers, tv sets etc.) into our own applications?
• The typical, very hard, AI challenges: How can we reallymodel creativity, originality etc.?
Propellor FilmTech Meetup #1 – 25 July 2017 35
Thank you!
Propellor FilmTech Meetup #1 – 25 July 2017 36
DKT kick-off meeting – 25 September 2015
Digital Curation Technologies• Support and optimise digital curation through language and
knowledge technologies• Develop innovative prototypes together with the SME partners• Further develop DFKI technologies and transfer them into
industry through platform for digital curation technologies
Georg Rehm und Felix Sasaki. “Digital Curation Technologies.” In Proceedings of the 19th Annual Conference of the European Association for Machine Translation (EAMT 2016), Riga, Lettland, Mai 2016
Georg Rehm und Felix Sasaki. “Digitale Kuratierungstechnologien – Verfahren für die effiziente Verarbeitung, Erstellung und Verteilung qualitativ hochwertiger Medieninhalte.” In Proceedings der Frühjahrstagung der Gesellschaft für Sprachtechnologie und Computerlinguistik (GSCL 2015), S. 138-139, Duisburg, 2015
Sprach- und Wissenstechnologien
Kuratierungstechnologien
Branchentechnologien
Plat
tform
tech
nolo
gie
Branchenlösungen
http://digitale-kuratierung.de