Mashup the Browser with Ubiquity and Jetpack
107
mitcho Michael Yoshitaka Erlewine TechMaine 17th Annual Conference Portland, Maine, December 10, 2009 mashup the browser with ubiquity and Jetpack
-
Upload
michael-yoshitaka-erlewine -
Category
Technology
-
view
4.438 -
download
0
description
Ubiquity and Jetpack are two experiments at Mozilla Labs which are revolutionizing the ways users can interact with and customize their browsing experience. Both projects directly tackle the disconnected nature of the web by introducing user-generated mashups at the browser level, making our increasingly web-centric experiences more productive and "humane." Users and developers can now easily extend the Firefox browser with little more than the JavaScript language already used on the web.In this presentation, you'll see how Ubiquity and Jetpack can transform your browsing experience and learn the basics of writing JavaScript code for Ubiquity and Jetpack. The presentation will also include discussion of the user-interface philosophy underlying Ubiquity and its multilingual natural language interface.
Transcript of Mashup the Browser with Ubiquity and Jetpack
mitchoMichael Yoshitaka Erlewine
TechMaine 17th Annual Conference
Portland, Maine, December 10, 2009
mashup the browser withubiquity and Jetpack
mitcholinguist, coder, teacher
Mozilla Labs, MIT
CC-BY-NC-SA: flickr.com/photos/jimgris/3538639095/
?mitcho.com @mitchoyoshitaka
envisioning the future of the web
envisioning the future of the web
• the you-centric web:
• social identity
• interacting with the web
• hackability
envisioning the future of the web
• the you-centric web:
• social identity
• interacting with the web
• hackability
envisioning the future of the web
• the you-centric web:
• social identity
• interacting with the web
• hackability
envisioning the future of the web
• the you-centric web:
• social identity
• interacting with the web
• hackabilitytoday
envisioning the future of the web
• the you-centric web:
• social identity
• interacting with the web
• hackability
you live your life online
your social graph is online
aza = window.people.find({name:“Aza Raskin”});
aza.send(“Greetings from TechMaine!”);
your browser should be yours
envisioning the future of the web
• the you-centric web:
• social identity
• interacting with the web
• hackability
mashup the browser withubiquity and Jetpack
three problems:
1. interaction
2. language
3. hackability
how do we interact with the web?
how do we interact with the computer?
CC-BY-NC-SA: flickr.com/photos/pixcat/2324175139/
1984
1997
2001
can you spot the difference?
• Following the GUI paradigm...
• add more buttons?
• add more menus?
The web is offering more and morefunctionality and API’s...
...but how do you use them?
CC-BY-SA: flickr.com/photos/daveduarte/2678853923/
from http://tecfa.unige.ch/~nova/img/lopez1.png
Aza RaskinCC-BY-NC: flickr.com/photos/santheo/2360961001/
ACM interactions 2008
CC-BY-SA: flickr.com/photos/seanosh/3307290530/
“Standard GUIs, with their drop-down menus, check buttons, and tree-lists, cannot compare to the range of options that a text interface effortlessly provides. With just five alphanumeric characters, we can choose one out of 100,000,000 possible sequences... It’s difficult to come up with a non-text-based interface that can perform as well.”
Aza Raskin on
text-based interfaces
CC-BY: flickr.com/photos/striatic/1629254/
?
CC-BY-SA: flickr.com/photos/seanosh/3307290530/
Aza Raskin on
text-based interfaces
“...maybe this confusion isn’t the fault of command-line interfaces in general; maybe it’s just the command lines we’re used to. If commands were memorable, and their syntax forgiving, perhaps we wouldn’t be so scared to reconsider these interface paradigms.”
CC-BY-SA: flickr.com/photos/seanosh/3307290530/
Aza Raskin on
text-based interfaces
“...maybe this confusion isn’t the fault of command-line interfaces in general; maybe it’s just the command lines we’re used to. If commands were memorable, and their syntax forgiving, perhaps we wouldn’t be so scared to reconsider these interface paradigms.”
a natural language interface
ubiquityubiquity.mozilla.com
DEMO
ubiquity
ubiquity
goal:accomplish more on the web
how?connect the web with language
ubiquity
verbs**JavaScript
ubiquityubiquity.mozilla.com
mashup the browser withubiquity and Jetpack
three problems:
1. interaction
2. language
3. hackability
The multilingual internet:1. The web is disconnected by language.
The Internet, 2005CC-BY: commons.wikimedia.org/wiki/File:Internet_map_4096.png
The Internet, 2005: jp, cn, tw, au, deCC-BY: commons.wikimedia.org/wiki/File:Internet_map_4096.png
The Internet, 2005CC-BY: commons.wikimedia.org/wiki/File:Internet_map_4096.png
The Internet, 2005: ut, it, pl, frCC-BY: commons.wikimedia.org/wiki/File:Internet_map_4096.png
The multilingual internet:2. The tools of the future must be multilingual.
AfrikaansShqipعربيঅসমীয়াEuskaraБеларускаяবাংলা (ভারত)Българскиcatalà中文 (简体)
正體中文 (繁體)
HrvatskiČe�štinaDanskNederlandsEnglish (British)
English (US)EsperantoEesti keelsuomiFrançaisFryskGalegoქართულიDeutschΕλληνικάગ"જરાતીעברית!ह#दी (भारत)
MagyaríslenskaBahasa Indonesia
GaeilgeItaliano日本語Kannada한국어KurdîLatvie�šulietuvių kalbaМакедонскиमराठी
Norsk bokmålNorsk nynorskoccitan (lengadocian)فارسیPolskiPortuguês (do Brasil)
Português (Europeu)ਪ"ਜਾਬੀromânăРусскийсрпскиසිංහලslovenčinaslovenskoEspañol (de Argentina)Español (de España)SvenskaTeluguไทยУкраїнськаTiếng ViệtCymraeg
• Writing a parser is hard
• Principles and Parameters approach
• a universal parser + individual language settings
localizing ubiquity
parser
花子にケーキを送って
花子に 送ってケーキをVNN
goal object
V ( goal, object )
parser
Argument identification via semantic role
CmdUtils.CreateCommand({ names: ["email", "mail"], arguments: [ {role: "object", nountype: noun_arb_text}, {role: "goal", nountype: noun_type_contact} ],...
localizing ubiquity
CmdUtils.CreateCommand({ names: ["email", "mail"], arguments: [ {role: "object", nountype: noun_arb_text}, {role: "goal", nountype: noun_type_contact} ],...
Argument identification via semantic role
localizing ubiquity
role English Japanese
object ø を o
goal to に ni, へ e
location at, on で de
LOCALIZING UBIQUITY
Argument identification via semantic role
DEMO:ubiquity in Japanese
localizing ubiquity
Minimal language descriptionsbranching: 'left',usespaces: false,joindelimiter: '',anaphora: ["これ", "それ", "あれ"],roles: [ {role: 'object', delimiter: 'を'}, {role: 'goal', delimiter: 'に'}, {role: 'source', delimiter: 'から'}, {role: 'position', delimiter: 'で'}, {role: 'position', delimiter: 'に'}, {role: 'instrument', delimiter: 'で'}, {role: 'alias', delimiter: 'として'},]
localizing ubiquity
CatalàDansk
EnglishEspañolFrançais
Italiano日本語
PortuguêsSvenska中文
Languagesettings:
ubiquity 0.5
CatalàDansk
EnglishPortuguês日本語
ubiquity 0.5
mashup the browser withubiquity and Jetpack
three problems:
1. interaction
2. language
3. hackability
the web should be generative
the browser should be generative
Firefox Addons
1. addons.mozilla.org (AMO)
2. over 6000 in the wild
3. HTML, JavaScript, XUL
4. Extend Firefox contest
Firefox Addons
1. addons.mozilla.org (AMO)
2. over 6000 in the wild
3. HTML, JavaScript, XUL
4. Extend Firefox contest
Firefox Addons
1. addons.mozilla.org (AMO)
2. over 6000 in the wild
my favorites: Tree Style Tabs, del.icio.us, AdBlock, Firebug
Firefox Addons
1. addons.mozilla.org (AMO)
2. over 6000 in the wild
3. HTML, JavaScript, XUL
4. Extend Firefox contest
Firefox Addons
1. addons.mozilla.org (AMO)
2. over 6000 in the wild
3. HTML, JavaScript, XUL
4. Extend Firefox contest
DEMO
Firefox Addons
+
largest extension platform ever
Firefox Addons
+
1.5 million downloads
but...
Firefox Addons
1. installs need restarts
2. not trivial to develop
3. requires manual upgrades
4. not designed for quick security review
Firefox Addons
1. installs need restarts
2. not trivial to develop
3. requires manual upgrades
4. not designed for quick security review
Firefox Addons
1. installs need restarts
2. not trivial to develop
3. requires manual upgrades
4. not designed for quick security review
Firefox Addons
1. installs need restarts
2. not trivial to develop
3. requires manual upgrades
4. not designed for quick security review
© John Slabyk
Jetpackjetpack.mozillalabs.com
Jetpack
the future of extending Firefox
Jetpack1. JavaScript, HTML, CSS
•
• simple, secure, modular API
2. instant install
3. automatic upgrade
Jetpack1. JavaScript, HTML, CSS
•
• simple, secure, modular API
2. instant install
3. automatic upgrade
Jetpack1. JavaScript, HTML, CSS
•
• simple, secure, modular API
2. instant install
3. automatic upgrade
Jetpack1. JavaScript, HTML, CSS
•
• simple, secure, modular API
2. instant install
3. automatic upgrade
Jetpack1. JavaScript, HTML, CSS
•
• simple, secure, modular API
2. instant install
3. automatic upgrade
Jetpack
DEMO 1:Helvetica
Jetpack
DEMO 2:You Are Here
IP Query API (XML):ipinfodb.com/ip_query2.php?ip=techmaine.com
Jetpack
DEMO 2:You Are Here
© 2005 en.wikipedia.org/wiki/File:Rocket_man02_-_melbourne_show_2005.jpg
Welcome to the future.
Jetpackjetpack.mozillalabs.com
a more personal web
a more generative web
a more natural web
an open web
“The best way to predict the future is to invent it”
—Alan Kay
mitchoMichael Yoshitaka Erlewine
by-nc-samitcho.com@mitchoyoshitaka
Thank you!Questions?
