MyLifeBits Jim Gemmell & Gordon Bell SDForum Distinguished Speaker Series February 19, 2004.
MyLifeBits: Attempting to realize the Memex Vision
description
Transcript of MyLifeBits: Attempting to realize the Memex Vision
11
MyLifeBits: Attempting to realize the Memex Vision
Gordon BellGordon Bell
February 2003February 2003http://research.microsoft.com/barc/MediaPresence/MyLifeBits.aspxhttp://research.microsoft.com/barc/MediaPresence/MyLifeBits.aspx
With Jim Gemmell & Roger LuederWith Jim Gemmell & Roger Lueder
22
Outline … MyLifeBits
Background…fulfilling the Memex visionBackground…fulfilling the Memex visionCyberizing everythingCyberizing everythingFile to database transitionFile to database transitionUse…beyond searchUse…beyond searchLong-term agenda and outlookLong-term agenda and outlook
33
MemexPosited by Vannevar Bush in “As We May Think”
The Atlantic Monthly, July 1945
““A memex is a device in which an individual stores A memex is a device in which an individual stores all his books, records, and communications, and all his books, records, and communications, and which is mechanized so that it may be consulted which is mechanized so that it may be consulted with exceeding speed and flexibility”with exceeding speed and flexibility”
Supports: Annotations, links between documents, Supports: Annotations, links between documents, and “trails” through the documentsand “trails” through the documents
““yet if the user inserted 5000 pages of material a day yet if the user inserted 5000 pages of material a day it would take him hundreds of years to fill the it would take him hundreds of years to fill the repository, so that he can be profligate and enter repository, so that he can be profligate and enter material freely”material freely”
44
Sketch of memex
Bush’s camera on the head
66
Capturing what you see
77
Memory Overload
As hard drives get bigger and cheaper, As hard drives get bigger and cheaper, we're storing way too much.we're storing way too much.
By Jim LewisBy Jim Lewis
There's a famous allegory about a map of the There's a famous allegory about a map of the world that grows in detail until every point in world that grows in detail until every point in reality has its counterpoint on paper; the twist reality has its counterpoint on paper; the twist being that such a map is at once ideally accurate being that such a map is at once ideally accurate and entirely useless, since it's the same size as and entirely useless, since it's the same size as the thing it's meant to represent.the thing it's meant to represent.
88
"The PC is going to be the place where you store the information and really the center of control“ Billg
1/7/2001MyLifeBits is a project to “cyberize” everything!MyLifeBits is a project to “cyberize” everything!
What? Recall of all articles, books, CDs, What? Recall of all articles, books, CDs, photos, video, communication (e.g. mail, phone), photos, video, communication (e.g. mail, phone), web web Why? …Why? …“because we can”“because we can”
Office: communicate, store, & workOffice: communicate, store, & work Home & Media Center: ambiance &entertainmentHome & Media Center: ambiance &entertainment Immortality for progeny. Memory aidsImmortality for progeny. Memory aids
Goal: to understand the 1 TByte PC c2006: Goal: to understand the 1 TByte PC c2006: need, utility, cost, feasibility and tools.need, utility, cost, feasibility and tools.
99
Gordon: Researcher, consumer,
computer system tester,
nerd wanna-be, and average man
Melissa: middle manager
Patrick: Consultant
Nicholas: Analyst
Sondra: Office manager
Knowledge worker scenarios
1010
The guinea pig Gordon Bell is digitizing his lifeGordon Bell is digitizing his life Has now scanned virtually all:Has now scanned virtually all:
Books written (and read when possible)Books written (and read when possible) Personal documents (correspondence including memos and email, Personal documents (correspondence including memos and email,
bills, legal documents, papers written, …)bills, legal documents, papers written, …) PhotosPhotos Posters, paintings, photo of things (artifacts, …medals, plaques)Posters, paintings, photo of things (artifacts, …medals, plaques) Home movies and videosHome movies and videos CD collectionCD collection And, of course, all PC filesAnd, of course, all PC files
Now recording: phone, radio, TV (movies), web pages… Now recording: phone, radio, TV (movies), web pages… conversations?conversations?
Paperless throughout 2002. 12” scanned, 12’ discardedPaperless throughout 2002. 12” scanned, 12’ discarded.. Only 30 GB!!!Only 30 GB!!!
1111
Capture and encoding
1212
I mean everything
1313
Input: tools, time, and cost Scanners: HP Digital Sender, flat beds with ADF, Scanners: HP Digital Sender, flat beds with ADF,
2-HP photo, faxing. (Duplex, color, feed-thru, etc.)2-HP photo, faxing. (Duplex, color, feed-thru, etc.) A good commercial scanner costs 2K-10KA good commercial scanner costs 2K-10K
Photos: $1 or 0.5-5 min. Photos: $1 or 0.5-5 min. Large posters: ~ 1-5 hr.Large posters: ~ 1-5 hr.Artifacts: ~ 10 min. including photoArtifacts: ~ 10 min. including photo
Scanning to TIF, PDF: <1 min/page or .10/page Scanning to TIF, PDF: <1 min/page or .10/page OCR: for MODI or PDF: ~3-5 pages/min (old data)OCR: for MODI or PDF: ~3-5 pages/min (old data) OCR: to recreate an editable “original” 10 min/page!OCR: to recreate an editable “original” 10 min/page!
OCR (Volume paper files): 400 pages/hr. 7 ppm.OCR (Volume paper files): 400 pages/hr. 7 ppm. Books: scanned at CMU ($10 - 100/book) in 1997Books: scanned at CMU ($10 - 100/book) in 1997 Videos: tbdVideos: tbd
1414
Music6.9 GB
1.8K files180 CDs
Working2.3 GB
432 folders2.9K files
Archive5.1 GB
477 folders18.7 K files
Video2.6 GB
10 hoursLow res
My Books98 MB
27.1K files & 42K .msg17.7 GB (by size) Files (by number)
.xls.jpg
.doc/html
.pdf .ppt/ppt albums
.tif
CyberAll Nov.1, 2001
Mail.7 GB
43K msgs
.doc/html.jpg
.gif
.xls
.ppt
.tif
.gif
1616
gbell wag: 67 yr, 25Kday life
1
1 0
1 0 0
1 , 0 0 0
1 0 , 0 0 0
1 0 0 , 0 0 0
1 , 0 0 0 , 0 0 0
1 0 0 -5 K B
M s g s
1 0 0 -5 0 K B
p a g e s
5 -1 0 0 K B
T i f s
0 . 1 -1 M B
B o o k s
1 0 -4 0 0 K Bj p e g s
4 0 K s1 K B p ss o u n d
0 . 1 -1 0 0 M Bs o n g s
1 -1 0 G B
V i d e o s
L i f e t i m e s t o r a g e ( G B )
1717
MyLifeBits organization: time and space
Timeline/Context(space)
Personal(some $s)
GB Co.(angel, etc.)
ProfessionalACM, etc., …@Microsoft.com,
New co’s.
Archival (time) Working
1818
MyLifeBits: Some Lives(t) PersonalPersonal
Parents, children, grandkidsParents, children, grandkids CGB himselfCGB himself Close friendsClose friends
GB $sGB $s Personal incl. several legal Personal incl. several legal
structuresstructures Investments & boardsInvestments & boards
Past companies/organiz’nsPast companies/organiz’ns DECDEC Carnegie-Mellon U.Carnegie-Mellon U. DEC, NSF, Encore, Ardent, DEC, NSF, Encore, Ardent,
GB_consulting, GB_consulting,
CGB@ MicrosoftCGB@ Microsoft MLBMLB ClustersClusters TelepresenceTelepresence WWW presenceWWW presence
Computer History MuseumComputer History Museum BOD memberBOD member Fund-raisingFund-raising CyberMuseumCyberMuseum
StartupsStartups Bell-Mason DirectorBell-Mason Director Diamond & Vanguard Brds.Diamond & Vanguard Brds.
1919
MyLifeBits is:
Memex and more (audio and video)Memex and more (audio and video) Universal store for all personal stuffUniversal store for all personal stuff Guiding principles for the system:Guiding principles for the system:
1.1. Full text search & Full text search & collectionscollections (> than hierarchy) (> than hierarchy)
2.2. Visualizations for search, display, insightVisualizations for search, display, insight
3.3. Annotations and links add value and essentialAnnotations and links add value and essential Increase search ability and value of information.Increase search ability and value of information. So make many kinds and them easy to create!So make many kinds and them easy to create! Stories are the ultimate annotationStories are the ultimate annotation
4.4. Keep the links when you author: “transclusion”Keep the links when you author: “transclusion”
2020
MLB database: size and content?
Database features are essential: Database features are essential: Consistency, Indexing, Consistency, Indexing, Pivoting, Queries, Speed/scalability, Backup, replication.Pivoting, Queries, Speed/scalability, Backup, replication.
Folders &Files were the starting point >> database into sets Folders &Files were the starting point >> database into sets aka “collections” that are identical to the folder structureaka “collections” that are identical to the folder structure
Outlook (msgs, attachments, calendar, contacts)Outlook (msgs, attachments, calendar, contacts) Web trails including voice message annotation Web trails including voice message annotation Journal (Outlook), trails: every document use & transactionJournal (Outlook), trails: every document use & transaction What about? What about?
Money (transactions, payees, etc.)…is their lifelog/trailMoney (transactions, payees, etc.)…is their lifelog/trail Streets and trips to cross-index to all docsStreets and trips to cross-index to all docs Attributes for photos for retrieval? Location, time, settingsAttributes for photos for retrieval? Location, time, settings Presentations as a report or trail. Each slide an object!Presentations as a report or trail. Each slide an object!
2121
2222
Media center 2
2323
CD
VCR
Cassette
Plasma Panel
DVD
MediaCenter
Computer
Set top
Set top
Kbd Mse
Wfr
Spkr
SpkrIR
Cable/Satellite
Ethernet SVHS-wide
5.1 digital
5 speakers
stereo
stereo
stereoVideo*
5.1 digitalcomp.
stereoVideo*
Video*
Cables/linksSpeaker 5+1Plasma 2 or 3Cable/Enet 2IR 8Stereo 45.1 digital 2Comp./S-video 3Plasma panel 1Power 10Kbd/mse 2Monitor II (opt.) 4Camera 2Total 42 – 46 Things 18+remotes
*Video = composite or S-videoCamera
Mic
Receiver
Legacy
Legacy
Legacy
Redundant
2424
Photos
2525
Caneel Bay Vacation Jan. 1998
Gordon, Gwen, Brig, Pam, Gordon, Gwen, Brig, Pam, Fiona, Bob, Laura and KolbeFiona, Bob, Laura and Kolbe
2626
Searching: the most useful app?
Challenge: What questions for useful results? Challenge: What questions for useful results? Lots of ways to look at what you retrieveLots of ways to look at what you retrieveNeed for breaking the returns into segmentsNeed for breaking the returns into segmentsSearching for an indexer and search engine: Searching for an indexer and search engine:
index service, Enfish, dtSearchindex service, Enfish, dtSearchStuff I’ve Seen MSR’s index & search… Stuff I’ve Seen MSR’s index & search…
evolving in the right direction. evolving in the right direction. Productizing would remove the pressure for Productizing would remove the pressure for
LonghornLonghorn
2727
2828
2929
3030
Detail view
3131
Resource explorerAncestor (collections), annotations, descendant
& preview panes turned on
3232
Interface to xls
3333
3434
Statistics of use
3535
Synchronized timelines with histogram guide
3636
Visualization Browsing & searching. “Get me what I want|need!”Browsing & searching. “Get me what I want|need!”
Help the user find things among possible items versusHelp the user find things among possible items versus Waiting for an ideal system that can find “what I want”Waiting for an ideal system that can find “what I want”
Publication: Conventional & web, presentations, Publication: Conventional & web, presentations, etc.etc.
Helps understand the nature of the content e.g. Helps understand the nature of the content e.g. histogram of objects in timehistogram of objects in time
Context: Links to help understand the relationship Context: Links to help understand the relationship between objects. Provides more search handles.between objects. Provides more search handles.
Information density: what is it? Information density: what is it? What is its relationship to others?What is its relationship to others?
Content important. Flash and form, less useful.Content important. Flash and form, less useful.
3737
Value of media depends on annotations
““Its just bits until it is annotated”Its just bits until it is annotated”
3838
System annotations provide base level of value
Date 7/7/2000Date 7/7/2000
3939
Tracking usage – even better
Date 7/7/2000. Opened 30 times, emailed to 10 Date 7/7/2000. Opened 30 times, emailed to 10 people (its valued by the user!)people (its valued by the user!)
4040
Get the user to say a little something is a big jump
Date 7/7/2000. Opened 30 times, emailed to 10 people. “BARC Date 7/7/2000. Opened 30 times, emailed to 10 people. “BARC dim sum intern farewell Lunch”dim sum intern farewell Lunch”
4141
Getting the user to tell a story is the ultimate in media value
A story is a “layout” in time and spaceA story is a “layout” in time and space Most valuable content (by selection, and by being well annotated)Most valuable content (by selection, and by being well annotated) Stories must include links to any media they use (for future navigation/search – Stories must include links to any media they use (for future navigation/search –
“transclusion”).“transclusion”). Cf: MovieMaker; Creative Memories PhotoAlbumsCf: MovieMaker; Creative Memories PhotoAlbums
Dapeng was an Dapeng was an intern at BARC intern at BARC for the summer for the summer of 2000of 2000
We took him to We took him to lunch at our lunch at our favorite Dim Sum favorite Dim Sum place to say place to say farewellfarewell
At table L-R: Dapeng, Gordon, Tom, Jim, Don, At table L-R: Dapeng, Gordon, Tom, Jim, Don, Vicky, Patrick, JimVicky, Patrick, Jim
4242
Value of media depends on annotations
Auto-annotate whenever Auto-annotate whenever possible e.g. GPS cameraspossible e.g. GPS cameras
Make manual annotation Make manual annotation as easy as possible. XP as easy as possible. XP photo capture, voice, photo capture, voice, photos with voice, etcphotos with voice, etc
Support gang annotationSupport gang annotation Make stories easyMake stories easy
no
ne
au
to
au
to-u
sag
e
use
r-ba
sic
use
r-story
Annotations
“Its just bits until it is annotated”
4343
4545
The Agenda for the Tbyte(s), Lifetime, PC:The killer app after office and mail.
1.1. Guarantee that data will live forever! “dear appy” problemGuarantee that data will live forever! “dear appy” problem2.2. Cheap, easy, and data-rich (e.g. time, place) capture:Cheap, easy, and data-rich (e.g. time, place) capture:
GPS and time everywhereGPS and time everywherePaper capture has to be as easy as discard (scanner/shredder)Paper capture has to be as easy as discard (scanner/shredder)E-book…e-magazines & journals need to have critical mass! E-book…e-magazines & journals need to have critical mass! Telephony and audio capture with indexingTelephony and audio capture with indexingMedia Center compatible for entertainment (photos, video, TV, radio)Media Center compatible for entertainment (photos, video, TV, radio)
3.3. One?One? dbase for all books, conversations, mail, web pages … dbase for all books, conversations, mail, web pages …vs. long-term use of hierarchical files. vs. long-term use of hierarchical files. Is dbase intuitive?Is dbase intuitive?
4.4. Annotations/meta-information add every-increasing valueAnnotations/meta-information add every-increasing valueEase of annotation because it aids search and becomes the contentEase of annotation because it aids search and becomes the contentContent analysis (critical for photo & video!)Content analysis (critical for photo & video!)
5.5. Information control: privacy, security, expunge/deniability,… Information control: privacy, security, expunge/deniability,… 6.6. New “killer apps”: alzheimer, immortality, surrogate memory?New “killer apps”: alzheimer, immortality, surrogate memory?7.7. Any GUI to improve use (e.g. time to learn, use, retention)Any GUI to improve use (e.g. time to learn, use, retention)
4646
The End
4747
The “dear appy” problemDear Appy, Dear Appy, How committed are you? How committed are you?
Please come back to me, Please come back to me, Lost and forgotten dataLost and forgotten data
Who’s responsible?Who’s responsible?mediamediaplatform, file, and databasesplatform, file, and databasesevolving standards and formatsevolving standards and formatsevolving and/or disappearing appsevolving and/or disappearing apps
4848
Digitizing our lives Right now, it is affordable to buy 100 GB/yearRight now, it is affordable to buy 100 GB/year In 5 years In 5 years 1TB/year is afforadable!1TB/year is afforadable! It’s hard to fill a terabyte/year just by keeping what you see or It’s hard to fill a terabyte/year just by keeping what you see or
hear, but you can:hear, but you can: Look at 9800 pictures Look at 9800 pictures a daya day (300 KB JPEGs) (300 KB JPEGs) Read 2900 documents Read 2900 documents a daya day (1MB files) (1MB files) Listening to audio or view compressed video 24 hours/day (it takes Listening to audio or view compressed video 24 hours/day (it takes
more than 256 kb/s to fill a TB in a year)more than 256 kb/s to fill a TB in a year) Watch 1.5 Mb/s video 4 hours each day.Watch 1.5 Mb/s video 4 hours each day.
As Bush said, we can “be profligate and enter material freely”As Bush said, we can “be profligate and enter material freely”
0
0.2
0.4
0.6
0.8
1
2002 2003 2004 2005 2006 2007
Dis
k ca
pac
ity
(TB
)