1 Past, Present and Future of User Interface Software Tools Brad A. Myers (work with Scott E. Hudson...
-
date post
19-Dec-2015 -
Category
Documents
-
view
213 -
download
0
Transcript of 1 Past, Present and Future of User Interface Software Tools Brad A. Myers (work with Scott E. Hudson...
1
Past, Present and Future ofUser Interface Software ToolsPast, Present and Future ofUser Interface Software ToolsBrad A. MyersBrad A. Myers(work with Scott E. Hudson(work with Scott E. Hudsonand Randy Pausch)and Randy Pausch)
Human-Computer Interaction InstituteHuman-Computer Interaction InstituteSchool of Computer ScienceSchool of Computer ScienceCarnegie Mellon UniversityCarnegie Mellon University
http://www.cs.cmu.edu/~bamhttp://www.cs.cmu.edu/[email protected]@cs.cmu.edu
MyersMyersCMU - HCI InstituteCMU - HCI Institute 2HFES’00HFES’00
Introduction
User Interface Software Tools Help developers design and implement user interfaces
Focus on Tools, but influenced by future UIs Today’s tools are highly successful
Window Managers, Toolkits, Interface Builders ubiquitous Most software built using them Are based on HCI research
Brad A. Myers. “A Brief History of Human Computer Interaction Technology.” ACM interactions. Vol. 5, no. 2, March, 1998. pp. 44-54. http://www.cs.cmu.edu/~amulet/papers/uihistory.tr.html
Future tools must be different
User Interface Software Tools Help developers design and implement user interfaces
Focus on Tools, but influenced by future UIs Today’s tools are highly successful
Window Managers, Toolkits, Interface Builders ubiquitous Most software built using them Are based on HCI research
Brad A. Myers. “A Brief History of Human Computer Interaction Technology.” ACM interactions. Vol. 5, no. 2, March, 1998. pp. 44-54. http://www.cs.cmu.edu/~amulet/papers/uihistory.tr.html
Future tools must be different
MyersMyersCMU - HCI InstituteCMU - HCI Institute 3HFES’00HFES’00
What Worked
Window Managers and Toolkits Event Languages Graphical, Interactive Tools Component Architectures Hypertext Object Oriented Programming
Window Managers and Toolkits Event Languages Graphical, Interactive Tools Component Architectures Hypertext Object Oriented Programming
MyersMyersCMU - HCI InstituteCMU - HCI Institute 4HFES’00HFES’00
Window Managers
Multiple (tiled) windows in research systems of 1960’s: NLS, etc.
Overlapping introduced in Alan Kay’s thesis (1969) Smalltalk, 1974 at Xerox PARC Successful because multiple windows help users
manage scarce resources: Screen space and input devices Attention of users Affordances for reminding and finding other work
Multiple (tiled) windows in research systems of 1960’s: NLS, etc.
Overlapping introduced in Alan Kay’s thesis (1969) Smalltalk, 1974 at Xerox PARC Successful because multiple windows help users
manage scarce resources: Screen space and input devices Attention of users Affordances for reminding and finding other work
MyersMyersCMU - HCI InstituteCMU - HCI Institute 5HFES’00HFES’00
Toolkits
A collection of widgets (“controls”) Menus, scroll bars, text entry fields, buttons, etc.
Toolkits help with programming Help maintain consistency among UIs
Key insight of Macintosh toolbox
Successful partially because address common, low-level features for all UIs
A collection of widgets (“controls”) Menus, scroll bars, text entry fields, buttons, etc.
Toolkits help with programming Help maintain consistency among UIs
Key insight of Macintosh toolbox
Successful partially because address common, low-level features for all UIs
MyersMyersCMU - HCI InstituteCMU - HCI Institute 6HFES’00HFES’00
Event Languages
Create programs by writing event handlers Many UIMSs used this style
Univ. of Alberta (1985), Sassafras (1986), etc.
Now used by HyperCard, Visual Basic, Lingo, etc. Toolkits with call-backs or action methods are related
Advantages: Natural for GUIs since generate discrete events Flow of control in user’s hands rather than programmer’s
Discourages moded UIs
Won’t work well in future
Create programs by writing event handlers Many UIMSs used this style
Univ. of Alberta (1985), Sassafras (1986), etc.
Now used by HyperCard, Visual Basic, Lingo, etc. Toolkits with call-backs or action methods are related
Advantages: Natural for GUIs since generate discrete events Flow of control in user’s hands rather than programmer’s
Discourages moded UIs
Won’t work well in future
MyersMyersCMU - HCI InstituteCMU - HCI Institute 7HFES’00HFES’00
Graphical Interactive Tools
Create parts of user interface by laying out widgets with a mouse Examples: Menulay (1983), Trillium (1986), Jean-Marie
Hullot from INRIA to NeXT Now: HyperCard, Interface Builders, Visual Basic’s
layout editor, resource editors, “constructors”
Advantages: Graphical parts done in an appropriate, graphical way Accessible to non-programmers
Create parts of user interface by laying out widgets with a mouse Examples: Menulay (1983), Trillium (1986), Jean-Marie
Hullot from INRIA to NeXT Now: HyperCard, Interface Builders, Visual Basic’s
layout editor, resource editors, “constructors”
Advantages: Graphical parts done in an appropriate, graphical way Accessible to non-programmers
MyersMyersCMU - HCI InstituteCMU - HCI Institute 8HFES’00HFES’00
Component Architectures
Create applications out of components which are separately developed and compiled In UI software, each component controls an area of the
screen Example: drawing component handles picture inside a
document
Invented by Andrew research project at CMU (1988) Now: OLE, OpenDoc, ActiveX, Java Beans
Create applications out of components which are separately developed and compiled In UI software, each component controls an area of the
screen Example: drawing component handles picture inside a
document
Invented by Andrew research project at CMU (1988) Now: OLE, OpenDoc, ActiveX, Java Beans
MyersMyersCMU - HCI InstituteCMU - HCI Institute 9HFES’00HFES’00
Hypertext
Ted Nelson named it in 1965 and developed a Hypertext system at Brown University
Important systems: NLS (1967), Hyperties (1986) World-Wide Web
Phenomenal success due to: Ease of use of Mosaic browser Support for embedded graphics Support for easy authoring
Ted Nelson named it in 1965 and developed a Hypertext system at Brown University
Important systems: NLS (1967), Hyperties (1986) World-Wide Web
Phenomenal success due to: Ease of use of Mosaic browser Support for embedded graphics Support for easy authoring
MyersMyersCMU - HCI InstituteCMU - HCI Institute 10HFES’00HFES’00
Object Oriented Programming
Success of OO owes much to UI software field Popularized by Smalltalk GUI elements (widgets) seem like objects
Have state, accept events (messages)
Rise parallels GUIs C++ with Windows 3.1
Success of OO owes much to UI software field Popularized by Smalltalk GUI elements (widgets) seem like objects
Have state, accept events (messages)
Rise parallels GUIs C++ with Windows 3.1
MyersMyersCMU - HCI InstituteCMU - HCI Institute 11HFES’00HFES’00
What Hasn’t Caught On
User Interface Management Systems Separation of UI and application is harder than expected
Formal Language-Based Tools Grammars and Transition networks are not appropriate
for direct manipulation
Constraints Declare once, system maintains Get to be unpredictable and hard to control
Model-Based and Automatic Techniques Results are not as good as hand-created Sometimes, no savings of effort
User Interface Management Systems Separation of UI and application is harder than expected
Formal Language-Based Tools Grammars and Transition networks are not appropriate
for direct manipulation
Constraints Declare once, system maintains Get to be unpredictable and hard to control
Model-Based and Automatic Techniques Results are not as good as hand-created Sometimes, no savings of effort
MyersMyersCMU - HCI InstituteCMU - HCI Institute 12HFES’00HFES’00
Future Prospects and Visions
Important Trends Ubiquitous Computing Move to recognition-based interfaces End-user customization and scripting
Important Trends Ubiquitous Computing Move to recognition-based interfaces End-user customization and scripting
MyersMyersCMU - HCI InstituteCMU - HCI Institute 13HFES’00HFES’00
Ubiquitous Computing
Computation embedded in many kinds of devices Digital pagers and cell phones, Palm Pilots,
CrossPads, laptops, wall-size displays, “smart” rooms Next wave: easy communication with radio
E.g., BlueTooth: www.bluetooth.com
Significant Implications for tools Moving target problem Tools for coordinating multiple, distributed,
communicating devices “Multi-computer” user interfaces
Computation embedded in many kinds of devices Digital pagers and cell phones, Palm Pilots,
CrossPads, laptops, wall-size displays, “smart” rooms Next wave: easy communication with radio
E.g., BlueTooth: www.bluetooth.com
Significant Implications for tools Moving target problem Tools for coordinating multiple, distributed,
communicating devices “Multi-computer” user interfaces
MyersMyersCMU - HCI InstituteCMU - HCI Institute 14HFES’00HFES’00
More problems: Varying Input and Output Today’s Desktop screens vary by a factor of 2.5 in
size and a factor of 4 in pixels Tomorrow’s screen will vary by factors of 100 in
size and a factor of 625 in pixels Cell phone to Stanford’s wall (3796 x 1436 pixels)
Today’s Desktop screens vary by a factor of 2.5 in size and a factor of 4 in pixels
Tomorrow’s screen will vary by factors of 100 in size and a factor of 625 in pixels Cell phone to Stanford’s wall (3796 x 1436 pixels)
MyersMyersCMU - HCI InstituteCMU - HCI Institute 15HFES’00HFES’00
Varying I/O Capabilities, cont.
Interaction techniques for desktop will not work No room on small devices Can’t reach menubar on wall-size devices
Want to run same application on different devices
Interaction techniques for desktop will not work No room on small devices Can’t reach menubar on wall-size devices
Want to run same application on different devices
MyersMyersCMU - HCI InstituteCMU - HCI Institute 16HFES’00HFES’00
Need for Prototyping Devices
User interface will be in hardware Rapid design and prototyping needed for hardware Pragmatics and usability cannot be evaluated from
a simulation on a screen
User interface will be in hardware Rapid design and prototyping needed for hardware Pragmatics and usability cannot be evaluated from
a simulation on a screen
MyersMyersCMU - HCI InstituteCMU - HCI Institute 17HFES’00HFES’00
Move to recognition-based interfaces
Speech, gestures, camera-based vision Multimodal interaction
User will pick which modality to use Use multiple modalities at same time
Today, programming these requires knowing about Hidden-Markov Models, grammars, feature vectors, etc. Need tools to hide these complexities
Speech, gestures, camera-based vision Multimodal interaction
User will pick which modality to use Use multiple modalities at same time
Today, programming these requires knowing about Hidden-Markov Models, grammars, feature vectors, etc. Need tools to hide these complexities
MyersMyersCMU - HCI InstituteCMU - HCI Institute 18HFES’00HFES’00
Fundamental Differences ofRecognition-based UIs
Input is uncertain Recognition can make errors Requires monitoring, feedback, correction
Do not produce discrete events GUI event model no longer works
Interpreting input requires deep knowledge of data Context of the application “Move the red truck to here” Need a architecture based on accessible application
data structures “Reflection”, “Open Data Model”
Input is uncertain Recognition can make errors Requires monitoring, feedback, correction
Do not produce discrete events GUI event model no longer works
Interpreting input requires deep knowledge of data Context of the application “Move the red truck to here” Need a architecture based on accessible application
data structures “Reflection”, “Open Data Model”
MyersMyersCMU - HCI InstituteCMU - HCI Institute 19HFES’00HFES’00
More Assumptions of Today’s Tools
Skill and Dexterity of users Older users Makes single, fixed library of widgets untenable
Non-overlapping and opaque components Preclude translucency, magic lens interactions
Skill and Dexterity of users Older users Makes single, fixed library of widgets untenable
Non-overlapping and opaque components Preclude translucency, magic lens interactions
MyersMyersCMU - HCI InstituteCMU - HCI Institute 20HFES’00HFES’00
Operating Systems Considerations
What is in the OS? Window Manager? Toolkit? Communication? Scripting
facilities?
Need ever increasing services for applications Need more access to low-level information
E.g., hardware buttons, whether on network
Ideally, API to support competition and research into these components
What is in the OS? Window Manager? Toolkit? Communication? Scripting
facilities?
Need ever increasing services for applications Need more access to low-level information
E.g., hardware buttons, whether on network
Ideally, API to support competition and research into these components
MyersMyersCMU - HCI InstituteCMU - HCI Institute 21HFES’00HFES’00
Conclusions
Research in tools necessarily trails innovation in UI design
Due to consolidation on desktop metaphor, significant progress in tools
UI design poised for radical changes New opportunities and challenges for tools
Research in tools necessarily trails innovation in UI design
Due to consolidation on desktop metaphor, significant progress in tools
UI design poised for radical changes New opportunities and challenges for tools
22
Past, Present and Future ofUser Interface Software ToolsPast, Present and Future ofUser Interface Software ToolsBrad A. MyersBrad A. Myers(work with Scott E. Hudson(work with Scott E. Hudsonand Randy Pausch)and Randy Pausch)
Human-Computer Interaction InstituteHuman-Computer Interaction InstituteSchool of Computer ScienceSchool of Computer ScienceCarnegie Mellon UniversityCarnegie Mellon University
http://www.cs.cmu.edu/~bamhttp://www.cs.cmu.edu/[email protected]@cs.cmu.edu
Talk given withSlideShow Commander
Result of the Pebbles research projecthttp://www.cs.cmu.edu/~pebbles
SlideShow Commander to be commercialized by:
Available soon!
Result of the Pebbles research projecthttp://www.cs.cmu.edu/~pebbles
SlideShow Commander to be commercialized by:
Available soon!
Synergy Solutions, Inc.Synergy Solutions, Inc.http://www.synsolutions.com/
Synergy Solutions, Inc.Synergy Solutions, Inc.http://www.synsolutions.com/
Brad Myers 23