ICS3211 lecture 08
85
ICS3211 - Intelligent Interfaces II Combining design with technology for effective human-computer interaction Week 8 Department of Intelligent Computer Systems, University of Malta, 2016
-
Upload
vanessa-camilleri -
Category
Education
-
view
469 -
download
0
Transcript of ICS3211 lecture 08
ICS3211 - Intelligent
Interfaces IICombining design with technology for effective
human-computer interaction
Week 8Department of Intelligent Computer
Systems,University of Malta,
2016
Interface Design I
Week 8 overview:
• Recap
• Interaction Design Process
• Using Processing
• Advanced Interface Technology
Learning OutcomesAt the end of this session you should be able to:
• Explore programming for visual design prototyping;
• Draw inferences about designing for different interfaces;
• Compare and contrast the different interfaces for use on the same application/game;
• List the research issues/gaps in the design for AR/VR applications;
• Describe some of the current research projects in AR/VR.
Cover
Download
Exhibition
» ownload Processing
» Play With Examples» Browse Tutorials
»
xhibition
Reference Libraries
Tools Environment
Tutorials Examples
Books
overview People
Foun dation
Shop
» Forum» GitHub» Issues>• Wild» FAQ
>• Twitt er»
Processing is a program.ming language, development environmentand online community. Since 2001, Processing has promoted
software literacy within the visual arts and visual literacy within technology. Initially created to serve as a software sketchbook and to teach computer program.ming fundamentals within a
visual context, Processing evolved into a development tool for professionals. Today, there are tens of thousands of students,
artists, designers, researchers, and hobbyists who use Processing for learning, prototyping, and production.
,. Free to download and open source
,. Interactive programs with 2D, 3D or PDF output
,. OpenGL integration for accelerated 3D
,. For GNU/Linux, Mac OS X and Windows
,. over 100 libraries extend the core software
,. Well docum ente d, with many books available
Keyfliesby Miles Peyton
I: p
!
Petting Zooby Minimaforms
Fragmented Memory by Phillip Steams
Experience Prototyping
The experience of even simple artifacts does not exist in a vacuum but, rather, in dynamic relationship with other people, places and objects.
Additionally, the quality of people’s experience changes over time as it is influenced by variations in these multiple contextual factors.
MORE PROCESSINGHow to do prototyping using Processing
Processing - starting out • https://processing.org/tutorials/gettingstarte
d/
• Open Source
• Interactive programs with 2D, 3D or PDF output
• OpenGL integration for accelerated 2D and 3D
• For GNU/Linux, Mac OS X, and Windows
• Over 100 libraries extend the core software
Basic Parts of a Sketch
/* Notes comment */!//set up global variables! float moveX = 50;!!
//Initialize the Sketch!(){!void setup
}!!
//draw every frame! void draw(){!}!
Sample Drawing
int m = 0;! float s =
0;!!
void setup(){!size(512,512);!background(255);!
}!!
void draw (){! fill(255,0,0);
!ellipse(mouseX,mouseY,s,s);!
}!!
void mouseMoved(){!s = 40 + 20*sin(++m/10.0f);!
}!
Drawing
• draw() gets called as fast as possible, unless a frameRate is specified• stroke() sets color of drawing outline• fill() sets inside color of drawing• mousePressed is true if mouse is down• mouseX, mouseY - mouse position
!void draw() { !!stroke(255); !!if(mousePressed) {!
!line(mouseX, mouseY, pmouseX, pmouseY);!!}!
!!!}!
Processing and Drawing
• Basic Shapesrect(x, y, width, height)!
ellipse(x, y, width, height)! line(x1, y1, x2, y2), line(x1, y1, x2, y2, z1, z2)!
• Filling shapes - fill( )fill(int gray), fill(color color), fill(color color, int alpha)!
• Curve• Draws curved lines
• Vertex• Creates shapes (beginShape, endShape)
Vertex Demo
void setup(){!size(400,400);!
}!!void draw(){!
background(255);! fill(0);! beginShape();!
vertex(0,0);! vertex(400,400);! vertex(mouseX,mouseY);!endShape();!}!
Curve Demovoid setup(){!size(400,400);!
}!!void draw(){!background(255);!
fill(0);!!int xVal = mouseX*3-100;!
int yVal = mouseY*3-100;!!
curve(xVal, yVal, 100, 100, 100, 300, xVal, yVal);!curve(xVal, yVal, 100, 300, 300, 300, xVal, yVal);!curve(xVal, yVal, 300, 300, 300, 100, xVal, yVal);!curve(xVal, yVal, 300, 100, 100, 100, xVal, yVal);!
!}!
Class and Objects
• see http://processing.org/learning/objects/• Object
• grouping of multiple related properties and functions
• Objects are defined by Object classes• Eg Car object
• Data• colour, location, speed
• Functions• drive(), draw()
Classes
• four elements: name, data, constructor, and methods.• Name
class myName { }!
• Data• collection of class variables
• Constructor• run when object created
• Methods• class functions
Class Usage
// Step 1. Declare an object.!Car myCar;!!void setup() { !// Step 2. Initialize object.!myCar = new Car();!
!}!
on theobject. !
void draw() { !background(255); !// Step 3. Call methods
myCar.drive(); ! myCar.display(); !
}!
Constructing Objects
• One CarCar myCar= new Car(); !
• Two Cars
!
!!// Creating!Car myCar1!Car myCar2
two car objects= new= new
Car();Car(); !
• One car with initial values
Car myCar = new Car(color(255,0,0),0,100,2); !
Modifying Constructor
Car(color tempC, float tempXpos, float
{
tempYpos, float tempXspeed) !
c= tempC; !xpos ypos
=tempXpos;=tempYpos;
!!
xspeed =tempXspeed; !
}!
!
Mouse Interaction
mouseX, mouseY);!
• Mouse position• mouseX, mouseY variables
• Mouse Interaction• mousePressed()• mouseReleased()• mouseDragged()
• Add in own codevoid mouseDragged(){!
line(pmouseX,pmouseY,}!
Keyboard Interaction
• Check keyPressed variable in draw() method!void draw(){!
pressed " +key);!! !if(keyPressed){!! ! !print(" you! !}!}!
"+key);!
• Use keyPressed() method!void keyPressed(){!! !print(" you're pressing!}!
Importing Libraries
• Can add functionality by Importing Libraries
• java archives - .jar files
• Include import codeimport processing.opengl.*;!
• Popular Libraries• Minim - audio library• OCD - 3D camera views• Physics - physics engine• bluetoothDesktop - bluetooth networking
Graphical Controls
height);!
• Use ControlP5 Library• http://www.sojamo.de/libraries/controlP5/
• Add graphical controls• Buttons, sliders, etc• Support for OSC (Open Sound Controller)
• Use ControlP5 classimport controlP5.*;!
addButton(name, value, x, y, width,• Event Handing
Interface Elements
• Interfascia• http://www.superstable.net/interfascia/
• GUI Library for Processing• Buttons• Check boxes• Textfields• Progress bar
Role Playing
Interaction Design Process
(Re)design
Identify needs
Build an interactive
version
Evaluate
Final Product
When to evaluate?
• Once the product has been developed• pros : rapid development, small evaluation cost• cons : rectifying problems
• During design and development• pros : find and rectify problems early• cons : higher evaluation cost, longer development
design implementation evaluation
redesign &reimplementation
design implementation
Four evaluation paradigms
• Quick and dirty
• Usability testing (lab studies)
• Field studies
• Predictive evaluation
Characteristics of approaches
Usability testing
Field studies
Predictive
Users do task natural not involvedLocation controlled natural anywhereWhen prototype early prototypeData quantitative qualitative problemsFeed back measures &
errorsdescriptions problems
Type applied naturalistic expert
Evaluation approaches and methods
Method Usability testing
Field studies
Predictive
Observing x xAsking users
x x
Asking experts
x x
Testing xModeling x
STORYBOARD
INITIAL SKETCHES
Pros: • Good for idea genera,on
• Cheap • Concepts seem feasible
Cons: • Not great feedback gained • Photoshop not fast enough
for making changes
First Draft
POST IT PROTOTYPINGCamera View with 3D
Second Dra.• Selection
highlighted in blue
Third Dra.• Home button added for easy navigation
to main menu
POWERPOINT PROTOTYPINGBenefits• Used for User Testing• Interactive• Functionalities work when following the story of Scenario 1• Quick• Easy arrangement of slides
User Testing• Participants found• 15 minute sessions screen captured• ‘Talk Allowed’ technique used
• Notes taken
• Post-Interview
VIDEO PROTOTYPE
• Flexible tool for capturing the use of an interface
• Elaborate simulation of how navigational aid will work
• Does not need to be realistic in every detail
• Gives a good idea of how the finished system will work
ADVANCED INTERFACE TECHNOLOGY
Advanced Interface Technology
• Wearable Computers• Augmented Reality• Virtual Reality• Invisible Interfaces• Environment Sensing• Physiological Sensing
Major changes in computing
Wearable Computing
▪ Computer on the body that is:▪ Always on▪ Always accessible▪ Always connected
▪ Other attributes▪ Augmenting user actions▪ Aware of user and surroundings
Wearable Attributes
View Through Google Glass
Research Problems• Hardware
• Power, networking, display
• User Interaction• User input, speech, gesture, gaze, etc• Novel interaction methods
• Social Acceptance• Privacy, social factors
• Novel Applications• Collaboration• Intelligent assistance
AUGMENTED REALITY
Augmented Reality Definition
• Defining Characteristics• Combines Real and Virtual Images
• Both can be seen at the same time• Interactive in real-time• The virtual content can be interacted with
• Registered in 3D• Virtual objects appear fixed in space
Augmented Reality Examples
Typical Demo Application
https://www.youtube.com/watch?v=UOfN1plW_Hw
Research Problems
• Low level hardware/software• Head mounted displays• Tracking systems
• User Interaction• Gesture based interaction• Multimodal input (speech, gesture)
• Novel Applications• Face to face collaboration• Authoring tools
VIRTUAL REALITY
Virtual Reality
• Immersive VR• Head mounted display, gloves• Separation from the real world
Occulus Rift
• 360 degree head tracking• 100 degree field of view
Wearable Virtual Reality
• Samsung Gear VR• See virtual reality on your phone• Using phone display, compass
AR vs VR
Research Problems
• Low level• Wide area tracking• Development tools
• User Interaction• Intuitive input (gesture, controllers)• Avatar control and representation• Techniques for navigation/manipulation
• Novel Applications• Massive multi-user environments• Content capture and sharing
INVISIBLE INTERFACES
Early Examples
• Interaction without devices:• BodySpace [Strachan 2007]: Functions to body position• Abracadabra [Harrison 2007]: Magnets on finger tips• GesturePad [Rekimoto 2001]: Capacitive sensing in clothing
• Palm-based Interaction• Haptic Hand [Kohli 2005]: Using non-dominant hand in VR• Sixth Sense [Mistry 2009]: Projection on hand• Brainy Hand [Tamaki 2009]: Head worn projector/camera
Unobtrusive Input Devices
▪ GesturePad▪ Capacitive multilayered touchpads▪ Supports interactive clothing
ImaginaryPhone
• Gustafson, S., Holz, C., & Baudisch, P. [2011]
Invisible Interfaces – Gestures in Space
• Gustafson, S., Bierwirth, D., & Baudisch, P. [2010]
• Using a non-dominant hand stabilized interface.
Project Soli
• Using Radar to support free-hand spatial input
Project Soli
https://www.youtube.com/watch?v=0QNiZfSsPc0 https://www.youtube.com/watch?v=jWNebDDmuXc
Research Gaps
• Free-hand interfaces using relative input• Combining invisible interface + mobile device• Multimodal interaction• speech + gesture input
• Affordances and discoverability• Interaction frameworks
Research Problems
• Hardware• Power, networking, display
• User Interaction• User input, speech, gesture, gaze, etc• Novel interaction methods
• Social Acceptance• Privacy, social factors
• Novel Applications• Collaboration• Intelligent assistance
ENVIRONMENT SENSING
Environmental Sensor
• New sensors track and capture real environment• Navigation, 3D modeling, user tracking
• Depth Sensors• Microsoft Kinect, Intel RealSense
• Integrated Devices• Google Tango
Google Tango
• Tablet based system• Android OS
• Multiple sensors• RGBD Sensor• IR Structured light• Inertial sensors
• High end graphics• Nvidia tegra chip
Research Problems
• Content creation• Creating better 3D models• Segmenting objects
• User Interaction• Interaction with real world• Interacting with multiple devices
• Novel Applications• AR notes/real world tagging• Social networking
PHYSIOLOGICAL SENSING
Physiological Sensors
• Sensing user state• Body worn devices
• Multiple possible sensors• Physical activity• Eye tracking, gaze• Heart rate• GSR• Breathing• Etc
Tobii Eye Tracker
• Wearable eye tracking system• Natural data capture• Scene camera capture• Recording/streaming eye gaze, 60 Hz sampling
Research Problems
• User Interaction• Implicit vs. Explicit interaction• Measuring cognitive
• Social Acceptance• Privacy, social factors
• Novel Applications• Collaboration• Intelligent assistance
