Designing for Effective End-User Interaction with Machine ... Designing for Effective End-User...

download Designing for Effective End-User Interaction with Machine ... Designing for Effective End-User Interaction

of 143

  • date post

    30-Jun-2020
  • Category

    Documents

  • view

    0
  • download

    0

Embed Size (px)

Transcript of Designing for Effective End-User Interaction with Machine ... Designing for Effective End-User...

  • Designing for Effective End-User Interaction with Machine Learning

    Saleema Amershi

    A dissertation

    submitted in partial fulfillment of the

    requirements for the degree of

    Doctor of Philosophy

    University of Washington

    2012

    Reading Committee:

    James A. Fogarty, Chair

    Daniel S. Weld

    Desney S. Tan

    Program Authorized to Offer Degree:

    Computer Science & Engineering

  • University of Washington

    Abstract

    Designing for Effective End-User Interaction with Machine Learning

    Saleema Amershi

    Chair of the Supervisory Committee:

    Associate Professor James A. Fogarty

    Computer Science & Engineering

    End-user interactive machine learning is a promising tool for enhancing human capabilities with data.

    Recent work has shown that we can create specific applications that employ end-user interactive machine

    learning. However, we still lack a generalized understanding of how to design effective end-user

    interaction with machine learning. This dissertation advances our understanding of this problem by

    demonstrating effective end-user interaction with machine learning in a variety of new situations and by

    characterizing the design factors affecting the end-user interactive machine learning process itself.

    Specifically, this dissertation presents (1) new interaction techniques for end-user creation of image

    classifiers in an existing end-user interactive machine learning system called CueFlik, (2) a novel system

    called ReGroup that employs end-user interactive machine learning for the purpose of access control in

    social networks, (3) a novel system called CueT that supports end-user driven machine learning for

    computer network alarm triage, and (4) a novel design space characterizing the goals and constraints

    impacting the end-user interactive machine learning process itself. Together, these contributions can

    move us beyond ad-hoc designs for specific applications and provide a foundation for future researchers

    and developers of end-user interactive machine learning systems.

  • i

    TABLE OF CONTENTS

    LIST OF FIGURES ................................................................................................................................... iii

    LIST OF TABLES ..................................................................................................................................... vi

    Chapter 1: Introduction ............................................................................................................................ 1

    1.1 Motivation .................................................................................................................................... 1

    1.2 Objectives and Hypothesis ........................................................................................................... 2

    1.3 Summary of Contributions ........................................................................................................... 4

    1.4 Scope ............................................................................................................................................ 5

    1.5 Outline.......................................................................................................................................... 6

    Chapter 2: Related Work ......................................................................................................................... 8

    2.1 Interactive Machine Learning as a Tool in Related Fields ............................................................ 9

    2.1.1 Information Retrieval ...................................................................................................... 9

    2.1.2 Recommender Systems ................................................................................................. 12

    2.1.3 Context-Aware Computing ............................................................................................ 14

    2.1.4 Programming by Demonstration .................................................................................... 15

    2.1.5 Adaptive and Intelligent Systems .................................................................................. 16

    2.1.6 New Uses of End-User Interactive Machine Learning .................................................. 18

    2.2 Exploring How to Design End-User Driven Machine Learning ................................................. 20

    Chapter 3: A Design Space for End-User Interaction with Machine Learning ....................................... 25

    3.1 Design Factors ............................................................................................................................ 27

    3.1.1 Interaction Goals and Contexts ...................................................................................... 27

    3.1.2 Constraints..................................................................................................................... 31

    3.2 Design Dimensions .................................................................................................................... 35

    3.2.1 System Feedback ........................................................................................................... 35

    3.2.2 End-User Control .......................................................................................................... 37

    3.2.3 Temporal ....................................................................................................................... 41

    3.3 Discussion and Limitations ........................................................................................................ 43

    Chapter 4: Web Image Classification with CueFlik ............................................................................... 45

    4.1 State of the Art in End-User Driven Image Search and Classification........................................ 46

    4.2 CueFlik ....................................................................................................................................... 47

    4.3 Design Factors Affecting End-User Interaction with CueFlik .................................................... 49

    4.4 Illustrating the System’s Understanding While Soliciting Effective Training Examples ............ 50

    4.4.1 Overview-Based Example Selection.............................................................................. 52

    4.4.2 Evaluation ..................................................................................................................... 58

    4.4.3 Results ........................................................................................................................... 60

    4.4.4 Discussion and Future Work ......................................................................................... 64

    4.5 Examining Multiple Potential Models ........................................................................................ 65

  • ii

    4.5.1 Supporting Model Comparison and Revision ................................................................ 67

    4.5.2 Evaluation ..................................................................................................................... 68

    4.5.3 Results ........................................................................................................................... 69

    4.5.4 Discussion and Future Work ......................................................................................... 70

    4.6 Summary .................................................................................................................................... 71

    Chapter 5: Access Control in Social Networks with ReGroup ............................................................... 72

    5.1 State of the Art in Social Access Control ................................................................................... 73

    5.2 Design Factors Affecting End-User Interaction with ReGroup .................................................. 75

    5.3 ReGroup ..................................................................................................................................... 76

    5.3.1 Example Usage Scenario ............................................................................................... 76

    5.3.2 Identifying Features ....................................................................................................... 77

    5.3.3 Suggesting People while Preserving End-User Control ................................................. 78

    5.3.4 Indirectly Training an Effective Classifier ..................................................................... 79

    5.3.5 Minimizing Frustration Caused by Unlearnable Groups................................................ 80

    5.3.6 Decision-Theoretic Filter Suggestions ........................................................................... 81

    5.3.7 Gracefully Handling with Missing Data ........................................................................ 82

    5.3.8 Implementation Details ................................................................................................. 82

    5.4 Evaluation ..................................................