Fast And Robust Interface Generation for Ubiquitous Applications - - PowerPoint PPT Presentation

▶
fast and robust interface generation for ubiquitous
SMART_READER_LITE
LIVE PREVIEW

Fast And Robust Interface Generation for Ubiquitous Applications - - PowerPoint PPT Presentation

Jan 23, 2024 197 likes •778 views

Fast And Robust Interface Generation for Ubiquitous Applications The S UPPLE Project University of Washington, Seattle Krzysztof Gajos, David Christianson, Raphael Hoffmann, Tal Shaked, Kiera Henning, Jing Jing Long, and Daniel S. Weld

slide-1
SLIDE 1

Fast And Robust Interface Generation for Ubiquitous Applications

The SUPPLE Project University of Washington, Seattle Krzysztof Gajos, David Christianson, Raphael Hoffmann, Tal Shaked, Kiera Henning, Jing Jing Long, and Daniel S. Weld

slide-2
SLIDE 2

Fast And Robust Interface Generation for Ubiquitous Applications

Automatic, On The Fly

The SUPPLE Project University of Washington, Seattle Krzysztof Gajos, David Christianson, Raphael Hoffmann, Tal Shaked, Kiera Henning, Jing Jing Long, and Daniel S. Weld

slide-3
SLIDE 3

Motivation

slide-4
SLIDE 4

Motivation

  • Variety of display devices & interaction contexts

makes hand-designed interfaces expensive ⇒ Adapt to device characteristics

  • Current interfaces: complex & “One size fits all”

⇒ Adapt to users and tasks ⇒ Automatic interface generation is a scalable solution

slide-5
SLIDE 5

Approach

  • Develop abstract representation for:
  • Interfaces
  • Display devices
  • Users
  • Automatically generate interfaces from the

abstractions

slide-6
SLIDE 6

SUPPLE Architecture

Display Device Model Target Device SUPPLE Interface Model Application

  • r

Appliance User Model User's Info Space

slide-7
SLIDE 7

SUPPLE Architecture

Display Device Model Target Device SUPPLE Interface Model Application

  • r

Appliance User Model User's Info Space

slide-8
SLIDE 8

Road Map

Motivation Modeling user interfaces in SUPPLE User interface generation as optimization Automatically adapting user interfaces A preliminary user study Adaptation in SUPPLE Customization support in SUPPLE Conclusions

slide-9
SLIDE 9
  • simple types:
  • media types:
  • containers:
  • derivative types:
  • vectors:
  • actions: τ1 → τ2

τ, Cτ

Modeling User Interfaces

vector(τ) int|f loat|string|bool {τi

i∈1...n}

image|interactiveMap

slide-10
SLIDE 10

Modeling User Interfaces

Light Level: τ: <int, [0,10]> Power: τ: bool Light: τ: { , } Light Bank: τ: { , , } Light ... Light ... A/V: τ: { , } Projector: τ: { , } Classroom: τ: { , , } Input: τ: <string, {data1,data2, video}> Vent: τ : <int, [0,3]> Power: τ : bool Screen: τ: bool

slide-11
SLIDE 11

Modeling User Interfaces

Light Level: τ: <int, [0,10]> Power: τ: bool Light: τ: { , } Light Bank: τ: { , , } Light ... Light ... A/V: τ: { , } Projector: τ: { , } Classroom: τ: { , , } Input: τ: <string, {data1,data2, video}> Vent: τ : <int, [0,3]> Power: τ : bool Screen: τ: bool

slide-12
SLIDE 12

Light Level: : <int, [0,10]> Power: : bool Light: : { , } Light Bank: : { , , } Light ... Light ... A/V: : { , } Projector: : { , } Classroom: : { , , } Input: : <string, {data1,data2, video}> Vent: : <int, [0,3]> Power: : bool Screen: : bool

Modeling User Interfaces

slide-13
SLIDE 13

Model (UI, Device, User) - Algorithm - Results - Personalization (Adaptation, Customization) - Arnauld - Consistency - Task Models

Click!

slide-14
SLIDE 14

Model (UI, Device, User) - Algorithm - Results - Personalization (Adaptation, Customization) - Arnauld - Consistency - Task Models

slide-15
SLIDE 15

Model (UI, Device, User) - Algorithm - Results - Personalization (Adaptation, Customization) - Arnauld - Consistency - Task Models

Click!

slide-16
SLIDE 16

Model (UI, Device, User) - Algorithm - Results - Personalization (Adaptation, Customization) - Arnauld - Consistency - Task Models

slide-17
SLIDE 17

Media Types

slide-18
SLIDE 18

Subtyping

Click!

slide-19
SLIDE 19
slide-20
SLIDE 20
slide-21
SLIDE 21

Road Map

Motivation Modeling user interfaces in SUPPLE User interface generation as optimization Automatically adapting user interfaces A preliminary user study Adaptation in SUPPLE Customization support in SUPPLE Conclusions

slide-22
SLIDE 22

Model (UI, Device, User) - Algorithm - Results - Personalization (Adaptation, Customization) - Arnauld - Consistency - Task Models

  • Driven by a Cost function ($) -- estimated

user effort to manipulate a rendering of the interface

  • Cost function derived from device model
  • Algorithm: branch-and-bound search with

full constraint propagation at each step

  • Performance: 0.2 - 2.0 seconds on a

desktop computer

User Interface Generation as Optimization

slide-23
SLIDE 23

User Interface Generation as Optimization

  • Flexible with respect to screen size
  • Versatile: Same algorithm for different

devices

  • Allows new concerns to be included in the

rendering process, e.g.:

  • Cross-device consistency
  • Adaptation to usage patterns
slide-24
SLIDE 24

Robustly Adapting to Different Screen Sizes

slide-25
SLIDE 25

Single Algorithm -- Many Devices

slide-26
SLIDE 26

Adapting To Usage Patterns

SUPPLE with an empty trace SUPPLE with a

“lights-heavy” trace

slide-27
SLIDE 27

Road Map

Motivation Modeling user interfaces in SUPPLE User interface generation as optimization Automatically adapting user interfaces A preliminary user study Adaptation in SUPPLE Customization support in SUPPLE Conclusions

slide-28
SLIDE 28

Adapting To Usage Patterns By Complete Makeover

SUPPLE with an empty trace SUPPLE with a

“lights-heavy” trace

Is this the only way to adapt?

slide-29
SLIDE 29

Content Eliding

slide-30
SLIDE 30

Visual Popout

slide-31
SLIDE 31

Split Interfaces

slide-32
SLIDE 32

Adaptation Strategies

Efficiency Confusion Efficiency Confusion

Complete Makeover

slide-33
SLIDE 33

Adaptation Strategies

Efficiency Confusion

Complete Makeover Content eliding

slide-34
SLIDE 34

Adaptation Strategies

Efficiency Confusion

Visual Popout Complete Makeover Content eliding

slide-35
SLIDE 35

Adaptation Strategies

Efficiency Confusion

Split Visual Popout Complete Makeover Content eliding

slide-36
SLIDE 36

Adaptation Strategies

Efficiency Confusion

Split Visual Popout Complete Makeover Content eliding

slide-37
SLIDE 37

Preliminary User Study

Split Interface Visual Popout Interface

slide-38
SLIDE 38

Preliminary User Study Results

  • Split Interface strongly preferred over

non-adaptive

  • Both adaptive interfaces slightly faster than

non-adaptive

  • But: Visual Popout interface often found

distracting

slide-39
SLIDE 39

Sneak Peak: Followup Study

  • In a different setting, we tested three

adaptation strategies including Split Interface and Visual Popout Interface

  • Users strongly preferred and were

significantly faster using Split Interface

  • Users strongly disliked

Visual Popout Interface with Mary Czerwinski and Desney Tan

slide-40
SLIDE 40

Split Interfaces in SUPPLE

  • Adapting by promoting hard-to-reach but

frequently used functionality

slide-41
SLIDE 41

Click!

Split Interfaces in SUPPLE

slide-42
SLIDE 42

Click! Click!

Split Interfaces in SUPPLE

slide-43
SLIDE 43

Click!

Split Interfaces in SUPPLE

slide-44
SLIDE 44

Split Interfaces in SUPPLE

Four extra clicks required just to print in landscape mode!

slide-45
SLIDE 45

Split Interfaces in SUPPLE

slide-46
SLIDE 46

System-

  • Vs. User-Initiated

Adaptation: Customization

  • Customizing any part of the interface with

drag and drop

  • Out of order undo
  • Generalization
slide-47
SLIDE 47

Customization

slide-48
SLIDE 48

Customization

slide-49
SLIDE 49

Customization

slide-50
SLIDE 50

Customization

slide-51
SLIDE 51

Customization

slide-52
SLIDE 52

Road Map

Motivation Modeling user interfaces in SUPPLE User interface generation as optimization Automatically adapting user interfaces A preliminary user study Adaptation in SUPPLE Customization support in SUPPLE Conclusions

slide-53
SLIDE 53

Conclusions

  • A powerful, flexible and practical tool for

automatically generating user interfaces for multiple devices

  • Adaptation and customization offer the possibility
  • f creating custom-made UIs for each user
  • Support for distributed operation and caching

makes it practical even on small devices

slide-54
SLIDE 54

Can I Have It?

  • We are awaiting permission to release

SUPPLE as an open source toolkit

  • Visit SUPPLE web site to sign up for the

user’s mailing list

Yes!

slide-55
SLIDE 55

Contributors

Daniel Weld Raphael Hoffmann Kiera Henning Jing Jing Long me Anthony Wu Dave Christianson Tal Shaked

slide-56
SLIDE 56

More Information

  • SUPPLE:

http://www.cs.washington.edu/ai/supple/ : supple interfaces

  • Krzysztof Gajos:

kgajos@cs.washington.edu

  • Daniel Weld:

weld@cs.washington.edu

slide-57
SLIDE 57
  • SUPPLE:

http://www.cs.washington.edu/ai/supple/ : supple interfaces

  • Krzysztof Gajos:

kgajos@cs.washington.edu

  • Daniel Weld:

weld@cs.washington.edu

More Information