CS-5630 / CS-6630 Visualization for Data Science Design and - - PowerPoint PPT Presentation

CS-5630 / CS-6630 Visualization for Data Science Design and Evaluation of Visualizations

Alexander Lex alex@sci.utah.edu

Tasks & Design

Problem-Driven vs Technique- Driven

problem-driven

top-down approach identify a problem encountered by users design a solution to help users work more effectively sometimes called a design study

technique-driven

bottom-up approach invent new visualization techniques or algorithms classify or compare against other idioms and algorithms

Purpose of the Nested Model

capture design decisions

what is the justification behind your design?

analyze aspects of the design process

broken apart into four different concerns

validate early & often

avoid making ineffective solutions

Nested Model for Visualization Design

Munzner 2009

Design Threats & Evaluation

Design Process

Understand 
 Domain Problem Map to 
 Abstract Task Identify Suitable
 Technique Data Type & Other Factors

Domain 
 Characterization

details of an application domain group of users, target domain, their questions, & their data

varies wildly by domain must be specific enough to continue with

cannot just ask people what they do

introspection is hard!

Domain Problem Characterization

Infinite numbers of domain tasks Can be broken down into simpler abstract tasks We know how to address the abstract tasks! Identify task - data combination: solutions probably exist

Example: Find Good Movies

I want to identify good movies in genres I like. Domain: general population, movie enthusiasts

Data & Task 
 Abstraction

the what-why, map into generalized terms identify tasks that users wish to perform

• r already do

find data types and good model of the data

sometimes must transform the data for a better solution

this can be varied and guided by the specific task

Example: Find Good Movies

What is a good movie for me?

Highly rated by critics? Highly rated by audiences? Successful at the box office? Similar to movies I liked? Specific Genres?

Data Sources: IMDB, Rotten Tomatoes, …

Encodings & 
 Interactions

the design of idioms that specify an approach

visual encodings interactions

ways to create and manipulate the visual representation of data decisions on these may be separate or intertwined visualization design principles drive decisions

Example: Find Good Movies

Combination of audience ratings and critics ratings, filtered by genre. Idiom: stacked bar chart for ratings filter interface for genre

Example

Goal: Control Data Quality for Gene Splicing Data Tasks: Judge Magnitude of sample Compare samples Compare groups

[Strobelt 2016]

Tasks

Analyze

high-level choices consume vs produce

Search

find a known/unknown item

Query

find out about characteristics of item by itself or relative to others

High-level actions: Analyze

Consume discover vs present

classic split: explore vs explain

enjoy: casual, social Produce Annotate, record Derive: crucial design choice

Analyze Consume

Present Enjoy Discover

Produce

Annotate Record Derive

tag

Mid-level actions: search, query

Search: what does user know?

target, location

how much of the data matters?

• ne, some, all

Search Query Identify Compare Summarize

Target known Target unknown Location known Location unknown

Lookup Locate Browse Explore

Example Compare (& Derive)

Low Level: Targets

Trends ALL DATA Outliers Features ATTRIBUTES One Many

Distribution Dependency Correlation Similarity Extremes

NETWORK DATA SPATIAL DATA Shape Topology

Paths

Examples

Trends: How did the job market develop since the recession overall? Outliers: Looking at real estate related jobs

Exercise: Task Abstraction

Your have been approached by a geneticists to help with a visualization problem. She has gene expression data (data that measures the activity of the genes) for 30 cancer tissue samples. She is applying an experimental drug to see whether the cancer tissue dies as she hopes, but she finds that only some samples show the desired effect. She believes that the difference between the samples is caused by differential expression (different activity) of genes in a particular pathway, i.e., an interaction network of genes. She would like to understand which genes are likely to cause the difference, and what role they play in that pathway. Objective 1: Task Abstraction Objective 2: Encoding Design

Task Abstraction

…only some samples show the desired effect.

• > derive two groups of samples

… the difference between the samples is caused by differential expression (different activity) of genes in a particular pathway. She would like to understand which genes are likely to cause the difference

• > identify those genes
• > compare gene expression of pathway genes between two groups
• > identify the outliers

Task Abstraction

She would like to understand which genes are likely to cause the difference, and what role they play in that pathway.

• > Locate the outlier in the network
• > Explore the topology

Encoding Design

Tabular Data, 30 samples, 30 genes Compare groups, spot outliers

Dimensionality Reduction? Scatterplot Matrices? Parallel Coordinates? Heat Maps? Bar Charts?

Doesn’t show raw data, 
 not great to compare groups. 30 Dimensions is too much -> Scalability 30 Dimensions is a lot, 
 coloring for comparison necessary Work! Spatial separation of groups. Work even better! 30x30 still feasible, 
 encoding advantage

Encoding Design

Network, 30 genes Explore Topology, Lookup Nodes

Matrix? Treemap? Node-Link Diagram?

Doesn’t work for topology tasks Doesn’t work for general networks Works well. 
 Combine with Table through highlighting.

Designing Visualizations

What is Design?

creating something new to solve a problem can be used to make buildings, chairs, user interfaces, etc. design is used in many fields many possible users or tasks

https://www.youtube.com/watch?v=hUhisi2FBuw

What is Design Not?

just making things pretty art – appreciation of beauty or emotions invoked something without a clear purpose building without justification or evidence

http://woodyart211.blogspot.com/2015/01/art-vs-design-comments.html

Form & Function

commonly: “form follows function” function can constrain possible forms

form depends on tasks that must be achieved

“the better defined the goals of an artifact, the narrower the variety

• f forms it can adopt” –Alberto Cairo

http://img.weburbanist.com/wp-content/uploads/2015/05/sculptural-furniture- main-960x481.jpg The Functional Art: An introduction to information graphics and visualization. New Riders, 2012.

Why does Design Matter for Vis?

many ineffective visualization combinations users with unique problems & data variations of tasks large design space

Why does Design Matter for Vis?

many ineffective visualization combinations users with unique problems & data variations of tasks large design space

When do we Design?

wicked problems

no clear problem definition solutions are either good enough or not good enough multiple solutions exist, not true/false no clear point to stop with a solution

examples of non-wicked (“tame”) problems

mathematics, chess, puzzles

Dilemmas in a general theory of planning. Rittel, H.W. and Webber, M.M., Policy Sciences, 1973.

Tacoma Narrows Bridge

Design Methods

Creativity Workshops

goals:

generate design requirements promote creativity

combined a variety of techniques:

wishful thinking constraint removal excursion analogical reasoning storyboarding

measured prototypes for appropriateness, novelty, & surprise

Ethan Kerzner, Sarah Goodwin, Jason Dykes, Sara Jones, Miriah Meyer A Framework for Creative Visualization-Opportunities Workshops IEEE Transactions on Visualization and Computer Graphics (InfoVis '18), to appear, 2018.

http://vdl.sci.utah.edu/CVOWorkshops/

serial parallel

Parallel Prototyping

Develop multiple designs in parallel Example: graphic web design

serial vs parallel design: create & critique

Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. Dow, S.P., Glassco, A., Kass, J., Schwarz, M., Schwartz, D.L. and Klemmer, S.R., Design Thinking Research, 2012.

Paper Prototyping

“create a paper-based simulation of an interface to test interaction with a user” received more suggestions than digital users requested more features to add hypothesis that paper prototyping stimulates creativity and interaction

Human-centered approaches in geovisualization design: Investigating multiple methods through a long-term case study. Lloyd, D. and Dykes, J., IEEE InfoVis, 2011. Methods to support human-centred design. Maguire, M., International Journal of Human-Computer Studies, 2001.

Five-Design Sheets

tailored to visualization design

in industry and classroom use sketching as a way to plan

the design sheets: #1 brainstorm solutions to a task #2-4 different principle designs #5 converge on design to implement http://fds.design/

Sketching designs using the Five Design-Sheet

• methodology. Roberts, J.C., Headleand, C. and Ritsos,

P.D., IEEE InfoVis, 2015.

VizIt Cards

different cards to assist with visualization design types of cards

domain inspiration abstract layout

aim to help students design, compare, collaborate, apply, and synthesize http://vizitcards.org

VizIt Cards: A card-based toolkit for infovis design education. He, S. and Adar, E., IEEE InfoVis, 2016.

Other Methods

interviews/observations qualitative analysis personas data sketches coding

Evaluation

Role of Evaluation / Validation

Goals:

avoid ineffective solutions justify solutions

Dimensions:

Perception vs System Is size a better visual channel than angle? Is my visualization system any good?

Unique vs Comparison Can I easily compare my vis to

• thers?

Is mine one of a kind? Usability Testing: Check for problems with system

Example: Three Linking
 Techniques

Perception / Comparison

Context-Preserving Visual Links Straight Visual Links Frame-Based Highlighting

Results

H1: Visual links lead to a better performance (are faster) than 
 conventional highlights. H2: Context-preserving visual links do not have a 
 negative impact on correctness

Average Search Time

1500 3000 4500 6000

H L CL

Average Misestimation

0.033 0.065 0.098 0.13

H L CL

Gaze Plots

Context-Preserving Visual Links Straight Visual Links Frame-Based Highlighting

Bipolar Daughter Father Obese

Example: Genealogies + Clinical Data

System / Unique Evaluation: Case Study, demonstrate usefulness for scientist

Genealogy with ~400 members rendered with Progeny

What evaluation methods are there?

Controlled experiment

Laboratory, Crowd-Sourced

Interviews / questionnaires

Unstructured, structured, semi-structured

Field observation, lab observation

Video / audio analysis Coding / classification of user behavior (speech, gestures)

Log analysis Algorithmic performance measurement

[Lam 2011]

What evaluation methods are there?

Heuristic evaluation

Judge compliance with recognized metrics/usability methods (the heuristics)

Usability testing, e.g., thinking aloud tests Wizard of Oz

Human simulates response of system Test functionality before it’s implemented

Eye tracker evaluation Expert evaluation Insight-based evaluation Case studies

Typical Metrics

Objective Metrics Task completion time Errors (number, percent,…) Percent of task completed Ratio of successes to failures Number of repetitions Number of commands used Number of failed commands Physiological data (heart rate,…) Numbers of insights …

Subjective Metrics Ratings Rankings User satisfaction Subjective performance Ease of use Intuitiveness Judgments Comments and Feedback …

Quantitative vs. Qualitative Evaluation

Quantitative Methods

Objective metrics, measurements Use numbers / statistics for interpreting data

Qualitative Methods

Subjective metrics Description of situations, events, people, interactions, and observed behaviors, the use

• f direct quotations from people about their experiences, attitudes, beliefs, and thoughts

Focused on understanding how people make meaning of and experience their environment or world

Internal vs. External Validity

Internal Validity

High when tested under controlled lab conditions Observed effects are due to the test conditions 
 (and not random variables)

External Validity

High when interface is tested in the field, e.g. handheld device tested in museum Results are valid in real world

The Trade-off

The more akin to real-world situations, the more the experiment is susceptible to uncontrolled sources of variation

Scope of Evaluation

Pre-design

• e. g., to understand potential users’ work environment and workflow

Design

e.g., to scope a visual encoding and interaction design space based

• n human perception and cognition

Prototype

• e. g., to see if a visualization has achieved its design goals, to see

how a prototype compares with the current state-of-the-art systems

• r techniques

Deployment

e.g., to see how a visualization influences workflow and work processes, to assess the visualization’s effectiveness and uses in the field

Re-design

• e. g., to improve a current design by identifying usability problems

Lam 2011

Added value should be obvious!

Develop new methods/interface/software that are so awesome, cool, impressive, compelling, fascinating, and exciting that reviewers, colleagues, users are totally convinced just by looking at your work and some examples. — Jarke van Wijk, 
 Capstone Talk @ IEEE VIS 2013

More on this Topic

CS 6540 - HCI (Fall) CS 6963 - Advanced HCI (Spring) ED PS 6010 - Intro Statistics and Research Design DES 5710 - Product Design and Development ANTH 6169 - Ethnographic Methods ED PS 6030 - Introduction to Research Design