Activity- -Based Serendipitous Recommendations Based Serendipitous - - PowerPoint PPT Presentation

▶
activity based serendipitous recommendations based
SMART_READER_LITE
LIVE PREVIEW

Activity- -Based Serendipitous Recommendations Based Serendipitous - - PowerPoint PPT Presentation

Oct 21, 2022 414 likes •722 views

Activity- -Based Serendipitous Recommendations Based Serendipitous Recommendations Activity with the Magitti Mobile Leisure Guide with the Magitti Mobile Leisure Guide System Codename: Magitti Designed and Prototyped by PARC for Dai Nippon

slide-1
SLIDE 1

System Codename: Magitti Designed and Prototyped by PARC for Dai Nippon Printing Co. Ltd.

Presenters

  • Victoria Bellotti
  • Bo Begole

The Other Co‐authors

Ed H. Chi, Nicolas Ducheneaut, Ellen Isaacs, Ji Fang, Tracy King, Mark W. Newman, Kurt Partridge, Bob Price, Paul Rasmussen, Michael Roberts, Diane J. Schiano, Alan Walendowski

Activity Activity-

  • Based Serendipitous Recommendations

Based Serendipitous Recommendations with the Magitti Mobile Leisure Guide with the Magitti Mobile Leisure Guide

slide-2
SLIDE 2

2

Overview

  • Background

and motivating fieldwork

  • System design
  • Evaluation

Recommendation Server

Consumer

Local Area

Context: Time, Location, etc. Restaurants, stores, events, etc. Mobile Device Preferences: Sushi, Bookstores, etc.

Filter and Rank Database Items Infer Activity

Feedback Feedback

Model Preferences

slide-3
SLIDE 3

3

About Dai Nippon Printing Co. Ltd.

  • DNP is a world leader in printing technology

and solutions

  • Affected by the shift from paper to

digital media

The Past: People carried magazines The Present: Most Japanese use a mobile phone to browse the Web and read/write E‐mail

  • DNP asked PARC to develop core technology

for new, consumer‐friendly digital media

  • All design to be driven by real need

motivated a lot of work to identify:

  • Best target users
  • Best solution for their needs

Traditional Publishing Modern Publishing

slide-4
SLIDE 4

4

Fieldwork 2 Confirm and Refine Fieldwork 2 Confirm and Refine

Evaluate design mock‐up in situ Refine design based

  • n user feedback

Fieldwork 1 Choose Best Idea Fieldwork 1 Choose Best Idea

Interviews,

  • bservations,

and scenario feedback Analyze results Refine concept design Future technology analysis

Finalized Concept Proposal Finalized Concept Proposal

Leisure guide concept proposal, “Magitti”

Contextual Publishing Concept Development

Technology Brainstorm Technology Brainstorm

Personas bring customer to life Share background domain info Brainstorm design ideas

Discover Target Users Discover Target Users

Assess many markets Develop scenarios and obtain feedback Choose the best

Young Adults at Leisure Activity-Aware Leisure Guide What to Build

slide-5
SLIDE 5

5

Details:

Persona Exploration

  • Activities
  • Develop day‐in‐the‐life scenarios (use resources such as magazines)
  • Role‐play day‐in‐the‐life (use props for realism and fun)
  • Goals
  • Develop empathy for end‐user customer
  • Find hidden needs
  • Foster creativity
slide-6
SLIDE 6

6

Details:

Brainstorm to Generate Ideas

  • Work as a team
  • No criticism
  • No ownership
  • Play off others’

ideas

  • Interpreters in mixed teams
  • Small teams and “choreography”

so everyone

  • Can reach board
  • Can make changes
  • Repeated process
  • First, issues and constraints
  • Then design ideas
  • Opportunities to present, reflect &

critique

  • With DNP we covered
  • Features and user experience
  • Business models
  • Competition
slide-7
SLIDE 7

7

Details:

Developing the Fruit of Our Labor

  • Clustering to combine many ideas
  • Take the 5 best
  • Develop scenarios for each one
  • Test with prospective user representatives in Tokyo
  • Choose best‐received
  • Develop mock‐up
  • Test again with user

representatives in Tokyo

  • Develop final concept
slide-8
SLIDE 8

8

Many User Studies During Concept Development and Early System Development

Observation Focus Groups Activity Sampling In-depth interviews Mobile-phone Diaries Notes Diary entries Location Survey responses 1000’s of Photos 40 Transcripts 10 Transcripts Time Time Fashion Identity Technology use Transportation Leisure activity type frequency Leisure activity venue types popularity Leisure activity type timing & probability 3000 activity & time reports 370 activity, time & location reports Planning Media use Information sources Information desired Social factors in leisure Knowledge

  • f locale

Observation reminders Practices Needs Priorities Surveys 670 Responses Problems Classifying Counting Coding Correlating Leisure activity type locations Coordination Activity type prediction Leisure activity types Form-factor Features Functions Interaction style Venue database classification Content

Study Methods Study Methods Informing Design of Informing Design of Data Data Analysis: Analysis: increasing abstraction

increasing abstraction

slide-9
SLIDE 9

9

From Fieldwork: Who Are the Users

  • Japanese youth are especially receptive to new technology
  • 19‐25 year‐olds spend 1.5 times more time in leisure activities

than 16‐19 year‐olds or 26‐33 year‐olds

  • Less school and work pressure
  • Ideal target for our design
  • Still very, very busy
  • School, jobs and little sleep
  • Relaxation is a priority
  • The system should do the work
  • Want to know what others think
  • Value opinions of real people
  • Include end‐user content
slide-10
SLIDE 10

10

From Fieldwork: What do they Do?

  • Outings often involve meeting friends
  • Often at “halfway point”

far from homes

  • Eager for local and localized info
  • Unfamiliar with locations they visit
  • Open to suggestions
  • May not plan the main activity
  • May not plan follow‐on activities
  • Motivation for Magitti
  • A city‐guide that assists in

exploration

10 20 30 40 50 60

1 2 3 4 5 6 7

1 = Not at All 7 = Extremely Well

Ratings of

“How well I know this neighborhood”

given by 170 young people stopped on the streets in diverse neighborhoods in Tokyo

slide-11
SLIDE 11

11

Overview

  • Background

and motivating fieldwork

  • System design
  • Evaluation

Recommendation Server

Consumer

Local Area

Context: Time, Location, etc. Restaurants, stores, events, etc. Mobile Device Preferences: Sushi, Bookstores, etc.

Filter and Rank Database Items Infer Activity Model Preferences

slide-12
SLIDE 12

12

User Interface

Map Pie Menu Details

slide-13
SLIDE 13

13

Akiko and Charles in “Any” Mode

Recommendations differ based on Personal Preferences

Akiko “Any” Mode Charles “Any” Mode

Magitti inferring “Eat”

slide-14
SLIDE 14

14

Implicit Interaction:

User Informs Magitti’s Modeling as a Side Effect of Purposeful Action

When user selects “Change Activity” to get more targeted recommendations, Magitti uses that selection to improve its model of user preferences

slide-15
SLIDE 15

15

Demo Video

slide-16
SLIDE 16

Recommendable Items Restaurant Reviews Store Descriptions Parks Descriptions Movie Listings Museum Events Magazine Articles … Recommendable Items Restaurant Reviews Store Descriptions Parks Descriptions Movie Listings Museum Events Magazine Articles …

EAT Straits Cafe 0.77 EAT Fuki Sushi 0.64 SEE

  • J. Gallery

0.60 EAT Tamarine 0.57 DO Sam’s Salsa 0.39 EAT Bistro Elan 0.38 BUY Apple Store 0.33 EAT Spalti 0.31

Filtering and Ranking Filtering and Ranking Activity Utility Information

slide-17
SLIDE 17

Recommendable Items Restaurant Reviews Store Descriptions Parks Descriptions Movie Listings Museum Events Magazine Articles … Recommendable Items Restaurant Reviews Store Descriptions Parks Descriptions Movie Listings Museum Events Magazine Articles …

EAT Straits Cafe 0.77 EAT Fuki Sushi 0.64 SEE

  • J. Gallery

0.60 EAT Tamarine 0.57 DO Sam’s Salsa 0.39 EAT Bistro Elan 0.38 BUY Apple Store 0.33 EAT Spalti 0.31

Filtering and Ranking Filtering and Ranking Activity Utility Information

Context Context

  • Time

Time

  • Location

Location

  • Email analysis

Email analysis

  • Calendar analysis

Calendar analysis History History

  • Prior population

Prior population patterns patterns

  • User Queries

User Queries

  • User Locations

User Locations

Eat Buy See Do Read

What you are doing now

slide-18
SLIDE 18

Recommendable Items Restaurant Reviews Store Descriptions Parks Descriptions Movie Listings Museum Events Magazine Articles … Recommendable Items Restaurant Reviews Store Descriptions Parks Descriptions Movie Listings Museum Events Magazine Articles …

EAT Straits Cafe 0.77 EAT Fuki Sushi 0.64 SEE

  • J. Gallery

0.60 EAT Tamarine 0.57 DO Sam’s Salsa 0.39 EAT Bistro Elan 0.38 BUY Apple Store 0.33 EAT Spalti 0.31

Filtering and Ranking Filtering and Ranking Activity Utility Information What you like What you like

Personal Preferences Personal Preferences

  • Explicit preferences

Explicit preferences

  • Rating of items inspected

Rating of items inspected

  • Analysis of content read

Analysis of content read

  • Behavior; where/when/what

Behavior; where/when/what Context Context

  • Time

Time

  • Location

Location

  • Email analysis

Email analysis

  • Calendar analysis

Calendar analysis History History

  • Prior population

Prior population patterns patterns

  • User Queries

User Queries

  • User Locations

User Locations

Eat Buy See Do Read

What you are doing now

slide-19
SLIDE 19

Recommendable Items Restaurant Reviews Store Descriptions Parks Descriptions Movie Listings Museum Events Magazine Articles … Recommendable Items Restaurant Reviews Store Descriptions Parks Descriptions Movie Listings Museum Events Magazine Articles …

EAT Straits Cafe 0.77 EAT Fuki Sushi 0.64 SEE

  • J. Gallery

0.60 EAT Tamarine 0.57 DO Sam’s Salsa 0.39 EAT Bistro Elan 0.38 BUY Apple Store 0.33 EAT Spalti 0.31

Filtering and Ranking Filtering and Ranking Activity Utility Information What you like What you like

Personal Preferences Personal Preferences

  • Explicit preferences

Explicit preferences

  • Rating of items inspected

Rating of items inspected

  • Analysis of content read

Analysis of content read

  • Behavior; where/when/what

Behavior; where/when/what Context Context

  • Time

Time

  • Location

Location

  • Email analysis

Email analysis

  • Calendar analysis

Calendar analysis History History

  • Prior population

Prior population patterns patterns

  • User Queries

User Queries

  • User Locations

User Locations

Eat Buy See Do Read

What you are doing now

slide-20
SLIDE 20

20

Predicting Activities from Population Priors

Mobile‐phone Diaries

Hourly activity report:

  • Who
  • Where
  • When
  • What
  • Info used & desired

Code each respondent’s activities over 7‐day week

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 6:00 7:30 9:00 10:30 12:00 13:30 15:00 16:30 18:00 19:30 21:00 22:30 0:00 Time of Day 5 10 15 20 S am p le C
  • u
n t (T
  • tal)
NOT SEE DO EAT OUT SHOP

Friday

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 6:00 7:30 9:00 10:30 12:00 13:30 15:00 16:30 18:00 19:30 21:00 22:30 0:00 Time of Day 5 10 15 20 S am p le C
  • u
n t (T
  • tal)
NOT SEE DO EAT OUT SHOP

Sunday

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 6:00 7:30 9:00 10:30 12:00 13:30 15:00 16:30 18:00 19:30 21:00 22:30 0:00 Time of Day 5 10 15 20 25 S am p le C
  • u
n t (T
  • tal)
NOT SEE DO EAT OUT SHOP

Saturday

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 6:00 7:30 9:00 10:30 12:00 13:30 15:00 16:30 18:00 19:30 21:00 22:30 0:00 Time of Day 10 20 30 40 50 60 70 80 S am p le C
  • u
n t (T
  • tal)
NOT SEE DO EAT OUT SHOP

Mon‐Thu

Predict

probability

  • f each

activity type

Aggregate all data

When there is no user When there is no user‐ ‐specific specific data, prior population data is used data, prior population data is used

slide-21
SLIDE 21

21

Predicting Activities from Email/SMS

  • How well do messages suggest activity?
  • We examined a public set of 10,000 SMS messages from National University
  • f Singapore students, similar to the Magitti target demographic
  • Approximately 11% of the messages contain information related to

leisure activities tomorrow what time you be in school? think me and shuhui meeting in school around 4. then duno still can see movie or not because duno if a rest want meet for dinner.

  • Keywords and linguistic structures are identified and sent to the activity

inference mechanism

ACTCAT=MOVIE, EAT :: ACTTIME=2007/05/26 16:00 :: UNCERTAINTY=10 minutes :: TENSE=FUTURE

slide-22
SLIDE 22

22

Learning Individual Patterns

Date/ Tim e Location Address Venue Type Activity Class

Sun, 2 7 Jan 2 0 0 8 1 1 :5 7 - 1 2 :4 5 3 7 °2 6 ’3 9 ”
  • 1 2 2°9 ’3 8 ”
3 8 9 Ram ona, Palo Alto Restaurant EAT Tue, 2 9 Jan 2 0 0 8 1 :2 2 - 1 :3 1 3 7 °2 3 ’1 1 ”
  • 1 2 2°9 ’0 2 ”
5 4 5 Ham ilton, Palo Alto Cafe EAT W ed, 3 0 Jan 2 0 0 8 1 1 :5 7 - 1 2 :4 5 3 7 °2 6 ’3 9 ”
  • 1 2 2°9 ’1 8 ”
1 4 3 Quarry Road, Palo Alto W algreens SHOP Fri, 1 Feb 2 0 0 8 1 3 :1 1 - 1 3 :3 7 3 7 °2 4 ’1 1 ”
  • 1 2 2°9 ’0 0 ”
8 5 4 University, Palo Alto Restoration Hardw are SHOP

… … … … …

2 4 6 8 10 12 14 16 18 20 22 24

Shopping Center

2 4 6 8 10 12 14 16 18 20 22 24

Downtown

EAT Most Likely SHOP Most Likely Unknown

Time

slide-23
SLIDE 23

50% 50%

Venue Likelihood: 1:00

Monday Tuesda

… 1 2 :0 0 to 1 :0 0 1 :0 0 to

Hector’s Cafe Astrid’s Grocery 12:00

Tim e

Location Visit

1 1 :5 7 - 1 2 :4 5 3 7 °2 6 ’3 9 ”

  • 1 2 2°9 ’3 8 ”

1 :2 2 - 1 :3 1 3 7 °2 3 ’1 1 ”

  • 1 2 2°9 ’0 2 ”

… … …

Context History Context History Weekly Behavior Patterns Weekly Behavior Patterns

$ $$ Grocery Cafe

… … …

$ $$ Grocery Cafe

… …

Predicting Activities from Learned User Patterns

BUY EAT

slide-24
SLIDE 24

24

Activity Inference Evaluation

Magitti Accuracy on Palo Alto Field Evaluation Data 62% 77% 82% 0% 20% 40% 60% 80% 100% Baseline (EAT) Time and Place Priors Priors + Learning

slide-25
SLIDE 25

25

Overview

  • Background

and motivating fieldwork

  • System design
  • Evaluation

Recommendation Server

Consumer

Local Area

Context: Time, Location, etc. Restaurants, stores, events, etc. Mobile Device Preferences: Sushi, Bookstores, etc.

Filter and Rank Database Items Infer Activity Model Preferences

slide-26
SLIDE 26

26

Preliminary Field Evaluation

  • 11 people, 32 outings (2.9 per person)
  • Shadowed one outing per participant
  • 60 places visited (1.9 per outing)
  • 30 restaurants, 27 shops, 3 parks
  • 16 outings accompanied by companion(s)

Using Magitti in a demo

slide-27
SLIDE 27

27

Overall Usefulness

  • Usefulness
  • Average of 35.0 recommendation list pages viewed per outing
  • People rated “helpfulness”

4.1 on 5‐point scale (5 high)

  • "Cool! I like that. I would never have found that place if it wasn't for

this.”

  • "It makes life more interesting. It allows you to get out of your daily

routine, almost as if you’re going to a different city.”

  • Serendipitous Discovery
  • 53% of places visited were new to the participants
  • On 67% of outings they went to at least one new place
  • On 69% of outings, they noticed another new place to visit later
slide-28
SLIDE 28

28

User Response

  • Predicting User Activity
  • People changed activity 5.1 times per outing
  • “It’s very nice that it recommends things without you

having to do anything, but sometimes you want to ask for specific things.”

  • Even when Magitti got it right, they still sometimes

switched, apparently because they wanted all the recommendations to be for that activity

  • Social Use
  • Five of eight users reported difficulty in sharing

experience with another person

  • Magitti user seen as disconnected from others and/or

controlling the outing

slide-29
SLIDE 29

29

Quality of Recommendations

  • Recommendations rated 3.8 on 1‐5 scale of "relevant

and of interest“

  • "Most of the time, the list contained a mix of useful

and not so useful recommendations“

  • Biggest factors to reduce confidence in

recommendations

  • Not seeing a nearby place in the list
  • Getting recommendations for places too far away
  • Lack of transparency of reasons for recommendations
slide-30
SLIDE 30

30

  • Information and

suggestions based on

  • Situation
  • Past behavior
  • Personal preferences

Replace Tedious Mobile Searching with Personalized Recommendations

Stop searching! Let information find you!

Magitti Team

Victoria Bellotti, Bo Begole, Ed H. Chi, Nicolas Ducheneaut, Ji Fang, Ellen Isaacs, Tracy King, Mark W. Newman, Kurt Partridge, Bob Price, Paul Rasmussen, Michael Roberts, Diane J. Schiano, Alan Walendowski