Italicizing Optional Instructions on Mobile Online Surveys Improves - - PowerPoint PPT Presentation

▶
italicizing optional instructions on mobile online
SMART_READER_LITE
LIVE PREVIEW

Italicizing Optional Instructions on Mobile Online Surveys Improves - - PowerPoint PPT Presentation

Jan 24, 2024 309 likes •499 views

Italicizing Optional Instructions on Mobile Online Surveys Improves Visual Filtering of Survey Content: An Eye Tracking Study Brian Falcone, Lawrence Malakhoff, Lin Wang Presenter: Alda Rivas U.S. Census Bureau AAPOR 73rd Annual Conference May

slide-1
SLIDE 1

Italicizing Optional Instructions on Mobile Online Surveys Improves Visual Filtering of Survey Content: An Eye Tracking Study

Brian Falcone, Lawrence Malakhoff, Lin Wang Presenter: Alda Rivas U.S. Census Bureau AAPOR 73rd Annual Conference May 17 2018 Denver, Colorado

This presentation is released to inform interested parties of research and to encourage discussion. The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau.

1

slide-2
SLIDE 2

Two studies:

First study: Effect of typographic cues on efficiency and accuracy with

which respondents complete a survey on a mobile device

Follow-up study: Incorporation of eye tracking to explore the

mechanism through which typographic cues affect efficiency.

2

slide-3
SLIDE 3

What is Typographic Cueing?

  • Reveals text content structure through

changes in weight, size, case, typeface,

  • etc. (Keyes, 1993)
  • Differentiates information categories
  • Use dark print for questions and light print

for response choices (Dilman, 2007)

  • Separate optional or occasionally needed

instructions from the question statement by font or symbol variations (Dilman, 2007)

3

Bold Question stem Italicized instructions

slide-4
SLIDE 4

Motivation

Gap of empirical evidence

– Support of theory. – Generalization of benefits of typographic cues to mobile interface.

Main research question:

– Does italicizing optional instructions make them more distinguishable?

  • Participants who are presented with optional instructions will have shorter

completion time on task because they are filtering out this information.

4

slide-5
SLIDE 5

Methodology

  • 2 x 2 Between-subjects design
  • 30 participants
  • Experimental task:

– 5 Question survey

  • 4 different conditions

– Survey completion time

  • Word recognition task:

– 10 words seen in survey – 10 words NOT seen in survey – Recognition score 5

Question stem Bolded Not Bolded (plain) Instructions Italicized

Bolded stem + Italicized instructions (n = 7) Plain stem + Italicized instructions (n = 8)

Not Italicized (plain)

Bolded + Plain instructions (n = 8) Plain stem + Plain instructions (n = 7)

D: Plain + Plain A: Bold + Plain C: Plain + Italics B: Bold + Italics

slide-6
SLIDE 6

Survey Completion Time

Italicized instructions result in faster survey completion times compared to plain (regular) text.

43.51 45.56 61.15 62.20

10 20 30 40 50 60 70 80 90 100 Bolded Q-Stem Plain Text Q-Stem Average Survey Completion Time (Seconds)

Italicized Instructions Plain Text Instructions

  • Bold – F(3,26)=.04, p = ns
  • Ital – F(3,26)=4.94, p < .05
  • Bold x Ital – F(3,26)= .00, p = ns

6

slide-7
SLIDE 7

Word Recognition

  • Bold - F(3,26)= .26, p = n.s.
  • Italics – F(3,26)= .97, p = n.s.
  • Bold x Ital – F(3,26)= .01, p = n.s.

Neither bold q-stem nor italicized instructions significantly affected the amount of words recognized.

34.89% 40.24% 27.48% 30.85%

0% 10% 20% 30% 40% 50% 60% 70% Bolded Q-Stem Plain Text Q-Stem

Average Corrected Recognition Score (HR - FA)

Italicized Instructions Plain Text Instructions

7

slide-8
SLIDE 8

Conclusions and Limitations

–Implementing a typographic cue (italicizing) for optional instructions does result in shorter completion time. –Utilized an indirect measure of visual attention (word recognition list). –Could not determine WHY individuals who used a survey designed with italicized instructions were faster.

8

slide-9
SLIDE 9

Follow-up Experiment Research question and hypothesis

– WHY did individuals who used a survey designed with italicized instructions complete surveys faster?

  • Italicized optional instructions results in lower visual attention and faster survey completion times

– Use a direct measure of visual attention by incorporating eye-tracking methodology

9

slide-10
SLIDE 10

Methodology

  • Participants: 16 Census Bureau Employees or

Contractors

– Setting: In-person tests at Census Bureau Headquarters in Suitland, Maryland

  • Task: Simulated Mobile Survey on PC using

Interactive PowerPoint

– Same 5 survey questions from previous behavioral study

  • Between-subjects design: 2 conditions

– Italicized Instructions VS Plain Text Instructions

10

slide-11
SLIDE 11

Eye Tracking Metrics and Analysis

  • Metrics:

– Total Fixation Duration – Fixation Count

11

slide-12
SLIDE 12

Eye Tracking Metrics and Analysis

  • Analysis:

– Area of Interest (AOI) Analysis:

  • Defined visual area for instructional text for each survey question.

Captures metrics within this AOI

– Summed total fixation duration and fixation count across all five survey questions for each participant

12

slide-13
SLIDE 13

Preliminary data: Fixation Count and Total Fixation Duration

  • Results:

– Average Fixation Count was nearly double when the instructions were plain text – Average Total Fixation Duration was nearly double when the instructions were plain text

13

20.96 42.14285714 5 10 15 20 25 30 35 40 45 Italicized Instructions Plain Instructions

Average Fixation Count Condition

Average Fixation Count Between Conditions

4.332 9.79 2 4 6 8 10 12 Italicized Instructions Plain Instructions

Average Fixation Duration (Sec) Condition

Average Total Fixation Duration Between Conditions

slide-14
SLIDE 14

Gaze Plots

14 Italicized Instructions (n=3) Plain Instructions (n=3)

  • Gazeplots:

– Gaze path and fixation duration for 3 participants from both conditions on the final survey question – Much more time was spent reading the instructions in the plain text group

slide-15
SLIDE 15

Heatmaps

  • Fixation Count Heatmaps:

– There is a clearly larger area of the instructional text being attended to by participants who saw plain text instructions

15 Italicized Instructions (n=3) Plain Instructions (n=3)

slide-16
SLIDE 16

Heatmaps

  • Fixation Duration

Heatmaps:

– There seems to be more time being spent fixating

  • n the area of the

instructional text being attended to by participants who saw plain text instructions vs italicized instructions

16 Italicized Instructions (n=3) Plain Instructions (n=3)

slide-17
SLIDE 17

Conclusions from preliminary data

Theoretical implications:

  • More time is spent reading the instructional text in the plain text condition

– This supports the theory that information deemed unneeded by the respondent is being visually filtered or skipped when text is italicized.

Practical implications:

– Evidence that typographic principles are applicable to mobile design.

Incorporating eye tracking to the design allowed us to directly observe the source of a cognitive benefit afforded by a design using typographic cues.

17

slide-18
SLIDE 18

Acknowledgement

  • Chistopher Antoun

– for major contributions to the design and analysis of the initial behavioral study that led to this follow-up research and helping with data collection

  • Elizabeth Nichols and Erica Olmsted-Hawala

– for contributing their knowledge and expertise towards the design of the initial behavioral

  • study. Also, a special thanks to Elizabeth for securing a booth for the Human Factors and

Usability Research (HFUR) group at the Census Innovation Fair where a bulk of the ET data was collected.

  • All Census Employees who volunteered to participate
  • MetroStar for software development
  • Sabin Lakhe for technical support

18