Impact of Reducing the Number of Contacts on Response Rachel - - PowerPoint PPT Presentation

▶
impact of reducing the
SMART_READER_LITE
LIVE PREVIEW

Impact of Reducing the Number of Contacts on Response Rachel - - PowerPoint PPT Presentation

Nov 11, 2023 290 likes •405 views

Quality or Quantity? The Impact of Reducing the Number of Contacts on Response Rachel Horwitz* Jordan Misra Beth Newman U.S. Census Bureau *Any views expressed are my own and not necessarily those of the U.S. Census Bureau. Background

slide-1
SLIDE 1

Quality or Quantity? The Impact of Reducing the Number of Contacts on Response

Rachel Horwitz* Jordan Misra Beth Newman

U.S. Census Bureau

*Any views expressed are my own and not necessarily those of the U.S. Census Bureau.

slide-2
SLIDE 2

Background

  • Survey:
  • National Survey of College Graduates
  • Longitudinal
  • 6-month data collection cycle
  • Web invite, paper questionnaire at week 8, telephone follow-up at week 12
  • Sponsored by the National Center for Science and Engineering Statistics within the National

Science Foundation

  • Sample cases can receive:
  • Prenotice
  • 6 unique invitation letters (2 with paper questionnaires)
  • 4 reminder letters (same content)
  • 1 reminder postcard
  • 3 reminder emails (same content)
  • Unlimited phone calls

2

slide-3
SLIDE 3

Background

  • Question: Can we reduce the number of contacts sample cases

receive while maintaining response rates and key estimates?

  • Goal: Reduce costs and burden

3

slide-4
SLIDE 4

Experiment

  • Contact Strategy Treatment Groups:
  • New letters/envelopes/emails
  • 10 unique letters
  • 6 emails (timing different than current procedures)
  • Infographic
  • Call limit of 10
  • Fully factorial design
  • Sample size: ~46,000

4

slide-5
SLIDE 5

Evaluation Measures

  • Response rates (AAPOR RR2)
  • Key estimates
  • Costs

5

slide-6
SLIDE 6

Results – New Materials

No sig diff - 72.9 vs 72.2

6

slide-7
SLIDE 7

Results – Infographic

No sig diff - 69.1 vs 67.2

7

slide-8
SLIDE 8

Results – Call Limit

No sig diff - 68.5 vs 67.8

8

slide-9
SLIDE 9

Results – Overall Best Strategy

  • New Materials, No Infographic, Call Limit
  • Similar response rates
  • No impact on data quality across 14 key estimates
  • Costs reduced - $7.94 savings/case
  • Additional findings
  • Larger, non-standard-sized envelopes and perforated envelopes

particularly successful

  • Email directly following mailing successful

9

slide-10
SLIDE 10

Acknowledgements

  • Michael White, David Hall, David Pysh, Greg Orlofsky, Amanda

Noss, Stephen Simoncini, Judith Pilkerton, Aliza Kwiat, Ian O’Brien, Taunisha Gates, John Finamore

10

slide-11
SLIDE 11

Thank you! Contact: rachel.t.horwitz@census.gov

11