Chapter 9. Survey Research Chapter 9. Survey Research survey - - PDF document

chapter 9 survey research chapter 9 survey research
SMART_READER_LITE
LIVE PREVIEW

Chapter 9. Survey Research Chapter 9. Survey Research survey - - PDF document

What are the topics appropriate to What are the topics appropriate to Chapter 9. Survey Research Chapter 9. Survey Research survey research methods? survey research methods? Survey research method is the most frequently used Survey


slide-1
SLIDE 1

1

Chapter 9. Survey Research Chapter 9. Survey Research

  • Survey research method is the most frequently used

Survey research method is the most frequently used mode of gathering data in the social sciences. mode of gathering data in the social sciences.

  • Topics covered in this chapter

Topics covered in this chapter

  • Topics appropriate to survey research

Topics appropriate to survey research

  • Guidelines for asking questions

Guidelines for asking questions

  • Questionnaire construction

Questionnaire construction

  • Types of survey research

Types of survey research

  • Strengths and weaknesses of survey research methods

Strengths and weaknesses of survey research methods

  • Secondary analysis

Secondary analysis

What are the topics appropriate to What are the topics appropriate to survey research methods? survey research methods?

  • Surveys can be used in descriptive, exploratory

Surveys can be used in descriptive, exploratory

  • r explanatory research
  • r explanatory research
  • With most surveys, individuals are the units of

With most surveys, individuals are the units of analysis analysis

  • Surveys are the best when the question is about

Surveys are the best when the question is about attitudes and orientations in a large population attitudes and orientations in a large population

What are the guidelines for asking What are the guidelines for asking questions? questions?

  • Open

Open-

  • ended vs. closed

ended vs. closed-

  • ended questions

ended questions

  • Open

Open-

  • ended

ended -

  • respondents are asked to provide their own answer to the

respondents are asked to provide their own answer to the

  • question. (Similar to essay questions in our exams)
  • question. (Similar to essay questions in our exams)
  • Adv.: flexible, all responses can be represented.

Adv.: flexible, all responses can be represented.

  • Disadv

Disadv.: difficult to process for computer analysis. Some respondents .: difficult to process for computer analysis. Some respondents may may give irrelevant answers. give irrelevant answers.

  • Closed

Closed-

  • ended

ended -

  • respondents are asked to select a answer from a list

respondents are asked to select a answer from a list provided by the researchers. (Similar to multiple choice questio provided by the researchers. (Similar to multiple choice questions in ns in

  • ur exams)
  • ur exams)
  • Adv.: provide uniformity of response, and easy to process for co

Adv.: provide uniformity of response, and easy to process for computer mputer analysis analysis

  • Disadv

Disadv.: relies heavily on researcher's structure of responses. .: relies heavily on researcher's structure of responses.

  • Make items clear

Make items clear

  • Don't assume persons who answer the questions have the same

Don't assume persons who answer the questions have the same knowledge as you have on your research topic. Explain questions knowledge as you have on your research topic. Explain questions and and the background clearly. the background clearly.

  • Example: GPA: 0

Example: GPA: 0-

  • 4 vs. 0

4 vs. 0-

  • 5 scales.

5 scales.

What are the guidelines for asking What are the guidelines for asking questions? questions? -

  • Continued

Continued

  • Avoid double

Avoid double-

  • barreled questions

barreled questions

  • Avoid putting many statements in one question and seek for one a

Avoid putting many statements in one question and seek for one answer. The

  • nswer. The

respondent may agree with one statement but disagree with the ot respondent may agree with one statement but disagree with the other part. her part.

  • Example: “Research methods class is interesting, therefore, mor

Example: “Research methods class is interesting, therefore, more sessions e sessions should be offered every semester.” Some students may agree with should be offered every semester.” Some students may agree with the second the second part because they are required to take the class so they want mo part because they are required to take the class so they want more sessions. But re sessions. But they may disagree with the first part. they may disagree with the first part.

  • Respondents must be competent to answer

Respondents must be competent to answer

  • You need to make sure that the respondents are able to answer yo

You need to make sure that the respondents are able to answer your questions ur questions reliably. reliably.

  • Example: asking that they were doing on at noon on Jan. 21, 1984

Example: asking that they were doing on at noon on Jan. 21, 1984; asking a ; asking a person who has only raised one child about how she dealt with te person who has only raised one child about how she dealt with teenage rivalry. enage rivalry.

  • Respondents must be willing to answer

Respondents must be willing to answer

  • Sensitive issues may require the researchers to convince the res

Sensitive issues may require the researchers to convince the respondents of pondents of anonymity. anonymity.

What are the guidelines for asking What are the guidelines for asking questions? questions? -

  • Continued

Continued

  • Short items are the best

Short items are the best

  • Respondents are often unwilling to spend too much time reading l

Respondents are often unwilling to spend too much time reading long

  • ng

questions. questions.

  • Avoid negative items

Avoid negative items

  • Negative questions can easily lead to misinterpretation.

Negative questions can easily lead to misinterpretation.

  • Example: “Do you feel that criminals should be prohibited from t

Example: “Do you feel that criminals should be prohibited from teaching in eaching in public universities?” The word “prohibited” may cause confusion public universities?” The word “prohibited” may cause confusion. A . A respondent may think criminals should not teach in public univer respondent may think criminals should not teach in public universities by sities by answering “no”. But the way the question is structured, a “no” a answering “no”. But the way the question is structured, a “no” answer would nswer would mean the opposite. mean the opposite.

  • Avoid biased items and terms

Avoid biased items and terms

  • The meaning of someone's response to a question depends in large

The meaning of someone's response to a question depends in large part on the part on the wording of the question. Questions that encourage respondents to wording of the question. Questions that encourage respondents to answer in a answer in a particular way are biased. particular way are biased.

  • Example: “Don't you agree with the Supreme Court decision that..

Example: “Don't you agree with the Supreme Court decision that...?” The way .?” The way the question is phrased makes it difficult for a respondent to s the question is phrased makes it difficult for a respondent to say “no”. ay “no”.

What are the key elements to What are the key elements to questionnaire construction? questionnaire construction?

  • A questionnaire should be spread out and uncluttered.

A questionnaire should be spread out and uncluttered.

  • Use contingency questions when necessary.

Use contingency questions when necessary.

  • Contingency questions are those that will only be answered by so

Contingency questions are those that will only be answered by some me respondents, depending on their answered to a previous question. respondents, depending on their answered to a previous question.

  • Format matrix questions so they are easily answered.

Format matrix questions so they are easily answered.

  • Example:

Example: Likert Likert scale type of questions scale type of questions – – several questions have the several questions have the same set of answer categories. same set of answer categories.

  • Be aware of issues with ordering items.

Be aware of issues with ordering items.

  • Earlier questions may have impact on respondents’ answers to lat

Earlier questions may have impact on respondents’ answers to later er

  • questions. Researchers need to be sensitive to this issue.
  • questions. Researchers need to be sensitive to this issue.
  • Include instructions for the questionnaire.

Include instructions for the questionnaire.

  • Short instructions help respondents make sense of the questionna

Short instructions help respondents make sense of the questionnaire. ire.

  • Pretest all or part of the questionnaire.

Pretest all or part of the questionnaire.

  • Ask friends and colleagues to fill out the questionnaire. Errors

Ask friends and colleagues to fill out the questionnaire. Errors are best are best found using this method. found using this method.

slide-2
SLIDE 2

2

What are the types of survey What are the types of survey methods? methods?

  • Self

Self-

  • administered survey

administered survey

  • Face

Face-

  • to

to-

  • face interview

face interview

  • Telephone survey

Telephone survey

  • Internet survey

Internet survey

What are the important aspects of What are the important aspects of self self-

  • administered surveys?

administered surveys?

  • Mail distribution and return

Mail distribution and return

  • Anything you can do to make the job of completing and returning

Anything you can do to make the job of completing and returning the the survey easier is helpful. survey easier is helpful.

  • Monitoring returns

Monitoring returns -

  • return rate graph

return rate graph

  • Both types can be used.

Both types can be used.

  • Follow

Follow-

  • up mailings: When the graphs get to the flat part, a follow

up mailings: When the graphs get to the flat part, a follow-

  • up

up mailing is a good idea mailing is a good idea

  • Acceptable response rate: Practices vary a lot. In

Acceptable response rate: Practices vary a lot. In Babbie Babbie’ ’s s view, 50% view, 50% response rat is ok; 60% is good, and 70% is very good. response rat is ok; 60% is good, and 70% is very good.

D ay T otal retu rned Returned every day Day

  • The role of the interviewer

The role of the interviewer

  • Neutral

Neutral

  • General rules for interviewing

General rules for interviewing

  • Appearance and demeanor should be similar to those being intervi

Appearance and demeanor should be similar to those being interviewed ewed

  • Be very familiar with the questionnaire

Be very familiar with the questionnaire

  • Follow question wording exactly, never rephrase the question

Follow question wording exactly, never rephrase the question

  • Record responses exactly, even grammatical errors. Never reword

Record responses exactly, even grammatical errors. Never reword responses as such rewording might change the meaning of the responses as such rewording might change the meaning of the response. response.

  • Be neutral when probing for responses. Never suggest an answer.

Be neutral when probing for responses. Never suggest an answer.

  • If multiple interviewers are used, the researcher need to coordi

If multiple interviewers are used, the researcher need to coordinate nate between different interviewers to control quality. Training and between different interviewers to control quality. Training and supervising are needed. supervising are needed.

  • CAPI (Computer Assisted Personal Interview) is widely used

CAPI (Computer Assisted Personal Interview) is widely used now. now.

What are the important aspects of What are the important aspects of face face-

  • to

to-

  • face interview surveys?

face interview surveys?

  • Most rules related to face

Most rules related to face-

  • to

to-

  • face interview

face interview surveys also apply to telephone surveys surveys also apply to telephone surveys

  • Computer

Computer-

  • assisted telephone interviewing

assisted telephone interviewing (CATI) is widely used these days (CATI) is widely used these days

  • Issues with sample representativeness

Issues with sample representativeness

  • No phones (no chance to be selected)

No phones (no chance to be selected)

  • Unlisted numbers (no chance to be selected)

Unlisted numbers (no chance to be selected)

  • Multiple phones (too high of a chance to be

Multiple phones (too high of a chance to be selected) selected)

What are the important aspects of What are the important aspects of telephone surveys? telephone surveys? What are the important aspects of What are the important aspects of Internet surveys? Internet surveys?

  • Issue of representativeness

Issue of representativeness

  • At this point of time, certain segment of the

At this point of time, certain segment of the population are less likely to be reached via internet population are less likely to be reached via internet

  • surveys. For example, the elderly and the poor.
  • surveys. For example, the elderly and the poor.
  • Depending on the nature of the survey (and thus

Depending on the nature of the survey (and thus different theoretical population), the different theoretical population), the representativeness of the sample varies. representativeness of the sample varies.

  • Cost:

Cost:

  • Face

Face-

  • to

to-

  • face is the most expensive. If it’s a nationwide survey, telepho

face is the most expensive. If it’s a nationwide survey, telephone ne survey can also be costly. Self survey can also be costly. Self-

  • administered surveys and Internet surveys are

administered surveys and Internet surveys are cheaper. cheaper.

  • Response rate:

Response rate:

  • Face

Face-

  • to

to-

  • face typically has the highest response rate, followed by teleph

face typically has the highest response rate, followed by telephone

  • ne
  • surveys. Self
  • surveys. Self-
  • administered surveys and Internet surveys tend to have low

administered surveys and Internet surveys tend to have low response rates. response rates.

  • Incomplete questions:

Incomplete questions:

  • Face

Face-

  • to

to-

  • face and telephone surveys have fewer incomplete questions becau

face and telephone surveys have fewer incomplete questions because se probing can be used by the interviewer. probing can be used by the interviewer.

  • Sensitive issues:

Sensitive issues:

  • Self

Self-

  • administered surveys and Internet surveys are the best because t

administered surveys and Internet surveys are the best because they can hey can

  • ffer anonymity. Telephone surveys are good too. Face
  • ffer anonymity. Telephone surveys are good too. Face-
  • to

to-

  • face is the worst

face is the worst

  • Complicated issues:

Complicated issues:

  • Face

Face-

  • to

to-

  • face is the best, followed by telephone surveys.

face is the best, followed by telephone surveys.

  • Personal safety of the interviewers:

Personal safety of the interviewers:

  • Self

Self-

  • administered, Internet, and telephone surveys can all avoid this

administered, Internet, and telephone surveys can all avoid this problem. problem.

How do these methods compare? How do these methods compare?

slide-3
SLIDE 3

3

What are the strengths and weaknesses What are the strengths and weaknesses

  • f survey research methods compared to
  • f survey research methods compared to
  • ther modes of observation?
  • ther modes of observation?
  • Strengths

Strengths

  • The use of standardized questionnaire enhances reliability.

The use of standardized questionnaire enhances reliability.

  • Characteristics of a large population can be described using

Characteristics of a large population can be described using large samples. large samples.

  • Many questions may be asked on a given topic

Many questions may be asked on a given topic -

  • leads to

leads to flexibility in analysis. flexibility in analysis.

  • Weaknesses

Weaknesses

  • Questionnaire standardization may yield superficiality

Questionnaire standardization may yield superficiality

  • Doesn't deal with context of social life

Doesn't deal with context of social life

  • Artificial

Artificial -

  • cannot measure action

cannot measure action

What are secondary analyses? What are secondary analyses?

  • When good existing data sets are available on the

When good existing data sets are available on the research topic, one should consider using these data research topic, one should consider using these data

  • sets. Such studies are then called secondary analysis.
  • sets. Such studies are then called secondary analysis.
  • Sources of existing statistics

Sources of existing statistics

  • The Statistical Abstract of the United States

The Statistical Abstract of the United States http:// http://www.census.gov/statab/www www.census.gov/statab/www/ /

  • ICPSR:

ICPSR: http:// http://www.lib.utah.edu/icpsr/index.html www.lib.utah.edu/icpsr/index.html (U. Of (U. Of

  • U. page), and
  • U. page), and http://

http://www.icpsr.umich.edu www.icpsr.umich.edu (ICPSR home (ICPSR home page) page)

  • Many exiting data are very large and complicated.

Many exiting data are very large and complicated. Extensive training in statistics and computer Extensive training in statistics and computer programming is usually required. programming is usually required.

What are the strengths and What are the strengths and weaknesses with secondary analysis? weaknesses with secondary analysis?

  • Strengths

Strengths

  • Efficiency

Efficiency -

  • time and money

time and money

  • Unobtrusive

Unobtrusive – – the researcher does not have an impact of the researcher does not have an impact of data collection. data collection.

  • Government data usually have very good sampling design

Government data usually have very good sampling design

  • Weaknesses

Weaknesses

  • The data may not reflect a particular measure that a social

The data may not reflect a particular measure that a social scientist has constructed. scientist has constructed.

  • The data may not have all the variables one needs.

The data may not have all the variables one needs.

An article applying survey An article applying survey reseach reseach method method

  • Medina, Jose, Joel

Medina, Jose, Joel Saegert Saegert, & Alicia Gresham (1996). , & Alicia Gresham (1996). Comparison of Mexican Comparison of Mexican-

  • American and Anglo

American and Anglo-

  • American attitudes toward money. Journal of

American attitudes toward money. Journal of Consumer Affairs, 30(1), 124 Consumer Affairs, 30(1), 124-

  • 145.

145.

  • Hypotheses:

Hypotheses:

  • H1: Mexican

H1: Mexican-

  • Americans will have lower scores than

Americans will have lower scores than Anglo Anglo-

  • Americans on the Power

Americans on the Power-

  • Prestige scale

Prestige scale

  • H2: Mexican

H2: Mexican-

  • Americans will have lower scores than

Americans will have lower scores than Anglo Anglo-

  • Americans on the Retention/Time scale

Americans on the Retention/Time scale

  • H3: Mexican

H3: Mexican-

  • Americans will have higher scores than

Americans will have higher scores than Anglo Anglo-

  • Americans on the Distrust/Anxiety scale

Americans on the Distrust/Anxiety scale

  • Mexican

Mexican-

  • Americans will have higher scores than Anglo

Americans will have higher scores than Anglo-

  • Americans on the Quality scale

Americans on the Quality scale

  • Measurements:

Measurements:

  • Dependent variables:

Dependent variables:

  • Money attitude scale: 4 variables, 31 indicators

Money attitude scale: 4 variables, 31 indicators

  • Independent variables:

Independent variables:

  • Self

Self-

  • reporting

reporting

  • Control variables:

Control variables:

  • Gender (male, female)

Gender (male, female)

  • Income (lower, medium, higher)

Income (lower, medium, higher)

  • Age (younger, medium, older)

Age (younger, medium, older)

  • Sample

Sample

  • Subjects selected from the former students’ roster

Subjects selected from the former students’ roster at a southwestern university that included a large at a southwestern university that included a large proportion of Mexican proportion of Mexican-

  • American constituents.

American constituents.

  • 6,863 questionnaires mailed out

6,863 questionnaires mailed out

  • 1,132 returned

1,132 returned

  • Response rate of 16.5%

Response rate of 16.5%

  • Sampling issues

Sampling issues

  • Non

Non-

  • response bias issues

response bias issues

slide-4
SLIDE 4

4

  • Results:

Results:

  • Power/Prestige: 1.97(Anglo), 1.92(MA), NS

Power/Prestige: 1.97(Anglo), 1.92(MA), NS

  • Reject H1

Reject H1

  • Retention/Time: 4.72(Anglo), 4.49 (MA), S

Retention/Time: 4.72(Anglo), 4.49 (MA), S

  • Support H2

Support H2

  • Distrust/Anxiety: 5.28(Anglo), 5.26 (MA), NS

Distrust/Anxiety: 5.28(Anglo), 5.26 (MA), NS

  • Reject H3

Reject H3

  • Quality: 3.96 (Anglo), 3.78 (MA), S.

Quality: 3.96 (Anglo), 3.78 (MA), S.

  • Reject H4, direction reversed

Reject H4, direction reversed

Additional things to do Additional things to do

Read Read Medina, Jose, Joel Medina, Jose, Joel Saegert Saegert, & Alicia Gresham (1996). , & Alicia Gresham (1996). Comparison of Mexican Comparison of Mexican-

  • American and Anglo

American and Anglo-

  • American

American attitudes toward money. Journal of Consumer Affairs, attitudes toward money. Journal of Consumer Affairs, 30(1), 124 30(1), 124-

  • 145

145 very carefully. At this point, you should be very carefully. At this point, you should be able to understand most of the issues in this research able to understand most of the issues in this research

  • article. Try to think about theories, unit of analysis, time
  • article. Try to think about theories, unit of analysis, time

dimension, major variables, sampling, and mode of dimension, major variables, sampling, and mode of

  • bservation used in this study. Are there things that can be
  • bservation used in this study. Are there things that can be

improved in ideal situations? What are the limitations of improved in ideal situations? What are the limitations of this study caused by its research design? this study caused by its research design?