University Planning and Analysis
Survey Consultation


So you want to do a survey...

Survey research can provide useful information on individuals' attitudes, experiences, and behaviors, provided that survey results are valid (i.e., accurate) and reliable (i.e., consistent). Doing a survey properly is not simply a matter of asking questions. The survey staff at University Planning and Analysis can, as time allows, help you think through any/all the necessary steps for your survey project. However, because UPA rarely has the time and/or resources to administer ad hoc surveys, if you are considering using a survey to collect information you need to think about who is going to:

  • Identify the goals and objects
  • Submit required materials to the NC State Institutional Review Board as necessary
  • Develop an appropriate research design
  • Design the questionnaire
  • Identify the population and select the sample
  • Prepare all materials and/or write all the programs necessary to administer the survey
  • Pre-test the questionnaire
  • Administer the survey, including sending out invitations and follow-up reminders to non-respondents
  • Analyze and interpret the results
  • Prepare reports and/or presentations
  • Use the results
  • Pay any expenses
If you have little or no experience in administering a survey, you are encouraged to consult with the staff at UPA (or some other experienced survey research team), and/or spend at least some time familiarizing yourself with the do's and don't of survey research. UPA has prepared a presentation that provides a relatively quick but thorough primer on survey research. A list of helpful references are also included at the bottom of this page.

This remainder of this page offers some quick tips on just a few areas of consideration when collecting quality survey data.

Before you start data collection...

  • See what's been done
    Conducting quality survey data requires a good amount of time, and often, money. Before initiating a new project, it is a good idea to determine whether the data you are interested in are already available.
  • Identify your purpose
    Before starting any survey project, you should be clear on why you are collecting data and what sorts of information you are interested in. By beginning your research with a clear purpose, you ensure that the questions you ask will generate the data you need.
  • Define a population
    Who are you interested in studying? Your sample, or individuals who answer your survey, should be similar to the larger group of people you want to draw conclusions about. If the sample is too dissimilar from your population, you cannot be sure that those surveyed represent the larger group.
  • Select a survey format
    Your population should determine the format of your survey. For instance, if you are interested in surveying welfare recipients, you would probably not want to develop an online survey, as poor persons often lack computer access.
  • Consider incentives
    As the use of survey research expands over time, people become less willing to devote their time and energy to any particular study. Providing incentives, such as cash, entry into a prize raffle, or gift cards, can sometimes help increase survey response rates.
  • Get IRB approval
    Research involving human subjects must undergo ethical review by NC State's Institutional Review Board. Please visit their website for more information.

Writing questions

  • Use appropriate language
    Use simple, basic English that is accessible to your population.
  • Avoid jargon and acronyms
    Do not assume that your survey population is familiar all the with technical terms and acronyms that you are.
  • Be direct
    Avoid vague words that are open to interpretation.
  • Be concise
    Wordy questions create more intellectual work for respondents. Simple question wording makes answering questions less mentally taxing.
  • Assume nothing
    Ask questions that presume very little about your respondents. For example, you would not want to ask a college student how many alcoholic drinks he/she consumes each week before first asking whether the student consumes alcohol at all.
  • Avoid double-barreled questions
    Double-barreled questions ask two questions in one. For instance, you might be interested in students' evaluation of a course's content and instructor. Yet if you ask How satisfied are you with the course content and instructor?, you are not sure whether a response reflects an assessment of content, the instructor, or both. You should ask two separate questions to collect the data you need.
  • Avoid leading questions
    Questions can be written in ways that "lead" respondents to answer questions in a particular way. For instance, you wouldn't want to ask students How effective are the improvements made to this course?, because the wording encourages an affirming response that may not necessarily be consistent with how students feel about the changes.
  • Manage survey length
    Respondents have constraints on their time and energy. By asking only what you absolutely need to know, you respect your respondents and increase the likelihood that they will finish your questionnaire.

Response options

  • Identify what your response options mean
    Instead of simply asking a respondent to evaluate some phenomenon on a numeric scale, identify what scale values mean. For instance, if you were interesed in how often a respondent reads for pleasure, you might create the response options 5 = Very Often, 4 = Often, 3 = Occasionally, 2 = Seldom, and 1 = Never. Defining values makes response options more understandable for both respondents and analysts.
  • Create mutually-exclusive response options
    If you want your respondents to select a single answer to a question, make sure that the response options are mutually-exclusive, or that only one answer applies. For instance, it would be inappropriate to ask students if they participated in research with faculty, an internship, OR study abroad, as some students will have done more than one.
  • Be exhaustive
    Write response options that cover all possibilities.
  • Balance your response options
    To avoid biasing results for questions that ask respondents to evaluate some phenomenon, ensure that there are equal numbers of positive and negative response options. If you were interested in faculty satisfaction with teaching assistants, you wouldn't want to ask whether they feel Very Satisfied, Satisfied or Very Dissatisfied, as faculty are more likely to select from the two affirming options than the single, extreme negative response.
  • Provide "not applicable" or "don't know" options as needed
    Some respondents lack the knowledge or experience necessary to answer certain questions.
  • Provide timelines
    If you are asking respondents to report the frequency of some event, establishing manageable timelines can be helpful. It might be difficult for a student to remember how many times they have exercised in a month, but they can likely report how many times they exercise each week.
  • Avoid response set
    If many similar questions and response options appear in a row, respondents can become fatigued and answer all questions in a set the same way. To avoid this, change formats between long series of questions.
  • Mind question order
    Answering early questions can sometimes affect respondents' responses to later questions. Keep in mind how certain topics may "prime" respondents to answer questions in certain ways. For example, asking staff about their job satisfaction after inquiring whether they have received a raise in the last 5 years may lead them to a different response than if they were asked about job satisfaction first.

References

For more comprehensive suggestions, refer to the following sources on survey research design and questionnaire development:

Bradburn, Norman M., Seymour Sudburn & Brian Wansink. 2004. Asking Questions: The Definitive Guide to Questionnaire Design. San Francisco, CA: Jossey-Bass. NCSU Library Call Number: H62 .B63 2004

Czaja, Ronald F. & Johnny E. Blair. 2005. Designing Surveys: A Guide to Decisions and Procedures. Thousand Oaks, CA: Sage Publications. NCSU Library Call Number: HA31.2 .C93 2005

Dillman, Don A. 2007. Mail and Internet Surveys: The Tailored Design Method. Hoboken, NJ: Wiley. NCSU Library Call Number: HM538 .D55 2007

Fowler, Floyd J. 1995. Improving Survey Questions: Design and Evaluation. Thousand Oaks, CA: Sage Publications. NCSU Library Call Number: HN29 .F67 1995

Rea, Louis M. & Richard A. Parker. 2005. Designing and Conducting Survey Research: A Comprehensive Guide. San Francisco, CA: Jossey-Bass. NCSU Library Call Number: HA31.2 .R43 2005


For more information on the survey development contact:
Dr. Nancy Whelchel, Assistant Director for Survey Research
University Planning and Analysis
Box 7002
NCSU
Phone: (919) 515-4184
Email: Nancy_Whelchel@ncsu.edu

Posted: August, 2011

Return to UPA Survey Page

Return to UPA Home Page