EVIDENCE USE IN EDUCATION
NOVEMBER 2021
https://ies.ed.gov/ncee/edlabs/regions/west/
1
Creating Eective Surveys
Best Practices in Survey Design
Purpose
When education leaders utilize survey results to inform their decisions, they are often left wondering if
the “right” survey questions were asked. To help state and local leaders improve existing instruments
or create better instruments for data collection, this quick reference guide highlights best practices in
designing effective surveys. Administering better surveys leads to more specific and accurate data,
and better data leads to better evidence with which to inform decisions.
This guide focuses on four areas of survey development:
¤ general best practices in survey design,
¤ common problematic survey questions (and how to avoid them),
¤ Likert scales and rating scales, and
¤ creating more inclusive surveys.
Intended Use
This quick reference guide is designed to help state and local education leaders and staff who are
asked to develop or modify surveys but may not have a background in survey design. This guide
will help you gain a more critical lens when contributing to survey design and development or when
reviewing survey results. If you’d like more details on survey design and other survey-related topics,
please refer to the list of resources at the end of this guide.
https://ies.ed.gov/ncee/edlabs/regions/west/
Creating Eective Surveys: Best Practices in Survey Design
2
General Best Practices in Survey Design
1. Define a clear, attainable goal for your survey
What are you trying to learn about from your survey? Start by crafting a goal before writing any questions.
Unclear, unattainable: “This survey will help us learn about everything parents are thinking about their child’s
math homework assignments.”
Clear, attainable: “This survey is designed to understand parents’ perceptions about the difficulty, frequency, and
variety of their child’s math homework last year. Responses will be used to shape the school’s guidance to math
teachers in the coming year.”
2. Don’t let your survey get too long
Long surveys lead to fatigued or annoyed respondents so they become less likely to provide truthful, detailed
answers or, more importantly, finish the whole survey.
3. Keep it brief, simple, and specific
Keep your survey brief and focused specifically on the exact data you wish to analyze. Ask only the questions
needed to achieve your survey goal, and ask them as clearly and simply as possible.
4. Save open-ended, challenging, and more personal questions for the end
Allow respondents to get comfortable with the survey by asking easier, more general, and less personal
questions upfront. Asking open-ended, challenging, and personal questions right away might feel jarring
or intrusive to a respondent.
5. Allow respondents to respond “not applicable (N/A)”
Whenever possible, include the option to indicate that a question does not apply to a respondent’s experience.
Otherwise, respondents might skip such questions, leaving you to wonder why.
6. Include a short introduction and a time estimate
Before asking respondents to answer questions, help them feel comfortable and able to accurately estimate their
commitment by welcoming them to your survey and indicating a realistic amount of time in which to complete it.
7. Test or pilot your survey and survey platform beforehand
Ask colleagues to test your survey first and give you feedback on clarity, ease of responding, and so on.
This applies to both online and paper-and-pencil surveys.
https://ies.ed.gov/ncee/edlabs/regions/west/
Creating Eective Surveys: Best Practices in Survey Design
3
Common Types of Survey Questions
Closed-ended questions include multiple-choice, rating-scale (e.g., rate from 1-5), and checkbox
(e.g., true/false) questions. These types of questions provide quantitative data and, generally,
are easier for respondents to answer. Including more closed-ended questions than open-ended
questions in your survey will reduce fatigue for respondents and increase the likelihood that they
will complete it.
Open-ended questions ask respondents for feedback in their own words (i.e., free response)
and can provide rich, qualitative data. Because open-ended questions can take much longer to
answer and analyze, include fewer of them and place them at the end of the survey.
Common Problematic Survey Questions (and how to avoid them)
Poorly written survey questions will reduce the quality and quantity of responses. Here are some common types of
problematic survey questions to avoid as you craft your data collection instrument:
Question Type Defined Example Suggested Fix
Leading question A question that signals,
prompts, or encourages
a certain answer
Were your friendly library
staff members helpful as
you engaged in summer-
school teaching?
On a scale from 1
(completely unhelpful)
to 5 (very helpful), rate
the helpfulness of library
staff members as you
engaged in summer-
school teaching.
Loaded question A question that includes
an unjustified assumption
or forces respondents to
agree with an assumption
How much will test
scores improve because
of your school’s new
reading program?
On a scale from 1
(decrease significantly) to
5 (increase significantly),
how do you expect
test scores to change
because of the school’s
new reading program?
https://ies.ed.gov/ncee/edlabs/regions/west/
Creating Eective Surveys: Best Practices in Survey Design
4
Question Type Defined Example Suggested Fix
Double-barreled question A question that asks for
an opinion about two
different items but allows
for only one response
How do you think
students’ test scores and
attendance will change
because of the new after-
school program?
Separate into two
questions:
1. How do you expect
test scores to change
because of your
school’s new after-
school program?
2. How do you expect
attendance to change
because of your
school’s new after-
school program?
Double-barreled answer An answer option that
presents two possibly
different opinions as a
response to one question
What was your
personal experience
with mathematics
in high school?
Rate from 1 (did not
like/did not succeed)
to 5 (passionate about/
excelled at).
What was your
personal experience
with mathematics
in high school?
Rate from 1 (negative) to
5 (positive).
Double-negative question A question that contains
two negative elements
intended to create a
positive element, which
can confuse respondents
Is it not uncommon for
teachers to coach a sport
after school?
How common is it for
teachers to coach a sport
after school?
https://ies.ed.gov/ncee/edlabs/regions/west/
Creating Eective Surveys: Best Practices in Survey Design
5
Likert Scales and Rating Scales
Closed-ended questions that incorporate a Likert scale or a rating scale are used frequently in surveys. When
written well, such questions are generally less exhaustive for respondents to answer and are an excellent source of
rich, quantitative data. Keep the following tips in mind as you craft your scale questions.
1. Generally, five rating choices are enough
Craft scales that yield only five possible answers (e.g., rate from 1 to 5, choose from five options). Provide more
rating choices only if absolutely necessary to gather more detail.
2. Maintain balance and objectivity
Make sure that each scale provides ratings that are distributed to provide an equal number of choices with a
positive or negative connotation.
3. Offer a neutral option
Include a neutral option to identify whether respondents feel neither positive nor negative about a topic. Allowing
them to express indifference, rather than skipping the question or answering “not applicable,” will give you a more
accurate representation of their perception. The neutral option should be in the middle of the scale.
4. Don’t forget to provide a “not applicable (N/A)” option
Enable respondents to opt out of responding by indicating that a question does not apply to their experience.
In the following example, the rating scale on the left is improved by changing the options to those on the right in
order to ensure balance and objectivity and to provide a “not applicable” option.
Please rate the helpfulness of library staff members as you engaged in
summer-school teaching.
Needs Improvement
1. Very helpful
2. Helpful
3. Somewhat helpful
4. Neither helpful nor unhelpful
5. Not helpful
Improved
1. Very helpful
2. Helpful
3. Neither helpful nor unhelpful
4. Unhelpful
5. Very unhelpful
6. Not applicable
https://ies.ed.gov/ncee/edlabs/regions/west/
Creating Eective Surveys: Best Practices in Survey Design
6
Using Scale Questions to Understand Change
Scale questions lend themselves well to understanding changes in opinions, perceptions, or
knowledge when asked, as identically as possible, both before and after an event. A pre- and
post-event survey can be completed either (1) in two phases, before and after an event, or (2) one
time after the event. Individual or aggregated responses can then be compared to understand
whether changes in opinions, perceptions, or knowledge have occurred.
Pre-event question:
Please select the option that best describes your level of
understanding of the topic(s) before you attended the webinar.
Post-event question:
Please select the option that best describes your level of
understanding of the topic(s) now that you have participated in the webinar.
Options:
(a) none at all, (b) a little, (c) a moderate amount, (d) a lot, (e) a great deal
CHANGE IN UNDERSTANDING - HYPOTHETICAL EXAMPLE
Level of Understanding Before and After Attending Webinar
25% 83%
20% 30% 40% 50% 60% 70% 80% 90%
Before After
Note: Percentages indicate the proportion of
participants who reported they knew “a lot” or
“a great deal” about the topics presented in
the webinar before and after attending.
https://ies.ed.gov/ncee/edlabs/regions/west/
Creating Eective Surveys: Best Practices in Survey Design
7
Creating More Inclusive Surveys
It is essential to consider the various identities and lived experiences of respondents. Ensure that all survey language
is inclusive for all respondents. Otherwise, they might be less likely to provide honest answers or to complete your
survey altogether. Consider the following suggestions as you design your survey questions.
1. Be thoughtful about demographic questions
Collect only the demographic information you need. When doing so, offer your respondents flexible and
robust options for identifying themselves. For example, demographic questions on gender should include a
spectrum of potential gender identities such as male, female, transgender, non-binary/non-conforming, and
prefer not to respond.
2. Make survey questions mandatory only if a response is necessary
Unless a response is imperative to utilizing the data, do not make questions mandatory. Requiring an answer to every
question can make respondents feel uncomfortable or unwilling to complete the entire survey if they would rather not
respond to one or more questions.
3. Be mindful of language used in your survey
Consider the inclusivity of your language and how it might be perceived by all respondents. Think about relevant
context, history, and the implications of how statements are phrased. Don’t include language that ascribes or
assumes a certain perception or viewpoint as something that should be “standard” or “normal” to your respondents,
as it is possible that a different viewpoint might be the common understanding in their lived experience.
4. Consult resources on inclusivity and bias-free language
If you are unsure, the resources included at the end of this guide can help you check that the language of your
survey is inclusive and bias-free.
Creating Eective Surveys: Best Practices in Survey Design
8
References and Additional Resources
American Psychological Association. (2019).
Bias-free language.
https://apastyle.apa.org/style-grammar-guidelines/bias-free-language
This resource offers general guidelines on bias-free language and specific guidelines that address individual
characteristics such as gender, racial and ethnic identity, sexual orientation, socioeconomic status, ability, and age.
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014).
Internet, phone, mail, and mixed-mode surveys: The tailored
design method
(4th ed.). Wiley.
This text covers all aspects of survey research, including survey design strategies, guidance on obtaining
high-quality feedback, and techniques to increase response rates.
Fisher, S. (2021, September 27).
How to create an effective survey.
Qualtrics. https://www.qualtrics.com/blog/
10-tips-for-building-effective-surveys
This introductory-level blog post is designed for audiences new to survey design and offers 15 practices for
designing effective surveys.
Irwin, C. W., & Stafford, E. T. (2016).
Survey methods for educators: Collaborative survey development (part 1 of
3)
(REL 2016–163). U.S. Department of Education, Institute of Education Sciences, National Center for Education
Evaluation and Regional Assistance, Regional Educational Laboratory Northeast & Islands.
https://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=4482
This guide outlines the crucial steps for educators and researchers to take when developing surveys for
education contexts.
Krosnick, J. A., & Fabrigar, L. R. (1997). Designing rating scales for effective measurement in surveys. In L. Lyberg, P.
Biemer, M. Collins, E. De Leeuw, C. Dippo, N. Schwarz, & D. Trewin (Eds.),
Survey measurement and process quality
(pp. 141–164). Wiley. https://doi.org/10.1002/9781118490013.ch6
This text chapter provides an in-depth description of designing and understanding rating-scale response questions.
Matsumoto, A. (n.d.).
How to create more inclusive surveys.
SurveyMonkey. https://www.surveymonkey.com/curiosity/
how-to-create-more-inclusive-surveys/
This blog post outlines principles to follow in order to ensure surveys are inclusive.
Pew Research Center. (n.d.).
Writing survey questions.
https://www.pewresearch.org/methods/u-s-survey-
research/questionnaire-design
This article discusses common pitfalls and best practices when designing questionnaires.
Walston, J., Redford, J., & Bhatt, M. P. (2017).
Workshop on survey methods in education research: Facilitator’s guide
and resources
(REL 2017–214). U.S. Department of Education, Institute of Education Sciences, National Center for
Education Evaluation and Regional Assistance, Regional Educational Laboratory Midwest.
https://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=4544
This toolkit offers a facilitator guide and workshop handouts that can be used to help conduct trainings
on survey methods.
This product was prepared under Contract ED-IES-17-C-0012 by Regional Educational Laboratory West, administered by WestEd. The
content does not necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names,
commercial products, or organizations imply endorsement by the U.S. Government.
https://ies.ed.gov/ncee/edlabs/regions/west/