Imagine you run a survey to find out whether students like your university’s new online library. You ask: “Don’t you agree that the new online library is a huge improvement?” Most people say yes — and you walk away thinking the library is a hit. But did you actually measure their opinion, or did your question do the answering for them?
That is the problem with biased survey questions. A biased question pushes respondents toward one answer, so the data you collect reflects the wording of your question rather than what people really think.
The good news is that biased questions follow predictable patterns. Once you know the seven traps below, you can spot them in your own questionnaire in minutes — and fix each one with a simple rewrite.
What Makes a Survey Question Biased?
A survey question is biased when its wording, structure, or answer options make one response more likely than others — regardless of what the respondent actually believes. A supervisor who spots a leading question in your questionnaire may question your entire results chapter, so fixing bias at the design stage is far easier than explaining it away later. Bias usually creeps in through leading language, hidden assumptions, confusing structure, or unbalanced options.
Step 1: Read Every Question Out Loud and Ask, “What Answer Does This Want?”
Before you fix anything, read your questionnaire from start to finish — out loud, if you can. For each question, ask yourself: if someone disagreed with the answer this question seems to expect, would they feel awkward saying so? If the answer is yes, the question is probably biased. Factual questions (“How many hours per week do you study?”) are rarely biased; opinion questions (“How satisfied are you with…?”) are where bias hides.
Step 2: Fix the 7 Traps, One by One
Work through your questionnaire trap by trap — checking for one type of bias at a time is faster than looking for all seven at once.
Trap 1: Leading Questions
A leading question contains a word or phrase that nudges the respondent toward a particular answer — words like don’t you agree, obviously, shouldn’t, or loaded adjectives like excellent and terrible.
Biased: “Don’t you agree that online classes are more convenient than face-to-face classes?”
Fixed: “How do online classes compare with face-to-face classes in terms of convenience?”
Why it works: The fixed version asks for a comparison instead of inviting agreement. Nobody has to argue with the question to give an honest answer.
Trap 2: Double-Barreled Questions
A double-barreled question asks about two things at once, so the respondent cannot answer both parts accurately. The classic sign is the word and joining two different ideas — or a question about a topic that has two separate parts.
Biased: “How satisfied are you with the quality of teaching and the availability of learning resources?”
Fixed: Split it into two questions: “How satisfied are you with the quality of teaching?” and “How satisfied are you with the availability of learning resources?”
Why it works: A student might love the teaching but hate the resources. One question forces them to average two different opinions into a single meaningless answer.
Trap 3: Loaded and Assumptive Questions
A loaded question smuggles in an assumption the respondent may not share — and answering at all means accepting it. The famous example is “When did you stop cheating on exams?” — there is no honest answer that does not admit to cheating.
Biased: “What do you dislike most about your supervisor’s feedback style?”
Fixed: “How would you describe your supervisor’s feedback style?” followed by an optional question: “Is there anything about the feedback you receive that you would like to change?”
Why it works: The fixed version does not assume the student dislikes the feedback. Satisfied students can say so — and unhappy students still get their say.
Trap 4: Double Negatives
Double negatives force respondents to untangle the logic before they can answer — and many will get it wrong. If a question contains not plus a negative word like never, rarely, or un-, rewrite it.
Biased: “Do you agree or disagree that the university should not discontinue the evening bus service?”
Fixed: “Should the university continue the evening bus service?”
Why it works: The fixed version has one clear idea and one clear direction. Respondents answer the question instead of solving a logic puzzle.
Trap 5: Vague or Ambiguous Wording
Words like often, regularly, recently, good, and fairly mean different things to different people. “Do you exercise regularly?” might mean three times a week to one person and once a month to another — and your data cannot tell the difference.
Biased: “Do you frequently use the library’s online databases?”
Fixed: “In a typical week, how many times do you use the library’s online databases?” (with options: 0, 1–2, 3–4, 5 or more)
Why it works: Specific numbers replace fuzzy words. Every respondent answers the same question — and you can compare their answers honestly.
Trap 6: Unbalanced Response Options
Even a neutral question becomes biased if its answer options are lopsided. A scale with three positive options and one negative option (“Excellent / Good / Fair / Poor”) pushes people toward the positive end. The same happens with yes/no questions that offer only one real choice, such as “Would you prefer the improved timetable or the current one?”
Biased: “How would you rate the new timetable? (Excellent / Very good / Good / Poor)”
Fixed: “How would you rate the new timetable? (Excellent / Good / Average / Poor / Very poor)”
Why it works: The fixed scale is symmetrical — two positive options, one neutral middle, two negative options. Respondents can express a genuinely negative opinion without jumping to the extreme end.
Also include a genuine neutral or “not applicable” option where it makes sense — forcing someone with no opinion to pick a side manufactures data that does not exist.
Trap 7: Social Desirability Pressure
Some questions make people want to look good rather than be honest. Almost nobody admits to cheating, littering, or ignoring lectures — at least not when the question makes the “good” answer obvious. This is called social desirability bias, and it is the hardest trap to fix because the pressure is invisible.
Biased: “Do you always complete your assignments honestly without copying from others?”
Fixed: “How often do students in your programme copy parts of assignments from others?” followed by an anonymous, self-report version: “Have you ever copied part of an assignment? (Yes / No / Prefer not to say)”
Why it works: Asking about “students in your programme” first lowers the pressure, and anonymity plus a “prefer not to say” option makes honest answers safer. For sensitive topics, make the survey fully anonymous and say so clearly at the top.
Step 3: Worked Example — Fixing a Real Questionnaire Section
Here is a short section from a student survey about campus food services, before and after a bias check.
Before (biased):
- Don’t you think the campus cafeteria serves excellent food at reasonable prices? (Yes / No)
- How often do you eat at the cafeteria and use the vending machines? (Daily / Weekly / Rarely / Never)
- What do you dislike most about the cafeteria’s hygiene and service? (Open answer)
- Do you agree that the cafeteria should not reduce its opening hours? (Agree / Disagree)
After (fixed):
- How would you rate the food served at the campus cafeteria? (Excellent / Good / Average / Poor / Very poor)
- How would you rate the prices at the campus cafeteria? (Very reasonable / Reasonable / Average / Expensive / Very expensive)
- In a typical week, how many times do you eat at the cafeteria? (0 / 1–2 / 3–4 / 5 or more)
- In a typical week, how many times do you use the vending machines? (0 / 1–2 / 3–4 / 5 or more)
- How would you rate the hygiene of the cafeteria? (Excellent / Good / Average / Poor / Very poor)
- How would you rate the service at the cafeteria? (Excellent / Good / Average / Poor / Very poor)
- Should the cafeteria reduce its opening hours? (Yes / No / No opinion)
The fixed version is longer — seven questions instead of four — but every answer now means something. The original four produced confident-looking data that mixed food quality with prices, eating with vending-machine use, and hygiene with service. Splitting double-barreled questions costs a little extra survey length and buys you data you can actually defend in front of an examiner.
Step 4: Pilot Test Your Fixed Questionnaire
Never send a survey to your full sample without testing it first. Ask five to ten people from your target group to complete it, then ask them: was any question confusing, did any question feel like it expected a particular answer, and was there any question you wanted to answer differently than the options allowed? Pilot testers catch the bias you are blind to — especially on your own topic, where your opinions quietly shape your wording. If two or more testers flag the same question, rewrite it.
Common Mistakes to Avoid
- Fixing the wording but not the options. A neutral question with lopsided answer choices is still biased. Check both.
- Using “agree/disagree” for everything. Agreement scales invite acquiescence bias — the tendency to agree regardless of content. Where possible, ask direct questions (“How satisfied are you…?”) instead of statements to agree with.
- Asking about the past with no time frame. “Have you used the counselling service?” could mean ever, or this semester. Always anchor recall questions to a period: “in the past 6 months”.
- Skipping the pilot test. No checklist replaces real respondents. Five pilot testers will find problems that hours of self-review will miss.
Frequently Asked Questions
How do I know if my survey question is biased?
Read the question and ask what answer it seems to expect. If disagreeing would feel awkward, or the question contains words like “don’t you agree” or “obviously”, it is probably biased. A pilot test with five to ten people from your target group is the most reliable check — ask them directly whether any question pushed them toward a particular answer.
What is the difference between a leading question and a loaded question?
A leading question suggests the answer through its wording (“Don’t you agree the new policy is great?”). A loaded question smuggles in an unproven assumption the respondent must accept to answer at all (“What do you dislike about your supervisor’s feedback?”). Both distort results, but loaded questions are sneakier because the bias sits in the assumption rather than the adjectives.
Can a survey be completely unbiased?
Probably not completely — every question involves choices about wording, order, and options. The goal is not perfection but care: remove the biases you can find, pilot test to catch the ones you missed, and report your methods honestly so readers can judge the results.
Should I use “agree/disagree” scales or direct questions?
Direct questions are usually better. “How satisfied are you with the library?” gives cleaner data than “The library services are satisfactory (agree/disagree)”, because agreement scales invite people to agree out of habit. If you do use agreement scales, balance the statements — include some worded positively and some negatively.
How many questions can a survey have before people stop answering carefully?
There is no magic number, but fatigue is real: long surveys produce rushed, careless answers toward the end. Keep a student survey under 15–20 minutes, cut any question you cannot explain a use for, and put the most important questions early. Splitting double-barreled questions makes your survey slightly longer — a worthwhile trade, because each answer becomes trustworthy.
The Bottom Line
Unbiased survey questions are not about clever tricks — they are about respect for your respondents. Ask one thing at a time, use neutral words, give balanced options, and let people say what they actually think, even when it is not what you hoped to hear. Run the seven-trap check and pilot test with a handful of real people. Your data — and your examiner — will thank you.