Quick answer
Survey research uses standardized questions to estimate attitudes, experiences, characteristics or reported behavior in a defined population. Begin with a decision and construct, confirm that a survey is the right method, define the target population and sampling frame, and choose probability or nonprobability recruitment with honest inference limits. Write short, neutral, single-concept questions with exhaustive response options and appropriate recall periods. Use qualitative discovery, cognitive interviews and a pilot to test comprehension, order, mode and logic. Monitor coverage, nonresponse, breakoff, speed and data quality; weight only with documented variables and report design effects and uncertainty. Publish question wording, field dates, sample, recruitment, mode, weighting and limitations.
What survey research is
Survey research collects standardized answers from a sample to describe or estimate characteristics of a defined population. It can measure reported behavior, experience, attitude, knowledge, intention and demographic or organizational characteristics at a stated time.
The questionnaire is only one component. Population definition, sampling frame, selection, contact, mode, response, processing, weighting and analysis all affect the estimate. A short questionnaire sent to the wrong people is not rescued by clean charts.
Surveys observe responses under a measurement process. They do not directly reveal objective behavior or causal effect. The researcher must define what each question can support and combine it with administrative, behavioral or experimental evidence when the decision requires more.
Decide whether a survey is the right method
Use a survey when the decision needs comparable measures across many units, an estimate for a population or repeated tracking. It works best when respondents can understand, retrieve, judge and report the requested information with acceptable burden.
Use interviews when constructs or language are not yet understood, observation when actual context matters, and experiments when the question is whether an intervention causes change. Existing records may answer factual questions more accurately and with less burden.
Write an analysis plan before the questionnaire. State primary outcomes, subgroups, comparisons, precision and decisions. This limits the temptation to collect every stakeholder's favorite item and search the resulting data for a favorable story.
Use a total survey error framework
Coverage error arises when the frame misses part of the target population. Sampling error arises because only a sample is observed. Nonresponse error occurs when response relates to the measures of interest. Measurement error comes from wording, recall, mode, interviewer and response behavior.
Processing, coding, imputation and weighting can add error. These components interact: a longer survey can improve construct coverage but increase breakoff; an expensive mode can improve access for one group and reduce candor for another.
Manage the largest decision risks rather than maximizing one visible metric. A huge opt-in sample can have tiny calculated sampling variance and substantial selection bias. Report uncertainty from design and adjustment, plus limitations that intervals do not capture.
Question
Define the decision, target population, construct and estimate needed.
- What will the survey change?
- What exactly must be measured?
Sample
Choose a frame, selection and recruitment design that reaches the intended population.
- Who can be selected?
- Who will be missing or less likely to respond?
Questionnaire
Translate constructs into clear questions and test how respondents understand them.
- Does wording measure one idea?
- Can respondents recall and answer?
Field and analyze
Monitor response and quality, process data reproducibly and adjust cautiously.
- Which groups are not responding?
- What did weighting change?
Report
Present estimates, uncertainty, methods and limitations in decision-ready form.
- What can be generalized?
- Which differences are real and material?
Define population, frame and sample
Name the unit and eligibility precisely. Category buyers in the past year, all residents with an evening travel need and current paying accounts are different populations. The screening window should match the decision and be answerable.
A probability sample gives known nonzero selection chances from a frame and supports design-based inference under assumptions. Nonprobability panels can be useful but require model-based adjustment and stronger caution about coverage and selection. Quotas do not make selection random.
Plan sample size from desired precision, design, subgroup needs, expected response and decision threshold. More responses do not repair a biased frame. Track recruitment source so blended samples can be analyzed and adjusted transparently.
- Decision and population explicit
- Constructs defined
- Method fit justified
- Frame coverage assessed
- Selection and recruitment documented
- Sample size precision-based
- Questions neutral and single-concept
- Recall period appropriate
- Cognitive test completed
- Pilot completed
- Nonresponse monitored
- Methods fully disclosed
Write questions people can answer
Use short, specific language familiar to the population. Ask one concept at a time and provide mutually exclusive, collectively appropriate response options. Distinguish no, none, not applicable, do not know and prefer not to answer where they mean different things.
Choose a recall period that matches event frequency and memory. Ask recent behavior before broad attitude when order could rationalize the account. Avoid leading premises, loaded words, double negatives and agree-disagree grids that invite acquiescence and straight-lining.
Question order creates context. Begin with engaging, relevant items, delay sensitive demographics and randomize response options when order has no substantive meaning. Preserve core wording in trackers so a question improvement does not masquerade as trend.
Survey research example
The transit study does not sample only app users because non-riders are central to the decision. It defines recent behavior separately from perceived safety and observed reliability, preventing one vague satisfaction item from carrying several meanings.
Operational data corroborates delay and incident conditions, while the survey estimates experience and belief. The agency can prioritize a test based on both, but it does not call correlation between safety concern and non-use a causal effect.
A hypothetical public-transit agency wants to know why evening workers do not use a new route and whether reliability or safety deserves the next investment.
Target residents with an evening travel need in the service area, including non-riders. Separate reported recent use, experienced reliability, perceived safety and intended use.
Combine an address-based frame with rider outreach, preserve selection probabilities and offer phone or web response so the current app database does not define the population.
Use interviews and cognitive testing to learn what riders mean by reliability and safety, then pilot recall periods, route descriptions, translations and questionnaire length.
Monitor response by neighborhood, schedule contacts across shifts, record dispositions and weight to defensible population controls while reporting design effects.
Present estimates and intervals alongside operational delay and incident data, then test the service intervention rather than claiming survey association proves causality.
A survey can estimate reported experience and belief in its target population. It cannot by itself establish that changing one service feature will cause ridership to rise.
Cognitively test and pilot the survey
Qualitative discovery identifies the vocabulary and response process. Cognitive interviews ask participants to think through what a question means, which memory they retrieve, how they choose an answer and whether options fit. Test across language, literacy and relevant experience.
A pilot exercises the whole system: invitation, consent, device layout, routing, randomization, translations, completion time, breakoff, data capture and export. Use pilot data to detect ceiling, floor, missing and implausible patterns, not to report an early result.
Freeze and version the production instrument. Emergency corrections need an audit trail and analysis of comparability. Test mobile and assistive technology, because a formally available survey can still exclude participants through interaction.
Manage response and data quality
Use understandable invitations, proportionate incentives, several contact attempts and modes or times suited to the population. Record standardized dispositions. A response rate is useful operational evidence but does not by itself measure nonresponse bias.
Monitor response composition, breakoff, item missingness, speed, duplicate patterns, device and open-text quality. Quality rules should be declared and reviewed; deleting respondents because their opinions look inconsistent can manufacture the desired result.
Protect privacy and minimize collection. Sensitive questions need purpose, safe placement, appropriate answer options and restricted access. Separate research from direct marketing so participation does not secretly enroll a respondent in sales activity.
Weight, estimate and compare responsibly
Weights can correct unequal selection and align observed characteristics to credible population controls. Document base weights, nonresponse adjustment, calibration, trimming and missing-data treatment. Report the effective sample and design effect where relevant.
Show estimates with uncertainty and practical magnitude. Multiple subgroup and item comparisons produce chance findings, so distinguish planned analysis from exploration and adjust inference or seek replication. Do not call overlapping percentages different without an appropriate test.
Inspect unweighted and weighted results, source differences and sensitivity to adjustment. Weighting cannot repair unmeasured selection when respondents and nonrespondents differ on the outcome after controls.
Report limitations and avoid common mistakes
Disclose sponsor, field dates, population, frame, sample, recruitment, mode, sample size, questionnaire, response or participation measures, weighting, uncertainty and analysis. Give readers enough information to understand who the estimate represents.
Common mistakes include surveying an email list and calling it the market, double-barreled questions, tiny subgroups, option-order bias, untested translations, excessive grids, treating intent as behavior, reporting weighted counts as respondents and hiding the full wording.
A survey is a designed measurement system, not a neutral form. Pilot it, preserve trend carefully and triangulate important decisions. The strongest result states what is estimated, for whom, under which design and with which uncertainty.
A biased question surveys itself, and a biased sample describes its own recruitment. Design the entire path from target population to reported estimate.
Frequently asked questions
What is survey research?
It is the standardized collection and analysis of reported characteristics, behavior, experience or attitudes from a sample to describe a defined population.
How large should a survey sample be?
Size depends on desired precision, sample design, population variability, subgroups, expected response and the minimum difference relevant to the decision, not a universal number.
What is the difference between probability and nonprobability sampling?
Probability sampling uses known nonzero selection chances from a frame. Nonprobability recruitment does not, so population inference depends more heavily on modeling and assumptions.
Why pilot a questionnaire?
A pilot tests invitation, logic, wording, mode, device behavior, timing, breakoff, data capture and processing before production errors become expensive.
Does a low response rate make a survey invalid?
Not automatically. Bias depends on whether response relates to the outcome after adjustment, but low response increases concern and should trigger analysis and transparent reporting.
Sources and further reading
- AAPOR: Best Practices for Survey Research ↗Professional guidance on planning, samples, questions, fieldwork, analysis and transparency
- Pew Research Center: Writing Survey Questions ↗Publisher methods guidance on wording, order, response formats, pretesting and trends
- AAPOR: Disclosure Standards ↗Professional standards for reporting sample, recruitment, mode, weighting and questionnaire details
- ISO 20252:2019 Market, Opinion and Social Research ↗International service and quality requirements for research, insights and data analytics