Questionnaire design and measurements

Have you considered if your survey questions capture the intended? How do you ensure reliable decision-making based on survey data? How do we measure intangible elements like experiences, feelings, and attitudes? RISE assists with questionnaire design and quality assurance through expertise in survey methodology and modern psychometric test theory.

When important decisions rely on data and measurements, it's crucial to use scientifically grounded and reliable methods. Surveys are increasingly used for data collection, but their questions and response scales often aren't designed based on research in survey methodology and question construction. Furthermore, surveys often lack thorough quality assurance or pre-testing of questions. We help design and modify surveys using scientifically based methods to ensure every part contributes to the survey's overall purpose.

Quality-Assured Survey Development

The most critical quality criterion for surveys and response scales is their ability to measure the intended variables, i.e., the survey's validity. At the same time, we must consider the respondent's experience. In a time of declining response rates and increasing survey fatigue, methods that promote high participation and honest, reliable answers are needed to accurately reflect reality. Understanding the survey response process and ensuring questions are understood consistently by all respondents is crucial. RISE has extensive knowledge in question technique, response scales, survey methodology, and test question methods. We can also assist with data collection methods, fieldwork, and nonresponse adjustment, as detailed in our Sampling-Based Statistical Survey expertise.

Quality-Assured Measures of Soft Values

Sometimes what we want to measure isn't directly observable, such as experiences, feelings, or attitudes. For this, psychometrically based survey questions with ordinal response categories are often used, e.g., 1) Completely agree, 2) Partly agree, 3) Hardly agree, 4) Do not agree at all. However, it's common to treat this response data as numerical to create indices or averages, which can lead to skewed estimates of the underlying value (experience, feeling, or attitude). Using item response theory (IRT) and measurement techniques, we can transform ordinal data into interval data by assigning a logit scale to response categories. This ensures the distances between categories are meaningful and accurate. RISE has significant expertise in IRT, making it possible to ensure the validity and comparability of measurements.

Implementation Steps

Needs Analysis and Planning: Initial discussions to understand needs and objectives. We can also provide expertise in Impact Measurement.
Construction Phase: Map theoretical frameworks within the measurement area. Design and develop questions and response scales, or correct existing ones. If questions are psychometrically based, we focus on ensuring their validity and reliability. We often recommend pilot studies to test the questions.
Analysis and Adjustment: Analyze collected data to ensure measures capture the intended information. Quality-check data and provide adjustment suggestions.
Reporting: Deliver final survey questions and response scales along with a report containing implementation and analysis recommendations.