From surveys to measurements with Rasch analysis

Society's need for knowledge about soft values such as health, well-being, behaviors, and abilities is increasing. It is important to be able to compare changes over time or between different groups or areas, to be able to make decisions about priorities, efforts, and effects. RISE has been working for several years to develop quality-assured measurements of latent traits, also called category based measurements. A latent trait or variable is the underlying experience or ability we want to measure, but which cannot be directly observed. For instance, if we want to measure a person's well-being we cannot measure it directly by measuring blood pressure or heart rate. However, we can measure well-being indirectly by asking questions about how the person is feeling, such as through a survey.

"By asking we get answers. But what does the answer mean? Is the person experiencing a high or low level of well-being, and how does it compare to other individuals or groups? How does the well-being of a group change over time? To get those answers we need to use basic metrology principles", says Magnus Johansson, researcher at RISE.

From survey to measurement

Part of going from survey to measurement is to ensure the quality of the survey or test. Do the questions provide sufficient and meaningful information about the latent trait we want to measure? It is also important to investigate whether the questions work equally well for different demographic groups, so that comparisons between measurement values can be made.

"We also need to test and analyze the questionnaire before it is used. For example, the respondents' level of well-being should be related to the probability that the respondent uses a higher response category. People with the same level of well-being should have similar response patterns to the individual questions. These are simple principles, but surveys are rarely tested to ensure this", says Magnus Johansson.

Converts answers into measurement values

Once the survey data has been collected, it is time for analysis. Here, Magnus Johansson and his colleagues often use Rasch analysis, which makes it possible to convert the answers in the survey into measurement values for the latent trait. The survey's measurement properties need to meet certain criteria for the conversion to be done correctly.

"Rasch analysis is the only method that, if used correctly and applying basic metrology principles, allows us to sum up and convert survey responses into metric values through a simple conversion table. Then we can go from saying that we asked people what they think about their well-being to saying that people's well-being is at a certain level."

Develops category based measurements

Magnus Johansson works together with his colleagues to spread knowledge and make the methodology available and accessible. An example is a tool that makes the statistical calculations more convenient. With the tool, it becomes relatively easy to carry out the analysis, document what has been done and enable others to reproduce the analysis. The tool is free of charge and the underlying source code and calculation models can be reviewed for those who are curious.

"A lot of knowledge is still required regarding preliminary work, interpretation of the results from the Rasch analysis, and what actions can be taken based on the results. But many of the calculations are basically done automatically. By calling simple functions, I get figures and tables that provide the central information", says Magnus Johansson.

The tool is under development but already available to use.

"I notice it's starting to pick up. Several people have gotten in touch and said that they learned how to do Rasch analysis with the help of the guide that gives examples of how the package can be used. The demand is increasing as more people realize that what we call measurements today are not actually quality-assured measurements. I hope that the tool can help reduce the threshold for working with quality-assured measurements", says Magnus Johansson.