Measurement of social interventions at city missions

Sveriges Stadsmissioner is the Swedish national organisation that brings together the local city missions around the country. Together, the ten independent city missions have 1,300 employees and many volunteers who work to fight poverty and homelessness. Within the national organisation, the local city missions collaborate on, advocacy work, collection and quality and method development. An important issue is the follow-up of effects. The interventions need to have an effect on the target groups, such as increasing the quality of life or well-being. But it is also important to be able to show funders that the interventions give positive effects and that the money makes a difference.

"It is difficult to measure social work, but we have taken a good step together with RISE. In 2024, we will continue to implement, develop, and spread the message about the importance of measuring impact", says Kenneth Humling, quality developer at Östergötlands Stadsmission.

Ombudsmanship and work integration

Among other things, the city missions wanted to develop methods to follow up the effects of ombudsmanship and work integration. Ombudsmanship is an intervention where the target group gets help in activities such as contacts with authorities. Work integration is an intervention aimed at getting the target group into employment.

"We chose these interventions because we work with them at all city missions. We wanted to ensure the quality of the measurements, get reliable metrics on the effects in the target group, but also support the implementation of the new methods in the various city missions", says Kenneth Humling.

Category based measurements

RISE has expertise in areas such as implementation research and so-called category-based measurements, which involve going from simple surveys to quality-assured measurements of experiences, feelings, behaviors, and abilities. With quality-assured measurements, it is possible to state how much the feeling of well-being or quality of life has increased, or how much it has increased in compared to other groups or over time.

"For quality-assured measurements, work is needed in several stages. Construction and testing of questions and surveys is an important part. Another important part is the analysis where you use Rasch analysis to obtain reliable measurement values", says Tomas Bokström, project manager at RISE, and continues:

"Then there is always a balancing act in dialogue with the partner, how far you want or can go. More accurate measurements require more work and is more complex, but we should not make more accurate measurements than what the partner needs. The most important thing is that the measurements become useful and provide the answers needed for the partner."

Organising for impact

In addition to questionnaires, tests of the measurement tools against the target groups and recommendations regarding measurement and analysis of the measurements, RISE also produced a training material regarding the integration of effect measurement, a material called Organising for impact. RISE also researched digital tools that might be suitable for carrying out and compiling the measurements. It's important to make it easy, filling out a survey often ends up way down the priority list.

"The focus of our activities is to support the participants in the efforts, we need to make it easy for them to answer. We also need to continue developing the questionnaires, for example we have seen that participants often answer that they feel much better than they actually do. Another aspect is that if we help explain the questions to the participant, we influence the result to some extent. There are also challenges with language barriers that we want to continue working on", says Kenneth Humling.