In 2022, SSB introduced a significant change by adopting a digital approach, encouraging participants to self-report via an app, which included a feature for receipt scanning. This change aimed primarily to reduce the respondent burden, counteract declining response rates, and meet a general need for modernization. An extensive series of tests was conducted to ensure the app's functionality met the diverse needs of respondents, focusing on user-friendliness and inclusivity, as this report describes.
The development of a new data collection tool was guided by the potential and challenges of integrating “smart technology” into survey processes. Feedback from our tests indicated that the majority of participants found the app convenient and easy to use, reflecting the prevalence of mobile technology use among working-age individuals in Norway. Many, but not all, are experienced mobile users and understand the interactive language of touch screens, scrolling, swiping, and the like. However, older participants showed a higher degree of hesitancy and encountered more technical issues, suggesting a digital divide that could impact participation rates among certain demographic groups.
With the transition from interviewer-administration to respondents’ self-completion of the survey in the app, we saw a clear need to simplify the task of responding. The length of the questionnaire and the diary-keeping period were reduced, and English was offered as an alternative language. However, it is a challenge to communicate in-app the survey tasks to do the respondents. We know that the more information required, the harder it is to communicate this information. For respondents the tests suggested that interpretation of the response task is a greater challenge than handling the app itself. Technology offers us many new possibilities to guide and direct respondents through the survey, but an app has limitations in conveying nuanced instructions. Interviewers have made a significant difference in communicating the task. They are crucial in recruitment and have a more central role in motivating, guiding, and providing technical support to respondents than has been common in telephone and web surveys. They have also been important in assuring respondents that it is safe to open the survey link received via SMS from SSB.
Even though the majority carried their mobile phones with them most of the time, they did not record their purchases as promptly as we had hoped. Many forgot receipts from stores, and there was often a delay between shopping and recording. This affects memory and results in less detailed reporting. Less frequent record-keeping, along with superficial reporting for other household members, leads to less detailed and accurate reporting. Scanned receipts are checked less frequently than desired. Overall, respondents show less motivation to report as accurately as before. Whether the volume and details of scanned data can compensate for less accurate manual recording remains inconclusive for now. Further analysis of survey data from the new Household Budget Survey 2022 will provide a basis for an independent evaluation and report.
Through extensive user testing, SSB has gained valuable insight and in-depth knowledge of how the new Household Budget Survey app functions when interacting with respondents. Here, we share a summary of this work.
NOTE: Please note that this report is a condensed translation of an internal publication at Statics Norway from 2023, see Brukertesting av ny forbruksundersøkelse (ssb.no). The full Norwegian version includes descriptions of each test round. In the English version this is not included as understanding of the Norwegian language and culture is a prerequisite for comprehension. The translation process was supported by AI technology and verified by SSB for accuracy.