The quality requirements for official statistics are described in Section 5 of the Statistics Act and the European Statistics Code of Practice (Eurostat, 2017).
Findings
The quality report’s main source of information is the annual quality evaluation of all producers of official statistics. The quality evaluation has now been conducted at statistical level for the first time. Self-assessments for all statistics in the national programme for official statistics 2021–2023 (the statistical programme) (SSB, 2021) and information from follow-up meetings, constitute the factual basis for this evaluation. A questionnaire on compliance with quality requirements was completed for the self-assessment.
Involving statisticians in the quality evaluation at statistical level has contributed to raise the statistics producers’ awareness of the quality requirements. This is an important and positive effect of the quality evaluation.
The results from the quality evaluation show good compliance with many of the quality requirements in the Statistics Act and the European Statistics Code of Practice. It is still necessary to improve awareness of, and compliance with some of the quality requirements. This includes the quality requirement for statistical confidentiality along with measurement and communication of uncertainty in the statistics in accordance with the quality requirement for accuracy and reliability. Compliance varies considerably among statistics producers for some requirements.
For approximately two-thirds of the statistics, the producers have indicated that statistical confidentiality needs to be maintained to prevent information being revealed about individuals or companies in the published statistics. To reduce this risk, it is important to use recognised methods and software for disclosure control. The quality evaluation shows that it is uncertain whether statistical confidentiality has been adequately safeguarded in about 20 per cent of previously published statistics. This indicates a need to raise awareness of and increase competence in statistical confidentiality.
The quality evaluation examines the extent to which statistical uncertainty is calculated, assessed, documented and communicated. Uncertainty is calculated in 23 per cent of the statistics. Several also assess uncertainty without making specific calculations. For half of the statistics, the accompanying explanations include possible reasons for statistical uncertainty. The results show clear potential for improvement, and improvements in this area will be valuable for many users and in the communication of the figures.
An indicator for timeliness has been presented for all official statistics for the first time. On average, the timeliness of official statistics overall is somewhat better than the requirement set by the Ministry of Finance for Statistics Norway. There is considerable variation in the timeliness of the statistics. The statistics producers generally consider users’ requirements and expectations for timeliness to be largely met.
The results from the quality evaluation show that quality indicators are seldom used in the production of official statistics to measure quality in statistical processes or output. This confirms findings from previous quality evaluations at institutional level.
Statistics Norway’s reports on quality in administrative information systems are designed to provide assessments and feedback on the quality of individual administrative data sources, but they are not necessarily well suited for tracking quality developments over time.
In 2022, the producers had planned a total of 113 quality improvement measures based on recommendations in previous quality reports, and by autumn 2022, a quarter of these had been implemented. A status report of 161 measures in January 2024 showed that 42 per cent of the measures had been implemented.
Overarching recommendations
The quality evaluation at statistical level has helped improve the factual basis for making recommendations on how to enhance quality. The most important new recommendations for ensuring compliance with the quality requirements in the Statistics Act and European Statistics Code of Practice are detailed below.
There is a need to increase the use of recognised methods and software to safeguard statistical confidentiality. Competence in statistical confidentiality needs to be improved, and one suggestion is to introduce compulsory basic training in statistical confidentiality for all new employees involved in the statistical production process and in relevant support functions.
Awareness of the uncertainty in statistics needs to be raised. All producers should consider how the calculation, documentation and communication of uncertainty can be improved.
Awareness of the quality requirements in the Statistics Act and European Statistics Code of Practice should also be raised, and it is recommended that courses on quality be made compulsory for all new employees working in statistics production and relevant support functions.
The quality reports drawn up by Statistics Norway based on data deliveries from administrative information systems need to be further developed to track the development of quality over time and to more easily adapt the report template as new needs arise. As a first step, a cost-benefit analysis of this need should be performed.
Several recommendations from earlier reports are still applicable for several of the producers. The most important of these are as follows:
- Improve user contact by creating established forums for statistical domains that currently lack these
- Conduct regular user surveys with systematic follow-up of the results
- Improve and standardise statistical documentation
- Increase the use of quality indicators to measure quality in statistical processes and output
Further development of the quality system
An important recommendation from earlier reports that is still applicable is to increase the use of quality indicators. This requires further development of the quality system through standardisation of data, metadata and log data structures, and the development of a common framework of indicators that can be used across statistics and producers. This is essential for expediting the efforts to increase the use of quality indicators.
The development of the quality system in relation to quality evaluations and quality reviews largely aligns with the plans outlined in last year’s report. The quality reviews have been streamlined, requiring fewer resources, and they are being performed over a shorter period of time than before. The number of reviews conducted each year has increased and will rise further in 2024. According to the plan endorsed by the Committee for Official Statistics, all producers of statistics must undergo a quality review for one of their statistics by the end of the current period in the national programme for official statistics (SSB, 2021). Achieving this objective is dependent on producers participating, allocating the necessary resources and submitting statistics for review. The reviews should also be evenly distributed throughout the statistical programme period, which ends in 2027.
The annual quality evaluation will continue at statistical level. From 2025, it will be possible to measure the development in compliance with the quality requirements that are relevant at statistical level. The five new producers who joined the statistical programme in 2024 will be integrated into the quality system and will begin conducting quality evaluations on a par with the other producers starting in 2024.