Towards universal childhood immunisation

Childhood immunisation programming is an essential part of every country's health programme to reduce vaccine preventable diseases. The Global Alliance on Vaccines and Immunisation (GAVI) was established to help fund and implement universal childhood immunisation. Funding provided by GAVI through its immunisation service support (ISS) is performance-based, with funds disbursed in proportion to the targeted or reported number of additional children immunised.

In 2008, Lim, Stein, Charrow and Murray published the article "Tracking progress towards universal childhood immunisation and the impact of global initiatives: a systematic analysis of three-dose diphtheria, tetanus, and pertussis immunisation coverage" in the Lancet. In the article, they raise a concern that fewer children have been immunised than officially reported and that this has significant health and financial consequences.

The main findings from the assessment of this study are as follows:

a) The study by Lim et al. (2008) estimates DTP3 coverage using officially reported coverage and survey data for 193 countries. Time-series analysis investigates the association between the presence of GAVI ISS and the difference between countries officially reported and survey based immunisation coverage.

b) In general, vaccination coverage based on administrative data was significantly higher than survey based vaccination coverage estimates. Furthermore, the study showed that 7.4 million additional children were immunised under ISS based on survey data compared to 13.9 million addition children reportedly immunised. This amounts to a difference of around US$140 million in support money.

c) We believe the amount of data analysed in the study is extensive and indicates that results are of a robust nature. The methodology (including the use of selfreported vaccinations) is validated with additional background information and studies supporting the authors' decisions.

d) The study shows that their new imputation method, bidirectional distancedependent regression (BDDR), performs similarly to the more commonly used multiple imputation method, validating its use. However, we believe in the absence of survey data, quick changes in immunisation coverage may not always be detected by the model, especially in recent estimates where there are no following surveys.

e) The study by Lim et al. (2008) has lumped together investment and reward payments for countries receiving GAVI ISS, which we believe may be problematic due to the differing nature of payment calculations.

f) Additionally, we believe vulnerable groups may be less likely to participate in surveys and similarly be difficult to reach for preventive health care programmes. This implies survey data may overestimate immunisation coverage.

In order to learn more about potential disparities between vaccination coverage reported in surveys and administrative data we have carried out case studies in four countries. Despite the existence of clearly defined administrative routines, the overall impression is that administrative data are subject to considerable uncertainty.

a) The countries selected for field studies all experienced high staff turnover and vacancies. There was a lack of relevant personnel and inadequate resources both in the organisations set up to give vaccines and to record the administrative data.

b) A lack of understanding for the importance of accurate reporting of vaccinations was often observed.

c) Measures of the target population (the number of children to be vaccinated) are often uncertain.

d) Cross border migration, and vaccinations of children in older age groups, can result in vaccinations given to children not included in the target population.

e) It is possible that population growth is higher than the projections used in the construction of coverage estimates. If the population growth is underestimated, the gap between survey data and administrative data will be overestimated.

Based upon the review and the case studies our recommendations for improving the quality of data reports on vaccination coverage are as follows:

a) Re-analyse all available data using methods similar to Lim et al. for chosen countries to improve the current knowledge base without collecting new data.

b) Implement small annual household surveys of selected vaccines (e.g. DTP) to provide general basis for vaccination coverage.

c) Small annual household surveys are also recommended to improve population estimates and thereby improve the estimation of the target group.

d) Replicate the study done in Mozambique by Mavimbe, Braa and Bjune (2005) on record keeping, reporting and the support mechanism to ensure quality data on vaccination. This ought to be done in selected countries to address differences between regions and over time. As a part of this, discussions on what appears to be success stories and failures would be initiated.

e) Implement a full analysis of the existing reporting systems in order to establish more user-friendly, simple and standardised systems.

f) Building Human resources. The quality of the statistics depends on the ability of the staff members who produce it. This both addresses subject matter training and introducing work modes were the employees do not fear their superiors. If you are afraid of your boss, you may report false not to get into trouble.

g) Evaluate the administrative and survey based vaccination coverage through time (for specific countries) to identify the points in time when different sources of data are becoming more in line with each other. Reasons for data becoming more similar can then be investigated.

h) Evaluate the effects of changes in data collection methods through time. This can give us more knowledge on which quality improvement measures that may be effective.

In order to further contribute towards improved data quality on vaccination coverage the following studies may be considered undertaken by Statistics Norway or others:

a) A literature review and re-analyses of data can improve the current knowledge base without collecting new data. We suggest a search for all available data on chosen countries and a re-analysis, using similar methods to Lim et al. (2008) (i.e. BDDR), adding more recent data. An analysis separating the investment phase from the reward payments would provide a clearer picture.

b) It is possible to measure the vaccination coverage in annual small surveys. Keeping the size of such surveys to a minimum makes them affordable and possible to process rapidly. If one vaccine can be seen as having a coverage representative for other vaccines or if it is particularly important, it can be used as a proxy for the coverage rate of other vaccines. Secondly, calculations of vaccination coverage are sensitive to the estimated population size. Small surveys (and other surveys between censuses) can also be used to improve the estimated population size. We suggest a pilot to evaluate what effects a small annual survey can have on the quality of vaccination coverage data.

c) Evaluation of different data collection methods. The data gathered for the Lim et al. (2008) study should be evaluated to identify points in time where survey and administrative data converge. And thus, reasons for data converging ought to be established. We also recommend investigating the effects of changes in data collection methods through time. It will e.g. be useful to identify changes in data collection that happens at the same time as changes in the time series. This can give us more knowledge on which quality improvement measures may be effective.