10 Conclusion
This guide has run through how to produce poverty estimates for out of sample areas using the Fay-Herriot model in R. Section A of the guide details how to collect the geospatial data from various online sources and harmonise the information into one single dataset. Section B explains the estimation procedure and the steps that must be taken to ensure precise, robust estimates. The combination of household data and geospatial data rather than census data (which is the method that has traditionally been used) enables the user to produce poverty estimates with more frequency and at a lower cost. Furthermore the public availability of geospatial data ensures that this method is available to a wider audience of agencies which should add valuable momentum to the poverty reduction conversation.
The motivation for this guide is therefore twofold:
From an academic standpoint:
Closing data gaps by providing estimates for previously hard to reach out of sample areas.
Integrating data by showing how to combine geospatial and household data.
Improving precision of estimates by providing methodology to produce a robust model that compares favourably to other previous estimation techniques
From a policy standpoint:
Lowering costs for government agencies, NGOs and thinktanks.
Influencing and improving policy decisions and social protection programs in order to drive poverty reduction. This is done by providing more frequent statistics at higher precision than previously available which allows policy makers to make more informed decisions.
Caveats
Despite these clear benefits, small area estimation is not a panacea. The out of sample estimates are only as good as your model, the assumptions the model makes and the underlying data that goes into the model. As a result these are some of the key considerations you need to take into account before undertaking small area estimation:
Sample size: The Fay-Herriot model assumes a large sample size for both the target and auxiliary variables. If the sample size in the small area is small, the model’s assumptions may be violated, leading to unreliable estimates.
Model specification: Choosing an appropriate model specification is crucial. The Fay-Herriot model assumes a linear relationship between the small area and auxiliary variables. If this assumption is not met, the model may provide biased estimates. It’s important to assess the linearity assumption and consider alternative model specifications if needed.
Auxiliary variable availability: The Fay-Herriot model relies on accurate and relevant auxiliary variables that are correlated with the small area variable of interest. If suitable auxiliary variables are not available or are not strongly correlated with the target variable, the model’s effectiveness may be limited.
Assumption of homoscedasticity: The Fay-Herriot model assumes that the variance of the small area random effects is constant across all areas. If there are substantial differences in the variance between areas, the model may not capture the heterogeneity adequately, leading to biased estimates.
Model extrapolation: The Fay-Herriot model relies on borrowing strength from neighboring areas to estimate small area parameters. However, if the small area is significantly different from the neighboring areas, the model’s estimates may not accurately reflect the true values.
It is therefore vitally important to perform model diagnostics throughout the process such as assessing the goodness-of-fit and assessing the predictive power of the model in order to evaluate the appropriateness of the Fay-Herriot model. Failure to perform proper diagnostics may lead to incorrect inferences.
It’s important to carefully consider these caveats and assess the suitability of the Fay-Herriot model in the specific context of your small area estimation problem. Alternative methods should be explored if the assumptions of the Fay-Herriot model are not met or if there are specific considerations unique to your data.