Meta-Analysis and Public Policy: Reconciling the Evidence on Deworming

Croke, Kevin, Joan Hamory Hicks, Eric Hsu, Michael Kremer, Ricardo Maertens, Edward Miguel, Witold Więcek
African DevelopmentResearch Methodology
SSRN-id2807693.pdf

The WHO recommends mass drug administration (MDA) in areas with >20% prevalence. Recent Cochrane meta-analyses endorse treatment of infected individuals but recommend against MDA. A theory-agnostic meta-analysis of the effect of multiple-dose MDA rejects the hypothesis of a common zero effect on child weight, mid-upper arm circumference, and height, and estimates significant average impacts. Estimates of implied treatment effect on infected children with MDA are not significantly different than those found for test and treat trials. These results suggest that MDA is a cost-effective intervention, particularly in the settings recommended by the WHO.

Destructive behavior, economic decision-making, and judgment under thermal stress

Ingvild Almås, Maximilian Auffhammer, Tessa Bold, Ian Bolliger, Aluma Dembo, Solomon M. Hsiang, Shuhei Kitamura, Edward Miguel & Robert Pickmans
w25785.pdf

Accumulating evidence indicates that environmental temperature substantially affects economic outcomes and violence, but the reasons for this linkage are only partially understood. While factors external to human beings (such as agricultural production) are known to respond adversely to high temperatures, extreme temperatures could also directly influence the internal mental processes governing decision-making. We study this by systematically evaluating the effect of thermal stress on multiple dimensions of economic decision-making, judgment, and destructive behavior with 2,000 participants in Kenya and the US who were randomly assigned to different
temperatures in a laboratory. We find that heat significantly affects individuals’ willingness to voluntarily destroy other participants’ assets, with pronounced increases among those experiencing heightened political conflict in Kenya. We find that other major dimensions of economic decision making are largely unaffected by temperature.

Preparing for an Aging Africa: Data-Driven Priorities for Economic Research and Policy

Madeline Duhon, Edward Miguel, Amos Njuguna, Daniela Pinto Veizaga, Michael Walker
African Development
eScholarship-UC-item-024117rq.pdf

Duhon, Madeline, et al. (2023) “Preparing for an Aging Africa: Data-Driven Priorities for Economic Research and Policy”,  https://doi.org/10.3386/w31750.

The over-60 population in Sub-Saharan Africa is expected to grow rapidly in the coming decades, tripling between 2020 and 2050. Despite this explosive projected growth, few countries in the region have implemented policies designed to support older populations. Further, little research in economics has specifically examined aging in Sub-Saharan Africa, though many opportunities exist for economists to generate research evidence to inform the design of effective policies in this area. This paper combines insights from a cross-disciplinary review with original data analysis to characterize the challenges and opportunities facing older Sub-Saharan Africans in domains such as health and financial security. Informed by these findings, the paper identifies directions for future economic research and discusses policy recommendations, including the need to reform health care systems and expand pension and other public support programs to prepare for an aging Africa.

Donor contracting conditions and public procurement: Causal evidence from Kenyan electrification

Catherine D. Wolfram, Edward Miguel, Eric Hsu, Susanna B. Berkouwer
African Development
WMHB donorprocurement-SUBMITTED.pdf

Wolfram, Catherine, Edward Miguel, Eric Hsu, Susanna B. Berkouwer. (2023) “Donor Contracting Conditions and Public Procurement: Causal Evidence from Kenyan Electrification”, National Bureau of Economic Research (NBER) Working Paper #30948.

There is limited causal evidence on the effects of different public procurement regulations on project quality and value-for-money for projects funded both by national governments and foreign aid donors. This paper uses both policy and experimental variation to study how two key contracting features—namely, contract bundling and monitoring—affect outcomes of a large economic development project. We leverage an unusual feature of Kenya’s nationwide electrification program: the quasi-random allocation of multilateral funding sources across nearby villages. African Development Bank (AfDB) projects used bundled contracts while the World Bank (WB) employed unbundled contracts together with strengthened inspections. To measure impacts, we collect on-the-ground engineering assessments, voltage and reliability data, household survey data on connection quality and usage, and analyze original contracts. The analysis suggests a stark trade-off: WB procedures delayed construction completion by 16 months on average relative to AfDB sites but improved construction quality by a sizeable 0.6 standard deviations. To disentangle the effects of contract bundling versus monitoring, we implement a randomized audits scheme that enhanced monitoring. The audits improve household connectivity, network size, and voltage at AfDB sites, but have no impact at WB sites, suggesting monitoring and unbundling contracts may be substitutes. Given the apparent trade-off, we investigate how net benefits depend on policymaker time preferences and infrastructure longevity (due to improved quality). Under plausible assumptions, WB processes could generate a net benefit ranging anywhere from +4% to -7% of total project value, indicating that neither procurement approach clearly dominates the other in this context.

Targeting Impact versus Deprivation

Johannes Haushofer, Paul Niehaus, Carlos Paramo, Edward Miguel, and Michael Walker
2022
African DevelopmentResearch Methodology
Targeting-impact-vs-deprivation MAIN-TEXT 2022-12-15.pdf

Haushofer, Johannes, Paul Niehaus, Carlos Paramo, Edward Miguel, Michael Walker. (2022). “Targeting impact versus deprivation”, National Bureau of Economic Research (NBER) Working Paper #30138.

A large literature has examined how best to target anti-poverty programs to those most deprived in some sense (e.g., consumption). We examine the potential tradeoff between this objective versus targeting those most impacted by such programs. We work in the context of an NGO cash transfer program in Kenya, utilizing recent advances in machine learning methods and dynamic outcome data to learn proxy means tests that target both objectives in turn. Targeting solely on the basis of deprivation is generally not attractive under standard social welfare criteria, even when the planner’s preferences
are highly redistributive.

Pre-results Review at the Journal of Development Economics: Lessons Learned

Self-Control and Demand for Preventive Health: Evidence from Hypertension in India

Bai, Liang, Benjamin Handel, Edward Miguel, and Gautam Rao
2020
HealthEducation and Human CapitalOther
EHP draft complete 2019-08-23-CLEAN.pdf
Self-control problems constitute a potential explanation for the under-investment in preventive health in low-income countries. Behavioral economics offers a tool to solve such problems: commitment devices. We conduct a field experiment to evaluate the effectiveness of different types of theoretically-motivated commitment contracts in increasing preventive doctor visits by hypertensive patients in rural India. Despite achieving high take-up of such contracts in some treatment arms, we find no effects on actual doctor visits or individual health outcomes. A substantial number of individuals pay for commitment but fail to follow through on the doctor visit, losing money without experiencing health benefits. We develop and structurally estimate a pre-specified model of consumer behavior under present bias with varying levels of naivete. The results are consistent with a large share of individuals being partially naive about their own self-control problems: sophisticated enough to demand some commitment, but overly optimistic about whether a given level of commitment is suffciently strong to be effective. The results suggest that commitment devices may in practice be welfare diminishing, at least in some contexts, and serve as a cautionary tale about their role in health care.

Does Mass Deworming Affect Child Nutrition? Meta-analysis, Cost-effectiveness, and Statistical Power

Croke, Kevin, Joan Hamory Hicks, Eric Hsu, Michael Kremer, and Edward Miguel
2016
HealthEducation and Human CapitalResearch Methodology
worms v93.pdf
The WHO has recently debated whether to reaffirm its long-standing recommendation of mass drug administration (MDA) in areas with more than 20% prevalence of soil-transmitted helminths (hookworm, whipworm, and roundworm). There is consensus that the relevant deworming drugs are safe and effective, so the key question facing policymakers is whether the expected benefits of MDA exceed the roughly $0.30 per treatment cost. The literature on long run educational and economic impacts of deworming suggests that this is the case. However, a recent meta-analysis by Taylor-Robinson et al. (2015) (hereafter TMSDG), disputes these findings. The authors conclude that while treatment of children known to be infected increases weight by 0.75 kg (95% CI: 0.24, 1.26; p=0.0038), there is substantial evidence that MDA has no impact on weight or other child outcomes. We update the TMSDG analysis by including studies omitted from that analysis and extracting additional data from included studies, such as deriving standard errors from p-values when the standard errors are not reported in the original article. The updated sample includes twice as many trials as analyzed by TMSDG, substantially improving statistical power. We find that the TMSDG analysis is underpowered: it would conclude that MDA has no effect even if the true effect were (1) large enough to be cost-effective relative to other interventions in similar populations, or (2) of a size that is consistent with results from studies of children known to be infected. The hypothesis of a common zero effect of multiple-dose MDA deworming on child weight at longest follow-up is rejected at the 10% level using the TMSDG dataset, and with a p-value < 0.001 using the updated sample. Applying either of two study classification approaches used in previous Cochrane Reviews (prior to TMSDG) also leads to rejection at the 5% level. In the full sample, including studies in environments where prevalence is low enough that the WHO does not recommend deworming, the average effect on child weight is 0.134 kg (95% CI: 0.031, 0.236, random effects estimation). In environments with greater than 20% prevalence, where the WHO recommends mass treatment, the average effect on child weight is 0.148 kg (95% CI: 0.039, 0.258). The implied average effect of MDA on infected children in the full sample (calculated by dividing estimated impact by worm prevalence for each study and applying a random effects model) is 0.301 kg. At 0.22 kg per U.S. dollar, the estimated average weight gain per dollar expenditure from deworming MDA is more than 35 times that from school feeding programs as estimated in RCTs. Under-powered meta-analyses (such as TMSDG) are common in health research, and this methodological issue will be increasingly important as growing numbers of economists and other social scientists conduct meta-analysis.

Analysis of statistical power reconciles drought-conflict results in Africa

Solomon M. Hsiang, Marshall Burke, Edward Miguel, Kyle C. Meng, Mark A. Cane
2016
African DevelopmentEnvironment and ClimateResearch Methodology
eScholarship UC item 77s421cd.pdf
Whether changes in climate substantially shape human conflict is a question of considerable recent interest to both academics and policymakers. Despite a substantial body of evidence for a strong association between climate and conflict, it remains widely claimed that large-sample empirical evidence linking climatic conditions and modern human conflict in Africa is mixed and thus any reported evidence for a strong association should be discounted. Theisen, Holterman, and Buhaug (henceforth THB) is one of the studies used to support this claim. Here we show that the results in THB are not inconsistent with earlier studies that report a substantial e ect of of climate on conflict. We demonstrate using power calculations and Monte Carlo simulations that even if a large association between climate and conflict existed in the data, the approach of THB would not be able to reliably distinguish this association from a null e ect, indicating that the approach taken by THB is statistically underpowered in this context. Therefore THB's analysis provides no basis for discarding earlier analyses and THB's conclusions drawn from this analysis overstate the extent to which they disagree with the literature. We also demonstrate that THB's stated advantage from using exceptionally high resolution data is unlikely to be realized in their analysis, since high resolution rainfall data was not actually collected in the majority conflict zones studied by THB. Although unremarked in the original analysis, the rainfall data in THB are interpolations of sparse and incomplete rainfall measurements.

Tracking, Attrition and Data Quality in the Kenyan Life Panel Survey Round 1 (KLPS-1)

Sarah Baird, Joan Hamory Hicks, and Edward Miguel
2008
African DevelopmentHealthEducation and Human CapitalResearch Methodology
attrition paper FINAL-CIDER aug08.pdf
Understanding the possible pitfalls of survey data is critical for empirical research. Among other things, poor data quality can lead to biased regression estimates, potentially resulting in incorrect interpretations that mislead researchers and policymakers alike. Common data problems include difficulties in tracking respondents and high survey attrition, enumerator error and bias, and respondent reporting error. This paper describes and analyzes these issues in Round 1 of the Kenyan Life Panel Survey (KLPS-1), collected in 2003-2005. The KLPS-1 is an innovative longitudinal dataset documenting a wide range of outcomes for Kenyan youths who had originally attended schools participating in a deworming treatment program starting in 1998. The careful design of this survey allows for examination of an array of data quality issues.

Reconciling Temperature-conflict Results in Kenya

Solomon M. Hsiang, Marshall Burke, and Edward Miguel
2013
African DevelopmentEnvironment and ClimatePolitical Economy and Conflict
Reconciling Tempurature-conflict Results in Kenya.pdf

Theisen (JPR, 2012) recently constructed a novel high-resolution data set of intergroup and political conflict in Kenya (1989-2004) and examined whether the risk of conflict onset and incidence responds to annual pixel-level variations in temperature and precipitation. Thiesen concluded that only extreme precipitation is associated with conflict incidence and that temperature is unrelated to conflict, seemingly at odds with recent studies that found a positive association at the pixel scale (O'laughlin et al., PNAS 2012), at the country scale (Burke et al., PNAS 2009), and at the continental scale (Hsiang et al., Nature 2011) in Africa. Here we show these ndings can be reconciled when we correct the erroneous coding of temperature-squared in Thiesen. In contrast to the original conclusions presented in Theisen, both conflict onset and conflict incidence are significantly and positively associated with local temperature in this new and independently assembled data set.

Social Engineering: Evidence from a Suite of Take-up Experiments in Kenya

Michael Kremer, Edward Miguel, Sendhil Mullainathan, Clair Null, and Alix Peterson Zwane
2011
African DevelopmentEnvironment and ClimateHealthEducation and Human Capital

Kremer, Michael, Edward Miguel, Sendhil Mullainathan, Clair Null, and Alix Peterson Zwane. 2011. "Social Engineering: Evidence from a Suite of Take-up Experiments in Kenya." Unpublished Working Paper.

Many effective health products and behaviors available through the private market are not widely adopted in less developed countries. For example, fewer than 10% of households in our Kenyan study area treat their water with dilute chlorine. Using a suite of randomized evaluations, we find that information and marketing interventions do little to boost use of chlorine. However, chlorine take-up is highly sensitive to price, convenience and social context, with more than half of households using chlorine when an individually-packaged supply is delivered free to the home. The highest sustained take-up is achieved by combining free, convenient, salient, and public access through a point-of-collection chlorine dispenser system and a local promoter. More than half of households treat their water and this use continues 30 months later even though promoters are paid only for the first six months. The estimated long-run costs of this intervention at scale, including administrative costs, are between $0.25 and $0.50 per person per year.