Reporting all results efficiently: A RARE proposal to open up the file drawer

David D Laitin, Edward Miguel, Ala' Alrababa'h, Aleksandar Bogdanoski, Sean Grant, Katherine Hoeberling, Cecilia Hyunjung Mo, Don A Moore, Simine Vazire, Jeremy Weinstein, Scott Williamson
2021
pnas.2106178118.pdf

Laitin, David, Edward Miguel, Ala’ Alrababa’h, Aleksandar Bogdanoski, Sean Grant, Katherine Hoeberling, Cecilia Hyunjung Mo, Don A Moore, Simine Vazire, Jeremy Weinstein, Scott Williamson. (2021) “Reporting all results efficiently: A RARE proposal to open up the file drawer”, PNAS, 118(52):e2106178118. doi: 10.1073/pnas.2106178118.

While the social sciences have made impressive progress in adopting transparent research practices that facilitate verification, replication, and reuse of materials, the problem of publication bias persists. Bias on the part of peer reviewers and journal editors, as well as the use of outdated research practices by authors, continues to skew literature toward statistically significant effects, many of which may be false positives. To mitigate this bias, we propose a framework to enable authors to report all results efficiently (RARE), with an initial focus on experimental and other prospective empirical social science research that utilizes public study registries. This framework depicts an integrated system that leverages the capacities of existing infrastructure in the form of public registries, institutional review boards, journals, and granting agencies, as well as investigators themselves, to efficiently incentivize full reporting and thereby, improve confidence in social science findings. In addition to increasing access to the results of scientific endeavors, a well-coordinated research ecosystem can prevent scholars from wasting time investigating the same questions in ways that have not worked in the past and reduce wasted funds on the part of granting agencies.

Skill Versus Voice in Local Development

Katherine Casey, Rachel Glennerster, Edward Miguel, Maarten Voors
2021
rest a 01082.pdf

Casey, Katherine , Rachel Glennerster, Edward Miguel, Maarten Voors. (2021) “Skills Versus Voice in Local Development”, The Review of Economics and Statistics, doi: 10.1162/rest_a_01082

Where the state is weak, traditional authorities control the local provision of public goods. These leaders come from an older, less educated generation and often rule in an authoritarian and exclusionary fashion. This means the skills of community members may not be leveraged in policy making. We experimentally evaluate two solutions to this problem in Sierra Leone: one encourages delegation to higher skill individuals and a second fosters broader inclusion in decision-making. In a real-world infrastructure grants competition, a public nudge to delegate lead to better outcomes than the default of chiefly control, whereas attempts to boost participation were largely ineffective.

Self-reported vs Directly Observed Face Mask Use in Kenya

Aleksandra Jakubowski, Dennis Egger, Carolyne Nekesa, Layna Lowe, Michael Walker, Edward Miguel
2021
jakubowski 2021 ld 210154 1627056043.52861-1.pdf

Jakubowski, Alexandra, Dennis Egger, Carolyne Nekesa, Layna Lowe, Michael Walker, Edward Miguel. (2021) “Self-reported vs Directly Observed Face Mask Use in Kenya”, JAMA Network Open, 4(7):e2118830, doi:10.1001/jamanetworkopen.2021.18830

Although countries in sub-Saharan Africa have so far avoided large-scale outbreaks of COVID-19, as of June 2021, the number of new cases has been surging in multiple nations. The official case count in Kenya was 179 876 confirmed cases in June 2021, including 1597 cases in Siaya county. One explanation for the relatively low spread of COVID-19 is the strict lockdown measures that were imposed early but were eventually eased because of their harsh socioeconomic consequences. Masks emerged as a prominent strategy to reduce SARS-CoV-2 transmission before widespread vaccinations and have been legally mandated in Kenya since April 2020. Yet compliance with public health guidelines is not guaranteed and motivating people to adopt new preventive health behaviors is inherently difficult. In this cross-sectional study, we examine the extent to which mask mandates are followed and quantify the bias of self-reported mask usage.

Evidence on Research Transparency in Economics

Edward Miguel
2021
Research Methodology
jep.35.3.193.pdf

Miguel, Edward. (2021). “Evidence on Research Transparency in Economics”, Journal of Economic Perspectives, 35(3): 193-214, DOI: 10.1257/jep.35.3.193.

A decade ago, the term “research transparency” was not on economists’ radar screen, but in a few short years a scholarly movement has emerged to bring new open science practices, tools and norms into the mainstream of our discipline. The goal of this article is to lay out the evidence on the adoption of these approaches – in three specific areas, open data, pre-registration and pre-analysis plans, and journal policies – and, more tentatively, begin to assess their impacts on the quality and credibility of economics research. The evidence to date indicates that economics (and related quantitative social science fields) are in a period of rapid transition toward new transparency-enhancing norms. While solid data on the benefits of these practices in economics is still limited, in part due to their relatively recent adoption, there is growing reason to believe that critics’ worst fears regarding onerous adoption costs have not been realized. Finally, the article presents a set of frontier questions and potential innovations.

Falling Living Standards during the COVID-19 Crisis: Quantitative Evidence from Nine Developing Countries

Dennis Egger, Edward Miguel, Shana S. Warren, Ashish Shenoy, Elliott Collins, Dean Karlan, Doug Parkerson, A. Mushfiq Mobarak, Günther Fink, Christopher Udry, Michael Walker, Johannes Haushofer, Magdalena Larreboure, Susan Athey, Paula Lopez-Pena, Salim Benhachmi, Macartan Humphreys, Layna Lowe, Niccoló F. Meriggi, Andrew Wabwire, C. Austin Davis, Utz Johann Pape, Tilman Graff, Maarten Voors, Carolyn Nekesa, Corey Vernot
2021
African DevelopmentHealth
eabe0997.full final-published.pdf

Egger, Dennis, Edward Miguel, Shana S. Warren, Ashish Shenoy, Elliott Collins, Dean Karlan, Doug Parkerson, A. Mushfiq Mobarak, Günther Fink, Christopher Udry, Michael Walker, Johannes Haushofer, Magdalena Larreboure, Susan Athey, Paula Lopez-Pena, Salim Benhachmi, Macartan Humphreys, Layna Lowe, Niccoló F. Meriggi, Andrew Wabwire, C. Austin Davis, Utz Johann Pape, Tilman Graff, Maarten Voors, Carolyn Nekesa, Corey Vernot. (2021). “Falling Living Standards during the COVID-19 Crisis: Quantitative Evidence from Nine Developing Countries”, Science Advances, 7(6), doi: 10.1126/sciadv.abe0997.

Despite numerous journalistic accounts, systematic quantitative evidence on economic conditions during the ongoing COVID-19 pandemic remains scarce for most low- and middle-income countries, partly due to limitations of official economic statistics in environments with large informal sectors and subsistence agriculture. We assemble evidence from over 30,000 respondents in 16 original household surveys from nine countries in Africa (Burkina Faso, Ghana, Kenya, Rwanda, Sierra Leone), Asia (Bangladesh, Nepal, Philippines), and Latin America (Colombia). We document declines in employment and income in all settings beginning March 2020. The share of households experiencing an income drop ranges from 8 to 87% (median, 68%). Household coping strategies and government assistance were insufficient to sustain precrisis living standards, resulting in widespread food insecurity and dire economic conditions even 3 months into the crisis. We discuss promising policy responses and speculate about the risk of persistent adverse effects, especially among children and other vulnerable groups.

A Framework for Open Policy Analysis

Fernando Hoces de la Guardia, Sean Grant, and Edward Miguel
2020
Research Methodology
Hoces-de-la-Guardia-et-al Open-Policy-Analysis 2020-10.pdf

Hoces de la Guardia, Fernando, Sean Grant, and Edward Miguel. (2020). "A Framework for Open Policy Analysis", forthcoming Science and Public Policy.

The evidence-based policy movement promotes the use of empirical evidence to inform policy decision-making. While several social science disciplines are undergoing a “credibility revolution” focused on openness and replication, policy analysis has yet to systematically embrace transparency and reproducibility. We argue that policy analysis should adopt the open research practices increasingly espoused in related disciplines to advance the credibility of evidence-based policymaking. We first discuss the importance of evidence-based policy in an era of increasing disagreement about facts, analysis, and expertise. We present a novel framework for “open” policy analysis (OPA) and how to achieve it, focusing on examples of recent policy analyses that have incorporated open research practices such as transparent reporting, open data, and code sharing. We conclude with recommendations on how key stakeholders in evidence-based policy can make OPA the norm and thus safeguard trust in using empirical evidence to inform important public policy decisions.

Twenty Year Economic Impacts of Deworming

Hamory, Joan, Edward Miguel, Michael Walker, Michael Kremer, and Sarah Baird
2021
Education and Human CapitalHealth
Twenty-year-economic-impacts-of-deworming.pdf
This study exploits a randomized school health intervention that provided deworming treatment to Kenyan children and utilizes longitudinal data to estimate impacts on economic outcomes up to 20 years later. The effective respondent tracking rate was 84%. Individuals who received 2 to 3 additional years of childhood deworming experience an increase of 14% in consumption expenditure, 13% in hourly earnings, 9% in non-agricultural work hours, and are 9% more likely to live in urban areas. Most effects are concentrated among males and older individuals. Given deworming’s low cost, a conservative annualized social internal rate of return estimate is 37%.

Deepening or Diminishing Ethnic Divides? The Impact of Urban Migration in Kenya

Kramon, Eric, Sarah Baird, Joan Hamory Hicks, and Edward Miguel.
2021
African DevelopmentPolitical Economy and Conflict
KLPS-Urbanization 190219.pdf

Kramon, Eric, Sarah Baird, Joan Hamory Hicks, and Edward Miguel. (2021). “Deepening or Diminishing Ethnic Divides? The Impact of Urban Migration in Kenya”, American Journal of Political Science, doi: 10.1111/ajps.12590.

The impact of urban migration on ethnic politics is the subject of longstanding debate. “First generation” modernization theories predict that urban migration should reduce ethnic identification and increase trust between groups. “Second generation” modernization perspectives argue the opposite: urban migration may amplify ethnic identification and reduce trust. We test these competing expectations with a three-wave panel survey following more than 8,000 Kenyans over a 15-year period, providing novel evidence on the impact of urban migration. Using individual fixed effects regressions, we show that urban migration leads to reductions in ethnic identification: ethnicity’s importance to the individual diminishes after migrating. Yet urban migration also reduces trust between ethnic groups, and trust in people generally. Urban migrants become “less ethnic” and more suspicious. The results advance the literature on urbanization and politics and have implications for the potential consequences of ongoing urbanization processes around the world.

Research Transparency is on the Rise in Economics

Swanson, Nicholas, Garret Christensen, Rebecca Littman, David Birke, Edward Miguel, Elizabeth Levy Paluck, Zenan Wang
2020
Research Methodology
Swanson-et-al 3S-Econ AEA-pandp.20201077-PUBLISHED 2020.pdf

This study provides a first assessment of awareness of, attitudes toward, perceived norms regarding, and adoption of open science practices within a broadly representative sample of active economics researchers. We observe a steep increase in adoption over the last decade, with an accelerating trend: as of 2017, 93 percent of economists had used at least one such practice—including posting data, sharing study materials, and study pre-registration—rising from 33 percent a decade earlier. We document extensive variation in adoption across economics subfields. Notably, most economists appear to underestimate the trend toward research transparency in the discipline.

Using survey questions to measure preferences: Lessons from an experimental validation in Kenya

Michal Bauer, Julie Chytilova, and Edward Miguel
2020
African DevelopmentResearch Methodology
Bauer-et-al EER 1-s2.0-S0014292120301252-main 2020-Published.pdf

Can a short survey instrument reliably measure a range of fundamental economic preferences across diverse settings? We focus on survey questions that systematically predict behavior in incentivized experimental tasks among German university students (Becker et al. 2016) and were implemented among representative samples across the globe (Falk et al. 2018). This paper presents results of an experimental validation conducted among low-income individuals in Nairobi, Kenya. We find that quantitative survey measures –hypothetical versions of experimental tasks –of time preference, attitude to risk and altruism are good predictors of choices in incentivized experiments, suggesting these measures are broadly experimentally valid. At the same time, we find that qualitative questions –self-assessments –do not correlate with the experimental measures of preferences in the Kenyan sample. Thus, caution is needed before treating self-assessments as proxies of preferences in new contexts.

A study of the impact of data sharing on article citations using journal policies as a natural experiment

Christensen, Garret, Allan Dafoe, Edward Miguel, Don A. Moore, and Andrew K. Rose
2019
Research Methodology
journal.pone .0225883.pdf

This study estimates the effect of data sharing on the citations of academic articles, using journal policies as a natural experiment. We begin by examining 17 high-impact journals that have adopted the requirement that data from published articles be publicly posted. We match these 17 journals to 13 journals without policy changes and find that empirical articles published just before their change in editorial policy have citation rates with no statistically significant difference from those published shortly after the shift. We then ask whether this null result stems from poor compliance with data sharing policies, and use the data sharing policy changes as instrumental variables to examine more closely two leading journals in economics and political science with relatively strong enforcement of new data policies. We find that articles that make their data available receive 97 additional citations (estimate standard error of 34). We conclude that: a) authors who share data may be rewarded eventually with additional scholarly citations, and b) data-posting policies alone do not increase the impact of articles published in a journal unless those policies are enforced.

Non-economic factors in violence: Evidence from organized crime, suicides and climate in Mexico

Baysan, Ceren, Marshall Burke, Felipe Gonzalez, Solomon Hsiang, and Edward Miguel
2019
Environment and ClimatePolitical Economy and Conflict
Mexico-JEBO-final 2019.pdf

Organized intergroup violence is almost universally modeled as a calculated act motivated by economic factors. In contrast, it is generally assumed that non-economic factors, such as an individual’s emotional state, play a role in many types of interpersonal violence, such as crimes of passion. We ask whether non-economic factors can also explain the well-established relationship between temperature and violence in a unique context where intergroup killings by drug-trafficking organizations (DTOs) and other interpersonal homicides are separately documented. A constellation of evidence, including the limited influence of a cash transfer program as well as comparisons with both other DTO crime and suicides, indicate that economic factors only partially mitigate the relationship between temperature and violence that we estimate in Mexico. We argue that non-economic psychological and physiological factors that are affected by temperature, modeled here as a “taste for violence,” likely play an important role in causing both interpersonal and intergroup violence.