Transparency and Reproducibility: Conceptualizing the Problem (Chapter 6) and Potential Solutions (Chapter 7)

Christensen, Garret, and Edward Miguel
2020
Research Methodology

A broad overview of research transparency and open science issues in the social sciences, across two chapters (Chapters 6 and 7).

Using survey questions to measure preferences: Lessons from an experimental validation in Kenya

Michal Bauer, Julie Chytilova, and Edward Miguel
2020
African DevelopmentResearch Methodology
Bauer-et-al EER 1-s2.0-S0014292120301252-main 2020-Published.pdf

Can a short survey instrument reliably measure a range of fundamental economic preferences across diverse settings? We focus on survey questions that systematically predict behavior in incentivized experimental tasks among German university students (Becker et al. 2016) and were implemented among representative samples across the globe (Falk et al. 2018). This paper presents results of an experimental validation conducted among low-income individuals in Nairobi, Kenya. We find that quantitative survey measures –hypothetical versions of experimental tasks –of time preference, attitude to risk and altruism are good predictors of choices in incentivized experiments, suggesting these measures are broadly experimentally valid. At the same time, we find that qualitative questions –self-assessments –do not correlate with the experimental measures of preferences in the Kenyan sample. Thus, caution is needed before treating self-assessments as proxies of preferences in new contexts.

Research Transparency is on the Rise in Economics

Swanson, Nicholas, Garret Christensen, Rebecca Littman, David Birke, Edward Miguel, Elizabeth Levy Paluck, Zenan Wang
2020
Research Methodology
Swanson-et-al 3S-Econ AEA-pandp.20201077-PUBLISHED 2020.pdf

This study provides a first assessment of awareness of, attitudes toward, perceived norms regarding, and adoption of open science practices within a broadly representative sample of active economics researchers. We observe a steep increase in adoption over the last decade, with an accelerating trend: as of 2017, 93 percent of economists had used at least one such practice—including posting data, sharing study materials, and study pre-registration—rising from 33 percent a decade earlier. We document extensive variation in adoption across economics subfields. Notably, most economists appear to underestimate the trend toward research transparency in the discipline.