Honeysuckle

CLEAR MEANS

Evaluating evidence-based policy

Over the past two decades, “evidence-based policy” has come to define the common sense of research and policymakers around the world. But while attempts have been made to create formalization schemes for the ranking of evidence for policy, a gulf remains between rhetoric about evidence-based policy and applied theories for its development.

In a 2011 paper, philosophers of science NANCY CARTWRIGHT and JACOB STEGENGA lay out a “theory of evidence for use,” discussing the role of causal counterfactuals, INUS conditions, and mechanisms in producing evidence—and how all this matters for its evaluators.

From the paper:

“Truth is a good thing. But it doesn’t take one very far. Suppose we have at our disposal the entire encyclopaedia of unified science containing all the true claims there are. Which facts from the encyclopaedia do we bring to the table for policy deliberation? Among all the true facts, we want on the table as evidence only those that are relevant to the policy. And given a collection of relevant true facts we want to know how to assess whether the policy will be effective in light of them. How are we supposed to make these decisions? That is the problem from the user’s point of view and that is the problem of focus here.

We propose three principles. First, policy effectiveness claims are really causal counterfactuals and the proper evaluation of a causal counterfactual requires a causal model that (i) lays out the causes that will operate and (ii) tells what they produce in combination. Second, causes are INUS conditions, so it is important to review both the different causal complexes that will affect the result (the different pies) and the different components (slices) that are necessary to act together within each complex (or pie) if the targeted result is to be achieved. Third, a good answer to the question ‘How will the policy variable produce the effect’ can help elicit the set of auxiliary factors that must be in place along with the policy variable if the policy variable is to operate successfully.”

Link to the paper.

• Cartwright has written extensively on evidence and its uses. See: her 2012 book Evidence Based Policy: A Practical Guide to Doing it Better; her 2011 paper in The Lancet on RCTs and effectiveness; and her 2016 co-authored monograph on child safety, featuring applications of the above reasoning.
• For further introduction to the philosophical underpinnings of Cartwright’s applied work, and the relationship between theories of causality and evidence, see her 2015 paper “Single Case Causes: What is Evidence and Why.” Link. And also: “Causal claims: warranting them and using them.” Link.
• Obliquely related, see this illuminating discussion of causality in the context of reasoning about discrimination in machine learning and the law, by JFI fellow and Harvard PhD Candidate Lily Hu and Yale Law School Professor Issa Kohler-Hausmann: “What’s Sex Got To Do With Machine Learning?” Link.
• A 2017 paper by Abhijit Banerjee et al: “A Theory of Experimenters,” which models “experimenters as ambiguity-averse decision-makers, who make trade-offs between subjective expected performance and robustness. This framework accounts for experimenters’ preference for randomization, and clarifies the circumstances in which randomization is optimal: when the available sample size is large enough or robustness is an important concern.” Link.

NEW RESEARCHERS

The Gold Standard and the Great Depression in Spain

Postdoctoral Fellow in Economic History ENRIQUE JORGE SOTELO researches the history of monetary policy. His PhD thesis analyzes the impact of currency depreciation on Spain’s financial sector between 1921-1935.

From the abstract:

“Spain experienced a severe economic contraction during the 1930s, especially during 1931, when depositors ran on banks and the exchange rate continued to deteriorate rapidly. My argument is in contrast with the traditional account that depicts Spain as having escaped the Great Depression because its currency was not convertible to gold and because it did not experience widespread bank failures. I argue instead that exchange rate depreciation multiplied the effects of the 1931 banking crisis, creating fragilities in the banking sector. It is a fact that the Banco de Espa˜na intervened as a lender of last resort during 1931, and that this was a necessary step, but this is not enough to explain why Spain was exceptional in terms of bank stability during the Great Depression. Instead, it was a combination of policy measures—that were either absent or came much later in a number of other countries—that kept the banking system afloat. This, however, did not translate in the resumption of economic growth or price stability. The Spanish economy contracted as sharply as the Italian, the French or the German did, and deflation kicked in from 1933. As a result, the Spanish experience does not seem to be a reliable counterfactual for other countries when discussing the links between exchange rate regimes and financial stability during the Great Depression.”

Link to the paper, link to Sotelo’s website.

Each week we highlight great work from a graduate student, postdoc, or early-career professor. Have you read any excellent research recently that you’d like to see shared here? Send it our way: editorial@jainfamilyinstitute.org.

+ + +

• On the blog, Kyle Moore explores the import of stratification economics for studying race: “The neoclassical and stratification approaches disagree over the causes of and remedies for racial disparities in socioeconomic outcomes and differ substantially in their understanding of income, education, wealth, and health. Link.
• From Marco Ranaldi, a proposal for a new way to measure inequality: income composition inequality, which “allows for (i) novel political economy analysis of the evolution of economic systems and (ii) the technical assessment of the relationship between the functional and personal distribution of income.” Link.
• “While the first business organizations to reach large size in the late nineteenth century did so through the route of vertical integration—formal ownership of assets and direct employment of workers—mid-twentieth-century franchising firms pioneered a new path to bigness, relying on restrictive contracts rather than formal integration to control their business organizations.” A brand new paper by Brian Callaci: “Control Without Responsibility: The Legal Creation of Franchising, 1960–1980.” Link.
• At Vox, an interview with Jessica Lovering of the Energy for Growth Hub about nuclear power—as technology and as policy. Link.
• The influence of 1930s populist radio demagogue Father Coughlin: “Exploiting variation in the radio signal strength as a result of topographic factors, I find that a one standard deviation increase in exposure to Coughlin’s broadcast opposing FDR reduced FDR’s vote share by about two percentage points in the 1936 presidential election.” By Tianyi Wang. Link.
• “Is there a gender-equality paradox in STEM?” New from the Harvard GenderSciLab, a paper by Sarah Richardson et al, and blog series by the authors. h/t Milo
• On the “true” fiscal multiplier during the European sovereign debt crisis. By Lucyna Gornicka et al. Link.
• “The knowledge commons, as we implemented it through the free and open source software licenses, and through those creative commons licenses that many of us worked on in different jurisdictions were a mistake, because they rest on the wrong assumptions, and are permeated by the wrong ideologies.” Balázs Bodó suggests that the knowledge commons is a “curse in disguise.” Link.
• “Peasant aristocrats? Wealth, social status and the politics of Swedish farmer parliamentarians 1769-1895.” New work by Erik Bengtsson and Mats Olsson. Link.
• Sam Asher and Paul Novosad on rural road development: “Nearly one billion people worldwide live in rural areas without access to national paved road networks. We estimate the impacts of India’s 40 billion USD national rural road construction program.” Among the findings: batter market connections, but no major changes in agricultural outcomes, income, or assets. Link.
• On Supply Chain Dive, a series on the impact of coronavirus on global supply chains. Link. h/t Francis
• A visualization of RePEc economists on Twitter, from Christian Mongeau. “Edges (links) show whether a user follows or is followed by another user. Colors represent the user’s main research area, based on an aggregation of their main NEP field.” Link.
• “One of the challenges that need to be overcome in policy evaluation studies is that there is hardly any experimental evidence related to R&D grants… more focus should be placed on researching the optimal design of policies. Which features of an R&D subsidy program are essential for maximizing its impact? And what types of firms benefit most from receiving a grant?” A short paper from Paul Hünermund and Dirk Czarnitzki suggests pilot studies on R&D. Link.
• From the AI Ethics lab, an interactive map shows ethics guidelines around the world. Link.
• “This paper exploits exogenous variation in the adoption of copyrights within Italy—as a result of the timing of Napoléon’s military victories—to examine the effects of copyrights on the creation of new operas. Although the primary purpose of copyrights is to encourage creativity, systematic evidence on the causal effects of copyrights continues to be scarce, primarily due to a lack of exogenous variation in modern copyright laws. Because opera is a public art form, new works are exceptionally well-documented, offering unique opportunities to observe changes in creativity. Difference-in-differences analyses show that basic copyrights increased both the number and the quality of operas, measured by their immediate success and durability.” By Michela Giorcelli and Petra Moser. Link.

Each week we highlight research from a graduate student, postdoc, or early-career professor. Send us recommendations: editorial@jainfamilyinstitute.org