History of risk assessment, and some proposed alternate methods
A 2002 paper by ERIC SILVER and LISA L. MILLER on actuarial risk assessment tools provides a history of statistical prediction in the criminal justice context, and issues cautions now central to the contemporary algorithmic fairness conversations:
“Much as automobile insurance policies determine risk levels based on the shared characteristics of drivers of similar age, sex, and driving history, actuarial risk assessment tools for predicting violence or recidivism use aggregate data to estimate the likelihood that certain strata of the population will commit a violent or criminal act.
To the extent that actuarial risk assessment helps reduce violence and recidivism, it does so not by altering offenders and the environments that produced them but by separating them from the perceived law-abiding populations. Actuarial risk assessment facilitates the development of policies that intervene in the lives of citizens with little or no narrative of purpose beyond incapacitation. The adoption of risk assessment tools may signal the abandonment of a centuries-long project of using rationality, science, and the state to improve upon the social and economic progress of individuals and society.”
Link popup: yes to the paper.
A more recent paper presented at FAT* in 2018 and co-authored by CHELSEA BARABAS, KARTHIK DINAKAR, JOICHI ITO, MADARS VIRZA, and JONATHAN ZITTRAIN makes several arguments reminiscent of Silver and Miller’s work. They argue in favor of causal inference framework for risk assessments aimed at working on the question “what interventions work”:
“We argue that a core ethical debate surrounding the use of regression in risk assessments is not simply one of bias or accuracy. Rather, it’s one of purpose.… Data-driven tools provide an immense opportunity for us to pursue goals of fair punishment and future crime prevention. But this requires us to move away from merely tacking on intervenable variables to risk covariates for predictive models, and towards the use of empirically-grounded tools to help understand and respond to the underlying drivers of crime, both individually and systemically.”
Link popup: yes to the paper.
- In his 2007 book Against Prediction popup: yes, lawyer and theorist Bernard Harcourt provided detailed accounts and critiques of the use of actuarial methods throughout the criminal legal system. In place of prediction, Harcourt proposes a conceptual and practical alternative: randomization. From a 2005 paper on the same topic: “Instead of embracing the actuarial turn in criminal law, we should rather celebrate the virtues of the random: randomization, it turns out, is the only way to achieve a carceral population that reflects the offending population. As a form of random sampling, randomization in policing has significant positive value: it reinforces the central moral intuition in the criminal law that similarly situated individuals should have the same likelihood of being apprehended if they offend—regardless of race, ethnicity, gender or class.” Link popup: yes to the paper. (And link popup: yes to another paper of Harcourt’s in the Federal Sentencing Reporter, “Risk as a Proxy for Race.”)
- A recent paper by Megan Stevenson assesses risk assessment tools: “Despite extensive and heated rhetoric, there is virtually no evidence on how use of this ‘evidence-based’ tool affects key outcomes such as incarceration rates, crime, or racial disparities. The research discussing what ‘should’ happen as a result of risk assessment is hypothetical and largely ignores the complexities of implementation. This Article is one of the first studies to document the impacts of risk assessment in practice.” Link popup: yes.
- A compelling piece of esoterica cited in Harcourt’s book: a doctoral thesis by Deborah Rachel Coen on the “probabilistic turn” in 19th century imperial Austria. Link popup: yes.
A new organization provides scholars access to Facebook data
“Social Science One implements a new type of partnership between academic researchers and private industry to advance the goals of social science in understanding and solving society’s greatest challenges. The partnership enables academics to analyze the increasingly rich troves of information amassed by private industry in responsible and socially beneficial ways. It ensures the public maintains privacy while gaining societal value from scholarly research. And it enables firms to enlist the scientific community to help them and produce social good, while protecting their competitive positions.”
Link popup: yes to the overview. A paper by GARY KING and NATE PERSILY details the conceptualizing, structuring, and funding of Social Science One, as well as its ethics standards. Link popup: yes. ht Will
Here’s the first data set Social Science One has arranged to make accessible to researchers:
“The data describes web page addresses (URLs) that have been shared on Facebook starting January 1, 2017 and ending about a month before the present day. URLs are included if shared by at least 20 unique accounts, and shared publicly at least once. We estimate the full data set will contain on the order of 2 million unique urls shared in 300 million posts, per week.”
Link popup: yes to the dataset, which was posted earlier this month and has been downloaded hundreds of times.
- Social science is increasingly preoccupied with, as SSO puts it, “enabl[ing] academics to analyze the increasingly rich troves of information amassed by private industry.” Just in the past six months we’ve spotlighted research in the same vein from Dan Herbst popup: yes (linking Transunion credit bureau data with student loan servicing files), Ozan Jaquette popup: yes (scraping travel data from college admissions counselors), and Matt Desmond popup: yes (combining formal eviction records with county-level data on landlord-tenant cases).
- Last month, professors at Princeton and Duke (with funding from Russell Sage and Sloan) hosted the second-annual Summer Institute in Computational Social Science popup: yes, a 13-day workshop that trains early-career scholars in digital field experiments, web scraping, and mass collaboration. Gary King and Sendhil Mullainthan were among the speakers. Syllabus and slides here popup: yes.
- Yale’s sociology department—the oldest in America—recently posted popup: yes multiple tenure track job openings for “applicants whose work uses quantitative methods to study social inequality.” There’s a new popup: yes (launched January 2018) Journal of Computational Social Science as well.
- Where Historians Work, an interactive database which “tracks the current employment status (as of 2017) of history and history of science PhDs who graduated from all 161 history PhD-granting departments in the United States from 2004 to 2013.” Link popup: yes.
- On a similar note, an interactive map from the Hamilton Project shows wage variation, sortable by occupation, across the country. Link popup: yes.
- Tort law and the internet: “IoT devices enable both familiar and new opportunities for harmful industry overreach, with the added twist that these practices can now cause physical harm and property damage. Simultaneously, there is little government oversight or routes of recourse for injured consumers under extant law.” Link popup: yes.
- UBI news: In Chicago, alderman Ameya Pawar has introduced legislation to create a UBI pilot. Link popup: yes. Obama mentioned his support for the idea in a speech in South Africa. Link popup: yes.
- An overview of ISAs in the Economist. Link popup: yes. ht Will
- For the mobility files: New work from Card, Domnisoru and Taylor suggests “an important causal role for school quality in mediating upward mobility.” Link popup: yes.
- Economist Trevor Logan with a paper on the marriage rates and patterns of enslaved African Americans in Louisiana, using unique hospital records. Link popup: yes.
- The Matthew effect in NIH research funding. “Prestigious institutions had on average 65% higher grant application success rates and 50% larger award sizes, whereas less-prestigious institutions produced 65% more publications and had a 35% higher citation impact per dollar of funding.” Link popup: yes.
- “Mexican manufacturing job loss induced by competition with China increases cocaine trafficking and violence, particularly in municipalities with transnational criminal organizations.” Link popup: yes.
- From Ken Opalo’s Africanist Perspective blog: “Is Somalia’s Al Shaabab better at tax collection than most low-income states?” Link popup: yes.
- “Can technological change contribute to political turnover?… High-yielding variety (HYV) crops strengthened the incentives and capacity of a politically excluded group, in this case agricultural producers, to seek greater political representation and played a role in the rise of agrarian opposition parties and decline of single-party dominance.” Link popup: yes.
- “Why randomized controlled trials inevitably produce biased results.” Link popup: yes. ht Sidhya
- Our friends and collaborators at the Stanford Basic Income Lab are hiring a project manager. Job description here popup: yes.
Each week we highlight research from a graduate student, postdoc, or early-career professor. Send us recommendations: firstname.lastname@example.org