Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment
<p>Algorithmic recommendations and decisions have become ubiquitous in today’s society. Many of these data-driven policies, especially in the realm of public policy, are based on known, deterministic rules to ensure their transparency and interpretability. We examine a particular case of algor...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | , , |
| Published: |
2025
|
| Subjects: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1852019099837661184 |
|---|---|
| author | Eli Ben-Michael (10803815) |
| author2 | D. James Greiner (21337843) Kosuke Imai (5329823) Zhichao Jiang (4724856) |
| author2_role | author author author |
| author_facet | Eli Ben-Michael (10803815) D. James Greiner (21337843) Kosuke Imai (5329823) Zhichao Jiang (4724856) |
| author_role | author |
| dc.creator.none.fl_str_mv | Eli Ben-Michael (10803815) D. James Greiner (21337843) Kosuke Imai (5329823) Zhichao Jiang (4724856) |
| dc.date.none.fl_str_mv | 2025-06-24T17:00:11Z |
| dc.identifier.none.fl_str_mv | 10.6084/m9.figshare.29039305.v2 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/dataset/Safe_Policy_Learning_through_Extrapolation_Application_to_Pre-trial_Risk_Assessment_sup_sup_/29039305 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Medicine Pharmacology Biotechnology Sociology Science Policy Mathematical Sciences not elsewhere classified Algorithm-assisted decision-making Decision-making under uncertainty Optimal policy learning Risk assessments Robust optimization |
| dc.title.none.fl_str_mv | Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment |
| dc.type.none.fl_str_mv | Dataset info:eu-repo/semantics/publishedVersion dataset |
| description | <p>Algorithmic recommendations and decisions have become ubiquitous in today’s society. Many of these data-driven policies, especially in the realm of public policy, are based on known, deterministic rules to ensure their transparency and interpretability. We examine a particular case of algorithmic pre-trial risk assessments in the US criminal justice system, which provide deterministic classification scores and recommendations to help judges make release decisions. Our goal is to analyze data from a unique field experiment on an algorithmic pre-trial risk assessment to investigate whether the scores and recommendations can be improved. Unfortunately, prior methods for policy learning are not applicable because they require existing policies to be stochastic. We develop a maximin robust optimization approach that partially identifies the expected utility of a policy, and then finds a policy that maximizes the worst-case expected utility. The resulting policy has a statistical safety property, limiting the probability of producing a worse policy than the existing one, under structural assumptions about the outcomes. Our analysis of data from the field experiment shows that we can safely improve certain components of the risk assessment instrument by classifying arrestees as lower risk under a wide range of utility specifications, though the analysis is not informative about several components of the instrument. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.</p> |
| eu_rights_str_mv | openAccess |
| id | Manara_2d6f3faa1f55e4237f497d8a581bd6cb |
| identifier_str_mv | 10.6084/m9.figshare.29039305.v2 |
| network_acronym_str | Manara |
| network_name_str | ManaraRepo |
| oai_identifier_str | oai:figshare.com:article/29039305 |
| publishDate | 2025 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Safe Policy Learning through Extrapolation: Application to Pre-trial Risk AssessmentEli Ben-Michael (10803815)D. James Greiner (21337843)Kosuke Imai (5329823)Zhichao Jiang (4724856)MedicinePharmacologyBiotechnologySociologyScience PolicyMathematical Sciences not elsewhere classifiedAlgorithm-assisted decision-makingDecision-making under uncertaintyOptimal policy learningRisk assessmentsRobust optimization<p>Algorithmic recommendations and decisions have become ubiquitous in today’s society. Many of these data-driven policies, especially in the realm of public policy, are based on known, deterministic rules to ensure their transparency and interpretability. We examine a particular case of algorithmic pre-trial risk assessments in the US criminal justice system, which provide deterministic classification scores and recommendations to help judges make release decisions. Our goal is to analyze data from a unique field experiment on an algorithmic pre-trial risk assessment to investigate whether the scores and recommendations can be improved. Unfortunately, prior methods for policy learning are not applicable because they require existing policies to be stochastic. We develop a maximin robust optimization approach that partially identifies the expected utility of a policy, and then finds a policy that maximizes the worst-case expected utility. The resulting policy has a statistical safety property, limiting the probability of producing a worse policy than the existing one, under structural assumptions about the outcomes. Our analysis of data from the field experiment shows that we can safely improve certain components of the risk assessment instrument by classifying arrestees as lower risk under a wide range of utility specifications, though the analysis is not informative about several components of the instrument. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.</p>2025-06-24T17:00:11ZDatasetinfo:eu-repo/semantics/publishedVersiondataset10.6084/m9.figshare.29039305.v2https://figshare.com/articles/dataset/Safe_Policy_Learning_through_Extrapolation_Application_to_Pre-trial_Risk_Assessment_sup_sup_/29039305CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/290393052025-06-24T17:00:11Z |
| spellingShingle | Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment Eli Ben-Michael (10803815) Medicine Pharmacology Biotechnology Sociology Science Policy Mathematical Sciences not elsewhere classified Algorithm-assisted decision-making Decision-making under uncertainty Optimal policy learning Risk assessments Robust optimization |
| status_str | publishedVersion |
| title | Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment |
| title_full | Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment |
| title_fullStr | Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment |
| title_full_unstemmed | Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment |
| title_short | Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment |
| title_sort | Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment |
| topic | Medicine Pharmacology Biotechnology Sociology Science Policy Mathematical Sciences not elsewhere classified Algorithm-assisted decision-making Decision-making under uncertainty Optimal policy learning Risk assessments Robust optimization |