Testimony on Technology In Relation to Automated Decision Systems
Automated decision systems (“ADS”) comprise any software, system, or process that aims to automate, aid, or replace human decision-making. ADS are widely used in administering government services; allocating resources; and making inferences about individuals, groups, or places. Their ubiquity across government agencies means that ADS have the potential to impact a person’s eligibility for welfare benefits, education opportunities, and even their very liberty. In most instances these tools are deployed opaquely without regulation, transparency, impact assessments, or independent audits. If left unchecked, they risk severely undermining the civil, human, and privacy rights of New Yorkers.
Two years ago, the Council recognized the need to regulate the government use of automated decision systems, and enacted Local Law 49 of 2018 to create an ADS Task Force. Unfortunately, the final ADS Task Force report offered New Yorkers little information on the actual use of ADS; as a result, we have joined other advocates and researchers in the publication of a Shadow Report that addresses some of the shortcomings and offers detailed recommendations to different stakeholders and institutions.
The public has a right to know about the automated systems used by our government, how widespread they are, and what type of decisions they make. A broad disclosure requirement for all agencies is long overdue. The NYCLU therefore offers qualified support for Intro. 1806-2019 as a first step towards accountability, equity, and due process.
Read the full testimony below.