City Council Passes First Bill in Nation to Address Transparency, Bias in Government Use of Algorithms

The New York City Council passed legislation today that assigns a task force to recommend ways to establish public accountability for the city’s use of algorithms. The bill is supported by the New York Civil Liberties Union and is the first in the nation to seek oversight and transparency regarding government use of algorithms, to look at how error and bias enter into their design and to recommend measures that ensure accuracy and fairness.

 

Algorithms are decision making systems automated by computers that include a series of steps or instructions designed to perform a specific task or solve a particular problem. Algorithms inform decisions that affect many aspects of society, including which school a child can attend, whether a person will be offered credit from a bank, what products are advertised to consumers and whether someone will receive an interview for a job.

 

“Algorithms are sometimes thought of as the perfect embodiment of objectivity, but the truth is that they are highly vulnerable to human bias,” said NYCLU Executive Director Donna Lieberman. “As algorithms become ubiquitous, this bill provides an opportunity for the city to grapple with the critical questions they present, and develop policies that promote transparency and prevent abuse.”

 

New York City already uses algorithms to help with a broad range of tasks: teacher evaluations, firefighting, identifying serious pregnancy complications and much more. Algorithms are also used across the country to predict where crimes will take place, who is likely to commit a crime and whether someone should be allowed out of jail on bail. A flawed algorithm can cause serious harm, as when a highly controversial DNA testing technique used by New York City’s medical examiner put thousands of criminal cases in jeopardy.

 

Today’s legislation (Int. 1696-a) will create a task force to review New York City agencies’ use of algorithms and the policy issues impacted, and develop a set of recommendations about whether algorithms should be made public, how to assess whether they are biased and the impact of such bias. The New York Civil Liberties Union testified in support of the bill in October.

 

“A flawed algorithm can lead to someone being trapped in jail for no good reason or not receiving a public benefit,” said NYCLU Legislative Counsel, Rashida Richardson. “This bill is the first in the nation to take such a broad view of the problem and recognize that, for algorithms to benefit society, they must be subject to public scrutiny and a mechanism to remedy flaws and biases.”