Testimony Before the New York City Council Committee on Technology on the Open Data Law

The New York Civil Liberties Union respectfully submits the following testimony regarding our recommendations for DoITT’s administration of Local Law 11 of 2012, the New York City open data law.

With 50,000 members and supporters, the New York Civil Liberties Union (NYCLU) is the foremost defender of civil liberties and civil rights in New York State and a longstanding advocate for government transparency. Open and accessible government is essential to a functioning democracy.

In advocating for the rights of New Yorkers, we often rely on publically-available data and records provided under the Freedom of Information Law. Access to information such as the NYPD’s Stop, Question and Frisk database, Department of Education data on student suspensions and others has lead to major policy changes; for example, in the most recent reporting periods, 80,000 fewer individuals were stopped and frisked by the NYPD, and nearly 20,000 fewer school suspensions were issued. Public debate about these practices was fueled by data demonstrating their ineffectiveness and unbalanced racial impact. The real-world effects of broad access to government data are hard to overstate.

Making these data available in an open format is vital, as the law states, to “make the operation of city government more transparent, effective and accountable to the public.” In this age of increasingly sophisticated database technology, it is incumbent upon the government to streamline access to computerized records. Freedom of Information requests are an important tool but are cumbersome to file and to answer; government agencies can better serve a transparency goal by proactively making datasets available. We wholeheartedly support the open data law and would like to see DoITT improve implementation and effectiveness of the law.

Failures of FOIL Compliance
Open records and publically-reported data are crucial to the work of the NYCLU and the transparency of our city. The web portal established by Local Law 11 could revolutionize how we access and use these data. The city council must continue to fund implementation of Local Law 11 and must work with DoITT to audit compliance by city agencies.

The NYCLU supports the open data law in part because of our negative experiences in receiving timely and substantive information from city agencies under the Freedom of Information Law (FOIL). These experiences are common to many advocates and members of the public as agencies struggle to timely respond to requests for large datasets. We anticipate that, when implemented fully, Local Law 11 will proactively address the transparency and accountability concerns central to FOIL by reducing paperwork and wait times associated with requesting records.
As an example, our efforts to obtain information about NYPD policies have often been met with refusals or outrageous delays. In 2007, after the NYPD neglected quarterly reporting required under local law, the NYCLU submitted a FOIL request to the department for access to its electronic stop-and-frisk database. The NYPD refused to provide the database, which forced the NYCLU to sue for the data. The ultimate release of these data, pursuant to a court order, has resulted in an explosion of public reporting about the large impact of stop-and-frisk on innocent New Yorkers and has fueled a citywide debate about civil rights and public safety. Organizations or individuals with fewer resources may never have obtained these data.

Importance of a Broad Reading of the Law
The NYCLU encourages the Council, DoITT and city agencies to adopt a broad reading of Local Law 11 to ensure its goals are met.

The declared legislative intent of the open data law is to increase government transparency, streamline communication both within government and with constituents, allow the public to identify efficient solutions for the government, promote strategies for social progress between the public and the government, and to create economic opportunities. These goals are best served by a broad and inclusive interpretation of the law, reaching all agencies and all public datasets.

Because technology perpetually outpaces the law, a broad interpretation of Local Law 11 will allow the law to continue to achieve its purpose as innovation changes the way government collects, analyzes, and stores data. An expansive reading of the agencies, offices, and types of information covered by the law will permit flexibility and transparency as digital data evolves.

Missing Datasets
DoITT has failed to compel city agencies’ full compliance in reporting all data mandated by the law. This is particularly evident with city agencies known to shirk compliance with open records laws, such as the New York Police Department and the Department of Education.

The New York Police Department
On September 22 of this year, DoITT released its NYC Open Data Plan that included a list of all datasets to be published pursuant to the law. The NYPD, an agency with the fifth largest budget in the city and perhaps the most public attention and interest of all government offices, submitted the names of only six data sources, shown below, which it deemed “public” as defined by the law.

These six data sources are image- and text-heavy reports published by the NYPD. While they contain important data summaries in graphs and tables, submitting entire reports to the Open Data Plan misses the mark of the law, which requires data to be reported in an “alphanumeric form reflected in a list, table, graph, chart or other non-narrative form, that can be digitally transmitted or processed… Such term shall not include…image files, such as designs, drawings, maps, photos…” PDF files are not easily machine-readable, and the use of the NYPD data within these reports would require intensive labor from anyone hoping to analyze these data in a way different than presented in the department’s reports.

Further, these reports are not raw data—they come pre-packaged, with a public relations angle from the NYPD. Researchers using them are left to deconstruct messages from the provided narratives and reconstruct their own conclusions. This is seen best in the Reasonable Suspicion Stops report, a precinct-by-precinct summary of selected variables from the Stop, Question and Frisk Database, which itself is raw data that the NYPD excluded from the Open Data Plan. This report appears to be a document designed for the news media in response to the NYCLU’s analysis of NYPD stop, question and frisk data.

Perhaps in an attempt to validate the need for and constitutionality of stops, the NYPD report includes the top crime suspected that lead to the stop. Yet, the report excludes the percentage of those stops that actually resulted in an arrest or summons. So while it may be valuable to know that a third of the people in East New York (Precinct 75) were stopped on suspicion of weapons possession, it is as important to know that less than one percent of stops resulted in a weapon found – lower than the 3.88 percent recovery rate for Murray Hill and Kipps Bay (Precinct 17). That statistic, easily gleaned from the raw data, is omitted from this report.

Finally, even if the reports the NYPD plans to provide under the open data law were raw data provided in a machine-readable format, the list they proffer is far from exhaustive of datasets “available for inspection by the public in accordance with any provision of law.” For example, the Stop, Question and Frisk Database, a database reporting information on each Terry stop made by an NYPD officer (not to be confused with the statistical summaries of the Stop, Question and Frisk Reports included above), is the one dataset that the NYPD already publishes that actually is a dataset and not a PDF of compiled narratives interspersed with tables of data summaries. Even though the NYPD reports these data on its website, the department excludes them from reporting under Local Law 11, perhaps to avoid publishing this controversial dataset in an open format. Currently, this database exists on the NYPD website as an SPSS file, which requires advanced coding skills or expensive software to open; it is not accessible to members of the public.

There are a number of other NYPD datasets excluded from the Open Data Plan that can and should be published, including the Stop, Question and Frisk Database, locations of crime incidents and reports, buildings enrolled in Operation Clean Halls, information on summonses issued for non-criminal violations, and operations of the school safety division. We list those we would like to see below.

1. Stop, Question and Frisk Database
   As mentioned above, this database is already published on the NYPD website. It easily fits the law’s definition of public data and if put into a machine-readable format, could easily be included on the law’s web portal.
2. Data reported to the city council under the police reporting law
   There are several datasets required by §14-149 and §14-150 of the Administrative Code of the City of New York. These include, for example, the number of calls, incidents, crime reports in the city; average response time by police unit; 911 operational time analysis and the number of police officers by precinct. The NYPD already compiles these data for the city council and should include them under Local Law 11 reporting.
3. Student Safety Act data on arrests, summonses, non-criminal incidents
4. School-based arrests
   Currently, the Student Safety Act requires reporting on “the number of individuals arrests and/or issued a summons by school safety agents or police officers assigned to the school safety division of the New York city police department.” This reporting has helped advocates and community members better understand what is happening in New York City schools, but the NYCLU hears many stories of students facing an arrest in school by a fully-uniformed officer outside of the School Safety Division of the NYPD. We expect that the Student Safety Act reporting may drastically undercount school-based arrests. These data are public and vital to holding the NYPD accountable in public schools. The NYPD should report these data by a total count and disaggregated by borough, race, gender, IEP status, ELL status and school.
5. School-based incident data
   Each year, hundreds of students are exposed to law enforcement actions but are not formally arrested by the NYPD. The Student Safety Act data does not account for these interactions – which may include handcuffing, use of alternative restraints or prolonged detention – even though the NYPD and the schools collect this information. Specifically, the NYPD School Safety Division’s Criminal Incident Report (CIR) captures information on whether an incident involved a weapon, a gang, or drugs. The CIR further states whether an incident occurred at scanning, whether a student was arrested or received a summons, and whether a student was issued a juvenile report. These data should be reported in aggregate and broken down any other category recorded on the CIR form.
6. Locations of impact zones
   The NYPD designates areas of the city where they believe a targeted dose of heavy police enforcement could help improve high crime rates. These zones, identified using Compstat, are subject to increased police presence, and perhaps, harassment. The NYPD can reveal what neighborhoods within precincts are targeted by Operation Impact without revealing details of enforcement that might compromise police activity.
7. Criminal court summonses
   The NYPD should report aggregate totals of criminal court summonses issued in the city, broken down by precinct, offense, year and race, gender and age of the recipient. The NYPD claims that they do not keep information on the race of people receiving summonses, and these data are nowhere publically available. But the NYCLU has strong reason to believe that summonses, much like low-level misdemeanor arrests, such as for marijuana possession, are disproportionately impacting young men of color. Publishing these data would be an important check on NYPD abuses in neighborhoods of color.
8. Location of serious crime incidents
   If I am considering moving to a new neighborhood or opening a business, it is nearly impossible to find out where crime is happening in New York City. If I hear a gunshot outside of my apartment, I can rarely find out if someone nearby has been hurt. The NYPD publishes weekly Compstat reports by precinct, but these are unable to capture neither a historical trend nor pointed locations for crime occurrences. Other cities, such as Chicago, post these data without compromising the safety of the city. The NYPD should make these data available including x- and y-coordinates and a historical trend.
9. Location of serious crime reports
   Similarly, the NYPD should report the locations where crime is reported to have occurred by x- and y-coordinates.
10. Information on an early warning system for officers reported through the CCRB
    The public should have access to what protocols the NYPD puts in place to trigger an alert when officers receive excessive complaints through the CCRB. The department should also publish data, in aggregate, on how it chooses to discipline officers once the CCRB decides an officer should receive discipline.
11. Costs to the city for bias-based policing practices
    The department should report the money it spends on policing practices, such as stop-and-frisk. For example, the department should report what it pays out for civil lawsuit resolutions.
12. Arrests made in and near public housing
    The department should report the number of arrests made in or around public housing units, for example, during top-to-bottom sweeps. These arrest data should be disaggregated by race, gender, age and charging offense.
13. List of Clean Halls buildings
    The NYPD should publish a list of buildings enrolled in Operation Clean Halls.
14. Information on surveillance activities
    The department should report how it spends the city’s resources on surveillance activities. These data should include total staff time spent, officers involved in surveillance operations, money spent, equipment bought and used for surveillance operations. These data should also include the number of open files the NYPD chooses to surveil each year.
15. Number of bench arrest warrants for criminal court summonses and number of arrests made based on executing bench arrest warrants for criminal court summonses
    These data should be reported by total number and disaggregated by precinct, charging offense, year and race, gender and age of the person arrested or issued a warrant.
16. Complaints made against School Safety Officers
    While the Civilian Complaint Review Board publishes information on the number of complaints received from the public, similar information is not available regarding complaints filed against School Safety Agents. The School Safety Division Investigations Unit, the first-line agency responsible for processing complaints against School Safety Agents, receives complaints regarding the use of excessive force, unlawful arrest, and other complaints that fall into the FADO category (Force, Abuse of Authority, Discourtesy, Offensive Language). Data on these complaints, including total number of complaints received, by category and geographic jurisdiction, is a necessary tool for the public to monitor the NYPD.

The Department of Education
The DOE is the city agency with the largest budget and the responsibility of educating over one million of the city’s children. They keep records on numerous student data points ranging from attendance records to school surveys to school discipline. In fact, the DOE is currently engaged in uploading all student records into a cloud maintained by the state in order to build a complex longitudinal dataset on student progress. The capacity and willingness to think expansively about data is obviously part of the culture at DOE. Yet, the department includes a mere 12 datasets in the NYC Open Data Plan.

1. Summer school enrollment
   The DOE includes in the Open Data Plan the audited register data, which details student enrollment on October 31 of the intended school year. These data do not include, however, enrollment for each summer school term. The DOE should report these numbers in the same way they report audited register data: disaggregated by demographic characteristics, such as gender, grade level, race, IEP status, as well as borough.
2. Graduation rates
   The DOE reports on its website graduation rates by cohort, gender, race and IEP status. It should include these data in Local Law 11 reporting.
3. School repair requests
   School repairs can be a political game and it’s important that students are not attending school in dilapidated buildings. The public should have access to aggregate numbers of requests for building and grounds requests by school.
4. Money spent on repairs
   Similarly, the DOE should report how much of its budget is spent on repairs by school.
5. List of impact schools
   The DOE selects certain schools as “impact schools” based on an evaluation of NYPD and DOE data, such as violent incidents at school, suspensions and attendance. The DOE should report the list of schools on this list each school year.
6. List of schools with permanent metal detectors
   The NYCLU estimates that there are permanent metal detectors in 232 schools housed within 76 school buildings across the city. We cannot provide an exact number, because the Department of Education refuses to respond to FOIL requests addressing this issue. Parents, researchers and advocates have a right to know where thousands of students are asked to enter metal detectors every day before school. The DOE should publish an annual list of these schools on the web portal.
7. Student Safety Act data
   The Student Safety Act (§8-1102 and §8-1103 of the Administrative Code of the City of New York) requires annual and biannual reporting on student suspensions to the city council. The Open Data Plan should include these data.

Recommendations
The NYCLU has the following recommendations for the Council today:

1. Continue to fund the open data law.
   The city council must continue to fund Local Law 11. Without funding, DoITT will not be able to maintain the web portal established by the law, and without the web portal, the law loses its best tool for sharing data from city agencies.
2. Interpret the law broadly.
   To ensure the transparency and accountability at the heart of the law, city agencies should report any public dataset they maintain. The default should be to include more data rather than reporting only the minimum datasets required by a narrow interpretation of the law. City agencies that comply fully and broadly with the law will save resources used in responding to FOIL requests by proactively promoting transparency.
3. Hold agencies accountable for reporting.
   DoITT and the city council must hold city agencies accountable for the datasets they plan to report. DoITT should review more closely the datasets in the Open Data Plan and reach out to underreporting agencies to encourage a broad inclusion of public datasets. This may require additional follow-up with agencies that are known to obfuscate open records requirements, such as the NYPD and the DOE. DoITT and the Council should conduct compliance audits, and work with the new administration to usher in a new era of transparency for New York City.