Gotham Gazette

The Place for New York Policy and politics

Opinion

Opinion

Mayor Must Veto Bill Passed by City Council That Will Only Make AI Bias Worse


tech workers appnexus

Technology at hand (photo: Edwin J. Torres/ Mayoral Photo Office)


For years, people have worried that one day robots would replace all of our jobs, but today, they already can decide if you get one. And even worse, they’re racist. That’s the dystopian reality of artificial intelligence tools screening job applicants, automating and amplifying racism and sexism. Racism runs rampant in human hiring, but AI is even worse. And thanks to a bill that the New York City Council just passed, the robots are likely here to stay. If signed or allowed to age into law, the legislation would give the green light to employers who use these biased tools and the tech companies that make millions by selling them.

In 2014, Amazon developed a new AI-powered tool to cut costs. But the system amplified Amazon’s own past bias, selecting men for technical roles and promotions. Candidates who went to women’s colleges were penalized because existing employees hadn’t attended those schools. Amazon was an early example, but it’s far from the only one. 

Globally, 55% of companies use AI hiring and evaluation software. Chances are that AI has reviewed your own applications, and you never even knew it. This civil rights catastrophe is what drove the City Council to take recent action, passing a new AI hiring bill. Sadly, the passed bill, if it becomes law by mayoral signature or inaction, is going to make things worse…a lot worse.

Months ago, advocates warned that the Council’s proposal was too weak to stop the tech companies selling racist and sexist technology. So the Council amended the bill, but made it even weaker. The result is civil rights in name only, a toothless requirement for companies to “audit” their algorithms without any enforcement to actually make sure they comply.

Algorithmic audits aren’t like the financial audits companies typically go through. Part of the reason is that the field is so new. Mandating an algorithmic audit today would have been like requiring financial audits before developing modern accounting rules. It can tell us a lot, or it can’t tell us almost nothing, and the choice is completely up to the auditor.

Even worse, on the rare occasion a company fails this weak audit, finding that software has “disparate impact”,  there are no consequences. The auditor only must make a “summary” of the audit available to the public. But AI is all in the details. A summary isn’t going to tell us what we need to analyze these tools for ourselves. The only way to understand AI is to look at every aspect of how it works, starting with the data used to train an algorithm, and continuing through every ramification of how the software is used in practice.

And what happens if a company ignores the audit requirements completely? Almost nothing. Rather than letting those who AI harms have their day in court, the law places the burden of enforcing all of this in the hands of the New York City Commission on Human Rights. The venerable agency does tremendous work for New Yorkers, but its staff was clear to the City Council that it doesn’t have the capacity to take on this new work. Rather than listening to the agency and advocates and letting workers sue companies that use biased AI, the Council kept enforcement with the Commission and avoided the possibility of real accountability.

The city should be blocking discriminatory software, not supporting it. But we are doing just that by allowing corporations to claim their discriminatory products earned our stamp of approval. Rather than blocking discrimination, this weak measure will encourage more companies to use biased AI. Vendors will tell companies that as long as they comply with these weak new rules, they’ll be safe to automate with AI. New York has the chance to become a leader in digital rights, but if it becomes law this bill will make us a leader in supporting discriminatory tech. 

We should applaud the City Council for acting on discriminatory AI, but this bill would worsen the problem. Any law that does not require deep transparency, implements a categorical ban on biased systems, and provides a private right of action will only allow terrible technology to automate racism and sexism into the fabric of New York City.

Mayor de Blasio must veto this bill and ask the Council to submit one that includes stronger protections. 

If this mayor and City Council refuse to act against the tech lobby, it will be up to their successors next year. Our current mayor may have one foot out the door, but biased AI is here to stay.

***
Albert Fox Cahn is the founder and executive director of the Surveillance Technology Oversight Project (S.T.O.P.), a New York-based civil rights and privacy group, and a visiting fellow at Yale Law School’s Information Society Project. Evan Enzer is a D.A.T.A. Law Fellow at S.T.O.P. and a graduate of the University of California, Berkeley, School of Law. On Twitter @FoxCahn & @EvanEnzer.

***
Have an op-ed idea or submission for Gotham Gazette? Email This email address is being protected from spambots. You need JavaScript enabled to view it.

Mayor Adams: Show New York You Care About Animal Rights


bp adams presser

Eric Adams at a dog run (photo: Eric Adams Flickr)


The election is finally over, and Eric Adams will be New York City’s next mayor, winning handily with 66% of the vote before the counting of absentee ballots. Sadly, the excellent animal issues platform put forth by the Republican/Independent candidate, Curtis Sliwa, might well be forgotten.

For the most part, the media ignored it – including at the two televised debates, with only one animal issue question, “will you ban horse-drawn carriages?” but no discussion. It was easier to mock Sliwa and his wife for the many cats they have rescued than to ask thoughtful questions about his ideas.

It is hoped that Mayor Adams will consider these progressive proposals and act on them – because both the animal shelter and horse-carriage issues have gone on for far too long without a solution and with our elected representatives turning a blind eye.

Adams has said he is “open to further discussion about prohibiting the operation of horse-drawn carriages.” His support for animal issues actually goes back many years and includes pets in housing, support of the Foie Gras ban, opposition to geese slaughter, and his trademark plant-based diet.

Through the Bureau of Veterinary and Pest Control Services, animal issues are still under the domain of the city’s Department of Health and Mental Hygiene (DOHMH), whose mission is to protect and promote the health of New Yorkers – people, not animals. And with a name like that, it’s no surprise.

The idea for a Department of Animal Welfare goes back to 1997 when the Shelter Reform Action Committee, of which I was co-chair, worked on a ballot initiative to create a separate Department of Animal Affairs, with its own Commissioner. It failed, largely due to then-Mayor Giuliani’s opposition. But what was true then, 24 years ago, is just as true today. Animals need their own representatives without this built-in conflict of interest.

The issues that need to be addressed include the animal shelter system, whose problems have been exposed in the media many times over the years and in an audit by the City Comptroller released September 15, 2020. But as Nathan Winograd says in his blog, NYC Audit Shows Continued Animal Neglect, some issues have never been properly addressed, such as the killing of thousands of healthy, adoptable animals each year; unsanitary conditions; inadequate adoption efforts; and poor leadership.

But instead of a complete overhaul with public input, DOHMH awarded Animal Care and Control (ACC), the quasi-governmental shelter system, with a 34-year contract worth over $1 billion after a badly-publicized hearing that drew the ire of activists saying it would limit competition and innovation — and of at least one City Council member, Robert Holden. How and why did that happen? Almost two years later, more Council members — Holden, Alicka Ampry-Samuels, Inez Barron, and Speaker Corey Johnson — spoke out demanding an investigation into the ACC after a New York Post expose.

Sliwa’s signature issue was to make the city’s animal shelter system No Kill – something other cities have done successfully. Not an easy feat but despite their claims, the ACC has failed to institute the proven policies and programs necessary to become truly No Kill, which means not killing healthy and treatable animals; accessible spay/neuter services; public/humane education and increased, responsible adoptions.  

The other ongoing issue that continues to plague this city is the horse-drawn carriage business and the desire by many to shut it down for public safety and humane reasons. The Committee for Responsible and Compassionate Tourism has explored this issue, including a fact-finding trip to Guadalajara, Mexico, the first city to transition to electric horseless carriages.

They’ve suggested phasing out horse carriages and replacing them with electric carriages, which will preserve jobs and provide more income for drivers since they will not be limited by weather restrictions that apply to the horses. Horses would be placed in responsible and caring homes. Since 2017, a growing number of cities around the globe are using electric horseless carriages with great success, with the town of Alcudia in Majorca being the latest, with the association of horse-carriage drivers actually proposing this idea to the administration.    

It is not a matter of whether the drivers love or treat their horses well. It is, instead, about the environment in which these horses are forced to work – the heavy traffic, congestion, cramped stalls, lack of turn-out to pasture – but mostly a horse’s nervous and sensitive nature, which can cause them to spook and bolt, and at 1,500 to 2,000 pounds cause serious injury or death to themselves or innocent passersby. 

New York City is the largest city in the country, of course, and it is irresponsible and a threat to public safety to continue to allow horse carriages to operate here.

Something must give on these difficult and challenging issues. There are solutions and it is hoped that Mayor Adams, a vegan who has acted positively on other animal issues, will answer this call. Animals have no voice of their own, and it is up to those who care to speak for them.

The way New York City eventually handles these issues in the new administration will be a mark of our collective character and capacity for decency, ethics, morality, and empathy. I truly hope we succeed.

***
Elizabeth Forel is co-founder, Committee for Responsible & Compassionate Tourism. On Twitter @CompassionNYC.

***
Have an op-ed idea or submission for Gotham Gazette? Email This email address is being protected from spambots. You need JavaScript enabled to view it.

Rediscovering the Power of Unions


2 unite here com

Labor movement (photo: Michael Appleton/Mayoral Photography Office)


It was called “Striketober.” While politicians in Washington bickered over infrastructure, jobs, and the social safety net, unionized workers across the heartland went on strike to get their fair share directly.

At one point in October, over 90,000 unionized workers issued strike authorizations — including 10,000 John Deere United Auto Workers (UAW) members, 37,000 nurses and other health care workers at Kaiser Permanente, and 60,000 film and television workers organized with  International Alliance of Theatrical Stage Employees (IATSE).

That trend has continued into November, as John Deere workers in Iowa, Kansas, and Illinois recently rejected the latest contract proposal from management and are continuing their strike.

From hospital staff in West Virginia, steelworkers in Ohio, and educators in Pennsylvania, workers are rising up against poverty wages, long hours, and dangerous working conditions.

In total, over 22,000 workers across the country are currently on strike. This includes over 3,000 student workers at Columbia University — currently the second largest strike in the country. Columbia, with its $14 billion endowment, is threatening to alter its pay schedule for student workers while docking wages and stipends for those on strike.

Decades ago, when unions were strong, it wasn’t uncommon to see millions of workers on strike each year. While New York has one of the highest rates of union membership in the country at 22%, union membership has rapidly declined over the last few decades there and beyond, from a peak of over 33% of American workers in 1944 to just under 11% in 2020.

And as unions weakened, inequality skyrocketed. The share of income going to the top one percent of Americans has doubled from 11% in the 1940s to over 22% today. It is this pervasive inequality that is driving tens of thousands of U.S. workers to strike.

At John Deere, workers have expressed growing frustration over proposals from management to slash pensions and wage increases while the company has enjoyed record-breaking profits. John Deere is projected to rake in up to $5.9 billion this year, far exceeding its previous high of $3.5 billion in 2013.

Meanwhile, CEO John May gave himself a 160% raise — and shareholders a 17% raise on dividends. As one worker described it, “We are the last thing they think about.”

President Biden’s Build Back Better agenda, while reduced in scope due to powerful corporate Democrats wielding their influence, includes a number of policies to strengthen worker protections and hold corporations accountable.

One provision in particular, lifted directly from the Protecting the Right to Organize (PRO) Act, would fine employers $50,000 to $100,000 for each violation of the National Labor Relations Act. The bill would also invest over $2 billion in enforcement of workers’ rights, helping government officials more effectively protect workers by investigating and enforcing labor laws.

That would be a big victory. Better wages, health care, paid leave, and real curbs on inequality may always face headwinds in Washington. But as workers across the country are rediscovering, a strong union can win those things directly.

***
Rebekah Entralgo is the managing editor of Inequality.org at the Institute for Policy Studies. On Twitter @rebekahentralgo@inequalityorg.

***
Have an op-ed idea or submission for Gotham Gazette? Email This email address is being protected from spambots. You need JavaScript enabled to view it.



Print Friendly and PDF

Editor's Choice


Gotham Gazette: New York Voters Approve $4.2 Billion Environmental Bond Act
New York Voters Approve $4.2 Billion Environmental Bond Act
Gotham Gazette: 
New York City Voters Approve Racial Justice Ballot Measures
New York City Voters Approve Racial Justice Ballot Measures
Gotham Gazette: 
Housing Development in New York City Slows to a Crawl as Officials Debate Tax Incentives
Housing Development in New York City Slows to a Crawl as Officials Debate Tax Incentives
Gotham Gazette: 
City Council Examines Low Voter Turnout, Moving Local Elections to Even Years City Council Examines Low Voter Turnout, Moving Local Elections to Even Years