Democrats want feds to target the 'black box' of AI bias

Democrats want feds to target the 'black box' of AI bias
In this Feb. 9, 2019, file photo, Sen. Cory Booker, D-N.J., is seen on a cell phone as he speaks during a meet and greet with local residents in Marshalltown, Iowa. Congress is starting to show interest in prying open the "black box" of tech companies' artificial intelligence much the same way the federal government checks under car hoods and audits banks. One proposal introduced Wednesday, April 19, and co-sponsored by Booker, would require big companies to test the "algorithmic accountability" of their high-risk AI systems, such as technology that detects faces or makes important decisions based on your most sensitive personal data. (AP Photo/Charlie Neibergall, File)

Congress is starting to show interest in prying open the "black box" of tech companies' artificial intelligence with oversight that parallels how the federal government checks under car hoods and audits banks.

One proposal introduced Wednesday and co-sponsored by a Democratic presidential candidate, Sen. Cory Booker, would require to test the "algorithmic accountability" of their high-risk AI systems, such as technology that detects faces or makes based on your most sensitive .

"Computers are increasingly involved in so many of the key decisions Americans make with respect to their daily lives—whether somebody can buy a home, get a job or even go to jail," Sen. Ron Wyden said in an interview with The Associated Press. The Oregon Democrat is co-sponsoring the bill.

"When the companies really go into this, they're going to be looking for bias in their systems," Wyden said. "I think they're going to be finding a lot."

The Democrats' proposal is the first of its kind, and may face an uphill battle in the Republican-led Senate. But it reflects growing—and bipartisan—scrutiny of the largely unregulated data economy—everything from social media feeds, online data brokerages, financial algorithms and self-driving software that are increasingly impacting daily life. A bipartisan Senate bill introduced last month would require companies to notify people before using on them, while also requiring third-party testing to check for bias problems.

Academic studies and real-life examples have unearthed facial recognition systems that misidentify darker-skinned women , computerized lending tools that charge higher interest rates to Latino and black borrowers, and job recruitment tools that favor men in industries where they already dominate.

Democrats want feds to target the 'black box' of AI bias
In this March 20, 2019, photo workers take photos with their phones as President Donald Trump speaks at Joint Systems Manufacturing Center in Lima, Ohio. Congress is starting to show interest in prying open the "black box" of tech companies' artificial intelligence much the same way the federal government checks under car hoods and audits banks. President Donald Trump's administration is also taking notice and has made the development of "trustworthy" algorithms a part of the White House's new AI initiative. (AP Photo/Michael Conroy, File)

"There's this myth that algorithms are these neutral, objective things," said Aaron Rieke, managing director at advocacy group Upturn. "Machine learning picks up patterns in society—who does what, who buys what, or who has what job. Those are patterns shaped by issues we've been struggling with for decades."

President Donald Trump's administration is also taking notice and has made the development of "safe and trustworthy" algorithms a major objective of the White House's new AI initiative . But it would do so mostly by strengthening an existing industry-driven process of creating technological standards.

"There's a need for greater transparency and data comparability," and for detecting and reducing bias in these systems, said Commerce Undersecretary Walter Copan, who directs the National Institute of Standards and Technology. "Consumers are essentially flying blind."

Dozens of facial recognition developers, including brand-name companies like Microsoft, last year submitted their proprietary algorithms to Copan's agency so that they could be evaluated and compared against each other. The results showed significant gains in accuracy over previous years.

But Wyden said the voluntary standards are not enough.

"Self-regulation clearly has failed here," he said.

Democrats want feds to target the 'black box' of AI bias
In this June 28, 2018, file photo Sen. Ron Wyden, D-Ore., ranking member of the Senate Finance Committee, speaks during a hearing on Capitol Hill in Washington.Congress is starting to show interest in prying open the "black box" of tech companies' artificial intelligence with oversight that parallels how the federal government checks under car hoods and audits banks. One proposal introduced Wednesday, April 10, 2018 and co-sponsored by Wyden would require big companies to test the "algorithmic accountability" of their AI systems that are helping make important criminal-justice decisions and affecting people's access to housing, credit and jobs. (AP Photo/Jacquelyn Martin, File)

In a bolder move from the Trump administration, the federal Department of Housing and Urban Development has charged Facebook with allowing landlords and real estate brokers to systematically exclude groups such as non-Christians, immigrants and minorities from seeing ads for houses and apartments.

Booker, in a statement about his bill, said that while HUD's Facebook action is an important step, it's necessary to dig deeper to address the "pernicious ways" discrimination operates on tech platforms, sometimes unintentionally.

Booker said biased algorithms are causing the same kind of discriminatory real estate practices that sought to steer his New Jersey parents and other black couples away from certain U.S. neighborhoods in the late 1960s. This time, he said, it's harder to detect and fight.

The bill he and Wyden have introduced would enable the Federal Trade Commission to set and enforce new rules for companies to check for accuracy, bias and potential privacy or security concerns in their automated systems, and correct them if problems are found. It exempts smaller companies that make less than $50 million a year, unless they are data brokers with information on at least 1 million consumers.

New York Democratic Rep. Yvette Clarke, who is introducing a companion bill in the House, said the goal is to fix problems, not just to assess them. She said it makes sense to give the FTC authority to regularly monitor how these systems are performing because it "has the finger on the pulse of what's happening to consumers."


Explore further

House passes bill to restore 'net neutrality' rules

© 2019 The Associated Press. All rights reserved.

Citation: Democrats want feds to target the 'black box' of AI bias (2019, April 10) retrieved 20 September 2019 from https://phys.org/news/2019-04-democrats-feds-black-ai-bias.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
11 shares

Feedback to editors

User comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more