What is the origin of the fears of racism associated with facial recognition technology?

Question

If one pays attention to technology-based news all the big players in facial recognition technology are now refusing to provide this technology to the US government and/or police. When news reports on this the generally imply the new refusals are caused by George Floyd's death and fears of racism associated with the technology, though they never elaborate on the specific fears.

I'm aware of the general concerns about the potential for abuse of facial recognition to lead to a 'Big Brother' police state, but I'm curious about the specific unclear claims about facial recognition being associated with racism; I'm not sure I understand that specific concern.

I'm aware of examples of early facial recognition technologies worked better on Caucasians than other races, but that seemed to be a failure to utilize a properly representative set of photos to train the technology, fixing this specific issue seems to only require an intent to get a more representative training set, it hardly seems an unfixable issue.

Beyond that I would think technology would be less racist than humans; my neural net doesn't have subconscious biases that cause it to make presumptions about an individual based off of the color of their skin, but study after study prove that humans, even those who otherwise do not show any outward sign of racism, are guilty of such biases. So why would facial technology be thought to lead to racist results?

Answer 1

People often have the mistaken belief that computers are inherently objective and unbiased – and while they may not hold prejudices themselves, the results that they produce reflect the biases and assumptions of their programmers. This is particularly clear with machine learning systems, where the predictions they give out are dependent on data set used to train the model.

For facial recognition software specifically, fears of racial bias stem in large part from a 2019 test carried out by NIST (US National Institute of Standards and Technology). This test evaluated 189 software algorithms from 99 developers and showed that facial recognition software has a false-positive rate that is orders of magnitude higher for Asians, African Americans, and Native Americans, compared to White people:

For one-to-one matching, the team saw higher rates of false positives for Asian and African American faces relative to images of Caucasians. The differentials often ranged from a factor of 10 to 100 times, depending on the individual algorithm. False positives might present a security concern to the system owner, as they may allow access to impostors.

Among U.S.-developed algorithms, there were similar high rates of false positives in one-to-one matching for Asians, African Americans and native groups (which include Native American, American Indian, Alaskan Indian and Pacific Islanders). The American Indian demographic had the highest rates of false positives. ...

For one-to-many matching, the team saw higher rates of false positives for African American females

These higher false positive rates mean that Asians, African Americans, and Native Americans are at significantly higher risk of being arrested and falsely accused of a crime based on a faulty facial recognition match. Because so many people trust computers’ objectivity, false positive matches can have severe consequences, and if they’re applied in a racially biased manner, as the data shows they currently would be, then the effect will be to introduce additional racial bias into the criminal justice system.

---

Another interesting takeaway is that, contrary to rs.29’s claim, this effect doesn’t represent some sort of inherent racial trait for these groups, as algorithms developed in Asian countries showed equivalent accuracy for Whites and Asians:

However, a notable exception was for some algorithms developed in Asian countries. There was no such dramatic difference in false positives in one-to-one matching between Asian and Caucasian faces for algorithms developed in Asia. While Grother reiterated that the NIST study does not explore the relationship between cause and effect, one possible connection, and area for research, is the relationship between an algorithm’s performance and the data used to train it

Answer 2

Let me set aside, for the moment, the question of whether current facial recognition technology (FRT) accurately distinguishes the facial features of non-whites. There is some evidence that it does not, but that is a technological problem which could (assumedly) be ironed out.

The more pressing problem is that technology does not think at all in the sense of making judgements. Saying that FRT is 'not racist' is exactly the same as saying that a handgun is 'not racist': they are both mechanical tools that have no opinion about how they are used or where they are pointed. But it's a clear fact that handguns disproportionately kill brown-skinned people in the US, whether used by police or private citizens, and it's likely that FRT would have the same net effect. Humans have to decide where FRT is used, humans have to decide what FRT results mean, and there is no reason to believe that the humans making those decision are any less fallible or biased than those humans who make the decision to fire handguns.

There are a number of concerns to raise here:

• Will FRT — deliberately or not — be deployed and used more heavily in minority communities, leading to greater surveillance and control of those populations?
• Will authorities — deliberately or not — use different standards for minority groups, accepting lower FRT confidence thresholds for those populations than for whites?
• Will 'science bias' — the tendency for people to overestimate the value of technological results — create a 'jump to conviction' effect in police and juries when FRT results agree with their own internal biases?

If any of these concerns prove true, then FRT would become a powerful part of the 'pipeline to prison' problem that minorities already face.

Answer 3

Every new technology used for policing is thought to lead to racist results, because it is assumed that policing as currently practiced leads to racist results.

Here's a blog post from 4 years ago that discusses a number of technologies that law enforcement was considering using in Northern California: https://www.aclunc.org/blog/together-we-can-put-stop-high-tech-racial-profiling

None of these technologies have the kinds of problems that facial recognition is widely thought to have, yet it is assumed that all of them will be used in racial profiling. The unstated assumption is that the police will use them to racially profile people, because that's something the police already do.

News coverage of "facial recognition" is different because big companies are researching it.

Nobody has heard of the company that made the Stingray (https://en.m.wikipedia.org/wiki/Stingray_phone_tracker). Everybody has heard of Microsoft. When Microsoft makes R&D decisions, especially regarding subjects that every other very large company in Silicon Valley is also researching, it's a lot more newsworthy.

Don't discount the possibility that this is just a face saving exit from wasted R&D investment

Face matching is a really, really hard problem, because faces are actually an extremely poor biometric. You tend to need photographs taken under ideal conditions, and people's faces change a lot over time. Even if the problems that are primarily racial were solved, effective face matching would still be very hard, and maybe not actually possible (at least not in the general sense).

"But... lots of companies have poured big money into researching it!" I hear you say. Yes, because R&D that big Silicon Valley companies do is a kind of arms race. Unless they have a good reason not to, everyone needs to be researching what everyone else is researching so they don't get left behind when someone else makes a big breakthrough.

You can't say, "we can't figure this out because it's really hard" because then you wasted a lot of money and are implicitly saying the other companies you're competing with are smarter than you are. But, if you cancel your Facial R&D program because "we are very concerned about the racial impact of this technology if it is used by law enforcement", then it's not a waste, it's just a really expensive PR stunt. When you're a big company with a lot of consumer facing products, the PR benefits of cancelling the research project might exceed the costs by several orders of magnitude. Who wants to buy a smartphone made by racist companies?

Answer 4

In addition to the other answers, there's also the issue of using facial recognition to predict criminality of an individual. For example: this The Intercept article. For clarity, the research paper was specifically published to highlight these fears, not as a serious attempt to predict criminality, but still serves to highlight the origin of fears around facial recognition.

Because some minority groups are already over-represented in crime statistics, this means that facial recognition could use that statistic to alert police whenever any member of that minority group is in a new part of town or an expensive store.

This has the potential to become a positive feedback loop, where increased police attention creates an increase in crime statistics, which means AI sends more police to follow members of that group.

Now you might be thinking that if they are committing more crimes, then it makes sense to police them more closely. I'll argue that there's already a discrepancy in the rate of crimes committed and actual convictions. For example, in the US European-Americans and African-Americans both use marijuana at relatively similar rates, but AAs are much more likely to be arrested and prosecuted for possession: see this Washington Post article.

Other policing practices, such as NYPD's zero-tolerance approach to "Broken Windows Policing", have led to lopsided statistics which would introduce bias into the AI's model.

The political issues around facial recognition cannot be divorced from it's potential uses and misuses. This is one origin of the fears around facial recognition. The remainder of my answer doesn't talk about facial recognition specifically because when you're talking about origins, then history is required.

Answer 5

The fears of racism associated with facial recognition technology aren't just in relation to the fact that early examples worked better on Caucasian faces. In fact, in An Other-Race Effect for Face Recognition Algorithms, Phillips et al. show that the racial bias extant in humans, in that people are generally better at recognising members of their own race than of other races, translates to racial bias in facial detection algorithms.

Psychological research indicates that humans recognize faces of their own race more accurately than faces of other races. This “other-race effect” occurs for algorithms tested in a recent international competition for state-of-the-art face recognition algorithms.

We report results for a Western algorithm made by fusing eight algorithms from Western countries and an East Asian algorithm made by fusing five algorithms from East Asian countries. At the low false accept rates required for most security applications, the Western algorithm recognized Caucasian faces more accurately than East Asian faces and the East Asian algorithm recognized East Asian faces more accurately than Caucasian faces.

Next, using a test that spanned all false alarm rates, we compared the algorithms with humans of Caucasian and East Asian descent matching face identity in an identical stimulus set. In this case, both algorithms performed better on the Caucasian faces—the “majority” race in the database. The Caucasian face advantage, however, was far larger for the Western algorithm than for the East Asian algorithm.

Humans showed the standard other-race effect for these faces, but showed more stable performance than the algorithms over changes in the race of the test faces. State-of-the-art face recognition algorithms, like humans, struggle with “other-race face” recognition.

So as well as the training data factor which you allude to in your question, there existed a clear bias relating to the population which developed the algorithm - the algorithms developed by competitors from East Asian countries were better at detecting East Asian faces, while algorithms developed by competitors from Western countries were better at detecting Caucasian faces.

The point is that these algorithms have been shown to retain some of the unconscious racial bias extant in human populations - something that can only partially be fixed by using a more diverse training set, as Phillips et al. show.

Answer 6

There is one problem that is at the base, before even starting machine learning.

It does not necessarily have a large practical effect, but it is unavoidable.

For illustration, imagine a face image of a low-resolution CCTV camera, or a face image that is small and hard to recognize in general.

Under low light conditions, there is less contrast between shadow and dark skin than between shadow and lighter skin.

It's plain physics, with no way around it.

If you have input images with low quality, there is a quality level where a white face can still be recognized or detected, but a black face can not.

The problem is that the signal to noise ratio is different, so there is actually less information in the darker image.

It may be possible to work around this by artificially discarding some information in the lighter case before it even gets to the neuronal network etc.
In situations where the image quality and the light can be controlled, this problem does not occur.

Answer 7

tl;dr– Those who think that law-enforcement is racist are liable to be concerned that anything that'd empower law-enforcement would further its perceived racism.

---

Some are concerned that empowering racists would further racism.

The concern's probably just:

1. Police do racist things.

2. Facial-recognition would help the police.

3. Therefore, facial-recognition would help the police do racist things.

And that's it.

Don't get me wrong – a lot of folks have tried to build arguments supporting this concern. But those seem more like post-hoc rationalizations built upon the pre-existing concern.

Since the concern's more basic, it seems unlikely that anyone could alleviate the concern by debunking a particular argument for it. Those who believe that the police are racist are liable to object to anything that empowers the police so long as they continue to mistrust law-enforcement.

---

Contrast: Defunding the police.

Recently systemic-racism in law-enforcement has been a major topic in the news. Some have suggested that police should be defunded.

This seems like the same phenomena: those who fear law-enforcement are liable to feel safer with it disempowered for the same reasons that they'd feel threatened by it being empowered.

Answer 8

Concerns regarding racism in facial recognition arise from its disproportionate accuracy across racial groups. The Gender Shades project by Joy Buolamwini and Timnit Gebru revealed lower accuracy for darker-skinned and female individuals. NIST found accuracy variances tied to race, age, and gender. These findings emphasize the necessity for diverse training datasets and equitable algorithm development to mitigate biases.