40

If one pays attention to technology-based news all the big players in facial recognition technology are now refusing to provide this technology to the US government and/or police. When news reports on this the generally imply the new refusals are caused by George Floyd's death and fears of racism associated with the technology, though they never elaborate on the specific fears.

I'm aware of the general concerns about the potential for abuse of facial recognition to lead to a 'Big Brother' police state, but I'm curious about the specific unclear claims about facial recognition being associated with racism; I'm not sure I understand that specific concern.

I'm aware of examples of early facial recognition technologies worked better on Caucasians than other races, but that seemed to be a failure to utilize a properly representative set of photos to train the technology, fixing this specific issue seems to only require an intent to get a more representative training set, it hardly seems an unfixable issue.

Beyond that I would think technology would be less racist than humans; my neural net doesn't have subconscious biases that cause it to make presumptions about an individual based off of the color of their skin, but study after study prove that humans, even those who otherwise do not show any outward sign of racism, are guilty of such biases. So why would facial technology be thought to lead to racist results?

  • 10
    Dangerous to believe such tech cannot be biased or that its use cannot lead to unfair or unjust outcomes. See Weapons of Math Destruction by Cathy O'Neil. Also it's fascinating to look at the accountability of members of the public vs. the lack of accountability of LEOs. – Lag Jun 16 at 10:20
  • 5
    "I'm aware of examples of early facial recognition technologies worked better on Caucasians than other races, but that seemed to be a failure to utilize a properly representative set of photos to train the technology, fixing this specific issue seems to only require an intent to get a more representative training set, it hardly seems an unfixable issue." - Consider that some people may not be confident that this will ever happen. – Zibbobz Jun 16 at 14:13
  • It's probably because of Jacky Alcine's famous gorilla photo. The more important question than this annoying racism hype would however be: Why do we need (a good, legitimate, non-fascist purpose) to track innocent people using facial recognition at all? Why, in any country that pretends to work on the assumption of innocence, and that pretends liberty and personal rights, do we need this? – Damon Jun 18 at 20:52
73

People often have the mistaken belief that computers are inherently objective and unbiased – and while they may not hold prejudices themselves, the results that they produce reflect the biases and assumptions of their programmers. This is particularly clear with machine learning systems, where the predictions they give out are dependent on data set used to train the model.

For facial recognition software specifically, fears of racial bias stem in large part from a 2019 test carried out by NIST (US National Institute of Standards and Technology). This test evaluated 189 software algorithms from 99 developers and showed that facial recognition software has a false-positive rate that is orders of magnitude higher for Asians, African Americans, and Native Americans, compared to White people:

For one-to-one matching, the team saw higher rates of false positives for Asian and African American faces relative to images of Caucasians. The differentials often ranged from a factor of 10 to 100 times, depending on the individual algorithm. False positives might present a security concern to the system owner, as they may allow access to impostors.

Among U.S.-developed algorithms, there were similar high rates of false positives in one-to-one matching for Asians, African Americans and native groups (which include Native American, American Indian, Alaskan Indian and Pacific Islanders). The American Indian demographic had the highest rates of false positives. ...

For one-to-many matching, the team saw higher rates of false positives for African American females

These higher false positive rates mean that Asians, African Americans, and Native Americans are at significantly higher risk of being arrested and falsely accused of a crime based on a faulty facial recognition match. Because so many people trust computers’ objectivity, false positive matches can have severe consequences, and if they’re applied in a racially biased manner, as the data shows they currently would be, then the effect will be to introduce additional racial bias into the criminal justice system.


Another interesting takeaway is that, contrary to rs.29’s claim, this effect doesn’t represent some sort of inherent racial trait for these groups, as algorithms developed in Asian countries showed equivalent accuracy for Whites and Asians:

However, a notable exception was for some algorithms developed in Asian countries. There was no such dramatic difference in false positives in one-to-one matching between Asian and Caucasian faces for algorithms developed in Asia. While Grother reiterated that the NIST study does not explore the relationship between cause and effect, one possible connection, and area for research, is the relationship between an algorithm’s performance and the data used to train it

| improve this answer | |
  • 8
    why then, you claim it is a problem of technology, and not of the data set? which to me seems clear from your last quote. – dEmigOd Jun 16 at 6:37
  • 63
    @dEmigOd You can't separate the technology from the data set. The training data is as much a part of an ML algorithm as the code itself. – Johanna Jun 16 at 7:34
39

Let me set aside, for the moment, the question of whether current facial recognition technology (FRT) accurately distinguishes the facial features of non-whites. There is some evidence that it does not, but that is a technological problem which could (assumedly) be ironed out.

The more pressing problem is that technology does not think at all in the sense of making judgements. Saying that FRT is 'not racist' is exactly the same as saying that a handgun is 'not racist': they are both mechanical tools that have no opinion about how they are used or where they are pointed. But it's a clear fact that handguns disproportionately kill brown-skinned people in the US, whether used by police or private citizens, and it's likely that FRT would have the same net effect. Humans have to decide where FRT is used, humans have to decide what FRT results mean, and there is no reason to believe that the humans making those decision are any less fallible or biased than those humans who make the decision to fire handguns.

There are a number of concerns to raise here:

  • Will FRT — deliberately or not — be deployed and used more heavily in minority communities, leading to greater surveillance and control of those populations?
  • Will authorities — deliberately or not — use different standards for minority groups, accepting lower FRT confidence thresholds for those populations than for whites?
  • Will 'science bias' — the tendency for people to overestimate the value of technological results — create a 'jump to conviction' effect in police and juries when FRT results agree with their own internal biases?

If any of these concerns prove true, then FRT would become a powerful part of the 'pipeline to prison' problem that minorities already face.

| improve this answer | |
24

Every new technology used for policing is thought to lead to racist results, because it is assumed that policing as currently practiced leads to racist results.

Here's a blog post from 4 years ago that discusses a number of technologies that law enforcement was considering using in Northern California: https://www.aclunc.org/blog/together-we-can-put-stop-high-tech-racial-profiling

None of these technologies have the kinds of problems that facial recognition is widely thought to have, yet it is assumed that all of them will be used in racial profiling. The unstated assumption is that the police will use them to racially profile people, because that's something the police already do.

News coverage of "facial recognition" is different because big companies are researching it.

Nobody has heard of the company that made the Stingray (https://en.m.wikipedia.org/wiki/Stingray_phone_tracker). Everybody has heard of Microsoft. When Microsoft makes R&D decisions, especially regarding subjects that every other very large company in Silicon Valley is also researching, it's a lot more newsworthy.

Don't discount the possibility that this is just a face saving exit from wasted R&D investment

Face matching is a really, really hard problem, because faces are actually an extremely poor biometric. You tend to need photographs taken under ideal conditions, and people's faces change a lot over time. Even if the problems that are primarily racial were solved, effective face matching would still be very hard, and maybe not actually possible (at least not in the general sense).

"But... lots of companies have poured big money into researching it!" I hear you say. Yes, because R&D that big Silicon Valley companies do is a kind of arms race. Unless they have a good reason not to, everyone needs to be researching what everyone else is researching so they don't get left behind when someone else makes a big breakthrough.

You can't say, "we can't figure this out because it's really hard" because then you wasted a lot of money and are implicitly saying the other companies you're competing with are smarter than you are. But, if you cancel your Facial R&D program because "we are very concerned about the racial impact of this technology if it is used by law enforcement", then it's not a waste, it's just a really expensive PR stunt. When you're a big company with a lot of consumer facing products, the PR benefits of cancelling the research project might exceed the costs by several orders of magnitude. Who wants to buy a smartphone made by racist companies?

| improve this answer | |
  • 2
    Plus uno for covering the business angle, too rarely considered especially for emotionally charged topics. – Jared Smith Jun 18 at 11:39
12

In addition to the other answers, there's also the issue of using facial recognition to predict criminality of an individual. For example: this The Intercept article. For clarity, the research paper was specifically published to highlight these fears, not as a serious attempt to predict criminality, but still serves to highlight the origin of fears around facial recognition.

Because some minority groups are already over-represented in crime statistics, this means that facial recognition could use that statistic to alert police whenever any member of that minority group is in a new part of town or an expensive store.

This has the potential to become a positive feedback loop, where increased police attention creates an increase in crime statistics, which means AI sends more police to follow members of that group.

Now you might be thinking that if they are committing more crimes, then it makes sense to police them more closely. I'll argue that there's already a discrepancy in the rate of crimes committed and actual convictions. For example, in the US European-Americans and African-Americans both use marijuana at relatively similar rates, but AAs are much more likely to be arrested and prosecuted for possession: see this Washington Post article.

Other policing practices, such as NYPD's zero-tolerance approach to "Broken Windows Policing", have led to lopsided statistics which would introduce bias into the AI's model.

The political issues around facial recognition cannot be divorced from it's potential uses and misuses. This is one origin of the fears around facial recognition. The remainder of my answer doesn't talk about facial recognition specifically because when you're talking about origins, then history is required.

| improve this answer | |
  • 2
    This answer does not address the question. It is speculative and has little to do with the specific technology of facial recognition. – mbsq Jun 16 at 7:30
  • ... and is centered around one extremely poor research paper that effectively argues for a kind of modern day phrenology. Which, nobody talking about "facial recognition" is suggesting be done; when people say "facial recognition" they really mean "facial identification" e.g. 1:1 or 1:N matching of faces. – Joe Jun 16 at 9:17
  • 3
    The political issues around facial recognition cannot be divorced from it's potential uses and misuses. This is one origin of the fears around facial recognition. The remainder of my answer doesn't talk about facial recognition specifically because when you're talking about origins, then history is required. – Fred Stark Jun 16 at 23:21
  • 1
    the research paper was specifically published to highlight these fears, not as a serious attempt to predict criminality, but still serves as evidence to the answer about the origin of fears around facial recognition – Fred Stark Jun 16 at 23:21
8

The fears of racism associated with facial recognition technology aren't just in relation to the fact that early examples worked better on Caucasian faces. In fact, in An Other-Race Effect for Face Recognition Algorithms, Phillips et al. show that the racial bias extant in humans, in that people are generally better at recognising members of their own race than of other races, translates to racial bias in facial detection algorithms.

Psychological research indicates that humans recognize faces of their own race more accurately than faces of other races. This “other-race effect” occurs for algorithms tested in a recent international competition for state-of-the-art face recognition algorithms.

We report results for a Western algorithm made by fusing eight algorithms from Western countries and an East Asian algorithm made by fusing five algorithms from East Asian countries. At the low false accept rates required for most security applications, the Western algorithm recognized Caucasian faces more accurately than East Asian faces and the East Asian algorithm recognized East Asian faces more accurately than Caucasian faces.

Next, using a test that spanned all false alarm rates, we compared the algorithms with humans of Caucasian and East Asian descent matching face identity in an identical stimulus set. In this case, both algorithms performed better on the Caucasian faces—the “majority” race in the database. The Caucasian face advantage, however, was far larger for the Western algorithm than for the East Asian algorithm.

Humans showed the standard other-race effect for these faces, but showed more stable performance than the algorithms over changes in the race of the test faces. State-of-the-art face recognition algorithms, like humans, struggle with “other-race face” recognition.

So as well as the training data factor which you allude to in your question, there existed a clear bias relating to the population which developed the algorithm - the algorithms developed by competitors from East Asian countries were better at detecting East Asian faces, while algorithms developed by competitors from Western countries were better at detecting Caucasian faces.

The point is that these algorithms have been shown to retain some of the unconscious racial bias extant in human populations - something that can only partially be fixed by using a more diverse training set, as Phillips et al. show.

| improve this answer | |
5

tl;dr Those who think that law-enforcement is racist are liable to be concerned that anything that'd empower law-enforcement would further its perceived racism.


Some are concerned that empowering racists would further racism.

The concern's probably just:

  1. Police do racist things.

  2. Facial-recognition would help the police.

  3. Therefore, facial-recognition would help the police do racist things.

And that's it.

Don't get me wrong – a lot of folks have tried to build arguments supporting this concern. But those seem more like post-hoc rationalizations built upon the pre-existing concern.

Since the concern's more basic, it seems unlikely that anyone could alleviate the concern by debunking a particular argument for it. Those who believe that the police are racist are liable to object to anything that empowers the police so long as they continue to mistrust law-enforcement.


Contrast: Defunding the police.

Recently systemic-racism in law-enforcement has been a major topic in the news. Some have suggested that police should be defunded.

This seems like the same phenomena: those who fear law-enforcement are liable to feel safer with it disempowered for the same reasons that they'd feel threatened by it being empowered.

| improve this answer | |
  • 2
    I think there's a lot of truth in this answer, but don't you think that you're making your point a bit too strongly? Distrust of the police and their history of racial bias is certainly a factor, but as many of the other answers have documented, there are real technical issues. I don't think you can say that this distrust is the only reason – divibisan Jun 16 at 22:11
  • @divibisan: Distrust strikes me as the primary reason. I mean, the effects of different AI-related technologies on society are very, very poorly understood by the vast majority of the population; if folks were worried for technical reasons, even technical reasons based on studies like those mentioned in your answer, such fears would seem malformed. – Nat Jun 21 at 0:12
4

There is one problem that is at the base, before even starting machine learning.

It does not necessarily have a large practical effect, but it is unavoidable.

For illustration, imagine a face image of a low-resolution CCTV camera, or a face image that is small and hard to recognize in general.

Under low light conditions, there is less contrast between shadow and dark skin than between shadow and lighter skin.

It's plain physics, with no way around it.

If you have input images with low quality, there is a quality level where a white face can still be recognized or detected, but a black face can not.

The problem is that the signal to noise ratio is different, so there is actually less information in the darker image.

It may be possible to work around this by artificially discarding some information in the lighter case before it even gets to the neuronal network etc.
In situations where the image quality and the light can be controlled, this problem does not occur.

| improve this answer | |

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .