Racial Bias in Algorithms

It's not racially biased. It doesn't even use race as a feature. It just so happens that blacks statistically have worse criminal records, and people with worse criminal records tend to commit more crimes in the future. A white person with the same record is not treated any differently. And the decision is based on objective data and statistics, not flawed human judgement. Human judges are incredibly biased and their predictions are barely better than chance. It's insanity that hu

You can have an accurate prediction which also reflects systemic bias.There was a story recently about NYC cops being given race based targets for arrests. If that data was fed into a system and predictions generated they could be both correct and racist.That's maybe an extreme example, I think the person you're replying to was trying to illustrate the same thing but with greater indirection between the racism and the arrest.To give a non-race example, I've heard that ugl

then won't the Heat List disproportionately output people from a certain race?This will only happen if that certain race commits more crimes (in the training data). If you take race out of a statistical predictor designed to learn crime, but race is a good predictor of crime, then the predictor might learn race at an intermediate step.Now there are statistical issues one might run into - e.g., early overfitting of what is essentially a bandit algorithm, and unaccounted f

If black Americans are more likely to commit local crimes and you are developing software for police to predict hotspots of crime, wouldn't race be another aspect of pattern recognition in this narrative?And would it be wrong for banks or loan programs to optimize on such pattern recognition if it were found that ethnicity could improve their forecasting? Or for software companies to forecast employee effectiveness on similar pattern recognition?

I think you've very severely missed the point. Differences like those you describe generally reflect biases in our socal structure, which are then (correctly, but problematically) reflected in statistical models and algorithms of all stripes. This becomes more problematic when we overinterpret the resulting findings, or uncritically incorporate the resulting predictions into real systems.A good example is the criminal justice system in the US. Minority communities are much more intensive

> For instance, at least here in the U.S., it is illegal for police to profile people based on raceIt must also be mentioned, sadly, that data shows that shows that police profile racially despite it being illegal, including, sadly, judges, who seem to punish minorities about whom there are negative stereotypes more harshly for similar crimes.> in the aggregate, have some predictive value.This data does have predictive value, for sure, studies have shown. But it's important t

the problem is there is a huge difference between:- what is statistically there- why it's statistically there (e.g. high crime rates in poor minority populated areas mean nothing if there is racist polic/laws even through they are statistically there)- if the statistics are misleading, biased and/or incomplete (e.g. bad English might statistically imply lower paying jobs in English speaking countries, things are much much less statistic relevant the more you move away fro

It seems to be that it's not about racism but rather probability. This concept is described here http://en.wikipedia.org/wiki/Sensitivity_and_specificity

The accuracy or inaccuracy of stereotypes reminds me of the regulations and ethical debates regarding the use of machine learning for example for creditworthiness. The algorithms often turn out to become "racist", because from the data they have, race is a good predictor.I'm still somewhat ambivalent on that, because I'm not convinced that statistics can be racist. But people pointed out to me that the way these statistics are collected might not be free of bias. Also, for

No, I have repeatedly and explicitly stated that protected classes like race should not be used as inputs into any sort of model. The notion that I have said that we should use race as an input to a model is completely false.What I am saying is that pointing to disparities in the outcome of these models to claim that the models are biased is not, on it's own, a reasonable conclusion. As you point out, people in Joshua Tree and South LA have higher rates of crime than average. So if our m