Virginia Eubanks for Slate describes the dangers of relying too heavily on black-boxed algorithms to create and enforce policies.
Policy algorithms promise increased efficiency, consistent application of rules, timelier decisions, and improved communication. But they also raise issues of equity and fairness, challenge existing due process rules, and can threaten Americans’ well-being. Predictive policing relies on data built upon a foundation of historical racial inequities in law enforcement. Remote eligibility systems run on the questionable assumption that lacking a single document—in a process that often requires dozens of pages of supporting material—is an affirmative refusal to cooperate with the welfare determination process.
