The Costs of Automated Decision-Making Systems
States increasingly use computers to administer government services, often taking human decision making out of the process. For instance, computers terminate individuals’ Medicaid, food stamps, and other welfare benefits. They detect fraud, paying only legitimate claims. Champions of automated systems extol their cost savings. Because these systems reduce the human role in executing government policy and programs, state governments can cut staff and close offices.
But, of course, to save money, we often need to spend money–systems need to be programmed to reflect current policy and monitored to ensure their accuracy. New York’s Department of Health recently learned that lesson the hard way. According to Government Technology, the Department failed to devote sufficient resources updating eMedNY, the state’s automated system used to detect Medicaid fraud and abuse. Because the agency lacked a “formal, structured process” for prioritizing and approving programming updates and changes, the system failed to catch approximately $200 million in fraudulent claims. Audits revealed millions in questionable claims from dentists. One dentist billed Medicaid for providing seven patients with 32 fillings each and then for pulling the same 32 teeth for each patient. Although these opaque automated systems pose many other problems, especially regarding the procedural protections owed individuals, see here, this one has seemingly caught the attention of elected officials.