Book Review: "Weapons of Math Destruction" by Cathy O'Neil

From Das U-Blog by Prashanth
April 18, 2017 - 7:49am
I've recently read Weapons of Math Destruction by Cathy O'Neil. It is a short but dense exposition into the various ways that computer algorithms can determine the courses of people's lives and exacerbate existing societal inequities and biases/prejudices, in areas like education, civic engagement, education, health at the workplace, and many others. It argues that while many algorithms used in big data can be used for good, many instead widen inequalities and reinforce various cycles of poverty because of their opacity, lack of accountability, poor use of statistics, and lack of critical examination by those in charge (who instead use results and predictions generated by such algorithms to fire workers, deny opportunities to potential employees, financially prey on poor people, and so on, making such predictions self-fulfilling prophecies as only confirmatory data is fed back in); in particular, many of these algorithms and models use questionable proxies to predict certain attributes or behaviors (especially when the desired attributes are hard to quantify but the proxies are easy), these models are rarely transparent in what inputs are collected and how they are manipulated to produce outputs, and further research and fine-tuning are rarely performed to correct models that most humans would recognize produce incorrect results (but which computers would miss). It concludes that extensions of existing regulations on use of health and financial data are needed to curtail the misuse of such algorithms, and that simultaneously data scientists need to be scrupulous about the ways that their work is used and developed.I rather enjoyed reading this book: it's pretty fast-paced, yet gives many detailed examples of the abuse of these algorithms to form a compelling narrative. Additionally, it in many ways follows the book The Attention Merchants by Tim Wu (which I have previously reviewed), because as that book shows the various ways that companies collect and sell customer data, this book shows the various ways that data can be used for the benefit of those companies (even if that works against some of those customers). There are only two issues that I have with this book. One is that the few times that politics comes up, the author's political bias (in favor of liberals in the US) is obvious; perhaps this is just due to the nature of the author's passionate crusa

Continue reading this article »