In this day and age of ever-evolving information digitization, we find ourselves at the intersection between technological advancements and a rise in ethical concerns. It’s time we start questioning the impact of pioneering head-strong into a data-driven society by noting both the benefits and consequences of using big data to drive collective decisions. These are the alarming details Cathy O’Neil discusses in her book, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. The title says it all. The age of algorithms has resulted in a reliance upon statistical models to predict human behavior, target customers, drive sales and model market trends. While some models may be useful, false positives, broken models and generalized practices can and do result in detrimental consequences.
As a mathematician, professor and former Wall Street quant, Cathy O’Neil raises a red flag warning us of the dangers of big data within a multitude of industries, including health insurance, universities and financial institutions. The ubiquity of computer algorithms is now seen in numerous practices across industries, whether that’s in stock trading, credit default predictions, loan qualifications, university rankings, just to name a few. These practices create an environment where individuals are simply evaluated through a series of data points then thrown into a bucket that determines whether they’re eligible for an opportunity, job or loan. Weapons of math destruction (WMD), another name for the faulty models, or what some might even argue to be worse – working models that result in unforeseen, negative repercussions affecting millions of lives – have the power to corrupt and provide misleading signals. What might seem like a standardized method to improve teacher performance within the classroom may turn out to be widespread lay-offs of qualified professors and teachers. O’Neil reminds us that the factors used to construct predictive models simply calculate a probability. Low classroom performance in a specific time and space is the result of a multitude of factors, and pointing fingers at a sole determinant is a faulty method to address the issue.
As I flipped through each page, I couldn’t help but reflect upon the challenges posed by O’Neil. I found this book not only to be a reminder of the importance of negative testing and careful evaluation, but also a warning to keep a wary eye on our progressive movement towards a future where decisions are driven by big data.