The slack brush notification tapped incessantly through my AirPods Pro as I headed into the co-working space. Yet again there was a flurry of activity as another piece of ‘urgent’ data was being discussed in any number of channels. A swarm of voices began chiming in. Run this test, run that test. Like ants in a pit of data, product managers, engineers and designers all zoomed around chasing an endless morass of noise.
And yet nothing changed. Overarching retention remained stagnant, user churn increased and the quality of users fell. This was something I witness a while back – a failure of teams to make proper use of data but instead were governed blindly by it (if there’s such a thing!)
Data is often lauded as the Holy Grail of decision-making. With the rise of sophisticated analytics tools and platforms, businesses have unprecedented access to vast amounts of data, promising insights that can drive growth, efficiency, and competitive advantage. However, amidst the allure of data-driven decision-making lies a hidden danger: the risk of relying too heavily on analytics.
The Pitfalls of Blindly Trusting Data
While data can indeed be a powerful asset, blind reliance on analytics without critical examination can lead to disastrous outcomes. What I witnessed in my intro was happening within a D2C scale-up. The teams were so intent on ‘winning’ from data that they were in fact blinded by it. They couldn’t prioritise what they should be looking at for long term success and the daily view of their metrics meant that they could never truly get a ‘past 30 days’ holistic view of their product direction. Here are a few real-world examples that illustrate the potential pitfalls at an even bigger scale!
- The Case of Volkswagen: In 2015, Volkswagen was embroiled in a scandal when it was discovered that the company had installed software in its diesel vehicles to manipulate emissions tests. This unethical behaviour stemmed from a relentless pursuit of data-driven performance targets, leading to a catastrophic loss of trust and billions of pounds in fines and settlements.
- Amazon’s Hiring Algorithm Bias: Amazon famously developed an AI-powered recruiting tool to streamline the hiring process. However, the algorithm exhibited bias against female candidates, favouring CVs that contained predominantly male terms. This highlights the dangers of relying solely on data without considering broader ethical implications and human judgement.
- The Flash Crash of 2010: High-frequency trading algorithms, designed to capitalise on split-second market movements, were implicated in the notorious Flash Crash of 2010, where the U.K. stock market plummeted nearly 1,000 points in a matter of minutes. Automated trading systems exacerbated the volatility, underscoring the need for human oversight in algorithmic decision-making.
The Role of Human Expertise
These examples underscore the importance of human expertise in complementing data-driven insights. While analytics can provide valuable information, human judgement is essential for contextualising data, identifying biases, and making ethical decisions. As a Chief Product Manager, it’s crucial to strike the right balance between data-driven strategies and human intuition.
How ProductMagic can help
We specialise in helping businesses navigate the complexities of data-driven decision-making. The main focus of strategy at ProductMagic is to drive sustainable growth and mitigate risks. Having been involved in health-tech for a while we look at various core pillars:
- Ethical Data Governance: We prioritise ethical considerations in data usage and governance, ensuring that your analytics initiatives align with regulatory requirements and ethical standards.
- Human-Centric Approach: While we leverage the power of analytics, we recognise the irreplaceable value of human expertise. Our consultants work closely with your team to integrate data insights with human judgement, fostering a culture of responsible decision-making.
- Customised Solutions: We understand that every business is unique. That’s why we tailor our solutions to your specific needs, whether it’s optimising marketing strategies, improving operational efficiency, or enhancing customer experience.
Thoughts
Yes, of course data-driven decision making should be how you inform decisions and prioritisation (link to article). But this is not without risks. There are so many other factors to augment your strategy. I also see an over-reliance on quant data where tooling can quickly provide dashboards. This means that the voice of the user is often misinterpreted – do not underestimate user interviews, going out in the field and getting a real world feel of your customer.