“I think if we launch this feature in our analytics product, it’s going to transform how analysts do their work and shift the data ecosystem entirely.” In 2015, there were a lot of promises of “disruption”, “transformation” and “radical shifts” in how we work with and digest data. Those promises sounded a lot like the statement above. In 2021, there are still promises of change, but few successful companies promise radical change in how people work - they focus on enabling and improving data consumers’ (and producers’) workflows.
What makes those companies so successful? They’ve realized they need to go to the customer, not ask the customer to come to them. A consistent theme I’ve observed across technical products is the assumption that they can get people to change. Yet people despise change. Think about what happens when your workflow is entirely disrupted - you feel completely out of sorts, frustrated and your engagement drops. The same goes in data - data consumers and producers have habits. Sure, you can ask them to change those habits, but you better have the best product ever built to see that happen. What’s more likely to succeed, both near and long term, is asking people to make smaller shifts in their existing workflow. They need a nudge, as Richard Thaler would describe it, not a shove.
To give people an effective nudge that affects a small change in their existing workflow, you need to fundamentally understand their workflow and what they are trying to do. A lot of existing BI dashboard tools are one size fits all. They expect people to come to them. Yet the downstream builders and consumers of dashboards are much more nuanced in how they choose and make decisions about their work. For a select few, that single-instance dashboard build might be helpful. But for most, that dashboard just won’t integrate into their workflow. There’s a reason nearly every company sees a dashboard graveyard, but one or two dashboards that people come back to repeatedly. Those successful dashboards fit into peoples’ workflows.
Of course, to affect people’s’ workflows in a meaningful way, we need to understand those workflows and build generally useful products that integrate into them. This is much easier said than done. Why? There’s the story we tell each other about our data workflows (e.g. the ones we write down in our final analyses and in published papers) and then there’s the real workflow (e.g. the one that is such a haphazard hot mess that we don’t like to speak about it regularly). Here are 5 ways to get started:
Don’t ask people how they work, spend time observing it. If you ask people about their workflow, you’ll get an absurdly rosy picture of what their day to day looks like. Decisions will seem clear, dashboards will look perfect, metrics fairly well defined. Until they trust you - then you’ll start to hear about the mess. Once they really trust you, all you’ll hear about is the mess. To get to the details you need, ask them if they’d be okay doing a working session with you on a problem they are working on, or potentially be okay recording their screen for you. The latter sounds weird, but it is fairly effective. The main idea is to get a real idea of what people actually do as they engage with problems.
Look for habits, not anomalies. I’ve seen a lot of data product pitches that focus on avoiding gigantic, costly mistakes that happen once in a great while. To be sure, these are things companies would love to avoid. But I’d rather invest in affecting the processes/tools that are already habits in the data producer or consumers’ workflows. It sounds nice to build things that prevent those expensive mistakes that analysts make once in a great while, but it’s even better to make a small change that affects workflows of hundreds of analysts.
Ask others what they would invest in. Its fairly common to hear the question “what should we prioritize fixing?” That generally produces one of two outcomes: a list so long that you need to devote a week to reading it, or a list with so few items that it can’t possibly represent reality. Instead, ask people what they would invest in, given a team and a budget. It sounds a bit strange, but when people feel like they are actively investing and making a decision, they’ll quickly get to the things they feel are most critical. Make others a data product owner/decision-maker (at least for a bit), and you’ll see some interesting results.
Don’t assume experiences generalize. I’m a firm believer that a lot of internal products are built because a PM talked to one or two people who said something was a huge issue. It might solve an issue for those 1-2 people, but there’s no guarantee the usefulness generalizes to a huge population. Once you have ideas or a proposal, don’t just look for sign-off from the people who helped create the idea. Look for sign-off from those who may not have been a part of the process at all. If your idea for a data product generalizes, it should be fairly easy for them to digest and comment on it.
Try to avoid using a general framework to represent a data workflow. In a desire to be fairly linear and clear, we often take processes that are complex and organize them into clear steps that give too much confidence to how orderly a process is. When you’re building a model of someone’s workflow, allow for some chaos and uncertainty. Allow them to come back to steps, skip steps, or stop mid workflow in frustration. Frameworks are nice to have, but they often don’t help right away.
Six years after I heard the first pitch about data products that would disrupt the data industry, I’m more convinced than ever that the core habits of data producers and consumers won’t fundamentally change. However, we can build data products that enable us to make those habits more consistent and efficient. To me, that’s the biggest transformation we have in front of us - improving the things we know to be important, not finding the next frontier of data analysis.
I like your way of pointing things out like 'allow them to come back to steps, skip steps and... Frustration'. Well, I learnt it the hard way. Thank you for making your blog informative.
You make interesting points about building data products, especially like the "a working session with you on a problem they are working on" idea which is very actionable!