If you go to your stored links on your browser right now, I’d bet that you have at least 5-10 links that go to dashboards, data tables, reports, experiment readouts or ML model summaries. We have no lack of data assets. Those assets at different points in time helped us make a decision, provide a recommendation, create an insight or bail on a project altogether. Those data assets existed at a particular moment in time to do a particular task. Then they make a one way trip to the “graveyard dashboard” or “land of forgotten experiments”.
These assets don’t necessarily feel like “products”, do they? They just feel like data “things” out there in space, linked to relevance by bookmarks on some of our browser tabs. Some of these assets should stay that way. However, others have value as potential products. I’d like to talk about why and what we gain from thinking about when a data asset becomes a product and steps we can take to support that transformation.
I want to make this really practical. If you’re reading this, find one of those links on your browser and go to the data asset. It could be a dashboard, a metric, an ML model, a script to clean data. Whatever you choose, make sure that you have the context around what it is, how it works and why it was built (even if you didn’t build it yourself). I’m looking at a metric definition and the SQL query that feeds it. Let’s explore some key questions.
What does the data asset do? This might seem like a stupid question. “It’s obvious what a metric does!” “How could you not see what a dashboard is supposed to do?” But I don’t think it is actually that obvious. Asking what a data asset does means that you need to understand what problem it addresses. Someone took the time to create the asset. Were they solving a problem that only existed in that moment? Is the “thing” the data asset does a capability that needs to live on for the organization? Understanding what the data asset does exposes if the need for that capability is recurring - one of the most important characteristics of the need for a data product.
What problems does the data asset address that other data assets or products do not? Most dashboards or metrics we build do not go into oblivion because they are useless (even if it is easy to feel like this if you work in data). Instead, they go into oblivion because a different data asset or product did the job better or more completely. It isn’t enough to know that the capability a data asset provides is recurring, we also need to know that a better version of it doesn’t exist elsewhere. If we can confidently say that this data asset does something unique that has a recurring need - we’ve got an even stronger case to develop the asset into a product under active development and management.
If the data asset serves a recurring need that another product or asset does not, is that need high enough priority for the business to devote resources at the moment? As the data landscape becomes more complex and company’s strategies diversify, the tools and capabilities needed from data become challenging to manage. Just because something is useful and is relevant for a problem does not mean it is worth resourcing. For some data assets, it isn’t a matter of if they should become products, but when they should become products. We only have finite resources in a business and should only commit to those data products that serve the company’s most critical outcomes.
Who needs to sponsor the data asset becoming a product? In some cases, the investment and overhead might be small. For example, managing a metric and ensuring it continues to be relevant for the business and has correct SQL underlying it may not be a huge lift. But it does take time. Now, imagine that the metric goes from 1 key metric to 10. Then to 20. Suddenly, you have someone’s entire role defined. Soon after that, a team needs to support this metrics library. It is critical to answer the question of who is going to go to bat for active investment in a data asset becoming an actively managed product. Simply saying “we should do that” or getting a head nod in a meeting isn’t enough. If you can’t get resources committed, there’s no chance the transition to a product happens.
When should a data product ultimately stopped being managed as one? It is much easier to say “we should invest in this data asset to make it the best it can be” than to say “we should stop investing in this data product we’ve built over the previous years”. If we want to make room for new data products, we probably have to leave behind some of those that are nearest and dearest to ourselves as data scientists. This is probably the most painful reality for data organizations: our resources are not infinite and the data product pool we want to manage has a hard cap to it. So, as you sit down with your dashboard, metric or ML model and think about what it would be like to have it as a first class product at your organization, also think about what you’d give up to make it that way.
While the exercise above might seem overly simple, I suggest you try it with a few colleagues. Ask them to find a few links stored in their bookmarks and use the framework above to assess if it is worth investing in the data asset as a product.
I am so grateful for your willingness to share this newsletter with others. The conversation and feedback are what keep it fun to write. If you enjoyed reading this, please share with others or subscribe if you haven’t already with the buttons above!
Thank you Eric for sharing your thoughts.
As you mentioned in number 5, choosing to stop maintaining a data product that is dear to our heart is very painful. And I think many organizations and of course people within them have a hard time doing it. How can someone lower in the business hierarchy (ie. Data Scientist) convince management that it's time to let go of the data product we made during the previous years?
Thank you for Sharing Eric.