If the last 2+ years of AI innovation or the last 10+ years of organizations leveraging data and data science to create new business opportunities has not yet convinced you to build a data estate, I’m not sure what will. We’re certainly not lacking any examples of data-powered organizations and companies like Microsoft are building next generation platforms like Fabric aimed at enabling AI use cases.
Before you go full steam ahead in your data estate, consider the following three topics.
Lead with Business Value
Your organization already has data being used to support a current business process. This data could be financial data from a payment system, purchasing intent from a marketing platform, or productivity data from HR systems.
All of this data is already fit for purpose within their respective operational system. It’s easy to assume that extracting and landing data in a data lake is sufficient. However, this is the quickest path to failure. You already have data doing what it’s supposed to be doing, supporting a business process, and capturing human interaction. You need to think of the value gained by extracting that data from the source system and stitching it with another data set.
Let’s imagine how data can create value in a payment system. For example, how might you prevent fraud? You’ll need to think about other data sets beyond what is captured at the point of sale. Past purchasing behavior, location information, data from a mobile application, etc., all stitched together can help build fraud detection algorithms. This is an excellent example of how building a data estate that supports real-time data integration, stitched data, and machine learning models would add value and prevent theft of customer funds.
Think Critically about the Latest Technology
Imagine you just attended a conference, and the main session featured a presentation on the most incredible technology you’ve ever seen. Wonderful! It’s going to solve all of your problems! You just need to pull the team off of core engineering work and have them pivot to this shiny new thing. After all, the technology capability is not part of your existing data estate, so you should build it immediately. Ship it!
That’s over the top, but you get the point. This isn’t unique to data & AI technology either. It’s a general problem in the technology industry, that people tend to gravitate to new, cool technology whether or not it might actually solve our problems. Now, you may need a real-time data integration capability when dealing with financial data, but maybe that’s not the right fit for what you are trying to accomplish in other areas. For example, if you are dealing with certain types of HR data, many of the analytics of these data would be sufficient to process on a daily, weekly, or even monthly basis. It’s essential to consider investing in the latest technology that supplements a gap in your data & AI capabilities and brings value to your overall estate.
Consider the Long-term Plan
How often do we make long-term compromises for the sake of today? Too often. That’s not to say we shouldn’t make trade-offs. It’s important to have healthy conversations about near-term needs and long-term impacts.
Long-term thinking with a data estate should not be about how you will build it or even what platform you select (we all have our preferences). The more significant thing to consider is how you will maintain it. Data gets stale, and inferior data flows into the estate. People change roles and companies. Technology evolves.
A strong data governance organization is critical, especially in this era of AI, where data plays a crucial role in creating new business value. Not enough governance leads to low-quality data and a lack of good data stewardship. “Garbage in, garbage out,” as they say. With the right governance model, you can create a comfortable, secure data environment that allows for innovation and creativity but ensures enough guardrails are in place for the longer term.
It’s a crazy world of data, so be sure you are creating business value with your chosen data & AI capabilities. Validate all new and emerging technology. And don’t forsake your data governance organization.