While considering consolidation of tools in the modern data stack space the focus should be on delivering a better user experience.Instead of assuming one tool is always the best, it's important to know who the users are and how they use the tools. The key factor for consolidation should be an enhanced user experience.
Major data platforms will focus on enhancing low-latency or real-time capabilities in the coming years. This evolution is seen as the next frontier after competing on traditional OLAP and slow data warehousing. The ability to work fast and efficiently in real-time scenarios will become a significant differentiator among these platforms, setting the stage for intense competition.
Data structures will evolve from being solely for the data team to becoming business infrastructure or a backend for the entire company. The increasing investment in data infrastructure by companies like Snowflake and Databricks reflects a strategic move to justify these platforms as essential for a wide range of business workflows, going beyond traditional data-centric use cases.
The modern data stack is an inevitable choice for businesses aiming to scale. Regardless of the complexity and the multitude of tools associated with the modern data stack, it is the most suitable and sustainable data model for companies as they grow. The sooner businesses align with this model, the more efficiently they can navigate complexities and avoid major overhauls during later stages of expansion.
Once a niche term, Reverse ETL has grown into a pivotal element of data strategy. Originating from the frustration of data silos, Reverse ETL bridges the gap between analytics and actionable insights. Reverse ETL plays a pivotal role in creating a single source of truth for organizations, promoting streamlined operations and informed decision-making.
With the expanding role of AI, there's a parallel need for more unstructured data. This evolution may trigger a surge in data validation and testing efforts to ensure the accuracy and trustworthiness of data, particularly as AI workflows become more integrated into business operations.
Census emerged from the frustration of data silos hindering leaders from getting a holistic understanding of their business. This led to the birth of Reverse ETL with the goal to create a single source of truth. This journey began by connecting Redshift to Salesforce at Figma, to bridge the gap between marketing and product insights.
“We're going to see more progress on this idea that data infrastructure is not just for the data team. It's actually business infrastructure. It's a backend for your company. It's a back office. And I think that's necessary because of what people are spending on this infrastructure. And at some point, whether it's CMO, CIO, CFO, CTO, right? You're gonna need to use this for more”
Boris Jabes
“I think you're gonna see real progress in the coming year with all of the major data platforms, they will start to have solutions around what you and I call low latency. So you can call it real time, you can go on streaming, you can call it whatever you want. But that is only going to get better and it's gonna be something that they use against each other to kind of demonstrate that they're able to do more scenarios.”
Boris Jabes
“I think we are going to see consolidation in this space, whether it's end to end platforms like 5X who help you manage it, or, you know, some level of acquisition. So I think consolidation is coming. I think we're going to see a lot more FinOps”
Tarush Aggarwal