Exploiting data to create competitive advantage is the name of the game. It is incumbent upon enterprise data leaders to provide the technical foundation for their companies to ensure that all useful data is available to all potential users in a timely and well-governed manner.
We call this need to deploy data to the far reaches of the organization pervasive intelligence. In contrast to the centralized world of business intelligence, pervasive intelligence takes as a given the decentralization of analytics and the democratization of analytic skills.
Pervasive intelligence requires a data movement architecture that reliably gets consumption-ready data to a wide variety of users, with the flexibility to support a variety of changing source data and infrastructure types.
The challenge is to create a platform for data movement whereby new pipelines can be designed and deployed quickly, scaled automatically, and monitored for performance. The system must also enable agility so that dataflows can be quickly updated when requirements or technologies change.
The key to mastering continuous dataflows and overcoming data supply chain challenges is to embrace DataOps. In the same way that DevOps tools and processes enable the continuous delivery of applications, DataOps allows enterprises to create reliable and continuous data movement architectures that frequently iterate in response to changes in infrastructure, upstream data providers, and analytics requirements.