For a while now, StreamSets has been touting the merits of DataOps as a new method for disciplined management of data topologies. In the same way DevOps changed how applications are developed, deployed and operated, DataOps will greatly improve data integration in the modern era. You can read our point of view here.
The fact is, we were a DataOps company before the term was coined in late 2015, as our founders recognized the serious operational challenges that unstructured, streaming data and hybrid cloud infrastructures would pose to enterprises used to static, batch structured data integration. Since our inception, we’ve been focused on enabling teams to operationalize data movement and we created the StreamSets DataOps platform to empower customers to capitalize on a DataOps approach.
As a milestone for notoriety, research firm Gartner recently published their annual Hype Cycle for Data Management (required Gartner account) and DataOps was included for the first time. Veteran Gartner research VP Nick Heudecker penned a supporting post on Gartner’s public blog which provides a useful summary of Gartner’s current point of view.
Is this a validation of DataOps as a “thing”? As the name of the document suggests, inclusion of DataOps is not an endorsement of the trend, but does note it as something that is emerging. For our part, StreamSets is already working with many customers today to help them on their journey with DataOps so we see it happening in the real-world.
In his blog, Nick states “you cannot buy DataOps” which, strictly speaking, is true since DataOps is a methodology not a product, though technology – and disciplined process and skilled personnel – are required to garner the benefits of DataOps. Much like other broader data management initiatives such as Data Governance, DataOps is not addressed by technology alone but rather by a combination of people, process and technology. That said, technology is absolutely necessary to making DataOps a practical reality. In fact, the right technology can act as a foundation on which the other two pillars can rest. That is the underlying motivation upon which StreamSets was founded and remains our vision to this day.
In the coming weeks and months we’ll be articulating our detailed view of what DataOps means (and doesn’t) and where and how it can applied to the greatest benefit. So watch this space as data management and integration evolve and interest in DataOps grows.