The DataOps Blog
Where Change Is Welcome
StreamSets Live: Demos with Dash!
45 minutes. Live demos. No (0) slides! Key takeaways: Visually build data pipelines like you code: build, verify, iterate, commit, rinse, repeat! Ingest data from…
StreamSets Transformer is Here
The emerging practice of DataOps encompasses many activities that an enterprise may execute today including ingestion, ETL, and real-time stream processing. The trick to DataOps is to execute these things in concert with global visibility, central control, and unparalleled insight into performance and security. The StreamSets DataOps platform provides a foundational capability for modern companies to build a DataOps practice…
Automating Kerberos KeyTab Generation for Kubernetes-based Deployments
A major challenge when deploying dataflow pipelines to run on Kubernetes is how to handle Kerberos principals and keytabs needed when pipelines write to secure Hadoop. One approach, of using Kerberos keytabs for principals of the form @ (without a host field), incurs security risks as a keytab for such a principal could be used on any host in the…
Announcing StreamSets Data Collector 3.10.0 and StreamSets Data Collector Edge 3.10.0
StreamSets is excited to announce the immediate availability of StreamSets Data Collector 3.10.0 and StreamSets Data Collector Edge 3.10.0. StreamSets Data Collector is open source under Apache License 2.0 and a powerful design and execution engine. It enables moving data between any source and destination, performing transformations, and push down analytics along the way. To download, click here. StreamSets Data…
How DataOps is Adding Value to Data Lakes
For those of you who joined us on June 6th, you dialed into a forward-thinking conversation between three industry experts. They waxed poetic about topics including big data, DataOps, governance, data science, and more in an effort to help modern data architects and analytics professionals better understand the emerging practices and themes around DataOps. If you missed the videocast you…
A New Definition of DataOps
This is short post, but relevant. Ever since DataOps was started (about 5 years) it hasn’t had a well-adopted and common definition. Wikipedia is partially OK at: DataOps is an automated, process-oriented methodology, used by analytic and data teams, to improve the quality and reduce the cycle time of data analytics. But it is not great and simply builds on…
