The DataOps Blog
Where Change Is Welcome
The Next Chapter for StreamSets
Arvind Prabhakar and I co-founded StreamSets in 2014 with an audacious vision: data should be the lifeblood of the enterprise.…
Announcing SOC 2 Type II Certification for the StreamSets DataOps Platform
The StreamSets DataOps Platform has successfully completed a SOC 2 Type II Compliance Audit. Our SOC 2 report came with zero exceptions, which is a testament to our established and rigorous systems and processes for handling data. With our commitment to empowering data engineers to build and run smart data pipelines for data integration comes an understanding of how important…
The Importance of Data Orchestration Pipelines for Organizations
Maryann Agofure is a Cloud engineer and technical writer who loves fried plantains. She’s currently on a cloud data engineering journey to gain a deeper understanding of data pipelines for better analysis, hence her interest in data orchestration, and seeking ways to grow her knowledge with big data tools. Her job as a Technical writer also helps her research and…
The Role of ETL in Data Integration
The relationship of data to today’s modern enterprise is becoming more sophisticated. As data sources continue to increase and data formats continue to evolve, ETL and its place in modern data integration are also changing. What was once a simple three-step process, ETL has become more nuanced and is often even replaced by a new data integration strategy, ELT. So,…
MLOps: How Data Teams Can Give ML Algorithms Life and Longevity
MLOps is emerging now more than ever in everyday workflows. Data Engineers are often tasked with the plumbing work to make data available for machine learning models. Even though statistically a small number of models get deployed to production, the ones that do often involve our time to manually update and test our previous workflows. But what is MLOps and…
5 Best Practices to Ensure a Seamless Data Migration
The list of issues that can derail a data migration can be scary. This is why, as far back as 2009, and as recently as 2021, Gartner has released statistics detailing the prevalence of failed, delayed, and over budget data migration projects. In 2009, 83% of data migrations failed or exceeded their budgets and schedules. And through 2024, Gartner predicts…