The DataOps Blog

The StreamSets DataOps Platform has successfully completed a SOC 2 Type II Compliance Audit. Our SOC 2 report came with zero exceptions, which is a testament to our established and rigorous systems and processes for handling data.  With our commitment to empowering data engineers to build and run smart data pipelines for data integration comes an understanding of how important…

Maryann Agofure is a Cloud engineer and technical writer who loves fried plantains. She’s currently on a cloud data engineering journey to gain a deeper understanding of data pipelines for better analysis, hence her interest in data orchestration, and seeking ways to grow her knowledge with big data tools. Her job as a Technical writer also helps her research and…

The relationship of data to today’s modern enterprise is becoming more sophisticated. As data sources continue to increase and data formats continue to evolve, ETL and its place in modern data integration are also changing. What was once a simple three-step process, ETL has become more nuanced and is often even replaced by a new data integration strategy, ELT.  So,…

MLOps is emerging now more than ever in everyday workflows. Data Engineers are often tasked with the plumbing work to make data available for machine learning models. Even though statistically a small number of models get deployed to production, the ones that do often involve our time to manually update and test our previous workflows. But what is MLOps and…

The list of issues that can derail a data migration can be scary.  This is why, as far back as 2009, and as recently as 2021, Gartner has released statistics detailing the prevalence of failed, delayed, and over budget data migration projects. In 2009, 83% of data migrations failed or exceeded their budgets and schedules. And through 2024, Gartner predicts…