The DataOps Blog
Where Change Is Welcome
3 Reasons to Love Your Data Engineer
Data scientists get a lot of press these days, and it’s not without good reason. Companies live and die by data and decisions are made with the high visibility work that data scientists and analysts do. But these ‘front-end’ data professionals have a secret… Their job is infinitely more difficult without a data engineer in the background. According to CIO…
EDW or EDH? Data Lake, Warehouse or Lakehouse?
As John Zada puts it, “There can seldom be just one explanation to things. As a result our paradigms can be over-simplistic, incomplete or inaccurate – removing the complexity from the world which is actually one of its defining qualities.” So can we really evaluate such impactful tools in such simplistic terms? Database to Data Warehouse: How It All Began …
Does Migrating Workloads to AWS Require Re-writes?
Migrating your data workloads to AWS without redesign headaches -- urban developer lore or critical capability for modern enterprises? In this blog we will investigate by building a data pipeline to Google Cloud Platform and then migrating the workload to AWS. Moving clouds may sound like moving mountains. But your ability to move data quickly and reliably between clouds ensures…
3 AWS Data Pipelines to Impress Your Boss
In this blog, you will learn how to design AWS Data Pipelines in StreamSets Data Collector, a fast data ingestion engine, that you can then use to impress your boss with in 2021 -- the year of the data engineer! Migrate An On-Prem Oracle Database to AWS Aurora RDS Consider the use case where you have a legacy data warehouse…
2021: The Year of the Data Engineer
2021 is going to be a good year. We may not be there quite yet, but there’s no denying things are looking up. As vaccines kick in across the world, focus will shift from fighting for survival to reopening, rebuilding, and strengthening fractured economies alongside our returning health. Remember our 3 scenarios for adjusting your data practice? It's clear that…