StreamSets Data Integration Blog
Where change is welcome.
A Birds-Eye View of a Modern Data Stack
The modern data stack is less like a stack and more like an ecosystem with many participants. This constellation of technologies coalesces around a few guiding principles. Three Guiding Principles The first principle of the modern data stack is complete customizability. Eschewing a one size fits all solution, the modern data stack allows for data teams to pick and choose…
3 Skills You Need to Succeed in Data and Analytics Today, According to Dr. Beverly
We recently kicked off our Women in DataOps series with Dr. Beverly Wright, a data and analytics thought leader with 30 years of experience. Beverly has spent many years teaching data science and analytics to undergraduates, master’s students, PhDs, and executives. She’s also spent more than a decade consulting for companies like Nielsen, another decade on the client side at…
Data Pipeline Architecture: Key Design Principles & Considerations for Data Engineers
Data pipelines are meant to transfer data from a source or legacy system to a target system. Easy right? Well not so much. As a Data Engineer, it’s our job to be responsible for multiple different data pipeline architecture decisions during the design phase. Answering questions like, what are the source/s and target/s for this data? Is this data coming…
Thinking Through the Basics of Your Data Governance Framework
Your company may not have a documented or formally defined data governance framework. But if data is created and used in your organization, you have a governance framework. Whether it’s effective or not… that’s another question. The challenge with data governance frameworks, and data governance in general, is tying together the elements of how your organization collects, manages, and archives…
Enterprise Data Integration: What it is, Why it Matters, and How to Approach It
Successful organizations learn from the past, are able to predict the future, and understand the shape of their present. Enterprise Data Integration is the magic that binds all these views together. What is Enterprise Data Integration? Enterprise Data Integration is the process of combining data from various sources and unifying that data in a sensible way. In the past, this…