StreamSets Data Integration Blog

A recent StreamSets research report examined data governance challenges in the modern enterprise. Based on survey data from over 650 data decision-makers and practitioners, we found that complex data architectures combined with line of business teams building their own data pipeline integrations (outside ITs purview) are leaving them struggling to understand, govern, and manage a fragmented data supply chain.  Check…

StreamSets has added support to run Transformer for Spark on a new type of cluster: Amazon EMR (Elastic MapReduce) Serverless. Choose it or any of the other available supported cluster types.  What is Amazon EMR Serverless? Amazon EMR Serverless is a feature of Amazon EMR that allows users to run big data processing workloads without having to provision or manage…

Legacy technology is an age-old problem in financial services. However, with upstart online banks and fintech companies using real-time data to lure customers away with just the right offer or superior experience, legacy technology environments are giving financial services organizations a new (and sizeable) data headache.  Complex Hybrid Environments Can Block Data Access Complete data is the key to unlocking…

Data transformation sits squarely in the middle of the complex data management ecosystem.  The data ingestion infrastructure you build is tailored to the types of transformations the system needs to run. And the storage and analytics layer depends on fast, reliable transformation.  So when you’re choosing a data transformation tool, it’s not as simple as selecting the “best” one from…

The business value of data analytics is well understood. Insights drawn from data can yield impressive business results in almost every industry context imaginable. A significant challenge remains, however: data is seldom all in the right place or format. Data integration is essential for getting the most out of dispersed, heterogeneous data sources. Some organizations have their data in OLTP…