Where change is welcome.
Calling all data professionals! Are you looking to streamline and automate reporting? StreamSets Academy has the perfect solution for you: the "Agile Reporting with StreamSets" self-paced course. Agile reporting refers to automated analytics creation that fuels reporting for internal…
In a 2013 Ted Talk, Dr. Kirk Borne, a data scientist and astrophysicist, spoke about the presence of known knowns, known unknowns, and unknown unknowns. Dr. Borne described them like this: Known knowns: Things you know about already. Known unknowns:…
Salesforce Data Pipelines is a Salesforce add-on that allows users to enrich and transform Salesforce data without taking it outside of Salesforce. It requires a separate fee and has limitations around scale, concurrency, and the maximum amount of recipes or…
When you focus on the theory of democratized data, it’s easy to get idealistic. In this world of theory, practical realities fall by the wayside. The backend, where data pipeline tools and solutions live, quickly becomes an afterthought. But, forced…
StreamSets has added support to run Transformer for Spark on a new type of cluster: Amazon EMR (Elastic MapReduce) Serverless. Choose it or any of the other available supported cluster types. What is Amazon EMR Serverless? Amazon EMR Serverless is…
Data transformation sits squarely in the middle of the complex data management ecosystem. The data ingestion infrastructure you build is tailored to the types of transformations the system needs to run. And the storage and analytics layer depends on fast,…
Source-to-target mapping in data engineering refers to the process of identifying and defining how data is transferred and transformed from a source system to a target system. It involves mapping the fields in the source system to the fields in…
StreamSets has added support for Oracle Data Guard to one of our most popular origins: Oracle Change Data Capture (CDC) What Is Oracle Data Guard? Oracle Data Guard is Oracles’ database disaster recovery solution. Oracle Data Guard allows for the…
Data movement forms a crucial aspect of data management and occurs via different methods, depending on the reasons for the move. One common method is data migration from a source to a target destination. Migration destinations may include database systems,…
A cron expression is a simple tool used to automate the scheduling of tasks such as database updates, batch processing, or regular system maintenance. Data engineers, system admins, IT professionals, and other software engineers commonly use cron expressions to streamline…