StreamSets Data Integration Blog
Where change is welcome.
What Is Source-To-Target Mapping?
Source-to-target mapping in data engineering refers to the process of identifying and defining how data is transferred and transformed from a source system to a target system. It involves mapping the fields in the source system to the fields in the target system and specifying any transformations that need to be applied during the migration process. For example, let's consider…
Ensuring Disaster Recovery and Reducing Costs With Oracle Data Guard
StreamSets has added support for Oracle Data Guard to one of our most popular origins: Oracle Change Data Capture (CDC) What Is Oracle Data Guard? Oracle Data Guard is Oracles’ database disaster recovery solution. Oracle Data Guard allows for the creation of one or more standby databases which can be used to protect critical business data in the event of…
How to Send Data from SQL Server to S3
Data movement forms a crucial aspect of data management and occurs via different methods, depending on the reasons for the move. One common method is data migration from a source to a target destination. Migration destinations may include database systems, data warehouses, or data lakes and involve the help of migration tools. Data lakes have become a popular destination for…
From Providing Data to Explaining Why: How Large Language Models Are Changing the Role of Data Practitioners
Unlocking the power of Large Language Models in data practice: From SQL troubleshooting to natural language querying with LlamaIndex. Image available to author via unique access to Midjourney the author assumes responsibility for the authenticity. Over the past few months, there has been a lot of talk about how ChatGPT, and other Large Language Models (LLMs), will change the world.…
Everything You Need To Know To Use Cron Expressions Like a Pro
A cron expression is a simple tool used to automate the scheduling of tasks such as database updates, batch processing, or regular system maintenance. Data engineers, system admins, IT professionals, and other software engineers commonly use cron expressions to streamline repetitive tasks. In the following sections, we’ll explain the basics of cron expression syntax, then provide a cheat sheet you…