Where Change Is Welcome
There are lots of tools for quick bulk replication to the cloud, but rarely does your journey end there. How do you keep that data current in the cloud while the data and its source systems are constantly evolving? Frequently…
StreamSets DataOps platform provides an end-to-end enterprise solution to maximize the value of your Snowflake Data Cloud. The platform can ingest data into Snowflake (using batch, streaming and change data capture data pipelines). With the preview of the StreamSets Engine…
StreamSets is excited to announce the immediate availability of StreamSets Transformer Engine 4.0.0. It is a modern ETL engine that enables developers and data engineers to build data pipelines and transformations that execute on Apache Spark. Highlights This is our…
Imagine asking Amazon Alexa or Google Home to run your ETL, data processing, and machine learning data pipelines. For example, "Start my data pipeline on Amazon EMR", “How many active jobs do I have running on Databricks?", or "Stop my…
Our analogy is, like when you have, one machine learning algorithm, the machine learning algorithm is training, training and training. So in the same way. We think together we are going to learn, learn, and learn with the different areas.…
Change Data Capture is becoming essential to migrating to the cloud. In this blog, I have outlined detailed explanations and steps to load Change Data Capture (CDC) data from PostgreSQL to Amazon Redshift using StreamSets Data Collector, a fast data…
Learn how to load data from S3 to Snowflake and serve a TensorFlow model in StreamSets Data Collector, a fast data ingestion engine, data pipeline for scoring on data flowing from S3 to Snowflake. Data and analytics are helping us…
What is Dataproc? Dataproc is a low-cost, Google Cloud Platform integrated, easy to use managed Spark and Hadoop service that can be leveraged for batch processing, streaming, and machine learning use cases. What is Google BigQuery? BigQuery is an enterprise…
Migrating your data workloads to AWS without redesign headaches -- urban developer lore or critical capability for modern enterprises? In this blog we will investigate by building a data pipeline to Google Cloud Platform and then migrating the workload to…
In this blog, you will learn how to design AWS Data Pipelines in StreamSets Data Collector, a fast data ingestion engine, that you can then use to impress your boss with in 2021 -- the year of the data engineer!…