Where Change Is Welcome
StreamSets is excited to announce the immediate availability of StreamSets Data Collector 3.11.0 and StreamSets Data Collector Edge 3.11.0. StreamSets Data Collector is open source under Apache License 2.0 and a powerful design and execution engine. It enables moving data…
StreamSets Cloud is a cloud service for designing, deploying and operating smart data pipelines, combining ease and scalability with the flexibility to execute pipelines anywhere - on-premise, or in a private or public cloud. It provides an integrated user interface…
In part 1, you learned how to extend StreamSets Transformer in order to train Spark ML RandomForestRegressor model. In this part 2, you will learn how to create Spark MLeap bundle to serialize the trained model and save the bundle…
Apache Spark has been on the rise for the past few years and it continues to dominate the landscape when it comes to in-memory and distributed computing, real-time analysis and machine learning use cases. And with the recent release of…
StreamSets is excited to announce the immediate availability of StreamSets Data Collector 3.10.0 and StreamSets Data Collector Edge 3.10.0. StreamSets Data Collector is open source under Apache License 2.0 and a powerful design and execution engine. It enables moving data…
StreamSets is excited to announce the immediate availability of StreamSets Data Collector 3.9.0 and StreamSets Data Collector Edge 3.9.0. StreamSets Data Collector is open source under Apache License 2.0 and a powerful design and execution engine. It enables moving data…
Guest post by Jeff Evans, Senior Software Engineer, StreamSets. The Field Mapper processor, introduced in Data Collector version 3.8.0, provides a flexible and powerful way to manipulate fields en masse in your records. It operates in one of three modes:…
In this blog post, I will present a step-by-step guide on how to scale Data Collector instances on Azure Kubernetes Service (AKS) using provisioning agents—which help automate upgrading and scaling resources on-demand, without having to stop execution of pipeline jobs.…
In my previous blog post, I demonstrated how to achieve low-latency inference using Databricks ML models in StreamSets. Now let's say you have a dataflow pipeline that is ingesting data, enriching it, performing transformations, and based on certain condition(s), you'd…
Data warehouses are a critical component of modern data architecture in enterprises that leverage massive amounts of data to drive quality of their products and services. A data warehouse is an OLAP (Online Analytical Processing) database that collects data from…