Where Change Is Welcome
Learn how StreamSets, a modern data integration platform for DataOps, can help expedite operations at some of the most crucial stages of Machine Learning Lifecycle and MLOps. Data Acquisition And Preparation Machine learning models are only as good as the…
Grok leverages regular expression language that allows you to name existing patterns and/or combine them into more complex patterns. Because Grok is based on regular expressions, any valid regular expressions (regexp) are also valid in grok. In StreamSets Data Collector,…
In this blog, we will review how easy it is to set up an end-to-end ETL data pipeline that runs on StreamSets Transformer to perform extract, transform, and load (ETL) operations. The pipeline uses Apache Spark for Azure HDInsight cluster…
StreamSets is proud to announce an expansion of its partnership with Databricks by participating in Databricks’ newly launched Data Ingestion Network. A key component of this integration is leveraging the recently released Databricks COPY command for bulk ingest into Databricks…
In this post, we will explore how to bulk ingest and process change data capture (CDC) information from Oracle 19c database using the enhanced Oracle CDC Client origin into Databricks Delta Lake in StreamSets Data Collector, a fast data ingestion…
StreamSets is excited to announce the immediate availability of StreamSets Data Collector 3.16.0. StreamSets Data Collector is a powerful design and execution engine. It enables you to build and deploy intent driven and data drift resilient smart data pipelines. Highlights…
Learn how quickly you can start ingesting and aggregating Clickstream logs on Amazon EMR using StreamSets Transformer Engine, and see how the data is analyzed in Elasticsearch, Kibana, and Amazon Redshift. What is Clickstream Analysis? Clickstream analysis by definition is…
Learn how to load a serialized Spark ML model stored in MLeap bundle format on Databricks File System (DBFS), and use it for classification on new, streaming data flowing through the StreamSets DataOps Platform. In my previous blogs, I illustrated…
UPDATED June 22, 2021 Here's the recording of the special edition introducing StreamSets Summer '21 from June 3, 2021. On March 25, 2020 I hosted my first StreamSets Live: Demos with Dash session and I want to thank everyone that…
Recently I attended an inspirational tech talk hosted by Databricks where the presenters shared some great tips and techniques around analyzing COVID-19 Open Research Dataset (CORD-19) freely available here. As stated in its description: "In response to the COVID-19 pandemic,…