The DataOps Blog
Where Change Is Welcome
Fun with Spark and Kafka: Slot Car Performance Tracking
Antonin Bruneau is a Solution Engineer at StreamSets, based in Paris, France. He recently created a fun demo system showing how the StreamSets DataOps Platform can collect, ingest, and transform IoT data. Over to you, Antonin... Traditional product blog posts explain how to solve complex problems you might find in your daily job, but today I want to show you…
Databricks Bulk Ingest of Salesforce Data Into Delta Lake
Learn how to leverage newly released Databricks COPY command for bulk ingest into Delta Lake using the hosted StreamSets Cloud service. StreamSets is proud to announce an expansion of its partnership with Databricks by participating in Databricks’ newly launched Data Ingestion Network. As part of the expanded partnership, StreamSets is offering additional functionality for StreamSets Cloud with a new connector…
Digging Deeper into StreamSets Cloud and SSH Tunnel
The recent blog post, Access Relational Databases Anywhere with StreamSets Cloud and SSH Tunnel, presented an overview of StreamSets Cloud’s SSH tunnel feature. In this article, aimed at IT professionals who might have to configure an SSH tunnel, I’ll look a little more deeply into SSH tunneling, and explain how I configured my home router and SSH server to allow…
Sentiment Analysis: Microsoft SQL Server 2019 Big Data Cluster And StreamSets DataOps Platform
Here's how to enable greater flexibility in interacting with Big Data and managing it in a way that makes it easy to use the data for machine learning and analytical tasks. It allows data team members to deploy scalable clusters of SQL Server, Apache Spark, and HDFS containers running on Kubernetes. In this blog post, I've highlighted the integration between…
Access Relational Databases Anywhere with StreamSets Cloud and SSH Tunnel
StreamSets Cloud makes it easy to build data pipelines integrating with cloud-based relational database services such as Amazon RDS, Google Cloud SQL and Azure’s databases. Now, with the January 13, 2020 release, StreamSets Cloud also supports on-premise databases accessed securely via an SSH tunnel. In this blog post I’ll explain how to configure data pipelines with SSH tunnel, and how…