StreamSets solutions architect Alex Woolford is a data engineer with deep experience building robust and scalable solutions using technologies such as the StreamSets DataOps Platform, Apache Kafka, and the Cloudera and Hortonworks Hadoop distributions. In his role at StreamSets, Alex provides our customers with expertise including architecture design, demonstration systems, prototypes, presentations, and product configurations. […]
Introduction I am very excited to announce the new Hadoop FS Standalone origin in StreamSets Data Collector 3.2.0.0. Data Collector has long supported the Hadoop FS origin, but only in the cluster mode. The Hadoop FS (HDFS) Standalone origin does not need MapReduce or YARN installed and can run in multithreaded mode, with each thread […]
Hello from your newly-appointed community champion and technical evangelist here at StreamSets! My name is Dash Desai and you will find me writing blog posts and cruising the community forums answering questions about StreamSets Data Collector as well as learning from community members. I will also be presenting at meetups and conferences so if you […]
The term DataOps is a contraction of ‘Data Operations’ and a play on the idea of DevOps. It seems to have been coined in a 2015 blog post by Tamr co-founder and CEO Andy Palmer. In this blog post, I’ll dive into what DataOps means today, and how enterprises can adopt its practice to create […]
When it comes to creating and managing your dataflow pipelines, the graphical user interfaces of StreamSets Control Hub and StreamSets Data Collector put the complete power of our robust Data Operations Platform at your fingertips. There are times, however, when a more programmatic approach may be needed, and those times will be significantly more enjoyable […]
Today we are pleased to announce the general availability of StreamSets Control Hub version 3.2. StreamSets has built the industry’s only DataOps platform. We call it DataOps because our platform makes it easy to iteratively update dataflows when technology changes. Release 3.2 enhances manageability and repeatability between Control Hub instances which is critical when understanding […]
Selvaraaju (‘Selva’) Murugesan is Senior Manager for Innovation and Data Analytics in the Australian Capital Territory (ACT) Government. Selva focuses on data management practices and data analytics, using StreamSets Data Collector to extract data from different databases, perform data cleansing on the fly and push data to the ACT Government’s Open Data Portal. Over the past few months, Selva has […]
Many StreamSets customers use Splunk to mine insights from machine-generated data such as server logs, but one problem they encounter with the default tools is that they have no way to filter the data that they are forwarding. While Splunk is a great tool for searching and analyzing machine-generated data, particularly in cybersecurity use cases, it’s easy […]
Spark Streaming + Data Collector + Secure Kafka When we first introduced cluster streaming mode with Apache Spark Streaming 1.3 and Apache Kafka 0.8 several years ago, Kafka didn’t support security features such as TLS (transport encryption, authentication) and Kerberos (authentication). In Spark 2.1, an updated Kafka connector was introduced with support for these features […]
Detect, Secure and Govern Sensitive Data Upon Ingestion StreamSets is excited to announce a new product for protecting data in motion. StreamSets Data Protector, as the name would imply, extends the value of the StreamSets DataOps Platform to enable users to detect, secure and govern sensitive data as it flows around your business. Leveraging StreamSets’ […]
Join our mailing list to receive the latest news from StreamSets.
