StreamSets Data Integration Blog
Where change is welcome.
Announcing Data Collector ver 1.3.0.0
With this release we have a number of exciting new features and integrations. And as usual, we've addressed a number of bug fixes. Integrations: Want to send data to Amazon Redshift? Use the new Kinesis Firehose destination to do it. If you deal with a lot of unstructured data, here's a MongoDB destination you can use. Testing Kudu within your Hadoop environment?…
Data in Motion: Simplifying Security & Building Custom Integrations
At the Strata+Hadoop World conference last week, I met with Pratik Verma, Chief Product Officer at BlueTalon, a Bay Area startup focused on big data security. As Pratik and I were talking, he explained some of the problems that arise when organizations collect more and more data, and they need to start thinking about exactly who should have access to that data. There's a…
Integrating StreamSets with Salesforce Wave Analytics
UPDATE - Salesforce origin and destination stages, as well as a destination for Salesforce Wave Analytics, were released in StreamSets Data Collector 2.2.0.0. Use the supported, shipping Salesforce stages rather than the unsupported code mentioned below! In my last blog entry I explained how you can write custom destinations to send data to systems not currently supported by StreamSets Data…
New Tutorial: Creating a Custom StreamSets Destination
One of the first things I hear after I explain the basics of StreamSets Data Collector is, "Cool, so can I ingest data from/send data to X?", for varying values of X. The short answer is, "Yes, you can!", while the longer answer involves checking the lists of origins (for ingesting data from X) and destinations (for writing data) included with the product,…
How Trend Micro Uses StreamSets – An Interview with the Threat Research Team
The Forward-Looking Threat Research team at Trend Micro were early adopters of StreamSets Data Collector. They use StreamSets to ingest data from a wide variety of sources to create a Threat Assessment Dashboard in Elasticsearch. In this interview, we talk with members of their team about how they evaluated StreamSets and implemented it in their production environment in a short…