Ingesting JSON Data Into Apache Kudu with StreamSets Data Collector

Ingesting JSON Data Into Apache Kudu with StreamSets Data Collector

Kudu

At the Hadoop Summit in Dublin this week, Ted Malaska, Principal Solutions Architect at Cloudera, and I presented Ingest and Stream Processing – What Will You Choose?, looking at the big data streaming landscape with a focus on ingest. The session closed with a demo of StreamSets Data Collector, the open source graphical IDE for building ingest pipelines.

In the demo, I built a pipeline to read JSON data from Apache Kafka, augmented the data in JavaScript, and wrote the resulting records to both Apache Kudu (incubating) for analysis and Apache Kafka for visualization.

Here's a recording of the session:

https://www.youtube.com/watch?v=LTONR-L40Xg

The Apache Kudu destination is new in StreamSets Data Collector 1.3.0.0, released this week and available for download.

This post was originally published on the Kudu blog.

Related Resources

Check out StreamSets white papers, videos, webinars, report and more.

Visit the Resource Library

Related Blog Posts

Receive Updates

Receive Updates

Join our mailing list to receive the latest news from StreamSets.

You have Successfully Subscribed!

Pin It on Pinterest