Adam Kunicki

Adam

Product Manager @ StreamSets

Kafka + TLS/Kerberos in Cluster Streaming Mode is coming!

Spark Streaming + Data Collector + Secure Kafka

When we first introduced cluster streaming mode with Apache Spark Streaming 1.3 and Apache Kafka 0.8 several years ago, Kafka didn’t support security features such as TLS (transport encryption, authentication) and Kerberos (authentication). In Spark 2.1, an updated Kafka connector was introduced with support for these features when used with Kafka 0.10 or newer.

Adam KunickiKafka + TLS/Kerberos in Cluster Streaming Mode is coming!
Read More

StreamSets Monitoring with Grafana, InfluxDB, and jmxtrans

The ability to monitor your critical infrastructure is a must, and we designed the StreamSets Data Collector (SDC) with this in mind: metrics are exposed through both the REST API and JMX. While there are many approaches to monitoring these metrics, let’s walk through a specific end-to-end example using jmxtrans to collect metrics, InfluxDB to store them, and Grafana to visualize them.

Adam KunickiStreamSets Monitoring with Grafana, InfluxDB, and jmxtrans
Read More