StreamSets Data Collector Edge™

Ultralight data ingestion at the edge.

The StreamSets Data Collector Edge (SDC Edge) enables at-scale data ingestion and analytics for edge systems. An ultralight, small-footprint agent, it is an ideal solution for use cases like Internet of Things (IoT) or cybersecurity applications that collect data from resource-constrained sensors and personal devices.

Edge Data Ingestion Challenges

  • Limited Memory, CPU and Connectivity


    Traditional tools are too resource-intensive and overwhelm edge devices.

  • Unexpected Changes (Data Drift) Draws Out Development Cycles

    Hard-wired pipelines are brittle, causing many iterations and failures when data changes.

  • Difficult to Provision and Operate at Scale

    Live management of thousands of edge pipelines is a Herculean effort, and hinders gaining data-driven insights.

Ultralight Agent Runs Anywhere

Small footprint, low resource requirements.

  • Less than 5MB footprint.

  • Utilizes little CPU resources (1-2% estimated).

  • Apache licensed open source binary.


One Solution Across Platforms and Protocols

No need for device-specific custom code.

  • Built in Go—portable across Android, Windows, Linux, iOS and more.

  • Supports leading messaging protocols including HTTP, MQTT, CoAP, and WebSockets.

Rich Functions in a Tiny Package

Not just a “data forwarder”.

  • Filter, route and execute expressions.

  • Detect and handle data drift.

  • At-least once and at-most once guarantees.

  • Multiple pipelines per agent.

  • Bidirectional support.

  • Architected for advanced analytics and ML.

hive and streamsets
hive and streamsets


Kafka and Streamsets
Kafka and Streamsets

jdbc with streamsets
jdbc with streamsets

httpclient with streamsets
httpclient with streamsets


Spark Logo
Spark Logo

Part of the StreamSets Data Operations Platform

Lifecycle management at scale.

  • Build pipelines on-prem or in the cloud.

  • Over 100 “drag-and-drop” connectors and processors.

  • Automated deployment.

  • Continuous performance management.

Common Use Cases

Internet of Things (IoT)

Reliably ingest and apply machine learning and other analytic techniques to data aggregated from huge populations IoT sensors and devices.


Ingest and apply advanced analytics to the vast quantities of data collected across a corporate network in order to detect imminent threats or attacks in progress.

Get started today

StreamSets runs on Linux or Mac OS X

Download Open Source
melissaStreamSets Data Collector Edge