Data Collector User Guide

Home
Pipeline Configuration

Getting Started
What's New
Installation
Configuration
Upgrade
Pipeline Concepts and Design
Pipeline Configuration
- Data Collector UI - Edit Mode
- Pipeline Types and Icons in Documentation
  In Data Collector, you can configure pipelines that are run by Data Collector and pipelines that are run by Data Collector Edge.
- Retrying the Pipeline
- Rate Limit
- Advanced Options
  Pipelines and most pipeline stages include advanced options with default values that should work in most cases. By default, each pipeline and stage hides the advanced options. Advanced options can include individual properties or complete tabs.
- Simple and Bulk Edit Mode
- Runtime Values
  Runtime values are values that you define outside of the pipeline and use for stage and pipeline properties. You can change the values for each pipeline run without having to edit the pipeline.
- Event Generation
- Webhooks
- Notifications
- Pipeline Statistics
- SSL/TLS Encryption
  Many stages can use SSL/TLS encryption to securely connect to the external system.
- Security in Amazon Stages
- Security in Google Cloud Stages
- Security in Kafka Stages
- Kafka Message Keys
- Implicit and Explicit Validation
- Expression Configuration
- Configuring a Pipeline
Data Formats
Origins
Processors
Destinations
Executors
Dataflow Triggers
Solutions
StreamSets Control Hub
StreamSets Data Collector Edge
Multithreaded Pipelines
Microservice Pipelines
Orchestration Pipelines
SDC RPC Pipelines
Cluster Pipelines
Data Preview
Rules and Alerts
Pipeline Monitoring
Pipeline Maintenance
Administration
Tutorial
Troubleshooting
Glossary
Data Formats by Stage
Expression Language
Regular Expressions
Grok Patterns

Pipeline Configuration

Data Collector UI - Edit Mode
Pipeline Types and Icons in Documentation
In Data Collector, you can configure pipelines that are run by Data Collector and pipelines that are run by Data Collector Edge.
Retrying the Pipeline
Rate Limit
Advanced Options
Pipelines and most pipeline stages include advanced options with default values that should work in most cases. By default, each pipeline and stage hides the advanced options. Advanced options can include individual properties or complete tabs.
Simple and Bulk Edit Mode
Runtime Values
Runtime values are values that you define outside of the pipeline and use for stage and pipeline properties. You can change the values for each pipeline run without having to edit the pipeline.
Event Generation
Webhooks
Notifications
Pipeline Statistics
SSL/TLS Encryption
Many stages can use SSL/TLS encryption to securely connect to the external system.
Security in Amazon Stages
Security in Google Cloud Stages
Security in Kafka Stages
Kafka Message Keys
Implicit and Explicit Validation
Expression Configuration
Configuring a Pipeline

© Apache License, Version 2.0.