Data passes through the pipeline in batches. This is how it works:
You can branch and merge streams in the pipeline.
You can drop records from the pipeline at each stage by defining required fields or preconditions for a record to enter a stage.
Field attributes are attributes that provide additional information about each field that you can use in pipeline logic, as needed.
Data Collector includes certain new features and stages with the Technology Preview designation. Technology Preview functionality is available for use in development and testing, but is not meant for use in production.
A test origin can provide test data for data preview to aid in pipeline development. In Control Hub, you can also use test origins when developing pipeline fragments. Test origins are not used when running a pipeline.