Design in Data Collector

In addition to designing pipelines in the Control Hub Pipeline Designer, you can also log into an authoring Data Collector to design pipelines and then add them to the Control Hub pipeline repository. You cannot design pipeline fragments in Data Collector.

You can add pipelines in the following ways:

Publish pipelines from registered Data Collectors
You can publish pipelines from Data Collectors that are registered with Control Hub.
Import pipelines from unregistered Data Collectors
You can import pipelines from Data Collectors that are not registered with Control Hub. Export the pipelines from the unregistered Data Collectors using the Export for Control Hub option, and then import the pipelines into Control Hub.
When you export a pipeline for use in Control Hub, Data Collector generates a JSON file named after the pipeline, as follows: <pipeline name>.json. The generated JSON file includes the definition of each stage library used in the pipeline.
Note: You can also export and import pipelines from registered Data Collectors. However, when a Data Collector is registered with Control Hub, it's simplest to publish the pipelines directly to Control Hub.

Publishing Pipelines from Data Collector

After you finish developing pipelines in a registered Data Collector, you publish the pipelines to the Control Hub pipeline repository. You can publish pipelines that are valid.

  1. Log in to the registered Data Collector.
  2. From the Home page, select pipelines in the list and then click the Publish Pipeline icon: . Or to publish a pipeline from the pipeline canvas, click the Control Hub Options icon (), and then click Publish Pipeline.
    The Publish Pipeline dialog box appears.
  3. Enter a commit message.
    As a best practice, state what changed in this pipeline version so that you can track the commit history of the pipeline.
    Note: If you are publishing multiple pipelines from the Home page, the same commit message is used for all of the pipelines.
  4. Click Publish Pipeline.
    When you log in to Control Hub, the Pipelines view lists the published pipelines.