Register Transformer

To register Transformer with Control Hub, you generate an authentication token and modify the Transformer configuration files.

You can register Transformer from the Transformer UI or from the command line interface.

A registered Transformer communicates with Control Hub at regular intervals. If Transformer cannot connect to Control Hub, due to a network or system outage, then Transformer uses the Control Hub disconnected mode.

Before you register Transformer, ensure that you have enabled HTTPS for Transformer.

Registering from Transformer

When you register from the Transformer UI, Transformer generates the authentication token and modifies the configuration files for you. You can also create Control Hub user accounts and groups during the registration process.

  1. Log in to Transformer using the following default credentials: admin/admin.
  2. Click Administration > Enable Control Hub.
  3. Enter the following information in the Enable StreamSets Control Hub window:
    Property Description
    Control Hub Base URL URL to access Control Hub:
    • For Control Hub cloud, set to https://cloud.streamsets.com.
    • For Control Hub on-premises, set to the Control Hub URL provided by your system administrator. For example, https://<hostname>:18631.
    Control Hub User ID Enter your Control Hub user ID using the following format:
    <ID>@<organization ID>
    Control Hub User Password Enter the password for your Control Hub user account.
    Labels for this Transformer Assign a label to this Transformer. Labels that you assign here are defined in the Control Hub configuration file, $TRANSFORMER_CONF/dpm.properties. To remove these labels after you register the Transformer, you must modify the configuration file.

    Default is all, which you can use to run a job on any of the registered Transformers.

  4. Click Enable Control Hub.
  5. Optionally, you can choose to create a Control Hub user account and group for each Transformer user account and group.
    The Create Control Hub Groups and Users window maps all existing Transformer user accounts and groups to Control Hub user accounts and groups. You can remove users or groups and can edit the IDs, names, and email addresses as needed.

    When you have finished reviewing the users and groups to create, click Create. Each new user is assigned a default set of Control Hub roles. Groups are not assigned any roles. After you log in to Control Hub, change those role assignments as needed to secure the integrity of your organization and data.

  6. Restart Transformer.

Registering from the Command Line Interface

When you register Transformer from the command line interface, Transformer generates the authentication token and modifies the configuration files for you. Transformer must be running before you can use the command line interface.

Start Transformer, and then use the system enableDPM command to register the Transformer.

Use the command from the $TRANSFORMER_DIST directory as follows:

bin/streamsets cli \
(-U <sdcURL> | --url <sdcURL>) \
[(-D <dpmURL> | --dpmURL <dpmURL>)] \
[(-a <sdcAuthType> | --auth-type <sdcAuthType>)] \
[(-u <sdcUser> | --user <sdcUser>)] \
[(-p <sdcPassword> | --password <sdcPassword>)] \
system enableDPM \
(--dpmUrl <dpmBaseURL>) \
(--dpmUser <dpmUserID>) \
(--dpmPassword <dpmUserPassword>) \
[(--labels <labels>)]

When using the system enableDPM command, the following basic options are required:

Basic Option Description
-U <sdcURL>

or

--url <sdcURL>
Required. URL of the Transformer.

The default URL is http://localhost:19630.

-D <dpmURL>

or

--dpmURL <dpmURL>

Required. URL to access Control Hub:
  • For Control Hub cloud, set to https://cloud.streamsets.com.
  • For Control Hub on-premises, set to the Control Hub URL provided by your system administrator. For example, https://<hostname>:18631.

The following table describes the enableDPM options:

Enable DPM Option Description
--dpmUrl <dpmBaseURL> URL to access Control Hub:
  • For Control Hub cloud, set to https://cloud.streamsets.com.
  • For Control Hub on-premises, set to the Control Hub URL provided by your system administrator. For example, https://<hostname>:18631.
--dpmUser <dpmUserID> Required. Enter your Control Hub user ID using the following format:
<ID>@<organization ID>
--dpmPassword <dpmUserPassword> Required. Enter the password for your Control Hub user account.
--labels <labels> Required. Assign a label to this Transformer. You can enter multiple labels separated by commas. Labels that you assign here are defined in the Control Hub configuration file, $TRANSFORMER_CONF/dpm.properties. To remove these labels after you register the Transformer, you must modify the configuration file.
For example, the following command registers a Transformer with Control Hub and assigns three labels to the Transformer:
bin/streamsets cli -U http://localhost:19630 -D https://cloud.streamsets.com system enableDPM --dpmUrl https://cloud.streamsets.com --dpmUser alison@MyOrg --dpmPassword MyPassword --labels Finance,Accounting,Development

Restart Transformer to apply the changes.

Using a Proxy Server

You can configure each registered Transformer to use an authenticated HTTP or HTTPS proxy server for outbound requests made to Control Hub. Define the proxy properties in the TRANSFORMER_JAVA_OPTS environment variable.

Modify environment variables using the method required by your installation type.

Add the following Java options to the TRANSFORMER_JAVA_OPTS environment variable:

  • https.proxyUser
  • https.proxyPassword
  • https.proxyHost
  • https.proxyPort

If the proxy server uses HTTP instead of HTTPS, use http.<property name> for each property.

For example, to configure a registered Transformer to use an HTTPS proxy server on host 138.0.0.1 and port 3138, define TRANSFORMER_JAVA_OPTS as follows:

export TRANSFORMER_JAVA_OPTS="${TRANSFORMER_JAVA_OPTS} -Xmx1024m -Xms1024m -Dhttps.proxyUser=MyName -Dhttps.proxyPassword=MyPsswrd -Dhttps.proxyHost=138.0.0.1 -Dhttps.proxyPort=3138 -server" 
Note: Oracle JDK disabled HTTP proxy authentication for HTTPS URLs in JDK 8 update 111. If Transformer runs on a machine with Java 8u111 or later, consider using an HTTPS proxy server. Or as a workaround, consider adding the following Java property to the TRANSFORMER_JAVA_OPTS environment variable, setting the property to an empty string:
-Djdk.http.auth.tunneling.disabledSchemes=''

However, use this workaround with caution since it exposes credentials by sending them through an unencrypted proxy.