Commands

The stf script installed as part of this project makes available a number of commands. For full details, the usage page is the definitive resource:

$ stf -h
usage: stf [-h] [--version] [-v] [--docker-image image]
           [--docker-image-dont-pull] [--docker-network network]
           [--sdc-resources-directory dir]
           [--testframework-config-directory dir]
           [--streamsets-sdk-directory dir] [--testframework-directory dir]
           {test,shell,build,start} ...

positional arguments:
  {test,shell,build,start}
                        Test Framework subcommands
    test                Run STF tests
    shell               Open a shell within the STF container
    build               Build STF Docker images
    start               Start a sub-system

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -v, --verbose         Be noisier (default: False)
  --docker-image image  Docker image to use for the STF container (default:
                        streamsets/testframework:master)
  --docker-image-dont-pull
                        Don't pull STF Docker image (default: False)
  --docker-network network
                        Docker network to which to attach the STF container
                        (default: cluster)
  --sdc-resources-directory dir
                        A directory containing resources to mount into the SDC
                        container (default: None)
  --testframework-config-directory dir
                        A directory containing STF configuration files to
                        mount into the STF container (default:
                        /Users/dima/.streamsets/testframework)

advanced arguments:
  Stuff for StreamSets devs, mostly

  --streamsets-sdk-directory dir
                        Path to the an `sdk` folder containing source code of
                        the StreamSets SDK for Python (default: None)
  --testframework-directory dir
                        A testframework directory to mount into the STF
                        container (for use when making STF changes that don't
                        really a rebuild of the image) (default: None)

Run 'stf <command> -h' for more information on a specific subcommand.

stf test

Tests are executed by cd-ing into a directory containing the test source and then running the stf test command. Any arguments after test are passed into the Test Framework container as arguments to an invocation of the pytest command.

stf shell

To get an interactive shell within the Test Framework environment, run stf shell. This command is particularly useful during the test development process, where a user may want to explore the streamsets.testframework package from an interpreter.

stf build

StreamSets publishes Docker images of StreamSets Data Collector and Data Collector stage libraries to Docker Hub. There are a number of images, however, which we are not allowed to distribute because of the terms of the underlying binaries’ EULA. stf build is the command to use to create these images locally.

stf start

The stf start command leverages the Docker orchestration capabilities of the StreamSets Test Framework to start StreamSets Data Collector. As an example, to start version 3.4.0 of StreamSets Data Collector with the AWS and JDBC stage libraries preinstalled, you could run

$ stf start sdc --version 3.4.0 --stage-lib aws jdbc
2018-09-19 07:33:41 AM [INFO] [streamsets.testframework.sdc] Pulling Docker image streamsets/testframework:master ...
2018-09-19 07:33:47 AM [INFO] [streamsets.testframework.sdc] Starting StreamSets Data Collector 3.4.0 container ...
2018-09-19 07:34:05 AM [INFO] [streamsets.testframework.sdc] SDC is now running. SDC container (3f3b1f951560:18630) can be followed along on http://localhost:32913