Installation ============ Prerequisites ------------- Prior to installing the StreamSets Test Framework, a number of prerequisites must be satisfied: Python ^^^^^^ The StreamSets Test Framework has been tested against Python 3.6 and this is the version we recommend be used when invoking ``stf`` commands. If your machine has a different version of Python 3+ installed, feel free to try it out, but if things seem broken, we recommend using `pyenv`_ to quickly and non-destructively switch to a different version of Python. .. _pyenv: https://github.com/pyenv/pyenv Docker ^^^^^^ Any recent version of Docker should work with the StreamSets Test Framework. .. note:: If using Docker for Mac, the StreamSets Test Framework requires that you go into your Docker Preferences and *deselect* the "Securely store Docker logins in macOS keychain" if you want to pull any Docker images stored on private repositories. This is needed in order to allow credentials entered via ``docker login`` to be passed into the STF container at runtime. Installation from PyPI ---------------------- To install the most recent stable release of the project, use your Python 3 installation's instance of `pip`_: .. _pip: https://pip.pypa.io .. code-block:: console $ pip3 install streamsets-testframework To do a simple check to verify that the installation was successful, simply run .. code-block:: console $ stf --version stf 4.0.0 Docker images ------------- The ``stf`` command launches most of its sub-commands in a Docker container started from the ``streamsets/testframework-4.x:latest`` `image on Docker Hub`_. .. _image on Docker Hub: https://hub.docker.com/r/streamsets/testframework-4.x/tags/ Many STF tests of StreamSets Data Collector support the automatic starting and stopping of SDC containers from Docker images. These tests will also handle orchestration of stage libraries, external libraries and environment libraries, as required by the tests, which are also packaged as Docker images. As part of our automated build process, StreamSets creates and pushes to Docker Hub images of `StreamSets Data Collector`_ and `its stage libraries`_ for stable releases and nightly builds. Because of license restrictions, though, there are a number of Docker images required to run certain STF tests that we do not distribute. These include images containing the JDBC drivers as part of `external libraries`_ and images containing libraries to interact with an environment. .. _StreamSets Data Collector: https://hub.docker.com/r/streamsets/datacollector/tags/ .. _its stage libraries: https://hub.docker.com/r/streamsets/datacollector-libs/tags/ .. _external libraries: https://streamsets.com/documentation/datacollector/latest/help/datacollector/UserGuide/Configuration/ExternalLibs.html?hl=drivers To build external libraries, simply run the ``stf build extras`` command: .. raw:: html

    $ stf build extras
    2018-09-19 07:33:41 AM [INFO] [streamsets.testframework.sdc] 2018-09-21 10:23:19 AM [INFO] [streamsets.testframework.cli.build] This will build images for version 3.4.0 ...
    ...
    
Similarly, to build environment libraries, run ``stf build environments``: .. raw:: html

    $ stf build environments
    2018-09-19 07:33:41 AM [INFO] [streamsets.testframework.sdc] 2018-09-21 10:23:19 AM [INFO] [streamsets.testframework.cli.build] This will build images for version 3.4.0 ...
    ...