==============
Advanced Usage
==============

This page highlights some of the more advanced functionality that users
might come across when developing tests with the StreamSets Test Framework (STF).

Fixtures
--------

STF includes implementations of a number of fixtures to enable their immediate
use in tests.

Environments
^^^^^^^^^^^^

The StreamSets Test Framework extensively uses `pytest fixtures`_ to simplify
access to objects needed within a test. As an example, most environments represented
by classes under :py:mod:`streamsets.testframework.environment` have corresponding
fixtures, which allows users to simply refer to them in the test function's
argument list and then use them within tests. This eliminates the need to import
the classes individually and also allows for their efficient reuse across tests.

.. _pytest fixtures: https://docs.pytest.org/en/latest/fixture.html

StreamSets Data Collector instances
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Fixtures are also used to represent instances of
:py:class:`streamsets.testframework.sdc.DataCollector` in the form of
:py:attr:`streamsets.testframework.conftest.sdc_builder` and
:py:attr:`streamsets.testframework.conftest.sdc_executor`. Along with the
benefits described above for environment classes, the use of these two fixtures
also allows regular functional tests to be run as upgrade tests. This behavior
is controlled by the invocation of ``stf test`` when the ``--sdc-version`` argument
is passed. If the string following this argument is a single SDC version,
the same Docker-based StreamSets Data Collector instance is used to build and execute
the pipeline; however, if two versions are passed (e.g. ``--sdc-version '2.7.2.0 > 3.4.0'``),
two SDC instances will be used with the first version given used to build pipelines
and the second used to execute the pipelines exported from the first.

:py:attr:`streamsets.testframework.conftest.sdc_common_hook`,
:py:attr:`streamsets.testframework.conftest.sdc_builder_hook`,
and :py:attr:`streamsets.testframework.conftest.sdc_executor_hook`
are three more fixtures whose reference implementation is a no-op. To use,
reimplement these fixtures in your test module, which will have the affect of acting on
the corresponding Data Collector instance fixture ahead of the
:py:meth:`streamsets.testframework.sdc.DataCollector.start` being invoked.
These are particularly useful for doing things like manipulating ``sdc.properties``,
which needs to be done before Data Collector is started if the changes are to take effect.
Also note that the common hook is executed before the builder/executor hook, allowing for a
composition of actions.

``configure_for_environment``
-----------------------------

Users familiar with the StreamSets SDK for Python will note that one of the
most visible differences between the SDK and STF is the addition of a number of
``configure_for_environment`` methods. For details about the functionality they implement,
please take a look at
:py:meth:`streamsets.testframework.sdc.DataCollector.configure_for_environment`,
:py:meth:`streamsets.testframework.sdc_models.Pipeline.configure_for_environment`,
and :py:meth:`streamsets.testframework.sdc_models.Stage.configure_for_environment`.