Protection Policies Overview

Protection policies alter and protect data. When you configure a job, you specify a read policy to protect data when reading from an origin system. You also specify a write policy to protect data when writing to destination systems.

When you do not specify a read or write policy, default policies are used to ensure the protection of sensitive data. You can designate one default read policy and one default write policy for the organization.

Protection policies use classification rules to identify the data to be altered and protected. Protection policies can protect data based on both StreamSets classification rules and custom classification rules. When necessary, policies can also protect unclassified data in known field paths.

You can configure a policy to route records with classified but unprotected fields to a security violation destination. Classified unprotected fields might exist in records because the policy deliberately leaves them visible for use, or they might also exist because a policy fails to properly protect the classified data.

When you write records with classified unprotected fields to a security violation destination, you can then review the problematic records and update protection policies to address outstanding issues. This helps to ensure that unprotected sensitive data does not become accessible to unauthorized users.

Creating a protection policy requires configuring the policy enactment type, then defining the procedures for the policy to perform. Procedures specify the sensitive data to protect, and how to protect that data.

For example, you have sensitive data that includes company IDs, social security numbers, and salaries, and you want that data obfuscated when written to less-secure systems. To do this, you create a write policy to be used for jobs that write to those systems. In the write policy, you configure procedures to alter and protect all sensitive data, including one for company IDs, one for social security numbers, and one for salaries. Those procedures are based on custom classification rules that identify company IDs and salaries, and the StreamSets classification rule for social security numbers.

It's important to test all policies and classification rules to verify that they work as expected before using them in production jobs.

For more information about how protection policies function as part of Data Protector, see Data Protector in Control Hub.

Working with Protection Policies

The Protection Policies view displays a list of all protection policies in the organization.

You can complete the following tasks in the Protection Policies view:

  • Create protection policies - Create new policies to alter and protect sensitive data.
  • View protection policy details - Displays all policy details such as the sampling type and security violation destination. Click the policy name to view policy details.
  • Edit protection policies - Edit policies to update policy configuration. To edit a policy, click the policy name to view policy details, then click Edit.
  • Set default protection policies - The organization uses one default read and default write policy. To set a policy as the default, click the policy name to view policy details, then click Set Default.
  • Delete protection policies - Delete policies that are no longer needed. To delete a policy, click the policy name to view policy details, then click Delete.
    Note: The policy and its procedures are immediately deleted and cannot be recovered.
  • View procedures associated with the policy - View procedures to create, edit, and delete procedures associated with the policy.

The following image shows a list of policies in the Protection Policies view. Each policy is listed with the policy name, enactment and sampling types, and the last modified time. When you click the policy name, policy details display additional options:

To work with procedures associated with the policy, click View Procedures.

Working with Procedures

Create, edit, and delete procedures for a protection policy from within the policy. You can work with the procedures for a policy after you create the policy.

To work with procedures, from the Protection Policies view click the name of a policy, then click View Procedures. This displays the Procedures view with the Procedures list. From the Procedures list, you can perform the following tasks:
  • Configure a procedure - Create or edit procedures as needed.
  • Delete a procedure - Delete a procedure when it is no longer needed. To delete a procedure, click the procedure name in the Procedures list, then click Delete.
    Note: The procedure is immediately deleted and cannot be recovered.

The following image shows the Procedures view with several procedures and the detail view of one of the procedures: