Catch Unprotected Records

A protection policy can catch unprotected records and write them to a security violation destination.

When catching unprotected records, any records with classified fields that are not protected by the policy are passed to the security violation destination. You can then review the unprotected records to determine if you need to update the protection policy.

Classified fields might be deliberately unprotected, because you want to make the field data available for use. Or, classified fields might be unprotected by accident, because the procedures in the policy do not adequately protect the data.

Note: Since data can be accidentally unprotected, use a destination system with the appropriate levels of security to catch unprotected records.
Catching unprotected records can be useful both during policy development and after putting a policy into production:
During development
Catching unprotected records during policy development makes it easier to determine if the problem is with the classification of records or the protection of records.

When reviewing data written to a security violation destination, you know that the data was classified but has a problem with protection. You can review these records and update the policy to ensure that the problems do not recur.

Then, you can review the data in the configured destination system for sensitive data that evaded detection. These records typically have a problem with classification, so you can update classification rules, then update related policies as needed.

Note: You might not want to catch unprotected records when developing and testing classification rules. When you use data preview to review how rules classify data, you want all records to pass to data preview, instead of routing classified unprotected records to a security violation destination.
In production
Catching unprotected records enables updating the rules and policies as needed. It also prevents sensitive data from being written to destinations where the wrong users can access it.

When you enable catching unprotected records, you define the destination to use and related destination properties. You also specify the classification score threshold, and any field paths or classification categories to ignore.

Tip: When classifying records, Data Protector uses all StreamSets and custom classification rules. When Data Protector catches unprotected records, you should review the records for potential updates to the policy and related classification rules, as well as updates to the Catch Unprotected Records properties.


Suppose you are configuring a read policy for your Human Resources team that catches unprotected records and uses a Local File System security violation destination. By default, all records with classified fields that are not protected are written to that destination.

However, you don't want this read policy to protect the company ID, names, or salary classification categories because you want to grant the HR team access to this data. To enable access, you add those categories to the list of classification categories to ignore. If you are also not concerned with any classifications with less than a .5 score, then you can set the classification score threshold to .5.

As a result, any unprotected fields classified with the specified categories or with low classification scores are not passed to the security violation destination, and remain visible to the team that uses this read policy to run jobs.