Field Attributes

Field attributes are attributes that provide additional information about each field that you can use in pipeline logic, as needed.

Some stages generate field attributes. For example, the Salesforce origin includes the original Salesforce data type in the salesforce.salesforceType attribute for every field.

You can create, modify, and evaluate field attributes in the pipeline. The Expression Evaluator processor enables creating and modifying field-level attributes. You can use field attribute functions to evaluate field attributes.

For example, you can use an Expression Evaluator processor to create a field attribute based on record data, then pass the record to a Stream Selector processor that routes data based on the value of the attribute.

Field attributes are automatically included in records written to destination systems only when you use the SDC RPC data format in the destination.

To include field attributes in the record data or to use field attributes in calculations, use the record:fieldAttribute and record:fieldAttributeOrDefault functions. For more information about field attribute functions, see Record Functions.

When using preview, you can enable viewing field attributes and record header attributes to aid with pipeline development.

Working with Field Attributes

Like record header attributes, you can use the Expression Evaluator processor to create or update field attributes. For example, when processing Avro data, origins place the precision and scale information for Decimal data into field attributes. If you want to increase those values before writing them to the destination system, you can use the Expression Evaluator processor to set the attribute value.

Field attributes are string values. You can also use record:fieldAttribute functions in any expression to include attribute values in calculations.

Important: Field attributes do not have field paths. When using an attribute in an expression, use just the attribute name surrounded by quotation marks after specifying the path to the associated field, as follows:
 ${record:fieldAttribute(<path to the field>,'<attribute name>')}

For example, all Decimal fields include automatically generated precision and scale field attributes. After reading data from the origin, you want to create a new Price Decimal field based on the existing cost data. You add an Expression Evaluator processor to create the Price field and define the precision and scale field attributes based on the existing Cost precision and scale.

The following image shows how to define the field attribute expressions for the Price field:

Field Attribute-Generating Stages

The following table lists the stages that generate field attributes to enable special processing:

Stage Description
Origins that process Avro data Include "precision" and "scale" field attributes for every Decimal field.
Origins that process XML data Can include field XPath information, XML attribute, and namespace declarations in field attributes.
Google BigQuery origin Includes original precision for Datetime, Time, and Timestamp fields in a field attribute.
Salesforce origin Includes data type information in field attributes.
Salesforce Lookup processor Includes data type information in field attributes.

Viewing Field Attributes in Preview

As with record header attributes, you can use preview to view field attributes at any point in the pipeline. To view field attributes, enable the Show Record/Field Header preview property.