Field Zip

The Field Zip processor merges list data from two fields into a single field.

You can use the Field Zip processor to merge two list fields or list_map fields. Use the processor to merge related lists of data.

The Field Zip processor merges list data with a corresponding list. If either list includes more items than the other, the extra list items are not merged.

When you configure the Field Zip processor, you specify the two fields to merge and the new or existing field to write the merged data. When writing to a new field, the processor retains all of the original list fields. When writing to an existing field, the processor overwrites the data in the existing field with the merged data. To pass only the merged data to the next stage, configure the processor to write to the existing root field, /. This overwrites the complete record with the merged data.

By default, the processor merges the field names and values from the lists. You can configure the processor to merge only the values.

If one or both fields in a record do not match the specified fields to merge, you can configure the Field Zip processor to pass the record as is without merging any fields, or pass the record to the pipeline for error handling.

Merging List Data

When merging data in list fields, the Field Zip processor merges each item in a list with the corresponding item in the other list. The processor merges the lists using the order of the lists, and returns a list of map fields.

Any item in a list without a matching item in the other list is not merged.

For example, say you have a record with two list fields:
order itemID cost
23523482
  • 2
  • 113
  • 954
  • 6502
  • 89.95
  • 8.95
  • 6.95

You configure the Field Zip processor as follows to merge the itemID and cost fields and write the data to the existing root field, /, which overwrites all incoming data with the merged data:

When you preview the pipeline and select the Field Zip processor, preview shows the following input and output data for the processor:

The output includes a single list of map fields. Each map has the field name and value in the itemID list merged with the corresponding field name and value from the cost list. Notice that the /itemID/3 input field is not included in the merged list because it has no corresponding cost information. Because the processor is configured to write to the existing root field, the processor does not retain any of the original list fields.

To configure the Field Zip processor to merge only the values, select Zip Values Only as follows:

When you preview the pipeline, the processor returns the same data without the original field names, as follows:

Merging List_Map Data

As with list fields, when merging data in list_map fields, the Field Zip processor merges each item in a list with the corresponding item in the other list. The processor merges the lists using the order of the lists, and returns a list of map fields.

The difference is, the lists in the map fields include different pairs of information, rather than the matching pairs of information from list fields.

For example, say you have a record with two list_map fields, each with a single map in the list:
basics additional
0
  • id: 23
  • color: 20005
  • info: <null>
0
  • inventory: 80
  • cost: $5.98

You configure the Field Zip processor as follows to merge the basics and additional fields and write the data to a new all field:

When you preview the pipeline, the Field Zip processor generates the following record:

Notice how each map in the merged all field contains a different set of fields. When processing list fields, the Field Zip processor returns the matching sets of fields. Also notice that the /basics/info input field is not included because it has no corresponding item in the other list_map field. Because the processor is configured to write to a new field, the processor retains the original basics and additional fields and writes the merged data to the new field.

To configure the Field Zip processor to merge only the values, select Zip Values Only as follows:

When you preview the pipeline, the processor returns the same data without the original field names, as follows:

Pivoting Merged Lists

Depending on your use case, you might want to add a Field Pivoter processor to your pipeline after merging list data with the Field Zip processor.

The Field Pivoter processor can pivot merged list data and create a record for each first-level item in the list.

For example, say you use the Field Pivoter processor to process the data generated in the first List example, above, with field names included in the merged list. You configure the Field Pivoter processor to write the pivoted data to a new field called Purchased_Item.

The pivot of the single record results in the following three records from the merged list:
Order ItemID Cost Purchase Purchased_Item
23523482
  • 2
  • 113
  • 954
  • 6502
  • 89.95
  • 8.95
  • 6.95
      • itemID: 2
      • cost: 89.95
      • itemID: 113
      • cost: 8.95
      • itemID: 954
      • cost: 6.95
    • itemID: 2
    • cost: 89.95
23523482
  • 2
  • 113
  • 954
  • 6502
  • 89.95
  • 8.95
  • 6.95
      • itemID: 2
      • cost: 89.95
      • itemID: 113
      • cost: 8.95
      • itemID: 954
      • cost: 6.95
    • itemID: 113
    • cost: 8.95
23523482
  • 2
  • 113
  • 954
  • 6502
  • 89.95
  • 8.95
  • 6.95
      • itemID: 2
      • cost: 89.95
      • itemID: 113
      • cost: 8.95
      • itemID: 954
      • cost: 6.95
    • itemID: 954
    • cost: 6.95

Configuring a Field Zip Processor

Use a Field Zip processor to merge two corresponding list fields in a record.

  1. In the properties pane, on the General tab, configure the following properties:
    General Property Description
    Name Stage name.
    Description Optional description.
    Required Fields Fields that must include data for the record to be passed into the stage.
    Tip: You might include fields that the stage uses.

    Records that do not include all required fields are processed based on the error handling configured for the pipeline.

    Preconditions Conditions that must evaluate to TRUE to allow a record to enter the stage for processing. Click Add to create additional preconditions.

    Records that do not meet all preconditions are processed based on the error handling configured for the stage.

    On Record Error Error record handling for the stage:
    • Discard - Discards the record.
    • Send to Error - Sends the record to the pipeline for error handling.
    • Stop Pipeline - Stops the pipeline.
  2. On the Zip tab, configure the following properties:
    Field Zip Property Description
    Fields to Zip Configure the fields to merge and the output field as follows:
    • First Field Path - Path to the first list or list_map field to use.
    • Second Field Path - Path to the second list or list_map field to use.
    • Zipped Field Path - Path to the new or existing field where the processor writes the merged list data.

      When writing to a new field, the processor retains all of the original list fields. When writing to an existing field, the processor overwrites the data in the existing field with the merged data.

    Zip Values Only Includes only field values when merging lists. When not selected, the processor includes field names and values.

    For more information about how the processor merges list data, see Merging List Data.

    Field Does Not Exist
    Action to take when one or both list fields do not exist in a record:
    • Include without Processing - Passes the record as is without creating the merged field.
    • Send to Error - Passes the record to the pipeline for error handling.