Data Formats Overview

Data formats - such as Avro, JSON, and log - are methods to encode data that adhere to generally accepted specifications.

The way that stages process data can be similar based on the stage type and the type of data being processed. For example, file-based origins such as Directory and SFTP/FTP Client will typically process data formats the same way. Similarly, message-based destinations such as Kafka Producer and JMS Producer generally process data formats the same way.

The documentation for each stage has a "Data Formats" section of documentation that contains processing details.

This chapter includes the details that are too complex to summarize in the Data Formats sections. If you have browsed and discovered this chapter and want to read on, feel free. But for stage-related details, see the stage documentation.

For information on the different data formats supported by each origin and destination, see Data Format Support.