Data Formats by Stage

Data Format Support

This appendix lists the data formats supported by Data Collector origins and destinations.

Origins

The following table lists the data formats supported by each origin.

Origin Avro Binary Datagram Delimited JSON Log Protobuf SDC Record Text Whole File XML
Amazon S3    
Amazon SQS Consumer    
Azure IoT/Event Hub Consumer              
CoAP Server    
Directory    
Elasticsearch * * * Not Applicable * * *
File Tail                
Google BigQuery * * * Not Applicable * * *
Google Cloud Storage        
Google Pub/Sub Subscriber      
Hadoop FS                
HTTP Client        
HTTP Server      
HTTP to Kafka * * * Not Applicable * * *
JDBC Multitable Consumer * * * Not Applicable * * *
JDBC Query Consumer * * * Not Applicable * * *
JMS Consumer      
Kafka Consumer  
Kafka Multitopic Consumer  
Kinesis Consumer    
MapR DB CDC * * * Not Applicable * * *
MapR DB JSON * * * Not Applicable * * *
MapR FS                
MapR Multitopic Streams Consumer    
MapR Streams Consumer      
MongoDB * * * Not Applicable * * *
MongoDB Oplog * * * Not Applicable * * *
MQTT Subscriber    
MySQL Binary Log * * * Not Applicable * * *
Omniture * * * Not Applicable * * *
OPC UA Client * * * Not Applicable * * *
Oracle CDC Client * * * Not Applicable * * *
RabbitMQ Consumer    
Redis Consumer    
Salesforce * * * Not Applicable * * *
SDC RPC                    
SDC RPC to Kafka                    
SFTP/FTP Client    
SQL Server CDC Client * * * Not Applicable * * *
SQL Server Change Tracking * * * Not Applicable * * *
TCP Server    
UDP Multithreaded Source * * * Not Applicable * * *
UDP Source * * * Not Applicable * * *
UDP to Kafka * * * Not Applicable * * *
WebSocket Client    
WebSocket Server    
Windows Event Log * * * Not Applicable * * *

Destinations

The following table lists the data formats supported by each destination.

Destination Avro Binary Delimited JSON Protobuf SDC Record Text Whole File XML
Aerospike * * * Not Applicable * * *
Amazon S3  
Azure Data Lake Store    
Azure Event Hub Producer          
Azure IoT Hub Producer          
Cassandra * * * Not Applicable * * *
CoAP Client          
Elasticsearch * * * Not Applicable * * *
Einstein Analytics * * * Not Applicable * * *
Flume    
Google BigQuery * * * Not Applicable * * *
Google Bigtable * * * Not Applicable * * *
Google Cloud Storage    
Google Pub/Sub Publisher  
Hadoop FS      
HBase * * * Not Applicable * * *
Hive Metastore                
Hive Streaming * * * Not Applicable * * *
HTTP Client    
InfluxDB * * * Not Applicable * * *
JDBC Producer * * * Not Applicable * * *
JMS Producer  
Kafka Producer  
Kinesis Firehose              
Kinesis Producer    
KineticaDB * * * Not Applicable * * *
Kudu * * * Not Applicable * * *
Local FS      
MapR DB * * * Not Applicable * * *
MapR DB JSON * * * Not Applicable * * *
MapR FS      
MapR Streams Producer    
MongoDB * * * Not Applicable * * *
MQTT Publisher          
Named Pipe          
RabbitMQ Producer    
Redis    
Salesforce * * * Not Applicable * * *
SDC RPC                
Solr * * * Not Applicable * * *
To Error * * * Not Applicable * * *
Trash * * * Not Applicable * * *
WebSocket Client