Products
StreamSets Platform
Power your modern analytics and digital transformation with continuous data.

Data Collector
Transformer for Spark
Transformer for Snowflake
Mainframe Collector
Control Hub
Connectors
Super iPaaS
DEMO
Spend Less Time Fixing and More Time Doing

Talk to an expert and eliminate data integration friction.

Request a Demo
Solutions
StreamSets Solutions
Powerful data engineering solutions for modern data integration across multiple cloud platforms.

Agile Reporting
Cloud Data Lake Integration
Cloud Data Warehouse Integration
Mainframe Data Modernization
Power Real-time Applications
Talk With StreamSets
Contact Us

Learn more about how StreamSets can help your organization harness the power of data.

Get in Touch
Partners
StreamSets Partners
Use data in more ways with a modern approach to data integration.

Amazon Web Services
Databricks
Google Cloud Platform
Hewlett Packard Enterprise
Microsoft Azure
Snowflake
Resources
Resources
Best practices and technical how-tos for modern data integration.

Getting Started
The Data Integration Blog
Webinars
Learn Data Management
Case Studies
Events
Community
WHITE PAPER
Data Integration Advantage

Building a Foundation for Scalable AI. See the state of AI in the enterprise.

Download Now
About Us
About Us
Modernizing data integration for continuous data under constant change.

Careers
Leadership
News
Software AG
Start Free Trial
Search

StreamSets Data Integration Blog

Where change is welcome.

Synchronize HDFS Data into S3 Using the Hadoop FS Standalone Origin

Cloud Data Migration

By Ji Sun Kim July 10, 2018

Introduction: from HDFS Data to S3

I am very excited to announce the new Hadoop FS Standalone origin in StreamSets Data Collector 3.2.0.0. Data Collector has long supported the Hadoop FS origin, but only in the cluster mode. The Hadoop FS (HDFS) Standalone origin does not need MapReduce or YARN installed and can run in multithreaded mode, with each thread reading one file at a time in parallel.