Sample Apache Spark Design Patterns
Jumpstart your pipeline design with sample design patterns and sample data
Choose a Spark Design Pattern for Your Data Pipeline
It has never been easier to unlock the power of fast ETL, machine learning and streaming analytics with Apache Spark. StreamSets Transformer is a modern ETL pipelines engine designed for developers and data engineers to build data transformations that execute on Apache Spark without Scala or Python skills. These are a few of our sample data pipelines to address the most common Apache Spark Design Patterns:
- Machine learning data pipelines using PySpark or Scala
- Slowly changing dimensions data pipelines
- Spark ETL on Azure
- Clickstream ingestion and analysis on AWS
Why Use Sample Pipelines for Spark Design Patterns?
When you use a cloud service for instant Apache Spark access, you get a tuned and management environment ready for data. With sample Apache Spark pipelines, you don’t have to have advanced skills to use it. StreamSets has created a rich data pipeline library available inside of StreamSets Transformer or from Github. Simply choose your design pattern, then open the sample pipeline. Add your own data or use sample data, preview, and run.
StreamSets smart data pipelines use intent-driven design. That means the “how” of implementation details is abstracted away from the “what” of the data. Use StreamSets Transformer to build data transformations that execute on Apache Spark for performing ETL, stream processing, and machine learning operations. Now, you can have the power of Apache Spark without having to code in Scala or PySpark.
