SAN FRANCISCO, June 9, 2021 — StreamSets, the provider of the industry’s first DataOps platform, today announced the preview of its new engine for running data pipelines using the new developer experience, Snowpark, from Snowflake, the Data Cloud company. StreamSets engine for Snowpark marries StreamSets’ powerful DataOps capabilities with the flexibility in Snowpark to use languages such as Java and Scala. Snowpark and the StreamSets engine for Snowpark provides a unique combination of power and ease-of-use for data engineers to build, operate and monitor smart data pipelines for the Snowflake Data Cloud.
Many Snowflake customers already use the StreamSets DataOps Platform to build and operate smart data pipelines that ingest data in streaming, batch or change data capture (CDC) modes into the Snowflake Data Cloud. Now, with Snowpark and the StreamSets engine for Snowpark, data engineers can go beyond SQL to express powerful data pipeline logic with the StreamSets DataOps Platform utilizing their preferred language, such as Scala and Java, through an intuitive graphical interface. Those pipelines can then execute directly on Snowflake’s Data Cloud, giving even more users the ability to achieve powerful data insights without having to manage additional processing systems.
Specifically, Snowflake customers can use the StreamSets DataOps Platform and the StreamSets engine for Snowpark to get even more out of their Data Cloud investments with the following key benefits:
- A high-performance data engineering platform harnessing the power and flexibility of Scala while executing entirely in Snowflake’s Data Cloud– eliminating the need for additional Spark-based systems that add cost, complexity and latency
- A powerful design environment that provides 10x productivity to data engineers building and orchestrating enterprise-grade data pipelines, combining the ease of drag-and-drop design with the extensibility of custom code and a comprehensive SDK
- A single experience for building, operating and monitoring all pipelines for Snowflake’s Data Cloud, including streaming, CDC and batch data pipelines ingesting data to the Data Cloud, as well as ELT pipelines executing workloads directly in the Data Cloud via Snowpark
- Smart data pipelines that are designed to deliver data on a continuous basis to support a DataOps practice, with built-in resiliency to changes in data structures, schema and semantics
- Data engineering delivered as a cloud service with no infrastructure to build or manage
“Snowpark is transformative for data engineers and developers who prefer languages other than SQL for development,” said Isaac Kunen, Senior Product Manager, Snowflake. “With StreamSets engine for Snowpark, data engineers will be able to build and operate Scala-based data pipelines designed for DataOps, bringing the power and performance of Snowflake’s Data Cloud to the savviest enterprise data engineers.”
“Our customers have been making amazing strides employing DataOps practices and democratizing their data with Snowflake,” said Judy Ko, Chief Product Officer, StreamSets. “We see Snowpark as the next big step, making sophisticated data engineering accessible to everyone, so all types of data, from all sources, can be delivered continuously through the Data Cloud.”
StreamSets engine for Snowpark is available for preview as of today. You can sign up by contacting trysnowpark@streamsets.com.
About StreamSets
At StreamSets, our mission is to make data engineering teams wildly successful. Only StreamSets offers a platform dedicated to building the smart data pipelines needed to power DataOps across hybrid and multi-cloud architectures. That’s why the largest companies in the world trust StreamSets to power millions of data pipelines for modern business intelligence, AI/ML and smart applications. With StreamSets, data engineers spend less time fixing and more time doing. To learn more, visit www.streamsets.com and follow us on LinkedIn.