skip to Main Content

The DataOps Blog

Where Change Is Welcome

Get Ready for Private Preview! Transformer Engine for Snowflake

By Posted in StreamSets News December 21, 2021

transformer-for-snowflake-office-hours

Transformer for Snowflake is the first enterprise data transformation engine built on Snowpark. Want to learn how the engine makes advanced, native data transformations for your Data Cloud possible? Join our technical experts on Office Hours.

It’s no surprise when organizations implement the Snowflake Data Cloud as their internal standard for their overall data strategy. Especially as Snowflake continues to advance its offerings.

Recently, Snowflake announced the public preview of Snowflake Snowpark. As part of the Snowpark Accelerated Partner Program, my team and I are excited about the game-changing capabilities Snowpark brings to our current and future users.

Apply for the Private Preview of the Transformer for Snowflake. Hands-on access is coming on January 3, 2022!

Expect Superior Performance & Extended Functionality with Transformer for Snowflake

The introduction of Snowflake Snowpark and the separation of storage and compute gives customers the ability to store massive amounts of data and run any number of complex queries without taking a performance hit. With Snowpark, you can also now run complex data processing jobs natively in the Snowflake Data Cloud which previously would have required external Spark clusters. 

For data scientists and data engineers this addition to the Snowflake Data Cloud provides greatly extended functionality. 

  • ML model training
  • Running custom libraries in your data pipelines
  • Running complex data processing logic such as data quality and standardization functions
  • And, much more.

What is the StreamSets Transformer for Snowflake?

Transformer for Snowflake allows data engineers to go beyond SQL to express powerful data pipeline logic with the StreamSets DataOps Platform. Utilizing Scala or Java through an intuitive graphical interface, users can choose no code, or drop-in code when they want.

Transformer for Snowflake provides all the benefits of StreamSets DataOps Platform with built-in monitoring and orchestration of complex data pipelines at scale. And it does so in the cloud, with no additional hardware required.

Learn What You Can Do During Private Preview

Private Preview for Transformer for Snowflake will be available in the StreamSets DataOps Platform by January 3, 2022. Sign up for your opportunity to be part of our early access program that will likely run through February.

I’ll personally be leading some live sessions and helping 1:1 to show some of the more advanced capabilities.

Here are just a few of the things I will be helping people apply to their efforts:

  1. Build fully functional pipelines that span SQL statements and advanced custom logic. 
  2. Learn how to import and use your custom Corporate libraries in data pipelines using UDF’s. (Let’s be honest, we all have these and they are so rich with business logic, yet so far away from the data!)
  3. Convert a Python notebook into a visual StreamSets pipeline that anyone can understand and follow. Think about it, no need to explain to others or decipher what others have written as it relates to complex data processing logic. 

Here is an image from inside our internal preview environment of the Transformer for Snowflake where we’ve converted a Python notebook that ran on Spark into a visual pipeline which ran 100% in Snowflake.

Snowflake SnowparkHere is another example, using the Transformer for Snowflake to perform Data Warehouse processing by using our Slowly Changing Dimension processor.

StreamSets Engine for Snowflake Snowpark

Transformer for Snowflake is the perfect coalescing of SQL and Spark which can now be harnessed with the StreamSets DataOps Platform, 100% hosted and executed serverlessly in our cloud.

Apply here and let us show you how your data processing is now only limited by your imagination when you use Transformer for Snowflake! I cannot wait to meet you and get started!

Note: This Private Preview is available to both existing Snowflake users who have access to Snowpark and new users. (If you don’t have Snowflake yet, please follow this link to sign up for your free trial of Snowflake.)

LET’S GO!

Try StreamSets Free

Back To Top

We use cookies to improve your experience with our website. Click Allow All to consent and continue to our site. Privacy Policy

[class^="wpforms-"]
[class^="wpforms-"]