skip to Main Content

The DataOps Blog

Where Change Is Welcome

Get Access to Transformer for Snowflake Today!

Engineering, Industry, StreamSets News

Transformer for Snowflake is the first enterprise data transformation engine built on Snowpark. Want to learn how the engine makes advanced, native data transformations for your Data Cloud possible? Join our technical experts on Office Hours. Today at StreamSets, we're thrilled to announce the launch of our Public Preview of Transformer for Snowflake. By entering Public Preview, all users of…

By March 21, 2022

How Your Data Ingestion Framework Turns Strategy into Action

Engineering

With Data Infrastructure predicted to grow to over 175 zettabytes (ZB) by 2025, the debate amongst data engineers is no longer how big the data they encounter will be. Instead, they talk about how best to design a data ingestion framework that ensures that the right data is processed and cleansed for applications that need them.  Data ingestion is the…

By March 21, 2022

What is Streaming Data Analytics? Use Cases, Examples, and Architecture

Engineering

Netflix’s ability to stream data killed Blockbuster video. All of a sudden, customers could access movies—late-fee free—from their couch. And there was no need to drive to the store, rewind the tape, and drive back. Also, for Netflix, a catalog of movie files was far cheaper to maintain and distribute than an inventory of DVDs and VHS tapes.  Streaming analytics…

By March 17, 2022

Testing and Automation with the StreamSets DataOps Platform SDK for Python

Engineering, StreamSets News

We are excited to announce the immediate availability of StreamSets DataOps Platform SDK for Python version 4.0.0.  It enables users to interact with StreamSets DataOps Platform programmatically using Python 3.4+. Highlights of the StreamSets SDK SDK Activation key is no longer required DataCollector and Transformer classes are no longer public because these are headless engines in StreamSets DataOps Platform Authentication…

By March 16, 2022

Hadoop (MapReduce) vs Apache Spark: A Deep Dive Comparison

Engineering

There is no denying the impact of distributed data and computing over the last 15 years. These innovations have shattered constructs and limitations in database design which have held largely constant over the decades previous. This enables analytics at a scale and speed never imagined possible. To understand how we got to machine learning, AI, and real-time streaming, we need…

By March 14, 2022
Back To Top