The DataOps Blog
Where Change Is Welcome
AWS Reference Architecture Guide for StreamSets
Using StreamSets DataOps Platform To Integrate Data from PostgreSQL to AWS S3 and Redshift: A Reference Architecture This document describes…
Python vs. Java: A Deep Dive Comparison
Did you know that Python and Java are two of the most commonly used programming languages today? Of the hundreds of languages to choose from, Python and Java continue to gain more and more user adoption, leading to more programs and platforms being written in Python and Java, and so forth creating a snowball-like effect in today’s programming ecosystem. Of…
Why & How to Use Data Enrichment to Activate Your Data Lake for Analytics
We all know the shift to the cloud is massive and has been accelerated by COVID; however, I think many of us (myself included) don’t take enough time to really look at the new cloud-native services for doing data enrichment and analytics. AWS Re:Invent last December was a great opportunity to talk to customers, prospects, and other technology companies. Meeting…
Transformer for Snowflake: Snowflake Transformation that Meets Cloud-First Expectations
Transformer for Snowflake is the first enterprise data transformation engine built on Snowpark. Want to learn how the engine makes advanced, native data transformations for your Data Cloud possible? Join our technical experts on Office Hours. It's likely you remember (or have at least heard) that databases in the past, beyond SQL, were extremely complicated to stay in a performant…
The JSON Validator: A Custom Processor to Ensure Your JSON Payload is Syntactically Accurate
Before we get to building your custom JSON validator, let’s talk about the author and their thoughts on why JSON has become so essential in data engineering. The author, Joel Klo, is a consultant at Bigspark, UK's engineering powerhouse, delivering next level data platforms and solutions to their clients. Their clients use modern cloud platforms and open source technologies, prioritize…
Get Ready for Private Preview! Transformer Engine for Snowflake
Transformer for Snowflake is the first enterprise data transformation engine built on Snowpark. Want to learn how the engine makes advanced, native data transformations for your Data Cloud possible? Join our technical experts on Office Hours. It’s no surprise when organizations implement the Snowflake Data Cloud as their internal standard for their overall data strategy. Especially as Snowflake continues to…