skip to Main Content

The DataOps Blog

Where Change Is Welcome

DataOps Summit 2021 On Demand

How to Become a Data Engineer (and Why You’re Lucky if You Do!)

StreamSets News

If you’re interested in how to become a data engineer, you’re in luck. Data engineers are in high demand, and resources for how to become a data engineer abound! Data engineers are technical professionals who understand how data analysts and data scientists need data, then build the data pipelines to deliver the right data, in the right format, to the right…

By March 16, 2021
Load Data Into Google BigQuery

How To Load Data Into Google BigQuery on Dataproc and AutoML


Load Data Into Google BigQuery and AutoML In this blog, we will review ETL data pipeline in StreamSets Transformer, a Spark ETL engine, to ingest real-world data from Fire Department of New York (FDNY) stored in Google Cloud Storage (GCS), transform it, and load data in Google BigQuery curated. Once the transformed data is made available in Google BigQuery, it…

By February 23, 2021
Optimizing Reuse For Data Quality

Optimizing Re-use for Data Quality to Scale Data Pipelines

StreamSets News

As more data enters an organization’s ecosystem for transformation and is shared with more and more organizations both within and external to the business, data quality processes and frameworks are essential. Without data quality, users will lose trust in the analytics, resulting in stalled user adoption and analytic silos. Pulling all of the information together correctly creates symbiotic results that…

Back To Top

We use cookies to improve your experience with our website. Click Allow All to consent and continue to our site. Privacy Policy