The DataOps Blog
Where Change Is Welcome
StreamSets Live: Demos with Dash!
45 minutes. Live demos. No (0) slides! Key takeaways: Visually build data pipelines like you code: build, verify, iterate, commit, rinse, repeat! Ingest data from…
Clickstream Analysis on Amazon EMR, Amazon Redshift and Elasticsearch
Learn how quickly you can start ingesting and aggregating clickstream logs using StreamSets Transformer running on Amazon EMR, and see how the data is analyzed in Elasticsearch, Kibana, and Amazon Redshift. Clickstream analysis by definition is the process of collecting, analyzing, and reporting aggregate information about webpage visits. StreamSets Transformer is an execution engine that runs on different "flavors" of…
3 Scenarios for Adjusting Your Data Practice to Business During the Pandemic
Okay, last work meetings are done for the day. Daughter is downstairs doing soccer lesson by Zoom, son is in his bedroom playing around with his iPad, rice cooker is doing its thing. Time to get this blog written. It’s clear that while the overall societal and macro-economic impacts of this pandemic have been unprecedented, the effects on any given…
StreamSets Named in InsideBigData Impact 50 List
Every year the insideBigData team puts together a list of companies making the biggest impact on this space, and for the 3rd year in a row StreamSets is one of these 50 prominent disruptors. StreamSets is ranked #22 alongside notable ecosystem pillars like Nvidia, Snowflake, Cloudera, DataRobot, and Databricks. Opinions are abundant in today's economy which is why insideBigData has…
Action in the Face of Uncertainty: Leading through Crisis
“These are the times that try men’s souls.” --Thomas Paine, The American Crisis Many of you, reading this blog, weathered the systemic disruptions of the dot-com bubble burst and the great financial crisis. The current environment is fundamentally different. While scientists had previously identified the possibility of such a pandemic, a 2020 business plan could not with certainty have “planned”…
Streaming Analysis Using Spark ML in StreamSets DataOps Platform
Learn how to load a serialized Spark ML model stored in MLeap bundle format on Databricks File System (DBFS), and use it for classification on new, streaming data flowing through the StreamSets DataOps Platform. In my previous blogs, I illustrated how easily you can extend the capabilities of StreamSets Transformer using Scala and PySpark. If you have not perused blogs…