The DataOps Blog
Where Change Is Welcome
Announcing StreamSets Data Collector 3.12.0 and StreamSets Data Collector Edge 3.12.0
StreamSets is excited to announce the immediate availability of StreamSets Data Collector 3.12.0 and StreamSets Data Collector Edge 3.12.0. StreamSets Data Collector is open source under Apache License 2.0 and a powerful design and execution engine. It enables moving data between any source and destination, performing transformations, and push down analytics along the way. To download, click here. StreamSets Data…
StreamSets Transformer: Design Patterns For Slowly Changing Dimensions
In this blog, we will look at a few design patterns for Slowly Changing Dimensions (SCD) Type 2 and see how StreamSets Transformer, the newest addition to the StreamSets DataOps Platform, makes it easy to implement them. While relatively static data like locations and addresses of entities, such as customers, change rarely (if at all) over time, in most cases…
Ingest data into Azure Synapse Analytics (formerly SQL DW) with StreamSets Cloud
Azure Synapse Analytics, the next evolution of Azure SQL Data Warehouse, combines enterprise data warehousing and big data analytics into a single analytics service. StreamSets Cloud's new Azure SQL Data Warehouse destination, released today, loads data into Azure Synapse. Loading data into Azure SQL Data Warehouse destination is a two-stage process. First, data must be written to Azure Storage, then…
StreamSets Data Collector: Simple Network Management Protocol And Management Information Base
This is a guest post by Clark Bradley, Solutions Engineer, StreamSets SNMP stands for simple network management protocol and allow for network devices to share information. SNMP is supported across a wide range of hardware such as conventional network equipment (routers, switches and wireless access points) to network endpoints like applications and internet of things (IoT) devices. An SNMP managed…
StreamSets Transformer: Your Questions Answered
StreamSets Transformer, a powerful tool for creating highly instrumented Apache Spark applications for modern ETL, is the newest addition to the StreamSets DataOps Platform. StreamSets enables next-generation ETL through the StreamSets Transformer tool. The product provides enterprises with the flexibility to create ETL pipelines for both batch and streaming data as well as clear visibility into their data processing operation…