skip to Main Content

The DataOps Blog

Where Change Is Welcome

Replicating Oracle to MySQL and JSON

StreamSets News

Yannick Jaquier is a Database Technical Architect at STMicroelectronics in Geneva, Switzerland. Recently, Yannick started experimenting with StreamSets Data Collector's Oracle CDC Client origin, building pipelines to replicate data to a second Oracle database, a JSON file, and a MySQL database. Yannick documented his journey very thoroughly, and kindly gave us permission to repost his findings from his original blog entry.…

By May 10, 2019

Field Mapper Processor: The Swiss Army Knife of Bulk Field Manipulation

Engineering

Guest post by Jeff Evans, Senior Software Engineer, StreamSets. The Field Mapper processor, introduced in Data Collector version 3.8.0, provides a flexible and powerful way to manipulate fields en masse in your records. It operates in one of three modes: Field paths: The location within the context of the entire record and is useful for moving fields around the record,…

By May 7, 2019

Binary Classification of Streaming Data using TensorFlow to ADLS Gen1 and ADLS Gen2

StreamSets News

Over the past decade, digital transformation has evolved such that every system and device has a digital trail: from IT servers, to factory equipment, to consumer electronics, to buildings, to cars. Increasing data volumes, rates, and variety have created increasing complexity, not to mention these new datasets must be analyzed in real-time. Fit-for-purpose data platforms allow for storing and applying…

By May 2, 2019

Sensor Data from Azure Event Hub to ADLS Gen2 and Azure SQL DWH

StreamSets News

Data that’s in flight is perceived to be more challenging to work with than "landed" data sitting quietly in some storage platform. With the high volume, and variety of data constantly streaming in from IoT sensors, you need a holistic DataOps approach that goes beyond traditional, stationary transactional data. The StreamSets DataOps platform provides an end-to-end event stream processing system…

By May 2, 2019
Back To Top