Sending IoT Data to ADLS Gen 2 and Azure SQL Data Warehouse with StreamSets

Sending IoT Data to ADLS Gen 2 and Azure SQL Data Warehouse with StreamSets

Analyze sensor readings in real time coming from Azure Event Hub to identify patterns and make decisions based on them.

Here, we calculate the average sensor value using a Windowing Aggregator Processor. The aggregator is calculates the average sensor value over a 5 second rolling window. If we detect a change in the sensor readings within the defined window of time, we must act immediately. If the average value per sensor calculated by the windowing aggregator is above a normal threshold, an alert is triggered and with a webhook, a notification is sent to the Operations team.

The pipeline will then send all aggregates to Azure SQL Data Warehouse and simultaneously archive all the sensor readings into Azure Data Lake Storage Gen2.

Receive Updates

Receive Updates

Join our mailing list to receive the latest news from StreamSets.

You have Successfully Subscribed!