Windows Event Log

The Windows Event Log origin reads data from a Microsoft Windows event log located on a Windows machine. The origin generates a record for each event in the log.

Use the Windows Event Log origin only in pipelines configured for edge execution mode. Run the pipeline on StreamSets Data Collector Edge (SDC Edge) installed on the Windows machine.

For example, you might use the Windows Event Log origin in an edge pipeline to read logs from a web or application farm of Windows servers. You install SDC Edge on each Windows machine that you want to read the logs from, and run the edge pipeline on each SDC Edge installation. You design the edge pipeline to pass the log data to a Data Collector receiving pipeline that runs on StreamSets Data Collector. The Data Collector receiving pipeline performs more complex processing on the data, and then writes the data to a big data system such as Hadoop. You can then analyze the data to detect security violations such as insider threats or illegal access to the Windows machines.

When you configure the Windows Event Log origin, you specify the Windows log to read from. You also specify whether the origin reads all events in the log or whether it reads only new events that occur after the pipeline starts.

When the pipeline stops, the Windows Event Log origin notes where it stops reading. When the pipeline starts again, the origin continues processing from where it stopped by default. You can reset the origin to process all requested files.

For more information about installing SDC Edge, designing edge pipelines, and running and maintaining edge pipelines, see Edge Pipelines.

Configuring a Windows Event Log Origin

Configure a Windows Event Log origin to read data from a Windows event log.

  1. In the Properties panel, on the General tab, configure the following properties:
    General Property Description
    Name Stage name.
    Description Optional description.
    On Record Error Error record handling for the stage:
    • Discard - Discards the record.
    • Send to Error - Sends the record to the pipeline for error handling.
    • Stop Pipeline - Stops the pipeline.
  2. On the Windows tab, configure the following properties:
    Windows Property Description
    Log Name Name of the Windows log to read from:
    • Application
    • System
    • Security
    Read Mode Determines how the origin reads the log:
    • All - Read all events in the log.
    • New - Read only new events in the log that occur after the pipeline starts.