Announcing StreamSets Data Collector 3.10.0 and StreamSets Data Collector Edge 3.10.0

Announcing StreamSets Data Collector 3.10.0 and StreamSets Data Collector Edge 3.10.0

StreamSets is excited to announce the immediate availability of StreamSets Data Collector 3.10.0 and StreamSets Data Collector Edge 3.10.0.

StreamSets Data Collector is open source under Apache License 2.0 and a powerful design and execution engine. It enables moving data between any source and destination, performing transformations, and push down analytics along the way. To download, click here.

StreamSets Data Collector Edge is a lightweight execution agent that runs on edge devices with limited memory, CPU, and/or connectivity resources. It enables reading data from an edge device or receiveing data from another dataflow pipeline. It supports messaging protocols including HTTP, MQTT, CoAP, and WebSockets. To download, click here.

Highlights

Let's review some of the highlights. For a complete list of enhancements, new features, bug fixes, and upgrade instructions, please refer to the Release Notes.

StreamSets Data Collector 3.10.0

This release includes the following new origins.

Origins
Processors

Groovy Evaluator, JavaScript Evaluator, and Jython Evaluator processors now support the following new features:

  • User-defined parameters—On the Advanced tab, enter parameters and values. In the script, access the value with the sdc.userParams dictionary.
  • Full-screen script editing—With your cursor in the script field, press F11 or Esc on Mac OS X to toggle full-screen editing.
Destinations

This release includes enhancements to the following destinations.

  • Cassandra—The destination has four new properties to help you debug: Connection Timeout, Read Timeout, Consistency Level, and Log Slow Queries.
  • RabbitMQ Producer—The destination has a new Set Expiration property, available when setting AMQP message properties on the RabbitMQ tab. Clear the Set Expiration property to disable expiration on messages that the destination sends.
Executors

This release includes the following enhancement.

JDBC Query—The executor can now run queries in parallel to improve throughput. On the Advanced tab, select the Enable Parallel Queries property to have the executor run queries simultaneously on each connection to the database.

Data Collector Configuration

This release includes the following Data Collector configuration enhancement.

The Data Collector configuration file sdc.properties contains a new property, stage.conf_com.streamsets.pipeline.stage.jdbc.drivers.load, where you can list JDBC drivers that Data Collector automatically loads for all pipelines.

Feedback and Contributions

If you'd like to suggest a feature, enhancement, or if you see something that needs to be fixed or made better, feel free to open a ticket by visiting—https://issues.streamsets.com.

Also note that StreamSets welcomes contributions from the community. For guidelines on contributing code, visit—https://github.com/streamsets/datacollector/blob/master/CONTRIBUTING.md

For more information about StreamSets Data Collector, visit our documentation. For more information about StreamSets Data Collector Edge, visit our documentation.

For any other questions and inquiries, please contact us.

Related Resources

Check out StreamSets white papers, videos, webinars, report and more.

Visit the Resource Library

Related Blog Posts

Receive Updates

Receive Updates

Join our mailing list to receive the latest news from StreamSets.

You have Successfully Subscribed!