Building a Slack Slash Command as a Microservice Pipeline

Building a Slack Slash Command as a Microservice Pipeline

Slack API logoOne of the drivers behind Slack‘s rise as an enterprise collaboration tool is its rich set of integration mechanisms. Bots can monitor channels for messages and reply as if they were users, apps can send and receive messages and more via a wide range of APIs, and slash commands allow users to interact with external systems from the Slack message box. In this blog post, I'll explain how I implemented a sample slash command as a microservice pipeline in StreamSets Data Collector 3.8.0, allowing users to look up stock item names and URLs from stock item numbers. Use this as the basis for creating your own slash command!

Slack Slash Command Basics

Quoting from the excellent Slack API documentation:

Slash Commands are initiated from the message box in Slack, but they aren't messages. A submitted Slash Command will cause a payload of data to be sent from Slack to an app, allowing the app to respond in whatever way it wants.

A user invokes a slash command by typing /commandname optionally followed by some text. Slack sends a signed HTTP form POST to the URL configured for that command. The form payload contains the identity of the requesting user, the command's name, any text following the command, and more. The app verifies the signature and responds to the POST with either plain text or JSON content that Slack can display to the user. The JSON content can include text, images, links and more.

To implement a slash command, then, your app needs a publicly accessible HTTP endpoint, it needs to parse incoming text, and it needs to return a response. It just so happens that a microservice pipeline in StreamSets Data Collector can do all of this.

Microservice Pipelines in StreamSets Data Collector

A microservice pipeline comprises a microservice origin, zero or more processors, and one or more microservice destinations. A microservice origin receives requests from client apps, parses the request content, and hands off a batch of one or more records for processing by the remainder of the pipeline, holding the request connection open while the batch is processed. Processors act on the message content, looking up data, applying transformations, etc, before passing the batch to the destination(s). There are a variety of microservice destinations; some pass data to an external system, but they can all return a response via the connection held by the origin. For more information on microservice pipelines, and a detailed tutorial, see the blog post Create Microservice Pipelines with StreamSets Data Collector.

While the initial implementation of microservice pipelines wrapped the outgoing batch for return to the microservice client, the microservice origins in Data Collector 3.8.0 have a Send Raw Response configuration option that simply returns records as-is, with no wrapping. This setting is the key to implementing a Slack slash command, and many other integrations.

I created a microservice pipeline to implement a Slack slash command to lookup stock numbers and return the stock item's name and URL. Here's how you would invoke it from the Slack message box:

And the response:

Here's the pipeline I built:

Slack slash command pipeline

Although the pipeline looks quite complex, it's actually pretty simple. Let's look at it stage by stage, starting with the origin on the left.

Since the incoming request arrives as an HTTP form post, the REST Service origin is configured with the Text data format:

Origin data format

As mentioned above, the Send Raw Response must be enabled in the HTTP Response tab; we're returning rich content to Slack, which requires the JSON data format:

Origin response config

Next, the Jython Evaluator verifies the signature according to the mechanism provided by Slack. The code is actually quite straightforward:

As noted in the code, SLACK_SIGNING_SECRET is configured as a runtime parameter, along with the parameters we will need for the database lookup:

runtime parameters

Since Slack also sends details of the user that invoked the command, you have the opportunity to perform additional authorization in the pipeline – for example, you might have a database table containing the ids of users authorized to invoke the slash command.

After the Jython script runs, a Stream Selector sends records fulfilling the condition ${record:attribute('signatureValid') == 'true'} for further processing, while records that fail the test have all of their fields removed and result in a 403 Forbidden response being returned to the client.

Valid requests are parsed by the Expression Evaluator using the str:splitKV() Expression Language (EL) function.

Parse form parameters

splitKV takes a delimited set of key-value pairs, such as an HTTP form POST payload, and splits it into a key-value map. For instance the input

is split into

Since HTTP form parameters are URL-encoded, we URL-decode the /text field. This is not crucial for this example, but is essential if your command text contains characters such as spaces or percent signs.

Now the JDBC Lookup processor can lookup the part number in the database. The SQL query is very straightforward:

If a matching stock item is found, them the /item_name record field will be populated. The second Stream Selector tests for the existence of the field with this expression:

If the /item_name field does exist, then the upper Expression Evaluator builds the response:

Constructing the response

Note the use of emptyList() and emptyMap() to build the required field hierarchy.

If the item isn't found, then the lower expression evaluator sets an error message:

Construct error message

The last step before returning the response is to remove all fields except the three that Slack is expecting:

Remove unwanted fields

Finally, the ‘200 OK' Send Response to Origin destination returns the record to the origin as a JSON object, along with a 200 HTTP status code indicating success.

200 OK response

We can test the pipeline with a simple Python script:

After setting the SLACK_SIGNING_SECRET runtime parameter to DUMMY_SLACK_SECRET and running the pipeline, we can run the test script:

Success!

Registering Your Slash Command Endpoint in Slack

The Slack API documentation explains how to set up a slash command as a Slack app – you'll need to supply the name of the command, its URL, description etc. Once you've created the Slack app, you can copy its Signing Secret and paste it into the pipeline's runtime parameters.

Putting it All Together

You can see the slash command in action in this short video:

https://www.youtube.com/watch?v=h2hc2fm7djY

 

Conclusion

With StreamSets Data Collector 3.8.0 you can implement arbitrary REST APIs as microservice pipelines. You can use standard processors to lookup data and manipulate record fields, while script evaluators can implement custom logic such as validating request signatures. Data Collector is Apache 2.0 licensed open source, so download it today and get started building your own microservice pipelines. Leave a comment to let us know what you come up with!

Attending the Strata Data Conference this week in San Francisco? Come visit the StreamSets team at booth 1231 in the expo!

Related Resources

Check out StreamSets white papers, videos, webinars, report and more.

Visit the Resource Library

Related Blog Posts

Receive Updates

Receive Updates

Join our mailing list to receive the latest news from StreamSets.

You have Successfully Subscribed!

Pin It on Pinterest