mths : sdrbrg ;

# bash: abort script on any error

For the longest of time it's been my understanding that set -e in Bash causes a script to terminate if any single command of the script fails, which is also exactly what it does, in most cases.

However, when you start using functions things change a bit. Consider the following example:

$ cat basherr
#!/bin/bash

set -e
function run() {
  false # <- this should cause the script to fail
  echo ", world!"
}
output=$(run)
echo "Hello$output" # <- this should never be executed

$ basherr ; echo $?
Hello, world!
0

I was recently working on a script that had a couple of functions and I wanted the script to abort if any single command failed. After some digging I found the -E option to set, which according to the documentation does the following:

If set, any trap on ERR is inherited by shell functions, command substitutions, and commands executed in a subshell environment. The ERR trap is normally not inherited in such cases.

Armed with this newfound knowledge I replaced set -e with set -E and a custom trap handler for ERR:

set -E
trap "exit $?" ERR

and lo and behold:

$ basherr ; echo $?
1

I could have saved myself some headache by reading the documentation of set more thouroughly as it does outline exceptions where set -e won't do what one might think.

10 Dec 2016 / bash

# helpers in the kitchen

When writing tests using RSpec, I prefer to use shared_context and shared_examples for integration setup and for testing shared behaviour, respectively.

When I started out with writing integration tests for my kafka cookbook using serverspec, I wanted to share tests between different suites as they were testing different init systems, but the tests could be heavily refactored to just depend on some shared variables rather than duplicating all of the test cases. It was however not immediately clear how one would go about sharing files between different suites.

After quite some digging I found a bit of information (I think in an old issue or pull request though I no longer have any links handy) that mentioned having a helpers directory in test/integration for sharing files between suites.

So it's just a matter of creating a helpers directory and the necessary busser specific subdirectory, adding some files and you're good to go. They'll even be available on the $LOAD_PATH, so it's easy to just require a spec_helper or the alike in the actual spec files.

Since v1.2.1 of test-kitchen it's also possible to create directories in the helpers directory (I tend to keep shared code in a support directory for example).

For reference my kafka cookbook is over here, and more specifically the helpers directory (and serverspec subdirectory) is over here.

19 Feb 2016 / test-kitchen chef rspec

# raw sns fanout to sqs

The other day I was messing around a bit with AWS SNS and SQS, more specifically I wanted to send messages to a SNS topic and have the messages fanout to a couple of SQS queues (as mentioned in the Common SNS Scenarios documentation).

It's simple enough to connect a SNS topic with a SQS queue:

$ aws sns subscribe --topic-arn <SNS_TOPIC_ARN> \
                    --protocol sqs \
                    --notification-endpoint <SQS_QUEUE_ARN>

However, the messages sent from SNS to SQS end up on a format that's pretty far from what I wanted:

$ aws sns publish --topic-arn <SNS_TOPIC_ARN> --message '{"hello": "world"}'
$ aws sqs receive-message --queue-url <SQS_QUEUE_URL>
{
  "Messages": [
    {
      "Body": "{\n  \"Type\" : \"Notification\",\n  \"MessageId\" : \"42e88f9d-6436-555e-aa9e-c0a7b128eeb8\",\n  \"TopicArn\" : \"arn:aws:sns:eu-west-1:123456789000:test\",\n  \"Message\" : \"{\\\"hello\\\": \\\"world\\\"}\",\n  \"Timestamp\" : \"2016-02-14T19:17:50.008Z\",\n  \"SignatureVersion\" : \"1\"\n}"
    }
  ]
}

I abbreviated the message a bit, but it's clear that the full SNS message has been JSON encoded and put into the Body of the SQS message.

So what I really wanted was to simply have the messages forwarded from SNS to SQS, without any surprises. Turns out it's actually possible to achieve exactly that, though I had a hard time finding it in the documentation (not exactly sure if I even found it in the documentation). The trick is to enable the RawMessageDelivery property of the SNS subscription, for example using aws:

$ aws sns set-subscription-attributes --subscription-arn <SNS_SUBSCRIPTION_ARN> \
                                      --attribute-name RawMessageDelivery \
                                      --attribute-value true
14 Feb 2016 / aws sns sqs