A Starling Adapter for SMQueue

Starling is a persistent, lightweight work queue implemented in Ruby that speaks the memcache protocol. I have been playing with it recently because I don't have the resources to look after — or the requirement for — a full-blown service bus. Starling is easier to install and configure than ActiveMQ, though nowhere near as fully featured. Both have their place, but comparing them is outside the scope of this article.

I knew I wanted a message bus to turn synchronous requests into asynchronous ones, pushing work off to background processes. What I didn't know was which message bus I would end up using. If you are familiar with the Gang of Four patterns book you have probably already spotted the relevant pattern here. SMQueue, which I am familiar with, provides a clean abstraction that makes it easy to swap out the message bus implementation while keeping your code identical. The catch: SMQueue didn't ship with an adapter for Starling.

"How hard," I thought, "would it be to write one?"

I blinked and suddenly it existed.

require 'rubygems'
require 'smqueue'
require 'starling'
require 'yaml'

module BarkingIguana
  module Messaging
    module SMQueue
      class StarlingAdapter < ::SMQueue::Adapter
        class Configuration < ::SMQueue::AdapterConfiguration
          DEFAULT_SERVER = '127.0.0.1:22122'

          has :queue
          has :server, :default => DEFAULT_SERVER
        end

        def initialize(*args)
          super
          options = args.first
          @configuration = options[:configuration]
          @configuration[:server] ||= Configuration::DEFAULT_SERVER

          @client = ::Starling.new(@configuration[:server])
        end

        def put(*args, &block)
          @client.set @configuration[:queue], args[0].to_yaml
        end

        def get(*args, &block)
          if block_given?
            loop do
              yield next_message
            end
          else
            next_message
          end
        end

        private
        def next_message
          ::SMQueue::Message(:headers => {},
            :body => YAML.load(@client.get(@configuration[:queue])))
        end
      end
    end
  end
end

Want to use it? You will need Starling running somewhere. After that, a producer is just two lines of code:

producer = SMQueue(:adapter => BarkingIguana::Messaging::SMQueue::StarlingAdapter, :queue => "some.queue.name")
producer.put "Quack quack"

And here is a consumer on the other side of the connection:

consumer = SMQueue(:adapter => BarkingIguana::Messaging::SMQueue::StarlingAdapter, :queue => "some.queue.name")
consumer.get do |message|
  puts message.body.inspect
  # => "Quack quack"
end

One thing worth noting: this adapter assumes YAML as the transport format. I would prefer JSON or XML, but YAML was the easiest to implement and I am not above taking the lazy path when it gets the job done.

There is also work to be done around failover — this adapter only supports a single server. I don't yet know enough about how Starling handles failover, and I would rather not rush into an implementation that turns out to be wrong.

If you can help with patches for other transport formats or failover support, please do.