Here’s a confession – over time I have come to like services that expose their data via a RESTful API. It’s scalable, it’s understandable and quite importantly – it’s elegant. Working with REST is a breeze, right?

Well, a few weeks ago, I found myself facing a rather mundane task – retrieving all the information about a post with a given ID from a certain web service (which happened to sport RESTful API). Yawn, huh? Well the thing was that in order to get the information I had to jump through two levels of indirection (commentable -> project -> project_owner’s email). And I had to do this three times to get all the data I needed. Oh and in CoffeeScript – so all asynchronous, no blocking.

Let’s take a look at our options. The most naive version of the implementation woud look something like this[ref]We’re skimping on error handing, the ‘real’ code would probably call an errback and return.[/ref]:

request.get('http://server.com/posts/42.json', (err, body, res) ->
    if err or res.statusCode isnt 200
        # Handle errors here...
        request.get('http://server.com/projects/#{body.project.id}.json'), (err, body, res) ->
            if err or res.statusCode isnt 200
                # Handle errors here.
                request.get('http://server.com/users/#{body.owner.id}.json'), (err, body, res) ->
                    if err or res.statusCode isnt 200
                        # Handle errors here.
                        email = body['details']['email-address']
...

A.K.A The Callback Pyramid Of Doom. Yuck. We could certainly do better.

Let’s extract all the error handling code into a single function[ref]Again, error handling is just…hinted. :-)[/ref].

request.get('http://server.com/posts/42.json', (err, body, res) ->
    checkErr(err, res)
    request.get('http://server.com/projects/#{body.project.id}.json'), (err, body, res) ->
        checkErr(err, res)
        request.get('http://server.com/users/#{body.owner.id}.json'), (err, body, res) ->
            checkErr(err, res)
            email = body['details']['email-address']
...

That looks slightly better, right? Well, if you take closer look, you will see that the “important vs. boilerplate” code ratio is actually quite horrible – more than half of the code is basically something we’ve already said, just repeated.

For a professional developer, that’s troubling. How could we do better? Is there a more declarative way to query the API without repeating ourselves? And can we make it generic so that it can be reused? A few minutes of quick googling didn’t show anything useful, so I sat down and implemented a tiny module called RestQ.

Here’s the code from above modified to use RestQ:

emailPromise = RQ.from('http://server.com/posts/42.json')
    .via('/projects/{project.id}.json')
    .via('/users/{owner.id}.json')
    .get('details.email-address')

emailPromise
    .then (email) -> ... , (err) -> ...

As you can see we’ve managed to eliminate most of the boilerplate and (by using promises) flatten out the callback pyramid. Parts of the URLs between the curly braces are expanded to body.project.id and body.owner.id respectively. The code is concise but readable.

Another nice thing about RestQ is that it allows us to write code like:

post = RQ.from('http://server.com/posts/42.json')
projectName = post.via('/projects/{project.id}.json').get('name')
authorName = post.via('/users/{author.id}.json').get('details.displayName')

Q.all([projectName, authorName]).then((pn, an) -> 
    ...
).fail((err) -> 
    ...
).end()

All without the hassle of dealing with nested callbacks. Pretty cool, eh?

Get the code from GitHub and try it out yourself!