Avoiding API Library Antipatterns

Paul Mison, 12 January 2009London Ruby User Group

Hi, Iʼm Paul Mison and Iʼm here to talk about Ruby, and APIs.

In particular, Iʼll be talking about Flickrʼs API. This is its documentation page. Flickr, as Iʼm sure you all know, is a site for sharing photos.

Photos like this one, in fact. by Ryan Forsythe

They offer a REST-style API, as well as XML-RPC and SOAP, which Iʼll ignore. However, you can take the lessons from this and apply it to many other popular modern web APIs, which have a similar structure.

The screenshots here are of their API Explorer tool, which is a nice way to see what arguments a method takes and what it returns. But you donʼt want to worry about XML: you just want kitten pictures!

RubyForge lists a lot of projects to talk to Flickr. (This was taken a while ago; I should probably have searched GitHub.)

In fact, I make out that there are seven libraries listed. It turns out that a lot of these are abandoned. Iʼll explain why.

First, hereʼs a Ruby Flickr library called flickr-fu. It works, and returns objects.

Letʼs try this perfectly valid Flickr API method - in fact, itʼs one of an entire subset of methods in the groups namespace - and we see the library is throwing an error. Why? Well, Thereʼs no groups method in flickr-fu because nobodyʼs written it yet.

This is part of the code for flickr-fu, for the people methods. Note all this code thatʼs just copying part of the response to the Person object. Now, imagine going through and doing that for the groups methods...

... and there are a few. Daunting, isnʼt it? I can understand why they havenʼt got around to it yet.

In fact, these are the areas that flickr_fu provides, according to commonthreadʼs github copy.

r2flickr, an extension of rflickr, which is listed on the API home page, does better, but even so, itʼs missing some of the newer parts of the API (such as the flickr.places methods).

This is the problem with abstraction.

This is the problem with abstraction. Once you start setting up objects like that, then you find that thereʼs a lot of work involved to add new ones, and Flickrʼs API is quite broad. This explains the abandoned modules- their authors got bored of writing boilerplate code.

Even when youʼve implemented all the existing methods, thatʼs not the end. “We added 1 new API method, and updated 7 others”. If youʼre looking after an API, thatʼs the last thing you want to see.

... especially since, for now, thereʼs no way to even track those changes yourself unless you write your own code to use Flickrʼs reflection methods - although Flickr are aiming to change that.

http://flickr.com/groups/api/discuss/72157604690623636/

Fixing It: The Request

There is a solution.

Fixing It: The Request

So, we can see why the OO approach leads to unfinished, or unmaintained, libraries. How do you avoid this?

http://flickr.com/photos/gspragin/2043558365/ Nothing to do with fixing, really, this picture, but I liked it anyway.

All a request to Flickr is is a signed call. This is from flickraw, and thatʼs all you need, including argument signing.

In fact, if you look at flickr-fu earlier, it has a similar internal method, send_request. If they just exposed it, I could use it for group or places calls. To be fair, it is documented as a public method, but you wouldnʼt think to look for it.

You can be a step cleverer, and offer the nice methods, even if you donʼt write boilerplate.

This is another part of flickraw, where it uses Flickrʼs own reflection methods - the ones mentioned earlier - to dynamically build the list of methods itʼs able to use. Smart, but takes a bit of effort.

http://flickr.com/photos/omaromar/417305666/

Fixing It: The Response

Now, what about the responses?

Back to flickr-fu. Look at all that code copying from the rsp XML object to properties of the Person object. What a drag, but I suppose it saves me having to do it. Still, surely thereʼs a better way?

Hereʼs some good news. Flickr supports returning its responses as JSON

Object

The O in JSON stands for Object, and this means that youʼre able to get things back that you can easily treat as, well, objects, without any of that painful XML traversing that we all know and love.

To return an API response in JSON format, send a parameter “format” in the request with a value of “json”

Getting back a JSON response rather than an XML one is trivial; you just change one parameter. (Sidenote probably not for the talk: you also want “nocallback” to be set, otherwise the JSON will be an executable bit of code rather than just an object, but itʼs still pretty straightforward.)

This is exactly what flickraw does, and as a result, it can automatically pick up any additions that Flickr make to their response format. And when you come to output it, you can just write clean Ruby code.

Conclusions

So, what have we learnt?

http://www.flickr.com/photos/artimagesmarkcummins/513969188

Offer simple access wrappers to users.

Instead of writing a method per allowed call, first write a generic method - youʼll probably need it anyway.

Abstraction sucks -unless youʼre clever

Rubyʼs dynamic.Let it do the work.

Rubyʼs a dynamic language. Use that to your best advantage - for example, building convenience methods on the fly.

Abstraction sucks -unless youʼre clever

Use JSON,not XML.

Use JSON, which can be easily turned into an object, rather than XML, to save yourself work converting things.

Thank you.

Questions?http://www.flickr.com/photos/antimega/2468011723/http://www.flickr.com/photos/kubina/279522886/http://www.flickr.com/photos/infrarad/182786930/http://flickr.com/photos/gspragin/2043558365/http://flickr.com/photos/omaromar/417305666/ http://flickr.com/photos/artimagesmarkcummins/513969188