Many programmers (or coders, or software engineers, or computer wizards, or
whatever term you would like to use; the arguments had over this could and
probably will fill a whole blog post on their own) find themselves involved to
a greater or lesser extent in the development of libraries and programming
utilities for the use of their colleagues in the discipline. There is no
denying that the development of good libraries is a noble and worthy pursuit,
although I have to say that from time to time I do wonder if perhaps some of
the effort expended on doing that could be diverted to creating useful tools
for non-technical types. Not the point though, so I’ll return to what I meant
to say.
My single real involvement in the open-source movement of any note whatsoever
(and believe me, it is of very minor note) is my occasional contributions to
the Python Requests library. My reasons for
choosing this library to contribute to are many-fold, and include
the friendliness of the maintainer
and the cleanliness and structure of the code. My primary reason, however, was
because I used it myself. And the reason I used it myself is very simple: its
API is brilliant (for those too lazy to Google: the definition of
API.
The Importance of Good APIs
When I started out writing code, not very long ago, I was in the position that
I suspect many new developers are in of being unable to tell the difference
between a ‘good’ API and a ‘bad’ one. The act of writing code was so much
conscious effort for me that I found no particular method of accessing a
resource ‘more intuitive’ than any other.
Initially this led me, with all of the arrogance of the man who doesn’t know
how little he knows, to dismiss the idea that there existed too much
variation in API design. My particular train of thought rested on the notion
that the API would fundamentally be shaped by the implementation, and that
there were probably not many possible implementations for a given
functionality (a notion I now consider to be laughable).
Then I tried to write a simple web scraper in Python.
I wanted to grab the entire archive of XKCD’s images, and save them with
numbered filenames. The experienced amongst you are already scoffing at the
simplicity of the task, but for me at the time it was a huge hurdle, and
required me to sit down and seriously think about how to do what I wanted to
do. The first step for me was to see what built-in modules Python had that
could help me do the job (and in fact this is still my intuition now). This
led me to
urllib2.
At the time, I had what I would consider a ‘barely functional’ knowledge of
HTTP (although, to be fair, I still think there’s a huge amount of HTTP I
don’t know, and even more I don’t fully understand). I knew about
HTTP verbs, so
I knew what query I needed to make to get the page I wanted. Despite that,
I spent ages sitting, working through that documentation, in order to get the
web data I wanted.
A few days later, I was browsing the web when I noticed someone praising
Requests. I was intrigued, and looked into it. When I saw
the comparison with urllib2, my jaw hit the
floor. This is brilliant!, I thought.
I didn’t know the half of it. Eventually, I swung around onto the GitHub repo
for Requests, and (thanks to the clean structure of the module) was able to
quickly find where the API was declared.
When I saw this,
I was astounded at how simple and easy to understand this was. The fact that
it uses an astonishingly tiny number of methods and is very well documented
is certainly helpful, but the more helpful feature is the fact that the API
has an almost perfect 1-to-1 mapping to the underlying behaviour. You want to
make an HTTP GET? requests.get. Want to make a POST? requests.post.
Clearly, even in my infantile programming state, I was able to see that there
was something about this API that was different to urllib2. Each did the same
thing, but somehow Requests was intuitive where urllib2 wasn’t. Clearly, this
API had whatever ‘x-factor’ makes APIs good.
The goodness of this API translated into immediate productivity improvements.
Use a quarter of the lines of code of urllib2 to get the same job done. The
fact that each method takes the same arguments means fewer references to the
documentation to write the correct code. The simplicity of Requests is such
that it makes investigation using the interactive interpreter (or, even
better, bpython) trivially easy, which
means you can prototype faster. For web-based Python work, I have never found
a library since that made my life so single-handedly easier than Requests did
and continues to do.
So, how did @KennethReitz manage to
compose so good an API when the authors of urllib2 failed?
API-Oriented Design
The answer is surprisingly simple: design the API first. Before writing any
library code whatsoever, work out how you’d like the user of the library to
interact with it. What methods will it use?
Will it use classes? Should the developer have
to set environment variables or write code to initialise the library? (Hint:
the answer to that last one is almost always ‘no’.) Sketch out on paper the
function calls you’d make on your ideal library to achieve a variety of
common tasks. This should include the names of the methods. Think carefully
about the functionality your library is providing. Does it wrap some protocol?
Should you emulate its syntax, or frame it in terms likely to be familiar to
the developers using it? (As an example of the second, consider all
ORMs.)
When you’ve made the design decisions for that, sit down and write the
function and class declarations required. Don’t put code in them, just the
declarations. Put that in a file (e.g. an api.py file, or a .h file for C)
and then leave it alone. Resist the urge to add anything to that at all.
You want a minimal API that exposes all the necessary functionality in the
simplest and most intuitive way possible. If you add to it to make writing the
library easier, then you have copped out and your API just got worse.
Once you have your API, write all of the necessary code to make it work. This
code doesn’t have to be at all intuitive: use magic,
use operator overloading, use import hooks, anything. Just make sure that your
API works.
The better the coder you are, the better your underlying code will be. As it
turns out, I am more than capable of debugging most minor problems people have
with Requests (and do my best to do so, to save smarter people from having to
spend the brain time on it), in large part because the codebase is so well
structured. But even the worst coders are capable of writing a passably good
API by hiding the complexity away from the API itself, and burying it in the
library.
I call this API-oriented design, although I’m sure someone else has the claim
to using that name first. If you do it, you will be writing libraries that
other developers will want to use. This is because the reality of the
situation is that most developers just want to use a library to do one small
part of their thing, and have no interest in developing the library or
understanding its structure.
So don’t force them to. Be intuitive and simple, and the rest will follow.