A little while ago I wrote a blog post talking about
Git hooks. As an example in
that post I wrote a post-commit hook that would minify and upload my static
files to S3.
This has spiralled a little bit out of control since then, and I finally drew
the line at maintaining that script. It was time to create something more
powerful. To this end I give you:
collectr.
About collectr
collectr has one goal: to control the management of static files. In its
current form it is a wrapper around the boto
library. It targets Amazon’s S3 as the back end data store. Its sole purpose in
life is to prepare and upload static files to S3 according to the user’s
wishes.
In the basic case, where all you want to do is upload some files in a directory
to S3, you can keep things simple:
import collectr
collectr.update('path/to/dir', 'bucket_name')But collectr allows much more than that. In its current early state, collectr
supports a few other behaviours: it can minify files, it can ignore files in
the directory, and it can apply S3 metadata. A very simple API controls this
functionality. For instance, you can apply metadata:
import collectr
directory = collectr.StaticDir('path/to/dir')
directory.metadata = {'Cache-Control': 'max-age=3600'}You can ignore files:
import collectr
directory = collectr.StaticDir('path/to/dir')
directory.ignore = ['.*.json']And you can minify.
Minification
Minification has the most functionality at the moment. To minify files, you
want to provide a dictionary. The keys of this dictionary should correspond to
the file extensions, and the values correspond to the command-line you want to
run to minify the files. This allows collectr to run using whatever
minification tool is your favourite (or alternatively, whatever you have
installed). For example:
import collectr
directory = collectr.StaticDir('path/to/dir')
directory.minifier = {'js': 'jsmin < {in_name} > {out_name}',
'css': 'yuicompressor -o {out_name} {in_name}'}Each of the files in the directory is checked, and if it has a matching
extension, the command-line is executed. It executes in its own shell, so
globbing and redirection should work fine. All you need is to remember to put
in both an in_name and out_name field in the format string. The out_name
is the same as the name of the file, with ‘-min’ appended to it. This will
likely become user-configurable in the future.
You might want to keep your pre-minified files in a separate directory. You can
do that: call the directory that contains the pre-minified files the input
directory like this:
import collectr
directory = collectr.StaticDir('path/to/dir')
directory.input_directory = 'other/dir'
directory.minifier = 'yuicompressor -o {out_name} {in_name}'The out_file format string variable will move from the first directory to the
second, so if you use this feature don’t forget to use out_file!
The above also shows the special-case where the minifier property is a string.
If you do this, collectr treats the string as a command-line to be applied to
both Javascript and CSS files. This is ideal for using with something like
yuicompressor.
You might have noticed that there’s nothing here that specifies minification.
This is correct: in fact, you can do any form of pre-processing you like using
this method. Enjoy the freedom!
Command-line tools
As a freebie, just to show a simple example of how you can replace your bash
scripts with something nicer, collectr comes with a simple script entitled
collect_static (hi there Django!). This
script is intended to be as close a match as possible to the script I wrote for
my post-commit hook. It’s roughly the same length, but the vast majority of it
is code for
argparse
and some if __name__ == '__main__' stuff. In fact, there is only one line of
code to handle all of the rest. collectr is really that easy.
The Future
Currently collectr is in version 0.0.4: that is to say, it’s only just started!
I’ll be working on this for the near future, and I’d like people’s help as
well. If you have an idea for a feature, I want to hear it: even better if you
have a Pull Request that implements that feature. I’m looking at supporting
other back ends and providing more advanced functionality, so let me know what
you want from something like this and I’ll provide it.
Feedback is always appreciated. Let me know what you think, and if you’re using
collectr!