One of Requests’ most popular features is its simple proxying support. HTTP as a protocol has very well-defined semantics for dealing with proxies, and this has lead to widespread deployment of HTTP proxies. The vast majority of these proxies are ‘transparent’: that is, they sit on the message path and quietly capture HTTP messages before […]
How I Made Porn 20x More Efficient with Python Video Streaming
Porn is a big industry. There aren’t many sites on the Internet that can rival the traffic of its biggest players. And juggling this immense traffic is tough. To make things even harder, much of the content served from porn sites is made up of low latency live streams rather than simple static video content. […]
Big Redesign Time
I first posted in this blog 14 months ago. In the history of humanity, that’s not all that long ago, but for me it feels like an age. When I wrote that original blog post, I was still at university (just). I knew I was going to be a software engineer, but didn’t have much […]
Run-time method patching in Python
Often, when programming, we may want to change some already set behavior. This can be accomplished by sub-classing whatever classes we have and overriding those methods we are not happy with. While this works with our own code, what happens if we want to change third party code? Of course we can edit it so […]
Pillow 2-1-0 is Out
Pillow is a popular fork of PIL by Alex Clark and Contributors. PIL is the Python Imaging Library by Fredrik Lundh and Contributors Pillow 2.1.0 is out! With this release, the Pillow team has finally removed support for “import _imaging”, thus completing the move of PIL modules into the PIL namespace [1]. Many thanks to […]
Heroku Buildpack for Numpy, Scipy and Scikit-Learn
(TLDR: https://github.com/dbrgn/heroku-buildpack-python-sklearn) Background At Webrepublic we just launched a Python based system that among other things does comparison of large texts using tf-idf vectors in a multi-dimensional vector space and measuring the cosine similarity between them (see http://stackoverflow.com/a/8897648/284318). For this, we needed scikit-learn. During the deployment process, I discovered that one does not simply deploy […]
IPython Notebook
Last PyCon US (2013) I attended, there were many interesting talks, but if I had to talk about what’s happening, what’s the trend, between other things I would mention IPython Notebook. In a few words: it’s an excellent tool, you can have an IPython console within your web browser where you can write code snippets […]
Using Jedi with YouCompleteMe
(In case you’re not familiar with it yet, Jedi is an awesome autocompletion library for Python that tries to understand your code. There are editor plugins available for Vim, Emacs and Sublime Text 2.) The “official” Jedi plugin for vim is jedi-vim, but there’s another vim autocompletion plugin that supports Jedi: YouCompleteMe. In contrast to […]
Caching In Python Requests
I think I’ve made it clear in the past that I think Requests is awesome. At this stage it’s become a mature, feature-filled library that is more than capable of replacing urllib2 and friends in almost every situation you might be interested in. There are very few things that urllib2 can do that Requests can’t […]
Enums in Python
Until now, if you wanted to use enumeration types in Python you had to fall back to a class-attribute approach: >>> class Color(object): … RED = 1 … GREEN = 2 … BLUE = 3 … >>> print(Color.RED) 1 This has different downsides, for example with representation (Color.RED is represented as an integer, not as […]
List Outdated Dependencies with Pip
In the past, if you wanted to review your installed pip packages to see if there are updates available, you either had to use tools like pip-tools or check all of them manually. But since version 1.3 (released on 2013-03-07), pip supports the checking of dependencies with the new list command. In contrast to pip […]
Writing A Persona Identity Provider
As I said in my last post, I recently added Persona Identity Provider functionality to this blog. If you follow me on Twitter (and really, why wouldn’t you?), you might have noticed that this wasn’t an entirely smooth process. Mozilla are a great organisation, and there are lots of great developers working on the Persona […]
New Pyramid Site
For the first time in 10 years, http://aclark.net is not powered by Plone. Nothing against Plone: it’s still one of the greatest loves of my life (inasmuch as you can love a software and community, as I do). Why This was not the result of a revolutionary plan, rather more of an evolution. It happened […]
User-Agent Strings (or, Don’t Make Me Come After You)
A very long time ago (read: ten years ago), we were in-between the so-called First and Second Browser Wars. Internet Explorer had killed Netscape Navigator by taking advantage of their desktop monopoly and Scrooge McDuck-like financial reserves to install a free copy of Internet Explorer on every single computer in the world (basically). Internet Explorer […]
Requests’ Two APIs
Kenneth Reitz’s excellent Requests library has been praised, rightfully, for its excellent API. In fact, its API is so good that it’s been praised in a literary context, as well as by almost every programmer who has come across it. There is no question that this API is one of the best you can find […]