The HTTPbis have spoken, HTTP/2.0 is happening. Major websites are beginning to
adopt it (hello there Twitter!), and the spec is beginning to get nailed down.
If you’re unfamiliar with HTTP/2.0 and all the fun things it brings you, Ilya
Grigorik has been doing an excellent job evangelising for it. Take a look at
this talk if you want a deep dive into HTTP/2.0.
If you want a shorter discussion,
this article is also a great
reference.
HTTP/2.0 brings a number of advantages: it saves time setting up and tearing
down TCP connections, it compresses HTTP headers, it provides flow control and
connection multiplexing. For certain applications in certain use-cases, these
can represent a huge advantage over HTTP/1.1.
It’s time for Python to get support for it. With that in mind, I’ve
spent the last few weeks working on my brand new project. I give you:
hyper.
Hyper
Hyper is an attempt to build a pure-Python HTTP/2.0 stack that perfectly
matches the http.client API from Python’s standard library. This will allow
hyper to be a drop-in replacement for http.client, providing all the
HTTP/2.0 goodness to all Python programs without the need for potentially
painful rewrites.
How does it work? Like this:
from hyper import HTTP20Connection
conn = HTTP20Connection('twitter.com:443')
conn.request('GET', '/')
resp = conn.getresponse()
print(resp.read())That’s all it takes. The same API as http.client, but with all the fun of
HTTP/2.0. For more examples, see the documentation.
If you’re really excited, just go right ahead and download it using pip.
Alternatively, take a look at the code
on GitHub: it’s available under the MIT
license, so everyone should be able to grab it and play with it.
Features
At the moment, hyper is in a very early alpha (version 0.0.1). I’ve proved
that the mainline use-case works fine, but I’m sure there are plenty of bugs
lying around, and there are several features I’d like to implement that I
haven’t.
Nevertheless, the core feature set is already in place:
- Can make HTTP/2.0 requests.
- Obeys the HTTP/2.0 flow-control mechanisms.
- Verifies TLS certificates.
- Streaming uploads.
- Streaming downloads.
There are plenty of features planned, with the headline one being transparent
HTTP/2.0 and HTTP/1.1 interworking. Currently, hyper will only work against
servers that fully support HTTP/2.0. I’m planning to implement an abstraction
layer that should prevent the user from needing to know whether the server
they’re contacting supports HTTP/2.0: hyper will transparently fall back to
HTTP/1.1.
The only feature I have no intention of implementing support for is one of
HTTP/2.0’s headline features: Server Push. I’ll discuss this omission later.
Warning
Please note that hyper is in a very early alpha. I do not recommend you use
it in production: things will go wrong. However, I’d like as many of you as
possible to take it out for a spin and report any bugs you find on the
GitHub page. The more bugs you find the
better hyper will get.
Restrictions
Currently, hyper requires Python 3.3 or higher. This unfortunate restriction
is caused by the fact that earlier versions of the standard library’s ssl
module are totally braindead and do not support the TLS
Next-Protocol-Negotiation extension, which HTTP/2.0 requires. Until I can find
a good way around this limitation, hyper will remain limited to at least
Python 3.3. If you don’t like this (I certainly don’t), I encourage you to
take your complaint up with the Python core development team.
Additionally, you currently need to find out whether your target server
supports HTTP/2.0 or not before you point hyper at it, or it’ll raise an
AssertionError. This is a necessary guard in this early release, but this
will be improved in future releases.
Sadly, hyper is also currently inherently single-threaded. This means it has
some unusual behaviours, but also that you need to implement your own
threading mechanism on top of it if you want to use it in multi-threaded code.
A suggested implementation is sketched out in the documentation, and I hope to
provide example code in the near future. More long-term, hyper will become
thread-safe and you’ll be able to simply have global connections.
The Omission of Server Push
I should briefly touch on the omission of Server Push from hyper. I’ve been
on record in the past as saying that Server Push is fundamentally incompatible
with the way most Python HTTP code is written. All the big Python HTTP client
libraries are built primarily to be used from synchronous code: they expect to
send a request and then block until the response comes back. They absolutely do
not expect to receive multiple responses to the same request, and provide no
API for receiving these responses.
I don’t believe it’s possible to write a pleasant API that correctly handles
Server Push without using either an event loop or having hyper spawn its own
background threads. Until asyncio becomes a standard, I don’t think hyper
should force you to do the first, and it’s obnoxious for a library to spawn its
own threads. This means that, for the moment, hyper does not support Server
Push. I do not believe this to be a big loss for 90% of the potential users of
hyper, so I judge it to be an acceptable limitation. If you strongly
disagree, please get in touch so we can talk about why you see this issue
differently to me.
Development
hyper is in early alpha and under active development
on GitHub. I’d love it if people took the
time to download and try out hyper. I’m particularly interested to see what
kinds of bugs people hit: you’ll definitely hit them, and a detailed bug report
would be really phenomenal.
All contributions are welcomed. Even if you try out hyper and find no bugs,
drop me a line on Twitter and let me know that
you used it: at this early stage I need all the feedback I can get.
Finally, I’d like to specially call out
Sriram Ganesan, who graciously volunteered some of
his time to work on hyper when it didn’t work at all. He provided some useful
infrastructure when I was too busy to do it myself, and he made my work
immeasurably easier: thanks so much Sriram!