Cpython excels in a very specific area; asynchronous (aka non-blocking) network servers. Back in the days before multi-threading was invented, asynchronous designs were the only available mechanism for managing more than one connection in a single process. Although it was possible to spawn off new processes to service incoming requests, that made communicating and sharing information and resources (e.g. database connections) between server components more complex and intractable.

With an asynchronous design, all of the incoming socket connections are managed by a single process. Current socket connections are kept in a simple data structure, such as a list, along with state information relating to the connection, and perhaps a handler, i.e. a piece of code that services the incoming requests. The single process used the unix select call (or its descendants such as poll) to monitor the state of every connection simultaneously, and then invoking the associated handlers when a network event of interest to them occurred (i.e. following a reactor pattern). And therein lies the problem.

When the asynch server calls the handler to service a request, that handler cannot delay before it returns; it must return very quickly, or it holds up the entire server. Returning quickly is easy if you’re doing trivial processing, e.g. an echo or chat server. But if you need to connect to a database, or even do something as simple as a DNS lookup, that requires forming another network connection, sending a request packet and waiting for a response. Which means that if your database or DNS library waits until the response arrives, hundreds of milliseconds or even seconds later, then the entire network server is held up, and clients must wait possibly seconds for service; clearly unacceptable. So with asynchronous servers, you must always use libraries that are aware they are operating in an asynchronous environment, are able to surrender control while they are awaiting a network response, and can add themselves to the server reactor, so that they can be notified when their response arrives.

While asynchronous architectures thrived in many areas, they proved to be overly complex for the mainstream programming market. Making all libraries asynch-aware, and tracking all of the interactions between them, was complex, difficult to understand, and thus error-prone. Multi-threading was invented to circumvent these problems. Instead of a single thread of execution controlling network service, separate threads of execution could be created, giving the illusion that there were multiple copies of the server code executing simultaneously, inside the same process, sharing memory, etc. Actually all the threads were running on the same linear processor, and the available timeslices were divided between them, in much the same way that most modern operating systems timeslice a single processor between multiple processes. The threads would generally run the same instructions, but would have their own program counter and call stack. Threads made programmers lives much easier, since threads could be designed to execute in a linear fashion. If the thread needed to pause at any stage, e.g. to await the response to a network request, it simply suspended itself, and a different thread would be selected to be given CPU time. At low cost.

Well, not quite.

There are problem with threads. Threads can have very large memory overheads, perhaps measuring in megabytes. Threads must synchronised, so that they don’t corrupt shared data structures, e.g. race conditions, which can be very hard to recognise when they happen, and complex to avoid in design. And threads can deadlock, which can be enormously difficult to debug.

The overhead of threads is such that it is rare to see a threading-based server outperform a well designed asynchronous server, on identical hardware. Threaded servers rarely service more than a couple of thousand network connections simultaneously; many can only service in the hundreds per second. There have been incredible increases in all forms of hardware capacity in recent decades; commodity boxes come with gigahertz processors, gigabytes of memory, terabytes of hard disk, and gigabits of network bandwidth. But although threaded-based servers have vastly improved in design, they still cannot match the performance of asynchronous servers, and address the C10K problem.

The C10K problem is a simple one; how to service 10,000 simultaneous network requests. I refer to you the C10K website for enormously detailed technical information on this complex problem. Suffice it to say that asynchronous architectures, with their much smaller memory usage, and lack of need for locking, synchronisation and context-switching, are generally considered to be far more performant than threaded architectures.

So where does python come in? It so happens that python is an excellent language for writing asynchronous servers. Python’s expressiveness and conciseness make it an excellent choice for developing simple and elegant solutions to the problems posed by asynchronous designs. Python was first published in 1991, years before java, and did not develop multi-threading until much later; python’s threading design borrows much from java’s design. Before threading came to python, asynchronous was the design of choice; none more so than Sam Rushing’s medusa, which is still considered the archetypal asychronous design, and was adapted into python as the asyncore module.

But asyncore is basic, and doesn’t provide non-blocking support for the protocols necessary in a real-world application; database protocols, DNS lookups, etc. But there are two heavyweights in the python asynchronous world that have the required support, in a robust form. Zope Corporation was there first with their venerable Zope product, arguably the grand-daddy of all python web frameworks. But dissatisfaction with Zope’s complexity lead to a new initiative; the Twisted Matrix. I won’t try to introduce Twisted here; take a look at the names of the sponsors on the Twisted website to see how important a framework it is.

And that’s why I’m excited; Twisted could soon be running on jython!

Back in the old days, jython 2.1 could not do aysnchronous socket operations. Jython 2.1 was written in the days when the only socket support in java was java.net, which was threading oriented. It was only in version 1.4 of the JVM that asynchronous support arrived, in the shape of the java.nio package. Now java has robust asynchronous support, after several years in the field.

I saw the possibility of using java.nio to enable jython to do asynchronous sockets a couple of years ago. I developed the support, and checked it into jython 2.2 last year, in the hope that it would draw the attention of developers of the big cpython asynchronous frameworks. Java has some distinct advantages over cpython in high performance network serving world, with the most obvious being the ability to run on all cores of multi-core processors simultaneously. Cpython is restricted in this regard by it’s Global Interpreter Lock, which prevents multiple threads running pure python code from running on multiple cores or processors simultanously. In java, and thus in jython, every single core in the multi-core system can be running its own reactor, actually simultaneously.

It’s looking more like those cpython asynchronous frameworks could be coming to jython soon.

The founder and chief architect of Twisted, Glyph Lefkowitz, logged three bugs (1119, 1120, 1121) against the jython socket module last week; I fixed them yesterday. I’m glad to say that they were all relating to corner cases and unusual usage; so far there have been no reports of serious bugs in the main functionality of the socket module. So hopefully the road to running twisted on jython is clear and obstacle free, at least at the level of socket communications.

And the Zope people have an ongoing project to get Zope running on jython; see the Zope-dev archives for ongoing updates.

Yes, exciting times indeed!

If you are interested in reading further about asynchronous I/O in the java.nio package, I thoroughly recommend Ron Hitchens book Java NIO.

1. Introduction
2. Can the browser handle GZIPped content?
3. GZIP vs ZLIB
4. Content-encoding vs Transfer-encoding
5. Content-length
6. Character translation on Windows and OS/2
7. OutputStream vs. Writer in jython servlets
8. Some links for related reading
9. Some sample CGI code
10. Some sample code for mod_python
11. Some sample code for jython servlets

Introduction

These are some notes on how to do HTTP compression, i.e. compress content before it is sent to the user. The benefits are

1. You transmit less bandwidth. If the amount of bandwidth you can serve is limited (i.e. you have fixed allocation of X gig/month), then compressing your content before you send it can dramatically lower your transmitted bandwidth.
2. Your client receives less bandwidth, meaning that pages load faster for them. This is particularly important for people using modems. (Although some modems already do compression. I have no idea how efficient modem compression is compared to gzip. See the Links section for a link to an article which discusses HTML compression vs. modem compression.)

Some points to remember

1. Although you save on bandwidth transmitted from your server, it comes at a cost of increased CPU usage on the server.
2. Compressing images probably isn’t worth it. Specialised image compression formats such as GIF, JPEG, and PNG are optimised for the content they carry, and probably cannot be much improved on. So it’s probably not worth compressing images using these techniques.
3. The best bandwidth saving will be achieved on textual files, i.e. mime types like <text/plain>, <text/html>, <text/xml>.

I’m not going to go into the details of the rationale or the mechanisms of compression here, I’m just going to stick to telling you how to do http compression in python. If you want to know about the nuts and bolts of compression and http, I suggest you read

1. RFC2616: Hypertext Transfer Protocol — HTTP/1.1
2. RFC1952: GZIP file format specification version 4.3

Can the browser handle GZIPped content?

Not all browsers can handle compressed content. Old browsers that support HTTP version 0.9 are unlikely to support compressed content. Compression was optional in version 1.0 of the HTTP spec, so only some HTTP 1.0 clients support it, most notably versions 4.x of Netscape Navigator. Support for compression is compulsory in HTTP 1.1, so all HTTP 1.1 clients should support compression. HTTP 1.1 clients include Internet Explorer 5.0 and up. Although I don’t know if Internet Explorer 4.0 is HTTP 1.1 or 1.0, I’m sure IE 4.0 supports gzip compression.

Fortunately, you don’t have to guess or keep a list of which browsers do or do not support compression, since the browsers themselves will inform you of it in their HTTP request. Browsers that support compression will inform you via the HTTP header Accept-Encoding (Check your browser’s headers here). Possible compression methods, as mentioned in RFC2616, include gzip, compress, deflate, and identity. Note that the last one, identity, means no compression. Note also that I’m only going to cover gzip in this document.

So if a browser supports compression, its HTTP request will include a header similar to one of the following

Accept-Encoding: gzip
Accept-Encoding: gzip, deflate

So you need to get the value of the Accept-Encoding header, and check if it contains gzip. If it does, then you can send gzip-compressed content to that browser. Here is some CGI code that checks the content of the header

import string
import os
 
acceptsGzip = 0
try:
    if string.find(os.environ["HTTP_ACCEPT_ENCODING"], "gzip") != -1:
        acceptsGzip = 1
except:
    pass

And here is some mod_python code that checks the content of the header.

import string
 
def acceptsGzip(req):
    """
        Checks if a browser request indicates that the browser will accept
        gzipped content in reply.
    """
    if req.headers_in.has_key('accept-encoding'):
        encodings = req.headers_in['accept-encoding']
        return (string.find(encodings, "gzip") != -1)
    else:
        return 0

And here is some jython servlet code that checks the content of the header.

import string
 
def acceptsGzip(req):
    """
        Checks if a browser request indicates that the browser will accept
        gzipped content in reply.
    """
    encodings = req.getHeader('accept-encoding')
    if encodings != None:
        return (string.find(encodings, "gzip") != -1)
    else:
        return 0

GZIP vs ZLIB

There are a few compression libraries that come with Python. The one which is used in HTTP compression is GZIP. ZLIB, although it is used by GZIP to do part of its job, should not be needed directly in your code.

When a file is compressed for transmission through http, it must be preceded by some special header bytes and followed by some special trailer bytes. Conveniently enough, the python GZIP module constructs exactly those headers and trailers. The details of what those headers and trailers are can be found in RFC1952.

In order to compress using the GZIP module, you’ll need to use code something like this.

import gzip
 
def compressBuf(buf):
    zbuf = cStringIO.StringIO()
    zfile = gzip.GzipFile(mode = 'wb',  fileobj = zbuf, compresslevel = 9)
    zfile.write(buf)
    zfile.close()
    return zbuf.getvalue()

Note that this code compresses into a buffer held in memory, rather than a disk file. This is done through the use of cStringIO.StringIO().

You can vary the compression level by changing the value of the compresslevel parameter, with compresslevel = 9 giving the best compression but consuming the most CPU cycles, and compresslevel = 1 giving the least compression and also consuming the least CPU.

Content-encoding vs Transfer-encoding

When you’re sending compressed content back to the browser, you have to inform the browser of the compression. This is done by the header Content-Encoding. So you should include a header in your response that looks like this.

Content-Encoding: gzip

You don’t need to read any further in this section. The note below about different methods of declaring encoding is just here for interest and completeness.

Note: There is another possible way to communicate the encoding: the Transfer-Encoding header. According to RFC2616, the difference between the two headers is as follows

1. Content-Encoding should be used when the encoding is a property of the content. So if you were serving a static file that is always compressed, then this is the header to use.
2. Transfer-Encoding should be used when the encoding is a property of the message used to transmit the content. So you were sending a static file that is normally uncompressed, and were compressing it just so as to minimise bandwidth during the transmission, then this is the header you should use.

However, RFC2616 is unclear what encoding to use if you are generating dynamic content, which may only be transient in memory and has no lifetime beyond the HTTP request-response pair it was contructed for. In such a situation, an argument could be made for using either Content-Encoding or Transfer-Encoding.

Rather than try to resolve that (potentially unresolvable) issue, I would just like to point out that majority of software "out there" seems to have opted for using Content-Encoding, and that’s the choice that I’ve made as well. You are, of course, free to choose otherwise.

Content-Length

You also need to tell the client browser the length of the compressed content you are sending. This is done by sending a Content-Length header, like this.

Content-Length: xyz

where xyz is the COMPRESSED length of the content.

There seems to be some "folk wisdom" out there on the ‘net that you should send the uncompressed length of the content. This is wrong! RFC2616 is quite clear about this. If you are interested, read sections 7.2.2 Entity-Length and 4.4 Message Length.

As far as I can see, the definitive statement on this matter is at the end of section 4.4 (I have underlined the relevant statement)

When a Content-Length is given in a message where a message-body
is allowed, its field value MUST exactly match the number of OCTETs
in the message-body. HTTP/1.1 user agents MUST notify the user when
an invalid length is received and detected.

Character translation on Windows and OS/2

If you’re working in CGI, and on Windows or on OS/2, you need to be careful about character translation.

As you may be aware, Windows and OS/2 are different from other platforms in the way that they represent an end of line. Whereas most platforms, including *nix, represent line ends as an ASCII linefeed (hex 0×0A, octal 012, python escape string ‘\n’), Windows and OS/2 represent end-of-line as a sequence of two characters, an ASCII carriage return followed by an ASCII linefeed (hex 0×0D 0×0A, octal 015 012, python escape string ‘\r\n’). Therefore, when you print anything in python, or write anything to sys.stdout, using code like this

print "Hello World!"
sys.stdout.write("Hello World!\n")

then both Windows and OS/2 filter the characters, and turn all linefeed characters into a sequence of carriage return followed by linefeed.

This is fine when you’re printing text. But when you’re trying to send binary information, particularly a compressed gzipped file, then this translation will corrupt the binary content, and your transmission of gzip compressed content will fail. Therefore, you have to disable this character translation in order for transmission of gzipped content to work.

When you’re working in CGI, the best way to do this is with the “-u” command line flag to python. This was kindly pointed out by Richie Hindle, who says

…… There is a much simpler way to switch off character translation of the standard
channels. The python interpreter accepts the -u switch to mean “make the
standard channels both unbuffered and binary.” This is tailor-made for
CGI – change your shebang line from “#!…python” to “#!…python -u” and
everything will work without changing your code (and without relying on
platorm-specific modules like msvcrt). Responsiveness may even improve
due to the lack of buffering – and that’s also true on platforms like Unix
which don’t do character translation.

A less convenient method is to execute some platform specific code to disable character translation inside your script. On Windows (MSVC), you should execute some code like this

import msvcrt
import os
import sys
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)

I don’t know what the equivalent code is on Windows (CygWin) or OS/2. If anyone does, email it to me, and I’ll include it in this page.

OutputStream vs. Writer in jython servlets.

Under the J2EE Servlet interface, you have a choice of two different ways to output generated content. They are

1. Through an OutputStream (obtained by calling ServletResponse.getOutputStream()). OutputStreams do not carry out character translation on their output.
2. Through a Writer (obtained by calling ServletResponse.getWriter()). Writers carry out character translation on their output, i.e. they will change the value of bytes in the content output by the servlet, to ensure that it meets the character encoding requirements of the client.

Since compressed gzip content is a (dense) binary format, none of the output bytes should be translated. If any bytes are translated, the output may be corrupted and the recipient may be unable to decode it. Therefore, you must output compressed content through a (non-translating) OutputStream object.

Some links for related reading

• W3C: Compression and Performance

Some sample CGI code

So, without further ado, here is some sample CGI code that will transmit compressed HTML to gzip capable browsers.

#! /path/to/python -u
 
import string
import os
import sys
import gzip
import cStringIO
 
def compressBuf(buf):
    zbuf = cStringIO.StringIO()
    zfile = gzip.GzipFile(mode = 'wb',  fileobj = zbuf, compresslevel = 6)
    zfile.write(buf)
    zfile.close()
    return zbuf.getvalue()
 
def testAcceptsGzip():
    acceptsGzip = 0
    try:
        if string.find(os.environ["HTTP_ACCEPT_ENCODING"], "gzip") != -1:
            acceptsGzip = 1
    except:
        pass
    return acceptsGzip
 
def sendHtml(buf):
    sys.stdout.write("Content-type: text/html\r\n")
    if testAcceptsGzip():
        zbuf = compressBuf(buf)
        sys.stdout.write("Content-Encoding: gzip\r\n")
        sys.stdout.write("Content-Length: %d\r\n" % (len(zbuf)))
        sys.stdout.write("\r\n")
        sys.stdout.write(zbuf)
    else:
        sys.stdout.write("\r\n")
        sys.stdout.write(buf)
 
myHtml = """<html><body><h1>hello compressed world!</h1></body></html>"""
sendHtml(myHtml)

Some sample mod python code

And here is some sample mod python code that will transmit compressed HTML to gzip capable browsers.

import string
import os
import sys
import gzip
import cStringIO
from   mod_python import apache
 
def compressBuf(buf):
    zbuf = cStringIO.StringIO()
    zfile = gzip.GzipFile(mode = 'wb',  fileobj = zbuf, compresslevel = 6)
    zfile.write(buf)
    zfile.close()
    return zbuf.getvalue()
 
def testAcceptsGzip(req):
    if req.headers_in.has_key('accept-encoding'):
        encodings = req.headers_in['accept-encoding']
        return (string.find(encodings, "gzip") != -1)
    else:
        return 0
 
def handler(req):
    req.content_type = "text/html"
    myHtml = """<html><body><h1>hello compressed world!</h1></body></html>"""
    if testAcceptsGzip(req):
        zbuf = compressBuf(myHtml)
        req.headers_out['Content-Encoding'] = 'gzip'
        req.headers_out['Content-Length'] = '%d' % (len(zbuf))
        req.send_http_header()
        req.write(zbuf)
    else:
        req.send_http_header()
        req.write(myHtml)
    return apache.OK

Some sample jython servlet code

And here is some sample jython servlet code that will transmit compressed HTML to gzip capable browsers.

import  javax.servlet.http.HttpServlet
 
import  cStringIO
import  gzip
import  string
 
def compressBuf(buf):
    zbuf = cStringIO.StringIO()
    zfile = gzip.GzipFile(mode = 'wb',  fileobj = zbuf, compresslevel = 6)
    zfile.write(buf)
    zfile.close()
    return zbuf.getvalue()
 
def acceptsGzip(req):
    encodings = req.getHeader('accept-encoding')
    if encodings != None:
        return (string.find(encodings, "gzip") != -1)
    else:
        return 0
 
class compressor(javax.servlet.http.HttpServlet):
 
    def service(self, req, resp):
        resp.setContentType('text/html')
        myHtml = """<html><body><h1>hello compressed world!</h1></body></html>"""
        if acceptsGzip(req):
            binarychan = resp.getOutputStream()
            zbuf = compressBuf(myHtml)
            resp.setHeader('Content-Encoding', 'gzip')
            resp.setHeader('Content-Length', '%d' % len(zbuf))
            binarychan.write(zbuf)
        else:
            textchan = resp.getWriter()
            textchan.write(myHtml)