from __future__ import *

Python 2.6 was released yesterday, which has tons of cool new features including a new json library and a new multiprocessing library. The json library is basically simplejson (from a few months ago) minus the Python 2.4/2.5 support and refactored to take advantage of the latest future-compatible features (such as the new str.format method instead of using % format interpolation).

The only downside is that I wasn't able to get the latest simplejson 2.0.1 performance enhancements into the first release of Python 2.6, but expect them for Python 2.6.1! Don't worry though, simplejson should install just fine with Python 2.6 if you need the speed. If not, you don't have any dependencies for JSON anymore, just change your imports:

# Use simplejson or Python 2.6 json, prefer simplejson.
try:
    import simplejson as json
except ImportError:
    import json

print repr(json.dumps({'key': 'serialize this!'}))
print repr(json.loads('{"key": "deserialize this!"}'))

One of my favorite features in json/simplejson is the shell command that will validate/pretty print JSON. Great for debugging, and it will be sweet to have it available on every box with Python 2.6+!

A really cool tip if you're using Mac OS X is that you can access the string version of the pasteboard from the shell with pbpaste and pbcopy. This will take the JSON from your pasteboard and replace it with a pretty-printed version! If you're using simplejson just change json.tool to simplejson.tool. I'm sure this would be super handy in a text editor macro too:

$ pbpaste | python -mjson.tool | pbcopy

Slides from my Ad Serving in Erlang talk at CUFP 2008 are available here: http://undefined.org/cufp2008/

I had a great time at ICFP and CUFP and it was really inspiring to see what other people are doing with functional programming. I look forward to attending in the future! I think I'll have to write an app in Haskell first though, so I can understand more of the ICFP talks ;)

If you're interested in Erlang I also recommend that you check out my other erlang posts particularly the video and slides from my Exploring Erlang @ C4[1] talk!

simplejson (documentation) is a simple, fast, complete, correct and extensible JSON (RFC 4627) encoder/decoder for Python 2.3+. It is pure Python code with no dependencies, but features an optional C extension for speed-ups.

simplejson 2.0.1 is a minor update:

• Fixes long encoding regression introduced in 2.0.0
• Fixes MinGW build regression introduced in 2.0.0

I didn't manage to do a simplejson 2.0.0 release notice here but that was a MAJOR update!

• optimized Python encoding path
• optimized Python decoding path
• optimized C encoding path
• optimized C decoding path
• switched to sphinx docs (nearly the same as the json module in python 2.6)

Even without the C speedups, it's several times faster. With the C speedups, it's WAY faster. I highly recommend that everyone update their frameworks to use the latest code.

For the primary benchmark I was concerned with (some internal data structures that Mochi uses) encoding is about 4x faster than cPickle (!), and decoding is just a tiny bit slower.

There's little excuse to use other libraries now. The fastest alternative I'm aware of is cjson 1.0.5, which is currently a little faster than simplejson 2.0.0 when decoding but is actually slower than simplejson 2.0.0 at encoding (the more important path for people vending web services). In both cases, simplejson provides more flexibility (e.g. the decoding object_hook), so a slight speed cost when decoding is probably to be expected.

There aren't really any API breaking changes, but when decoding a str input it will return str objects instead of unicode if the str is all ASCII with no escaped characters. I'm not aware of any scenario other than doctests where this could be a problem.

simplejson is a simple, fast, complete, correct and extensible JSON (RFC 4627) encoder/decoder for Python 2.3+. It is pure Python code with no dependencies, but features an optional C extension for speed-ups.

simplejson 1.9.3 is a minor update:

• Decoding is significantly faster (for our internal benchmarks)
• Misc. bug fixes

simplejson is a simple, fast, complete, correct and extensible JSON (RFC 4627) encoder/decoder for Python 2.3+. It is pure Python code with no dependencies.

simplejson 1.9 is a minor update:

• Rewrote test suite with unittest and doctest (no more nosetest dependency)
• Better PEP 7 and PEP 8 source compliance
• Removed simplejson.jsonfilter demo module
• Misc. bug fixes

There is one backwards incompatible changes in this release:

• simplejson.jsonfilter is no longer included

I had the privilege last night of going to Google's Campfire One where the Google App Engine was launched, which is basically a service that I've been secretly hoping that Google would release for the past three years. App Engine is going to change everything -- as soon as they come up with a pricing model, anyway. I'm sure whatever it is will be more than competitive with Amazon's offerings, which isn't really worth any price given that they can't keep it fully operational. Yesterday was the perfect day for EC2 to fall over again, they might as well shut it off altogether once Google gets this service into production ;)

So why does Google App Engine change everything? I don't have a lot of time to spend here but a few key points:

• Single sign-on for Google users. Everyone with a gmail account is already registered for your service. You have no idea how cool this actually is :)
• BIGTABLE. My god. I would've spent in excess of $100k to have access to this part of Google's infrastructure and saved money. Scaling SQL databases sucks. If you have the kind of access patterns that we do, databases designed for OLTP are simply not suitable and it's a real pain to try and make it work. The fact that schema is managed directly in the code and that schema upgrades look awfully painless is a huge extra bonus.
• Works locally, deploys globally
• Python and WSGI!
• No more going to the data center, provisioning bandwidth from telcos, etc.
• Payment is surely coming. No more PayPal or Verisign or whatever.

In the same way that Google Apps (and Mac OS X) have enabled people to run without IT departments, Google App Engine is going to let them go big without an ops department. With the current imposed limits I can't prove this theory at Mochi Media, since everything we do is beyond the scale of their current quotas, but maybe I'll allocate some of my "infinite spare time" to ditch this Wordpress crap and try it out for my blog while they sort that out :)

The minus is that this project is actually probably pretty horrible for open source. Yahoo and the rest of the Hadoop team have their work cut out in making that stack competitive with this. If they don't, Google is going to own scale for a while. While MySQL and PostgreSQL still have some years left in them as people learn how to write scalable apps, I can't see that model lasting very long now that you don't have to be in Google's employ to use better solutions to the data problem.

simplejson is a simple, fast, complete, correct and extensible JSON (RFC 4627) encoder/decoder for Python 2.3+. It is pure Python code with no dependencies.

simplejson 1.8.1 is a major update:

• Optional C extension for accelerating the decoding of JSON strings
• Command line interface for pretty-printing JSON (via python -msimplejson)
• Decoding of integers and floats is now extensible (e.g. to use Decimal) via parse_int, parse_float options.
• Subversion and issue tracker moved to google code: http://code.google.com/p/simplejson/
• Misc. bug fixes

There is one incompatible change:

• "/" is no longer escaped, so if you're embedding JSON directly in HTML you'll want to use .replace("/", "\\/") to prevent a close-tag attack. Sorry, but the in-HTML use case just isn't worth the bloat in everyone else's documents.

So it seems that I made the wrong decision for our uplink at Mochi Media. We have a gigabit line to Cogent at 365 Main. They've been mostly good so far (and cheap), but their recent spat with Telia has cut us off to parts of Europe (at least Sweden and Denmark). Not cool.

Anyone have any recommendations on other providers we should talk to? I don't want us to be dependent on Cogent for a minute longer than we have to be.

The video of the Exploring Erlang talk I gave at C4[1] last August is up on the interwebs! It only took seven months :)

Video:

http://rentzsch.com/c4/c41VideosAvailable

Slides and source:

http://bob.pythonmac.org/archives/2007/08/11/exploring-erlang-c41/

The float printing options that ship with Erlang all suck. You either don't have sufficient precision, or you have an extremely verbose representation:

1> io:format("~w ~.18g ~g ~s~n", [0.1234567, 0.1234567, 0.1234567, float_to_list(0.1234567)]).
0.123457 0.123456700000000003 0.123457 1.23456700000000002548e-01
2> io:format("~w ~.18g ~g ~s~n", [0.1, 0.1, 0.1, float_to_list(0.1)]).
0.100000 0.100000000000000006 0.100000 1.00000000000000005551e-01

mochiweb includes an implementation of the algorithm from the "Printing Floating-Point Numbers Quickly and Accurately" paper in the mochinum module (which is standalone, if you just want to use it without the rest of mochiweb):

3> io:format("~s~n", [mochinum:digits(0.1234567)]).
0.1234567
4> io:format("~s~n", [mochinum:digits(0.1)]).
0.1

Problem solved :)