Null Disquisition

Enable WebGL in ChromeOS

Once you've enabled Developer Mode, you can startup a Chrome instance with WebGL enabled. Run "shell" from within crosh (ctl+alt+t) and then run:

/opt/google/chrome/chrome --enable-webgl

Then try loading up a WebGL demo: Cooliris Wall

YMMV

Cr-48

(download)

Click here to download:

cr-48-tHpjrnnxaIzxGvvmgrEi.zip (377 KB)

First Snow

(download)

Click here to download:

p69.mov (4.74 MB)

MongoDC - Afterthoughts

Cool conference, clearly lots of smart ppl working on this product. I think they are trying to do too much too soon (geospatial indexing, full text, etc). I'd like to see some of core pieces bolstered before features like this. Maybe they can work on lowering mongoDB's pH and getting closer to ACID.

MongoDC - ACID

So, what about command isolation? "We don't do isolation". What about command level atomicity? Nope, only document level atomic updates.

How do you guarantee consistency with replica sets in a sharded environment? Only read from the master.

And we all know single server durability is not yet possible.

ACID fail

MongoDC - First update

Some good talks here at MongoDC. Starting to realize that MongoDB isn't really a DBMS at all, but really a structured document store with flexible indexing and querying

MongoDC - eCommerce

Spoke with the CTO of Totsy about the semantics of their eCommerce app. They use a document per inventory item rather than keeping a "count" attribute for each inventory. This approach reduces resource contention on inventory documents from atomic document updates, but increases the number if documents and redundant data.

Seems the theme of the day is: redundant data

Stone Mt, NC

WebSockets in Python

Since the dawn of AJAX, web developers have longed for persistent server-side connections. For a while Comet was hailed as the bastion of “server push”, but deep down we knew it was just a hack. Now finally, years later, we have an API and a protocol being standardized for socket connections between the browser and the server – aptly named, WebSockets.

WebSockets are bi-directional communication channels that run on single TCP sockets allowing communication between the client and the server. Since they behave like regular INET sockets, we should be able to easily implement them with existing tools. However, when I was looking for example implementations in Python, I didn’t find anything that quite satisfied me.

Python sockets module

Now don’t make the mistake of thinking I’m a systems programmer. I have never written a low-level network application like this, and in fact this is my first time playing with sockets or select in Python. The root of all of this is the WebSocket itself, which is just a socket.

import socket
websocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
websocket.bind(("localhost", 9999))
websocket.listen(5)

That’s all you need to get the WebSocket up and running. Granted, it’s not very useful since you can’t connect to it (no handshake), but it’s a WebSocket nonetheless. When a client connects to the socket, it initiates the handshake with the following

GET / HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: localhost:9999
Origin: file://
Sec-WebSocket-Key1: x   d3L703 2  {63 k  L1( 90
Sec-WebSocket-Key2: ^    14   +40Z7R<12om I8  0[

??????????????

And expects a response in a similar form:

HTTP/1.1 101 Web Socket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
WebSocket-Origin: file://
WebSocket-Location: ws://localhost:9999/
Sec-Websocket-Origin: file://
Sec-Websocket-Location: ws://localhost:9999/

??????????????

The “?” are random bits used in the challenge/response part of the handshake. Interesting note: In addition to failing to do the Challenge/Response, Chrome looks for the “Websocket-X” headers, while Safari (correctly) looks for the “Sec-Websocket-X” headers.

Here’s my full standalone WebSocket server: http://gist.github.com/512987

I won’t delve into the details of the implementation, namely because I’m sure it’s suboptimal. I was pretty happy with Challenge/Response piece. I read the spec from IETF and implemented it, nice and simple. Aren’t open standards great? I ended up having to do the handshake because Safari 5 won’t let you use a WebSocket otherwise.

Stay tuned for everyone’s favorite asynchronous demo: a chat program!

-David

JSON Encoding mongoDB Documents in Python

One thing that kept puzzling me about pymongo was that I couldn’t a serialize a document as JSON. Aren’t these things just fancy JSON objects on the backend? Well, they are – but ObjectId is part of mongoDB extension of JavaScript so there is no JSON-equivalent. And since Python only knows about the standard JSON spec, it won’t know what to do with ObjectId instances. When attempting to encode a Python dictionary which has an ObjectId as one of its values, I get a TypeError saying ObjectId “is not JSON serializable”.

My solution is to extend the JSONEncoder included in Python’s json module (in 2.6 or later)

It adds a special case to handle encoding an ObjectId into a literal “ObjectId” in the encoded JSON.

Custom JSON encoders can be used when issuing json.dump or json.dumps by specifying cls

json.dump(obj, cls=MongoEncoder)

E.g.,

>>> import json
>>> from pymongo.objectid import ObjectId
>>> from mongoencoder import MongoEncoder
>>> x = {'a':1,'b':"foo",'c':ObjectId()}
>>> print x
{'a': 1, 'c': ObjectId('4c4f4f5e2554c813e4000001'), 'b': 'foo'}
>>> print json.dumps(x)
Traceback (most recent call last):
[...]
TypeError: ObjectId('4c4f4f5e2554c813e4000001') 
  is not JSON serializable
>>> print json.dumps(x, cls=MongoEncoder)
'{"a": 1, "c": ObjectId("4c4f4f5e2554c813e4000001"), "b": "foo"}'

Viola! Enjoy