Played a bit with Microsoft Bond today. It’s a framework for data serialization, similar to Protocol Buffers.
My goal was to find out how much information I could fit into an Orleans grain state persisted Azure Table Storage. A Table Storage row is limited to 1MB, but the limit in Orleans is slightly lower at 983040 bytes (or fifteen 64KB columns) per grain state.
The default, version-tolerant serializer uses JSON. It’s familiar and easy but JSON is not a very compact serialization format and text is stored encoded in UTF-16 in Table Storage so we actually have only 491520 characters at our disposition.
With JSON I was initially able to store about 10K items in the state. Using a custom JSON converter to serialize the items as arrays (instead of objects), I was able to increase the number of items to 40K.
Then with protobuf-net instead of JSON, I could increase the number of items that fit into the state to about 60K.
With Bond, the number of items went up to 100K. The Bond serializer in Orleans uses the Simple Binary protocol
Simple Binary A binary, untagged protocol which is a good choice for storage scenarios as it offers potential for big saving on payload size.
Advantages of Bond: 10x increase in storage capacity compared to JSON in this scenario.
Inconvenients: A tool is required to generate C# classes from Bond schemas.