In an earlier journal entry I explained the switch from storing maps in revel as large directories of files, to a one file per map, monolithic storage of rooms.
This transition is also going to be handy for content delivery and filtering based on what parts of the game (assuming a player opts to buy it in parts) they own.
I recently switched the Revel Editor over to using the frag-file format (after the 0.15 release on Sunday, heh heh.) and converted the current state of all maps into frag files.
There have been some interesting revelations concerning this.
It's not that much faster, but it's a start
In conjunction with clipping the map area of Garranshall a bit; and some improved performance from the frag file; I can render minimaps in passable time (about one minute for garranshall):
Ain't it purdy? well at least it auto-generates for me.
There are still a lot of cookie-cutter stubbed in areas, but those will get refined as time goes on.
While this image doesn't look like much, it is created by processing roughly 3.5million tile values.
For a sense of scale, each pixel is about the size of an in-game person.
Having a uniform concept of saved and unsaved is nice
There was always a bit of a weird issue when editing maps; they were loaded and edited at the file-level; where the rest of the workspace was in memory and therefore isolated from disk until saved.
This caused weird potentials such as, saving the workspace but forgetting to save an edited room; or vice versa.
Now that frag files remain opened collectively in the editor, shared and saved when the workspace gets saved; everything is on the same page now.
JSON is great, but a bit heavy for big arrays of data
It would seem my remaining speed issues deal mostly with the fact that the storage format for rooms is still JSON; aside from being much larger than it needs to be (when compared with binary equivilent data); parsing it takes a good bit of time, especially for the 4800 integers each room holds.
Given that room data is completely transient and never stated; the format I choose to use is pretty open; it was of course easy to use JSON during the prototyping phase; rooms could be built in a text editor quickly, and new fields could be added readily.
Given that things are more mature now, moving to a specialized binary format (base64 encoded for transfer sake) might be the way to go.
Currently a rather empty room file is about 10k in JSON; a hypothetical equivilent in binary would be about 5k, roughly 50% savings (mostly due to the ',' being present for each array element in JSON); even larger savings could come from RLE encoding layers; though this would sacrifice processing speed for data size.
More testing and consideration is needed before I make a concrete plan; my main goals are speed and ease of editing; reduced bandwidth/cpu load from server to client; and of course easy, solid implementation, especially at the browser level.
