Updated 1 Bit Voxels

_posts/2015-06-03-1bit-voxels.md → _posts/2015-06-03-1-bit-voxels.md

@@ -2,16 +2,16 @@
 layout: post
 category : voxel
 tagline: "or not..."
-tags : [voxel, graphics, sparse, octree]
+tags : [voxel, graphics, sparse, octree, bloom-filter]
 ---
 {% include JB/setup %}
 
 
 ## Introduction
 
-A recent Reddit post has inspired me to write a short entry on a method I developed to reduce the data needs of my voxel structure as far as possible. My goal was the commonly cited "1-Bit-Per-Voxel". From a pure information theory standpoint I strongly believed that this was impossible without encoding the data somewhere else in the algorithm. I still do, but I tried nonetheless to see how close I could get.
+[This recent Reddit post](http://www.reddit.com/r/VoxelGameDev/comments/37rc9m/what_did_you_wish_you_knew_when_you_started_out/) has inspired me to write a short entry on a method I developed to reduce the data needs of my voxel structure as far as possible. My goal was the commonly cited "1-Bit-Per-Voxel". From a pure information theory standpoint I strongly believed that this was impossible without encoding the data somewhere else in the algorithm. I still do, but I tried nonetheless to see how close I could get.
 
-I am currently working on a voxel based game (no, it's neither sandbox, nor first person) with the previous goal of realistic rendering of voxels sized in milimeters. Obviously this required lots and lots of data. A compression algorithm was needed, but there are few that suited real-time rendering. I started with [Gigavoxels](http://maverick.inria.fr/Publications/2009/CNLE09/) and [Sparse Voxel Octrees](https://research.nvidia.com/publication/efficient-sparse-voxel-octrees). They formed the basis, but still required lots of data. 
+I am currently working on a voxel based game (no, it's neither sandbox, nor first person) with the goal of realistic rendering of voxels sized in milimeters. Obviously this required lots and lots of data. A compression algorithm was needed, but there are few that suited real-time rendering. I started with [Gigavoxels](http://maverick.inria.fr/Publications/2009/CNLE09/) and [Sparse Voxel Octrees](https://research.nvidia.com/publication/efficient-sparse-voxel-octrees). They formed the basis, but still required lots of data. 
 
 After months and months of trying different things, I finally found a method that seemed promising. And it worked! At least at compressing data. My best attempt so far? 1.8-Bits-Per-Voxel compression. I felt like on an episode of Silicon Valley, except that my algorithm sucked. And here I wish to tell you why.
 
@@ -31,15 +31,17 @@ This will result in a very blocky representation of said triangle. To improve it
 
 Marching cubes and countour data both have their own problems. Furthermore I explicitly wanted square voxels. Ever since the good old Novalogic [Comanche](http://www.novalogic.com/novaworld/nl_clone/c3g/screens/Snap0009.JPG) and [Delta Force](http://www.novalogic.com/images/DF/screen4_large.jpg) days I just love the look and feel of voxels.
 
-So, more Voxels it is! But, adding more depth to our tree requires alot of data. We need to store pointers and as we go deeper, these pointers will need more space as well. So, it seemed obvious, at some level down the octree, I stopped using octrees. The octrees job was now to localise where in space we are and use some other data structure to fill that space with voxels (see red voxels below)
+So, more Voxels it is! But, adding more depth to our tree requires alot of data. We need to store pointers, colors... And as we go deeper, these pointers will need more bits to address the far away nodes. So, it seemed obvious, at some level down the octree, I stopped using octrees. The octrees job now was to localise where in space we are and use some other data structure to fill that space with voxels (see red voxels below)
 
 ![Sparse Voxel Octree of depth 3 and 2 more levels]({{ site.url }}/assets/images/sparse-bloom.png)
 
-This would be equivalent to 2 more Octree levels, except it is now of a different data structure which we can traverse using a [3D Bresenham's Algorithm](http://en.wikipedia.org/wiki/Bresenham%27s_line_algorithm).
+This would be equivalent to 2 more Octree levels, except it is now of a different data structure! A simple bit array would do the trick, but I wanted something fancy! 
+
+This data structure could then be traversed using a modified [3D Bresenham's Algorithm](http://en.wikipedia.org/wiki/Bresenham%27s_line_algorithm) in the ray-tracing routine. Simple.
 
 ## Bloom Filters
 
-And that structure is a bloom filter. Basically a bloom filter is nothing more than a lossy way of looking up hashes. Each point in space, when hashed in the bloom filter will return 1 of the following 2 outcomes:
+And that structure is a [Bloom filter](http://en.wikipedia.org/wiki/Bloom_filter)! Basically a bloom filter is nothing more than a lossy way of looking up hashes. Each point in space, when hashed in the bloom filter will return 1 of the following 2 outcomes:
 
 * This Spot in Space is NOT a Voxel
 * This Spot in Space might be a Voxel
@@ -48,13 +50,15 @@ Yes, might. The bloomfilter will only confirm the non-existence of a voxel. But
 
 However! Since we "pre-filtered" our space with a sparse voxel octree, the overall shape of our object we voxelized should still be the same! Furthermore, we can scale Bloom Filters in such a way, that the false positive rate is very small. Seems good!
 
+What we still need is a Hashing function. I decided to go with [MurmurHash3](https://code.google.com/p/smhasher/wiki/MurmurHash3). The murmur series is really fast and has a great distribution as was visualized in this [great StackExchange answer](http://programmers.stackexchange.com/questions/49550/which-hashing-algorithm-is-best-for-uniqueness-and-speed).
+
 ## Some Pictures
 
 So, what can be done? Let's first take a look of an early in-game picture I took a while back using said bloom filtered voxels
 
 ![7 Bit]({{ site.url }}/assets/images/7bit.jpg)
 
-At first glance, looks like a standard Landscape scene rendered with Voxels. This is in fact a 8192 x 8192 sized Terrain fully loaded inside the VRAM using the presented method. My pure Sparse Octree implementation took around 400 MB (non-optimized). The bloom filtered version here? 56 MB. That's roughly 7 Bit-Per-Voxel. Not exactly 1-Bit-Per-Voxel territory, but getting there. 
+At first glance, looks like a standard Landscape scene rendered with Voxels. This is in fact a 8192 x 8192 sized Terrain fully loaded inside the VRAM using the presented method. My pure Sparse Octree implementation took around 400 MB (non-optimized). The bloom filtered version here? 56 MB. That's roughly 7 Bits-Per-Voxel. Not exactly 1-Bit-Per-Voxel territory, but getting there. 
 
 The Catch? If you look closely, you will see minor artifacts (false-positive voxels floating around). Furthermore, a big problem with the bloom filtered voxels is, you can't store any data in them. Merely a "might be here" and "is not here". All the color data is stored in the Sparse Voxel Octree overlay ontop of the bloom filter. You can see this, as there are alot of 4 x 4 patches in the terrain of the same color. This could be circumvented by introducing procedural texturing or similar. 
 
@@ -65,18 +69,20 @@ The details of this are as follows:
 * 10% [false positive rate](http://en.wikipedia.org/wiki/Bloom_filter#Optimal_number_of_hash_functions)
 * 5 times hashed with [MurmurHash3](https://code.google.com/p/smhasher/wiki/MurmurHash3)
 
-Yes, the false positive rate does seem rather high. But, visually, it seemed acceptable to me. 
+Yes, the false positive rate does seem rather high. But, visually, it seemed acceptable to me. The seeds for the Hashing were tweaked in such a way to receive the best looking result. No magical numbers I can give, just trial and error.
 
 And the great thing is? It scales! If I wanted a lower false positive rate and higher detail, I simply would need to increase the Bloom Filter size! Voila! Almost like JPGs.
 
-I did say in the beginning that I managed a 1.8-Bits-Per-Voxel compression. Well. Here is a picture of that
+I did say in the beginning that I managed a 1.8-Bits-Per-Voxel compression at some point. Well. Here is a picture of that
 
 ![1 Bit]({{ site.url }}/assets/images/1bit.png)
 
-And now forget it ever saw the light of day. Yes, some tricks can be applied to reduce the artifact count significantly. For example in the above picture, all "free floating" voxels were not rendered, as I assumed that those must be false-positives. Booo...
+And now forget that this abomination ever saw the light of day (Voxelized with [Binvox](http://www.cs.princeton.edu/~min/binvox/)). Yes, some tricks can be applied to reduce the artifact count significantly. For example in the above picture, all "free floating" voxels were not rendered, as I assumed that those must be false-positives. This was done by doing a hash look up for all neighbouring voxels. Slow, but it did improve image quality quite a bit. 
 
 
-## Some Conclusion
+## Some Conclusions
+
+At first, I was super excited with this method. I believed for sure that I could reach 1-Bit-per-Voxel territory. But I tried and tried and failed. Anyways,
 
 The Good
 
@@ -96,7 +102,9 @@ And the Ugly
 * Artifacts... Artifacts everywhere
 
 
-So, DON'T use it. It's bad. But I do hope that it might have given someone out there the inspiration to come up with a really usable compression algorithm for our little voxels. Currently I am using something based on this idea, but far less memory efficient.
+So, DON'T use it. It's bad. But I do hope that it might have given someone out there the inspiration to come up with a really usable compression algorithm for our little voxels. 
+
+Currently I am using something based on this idea, but far less memory efficient. It's working really well, and if it turns out as hoped, I'll do another post. But sadly, I had to abandon my realism based voxel dream.
 
 ** Cheers! **