Hybrids Of Steel

This post is actually gonna be dedicated to reflective lighting term computation. I'd like to talk a bit about specular reflections. A number of approaches to solve the problem of reflections have been developed and all of them are based on 2 concepts. First of them works only with direct lighting term considering light to shade each point directly, with no contribution form other points forming objects of the scene. In this case we have pretty simple (and certainly fast) equations that are used to determine lighting term. But we have no ability to produce reflection because we don't have any information about surrounding geometry for every reflective point of the world. A well-known approach of cubemap (or environment map) reflections is widely used in this case. I'm pretty sure that everyone is familiar with the idea and I won't spend much time and screen space
on it, so here are some basic ideas:

• Cube textures are used to store information about surrounding objects per scene object
• Or may be even per a number of objects
• Allows fast and realistic reflections to be obtained
• Allows HDR on modern hardware
• A cubemap may be created in 1 pass on DX10+ HW ( 6+ MRTs and GS cloning are available)
  

This technique is straightforward, fast and used almost everywhere. It's main drawback is pretty obvious - it doesn't allow us to produce self reflections on objects as all the data from the cubemap is used on per-object basis. That means that we have all the surrounding objects in the cubemap except the reflective object itself and thus we're not able to fetch it's reflection from the cubemap. This leads to loss of realism of the scene since missing details can be easily noticed, especially when user knows where to look for them :) Such an absence of reflections can be easily noticed on vehicles as they have lots of overhanging parts such as rear view mirrors or aerodynamic spoilers or whatever else can be attached to a vehicle. Such failures IMO are more noticeable then, for example, SSAO so it's a good point to look around for a way of solving them.
And here the second concept comes into the game. The Ray tracing. It's natural to talk a lot here as well but I'll discuss points that are most valuable in my opinion.

• As we trace a ray through the scene we can collect contribution of multiple objects to final color
• We are actually able to balance speed/quality by setting a limit for the number of bounces
• We have to find an efficient way to store scene data since it can require very large amount of memory
• Since ray direction after each bounce is generally unpredictable we're gonna have some trouble with cache locality

So the idea is pretty common once again. We reflect a ray from the reflective object's surface just like in the case of cubemap reflection. But instead of just fetching a color from a cubemap using obtained ray direction, we march along the ray direction looking for an intersection with any geometry. When intersection is found, we're reflect once again and so on. After each reflection we also take into consideration diffuse color of the surface we're reflecting from to a final color.
Okay, thats seems rather easy even comped to 'classical' cubemap approach. The only problem unsolved is how are we actually supposed to store all those colors and normals for all the objects of the scene? Well, cubemaps seem to be a good choice. We just have to store multiple cubemaps to simulate 'depth' of the scene relatively to it's center, which is, at this point, our reflective object. Yeah, my new shiny car is a center of the Universe! :) Ok, whatever. To be able to store back facing polygons of scene objects (they are also reflecting our light rays!) we have to store 4 cubemaps per slice - 2 for normals ( one for front facing geometry and one for back facing ) and 2 for albedos ( yep, once again one per facing sign ) . But how do we determine what is a 'slice'? In general case we need some kind of depth peeling algorithm. The problem of depth peeling is that we don't now in general case how many iterations it will take us to peel all the layers. It's critical for out ray traced reflections idea, because each peeled layer takes 2 cubemaps  and it's easy to run out of memory budget. But it turns out that we don't need that general case. The aim of our work is to get object's self reflections. These can be obtained by tracing a ray against the object's geometry. That means that we need only 2 layers for self reflections (front faces + back faces) plus one more layer to store all the rest of the scene just like if we were making that 'classical' cubemap reflections. We don't even need to perform honestly that depth peeling algorithm as everything is already determined. So the brief summary:

• Create 6 cubemaps
• Render back facing polygons of reflective object to first pair of them storing color and normal
• Render front facing polygons of reflective object to second pair of them storing color and normal
• Render front facing polygons of the rest scene to the last pair of them storing color and normal
• You now have everything you need to race rays reflected from reflective object to gather reflection
• The last one seem pretty weird-written to me but I hope you've got it right :)

Pros:

• All the data is stored in a GPU-friendly and easy to access form
• Using fixed maximum number of iterations and dynamic branching of modern GPUs we're able to do the tracing in real time maintaining reasonable FPS
• That won't be so hard to integrate such a technique to an existing rendering pipeline as it's based on popular and well known conceps
• We can improve performance reducing size of the cubemaps

Cons:

• We still need large amount of RAM to store all our cubemaps
• We sacrifice quality to simplify algorithm. For correct and detailed multiple reflections we have to implement depth peeling 
• The cache locality problem is not solved. New ray's direction after reflection becomes even more random after 2nd or 3rd order reflections
• It takes a lot of time to prepare those 6 cubemaps for one reflective object so we need to optimize this somehow. Spreading the work into several frames may cause reflections to display in fits and starts and/or increase required amount of RAM if one is going to fix it using second (for each of the 6 of course) cubemap and lrp instruction in the shader. So this is a problem to think about. 

Well, I hope I haven't missed something really important and I'm pretty sure that I've missed something. So if you will point it out in a comment I'll try to give an answer or add a note in the post. Anyway the idea of using ray tracing for reflection computation should be clear now.
The diffuse term can still be obtained in the 'default' way, that means no rays, no tracers, but only good old Dot(N,L). Ray traced reflection can be then added using Fresnel's term as an 'intensity' coefficient. I personally do lerp Diffuse + Specular with Reflection. That seems to be rather nice.

As a reference, I'd like to advice an article named "Robust multiple specular reflections and refractions" by Tamás Umenhoffer, Gustavo Patow and László Szirmay-Kalos. You can find it in nVidia's GPU Gems 3, online version is available here. The article explains the idea in a strict form, provides necessary formulas, code snippets and optimization hints so I consider it to be very useful and a good place to start. All my work on multiple reflections is based on this article and I'm a bit disappointed that I'm not the one who came up with the idea first :)

Here are some results of my WIP implementation:

As I promised earlier, here's a link to The Gallery where you can fin more screenshots of the engine and it's applications.