Storm ReBorn

So a year and a half passed since the last post here and the blog seems pretty like dead. I've decided to resurrect this one instead of creating new one. All the previous posts made by my friend and co-developer G@rd will be left intact as part of the history. I personally prefer to write in English 'cause it helps me a lot to improve my English. This will also allow foreign visitors to read the log. I hope that this attempt to record our development process will not fail, a least as fast as previous one :)

Okay, enough introduction. Here's what we've done during this year of silence. First of all, we've destroyed completely our previous work :). The reason of such a genocide is pretty usual. We found the architecture of our engine lame and unusable. Our first approach was based on the integration concept. This means that we had a monolith heavy core with a game client attached to it. The main pros of this approach are

• easy integration (all the parts of the engine are already put together by design)
• fast interaction (no LoadModule, everything is static)

The main cons, however, came out as

• extremely long build time (up to 10 minutes on Core2Duo with 2.0 GHz freq, as we had to recompile lots of facilities that depended on the modified one)
  
• difficulties with tool programming ( we had to rip chunks off the engine to attach some features to various tools or to integrate some tools within engine)

May be, all the cons were caused by our lame design or bad implementation. This does not really matter now. Something just had to be changed so we've started from scratch.

First of all, the terminology. Lets consider 'the module' to be an entity that solves one problem domain such as, for example, object rendering, and encapsulates all required feature subset, for example shader support, providing an interface to control it's actions, for example ::Draw(IObject*);

The new design just had to solve issues of the previous one possibly without adding new problems. It was natural to split monolith core to a set of smaller modules placed in separate binaries. That binaries should be preferably separated, that means that we have to build them as DLLs, not LIBs linked into the EXE. That would allow us to patch rather large engine parts with no need to recompile everything. We can build some DLLs and send to each other as patches so all the team mates do not have to deal with code, SDKs and compilation of parts of the engine they do not develop. The drawback of this approach is linkage. We found reasonable to use MS VC's __declspec(dllexport) modifier to export some symbols to LIBs associated with final DLLs. Such a modifier placed on a public function causes compiler to create a LIB file with the DLL (which is a primary build target). As a result, we can link to the DLL through supplied LIB. This seems to be a lot faster then true dynamic library loading.

The core was split up into a number of modules. The first, and the most low level if Foundation. It's aim is to provide proxy between our engine and OS. The engine should not know about OS specific APIs, synchronization primitives etc. This abstraction layer provides us with platform independency. Furthermore, the Foundation contains some basic mechanics required by both engine and tools, such as String representation classes, configuration manager, optimized memory allocator, I/O, math, SIMD optimizations, etc. Such approach allows us to build engine tools on the same basement as the engine that seems to be rather convenient. The Foundation is to be used as a base for each of higher level engine modules as well.

The next layer contains the Console module. It is to be built upon the Foundation module. The Console provides debugging IO and basic scripting integration via callbacks. It doesn't know about scripting module implementation, but is able to send input data to it and accept results.

The Networking and Scripting modules are built at the same abstraction layer as the Console and both use the Foundation. Together the Console, the Networking module and the Scripting module provide basic functionality such as

• developer commands and stats
• remote debugging
• action automation via scripts
• script debugging

This set seems to be enough for comfortable work with the engine.

The next layer consist of the Renderer, the Physics simulator and the Sound manager. These modules application is pretty straight-forward and their internals are not the subject of this post. The only thing that matters now is that these guys are to draw something, move/bounce/collide something and to make some noise respectively.

The last but not least, and the most hight level is the Scene. It is supposed to provide high-level set of functions to create, manage, destroy in-game objects. It uses all the layers below to simulate, draw, sound and synchronize via net all the game world objects. It allows debugging through the console and scripting, too.

The other modules are to be built are the GUI module and the AI module. These are planned, but are not heavily developed right now as we are working on low level routines to do our best in optimization and interface improvements.

Summarizing all the above I should say that new architecture approach seems to be logical, straight and hierarchical. It allows easy building and upgrading binaries. It provides unified basement for both the engine and the tool set. All the modules encapsulate everything related to their problem domain and provide interfaces for interaction.

I'll try to give more details on particular implementations later as I don't want to make the 'new first post' too heavy.

Before the conclusion I'll say a few words about target platforms and hardware requirements. Out first engine was targeted to PC platform with MS Windows XP and DirectX 9.0c compatible GPU. We've set a shader model 3 support as a requirement. With the new engine we target SM 4-5 GPUs installed on PCs under control of Windows 7. This version of Windows seems to be very promising from technical side and beloved by the community. There's no doubt that it will be much more popular then Vista was, so lots of users will have it on their PCs. This means that users will have DirectX11 (shipped with 7). DirectX 11 will also be available on Vista, so the audience that already has Vista and is not going to upgrade in near future will be also able to run our games.
So in summary, we are now

• using DirectX11 as primary API for graphics
• targeting PCs with Windows Vista and 7
• dropping XP support (at least temporary until the engine grows to alpha)

The new architecture design allows us to continue support of Windows XP with D3D9 as primary GAPI absolutely painlessly, but we have not enough resources to develop this branch. This means that it's not dropped completely, but frozen for a long time. We have some doubts on XP as a gaming platform by the time we finish our development, so there's no warranty that this branch will be ever finished.

From the other side, D3D11 as primary GAPI provides us with an ability to use new 'feature level' hummm.... feature of the API. It removes all the nasty D3DCAPS.. flags and provide strict sets of features required for any target ( 9_1, 9_2, 9_3, 10_0, 10_1, 11 ). We are now able to use one single GAPI for all devices under Windows Vista + OS and all the unsupported features will be disabled automatically. This means that technically we are able to continue support of D3D9 SM 3 level devices.

In conclusion (finally) I'd like to say that the project is no dead and it's reconstruction form scratch allowed us to add lots of new features, to make it much more comfortable to use and
to speed up development. Currently our re-developed engine is in early pre-alpha stage but already has some highly improved main killer-features of it's previous generation among of new ones, such as

• geometry clip map terrain (up to 5 times faster implementation and up to 4 times less memory consumption)
• deferred shading render engine ( up to 8 times faster, thanks to wise state management and insane optimizations )
• completely new resource management system with multi threaded loading (intensively uses multicore CPUs for resource cooking and management)
• completely new virtual file system with support of LZMA compression
  
• much, much more to come...

In near future we plan to spread our scene graph into several threads to speed up culling and animation, improve object management, add some new eye candies to the renderer.
The video and screen shots gallery will appear a bit later when we complete tests of our current feature set.