Dirty hands coding

Natvis-like experience with GDB pretty printers. Introduction

2026-07-31T22:01:00+07:00

Over the years of working on TheDarkMod game, we have amassed 1K lines of natvis code, 95 type definitions in total. Natvis has become an inherent part of good debugging experience, but it is available only on Windows. This story is about my attempt to extend this experience to the Linux side.

This multipart article explains how to reimplement most of the inherent natvis features on top of GDB pretty printers, provides a declarative approach to pretty printers similar to how natvis works, gives a solid deep-printing implementation similar to that of MSVC, and discusses the major issues of GDB value inspection.

TheDarkMod

TheDarkMod is the game started off the Doom 3 engine, which was made in years 2000-2004. Thus, the codebase has some properties which might seem unconventional for many C++ programmers:

Not Invented Here is a norm in gamedev. STL was not used by Doom 3, so custom containers are used almost everywhere in the code: idList instead of std::vector, idStr instead of std::string, etc.
C with classes is the general way of doing things, some parts of the renderer code were inherited from the older idTech engine and are in pure C. It is very typical to see intrusive linked lists with the next pointer being an explicitly declared member of a class.
Entity hierarchy is a wide inheritance hierarchy containing all kind of game objects. Entity-Component-System architecture was not known yet. idEntity base class has 49 directly derived classes. It is also a god object: 109 direct members and sizeof = 2376 bytes.

Here are some links to the code:

sys/msvc/natvis: the original natvis definitions for MSVC
sys/linux/gdbpp: work-in-progress GDB pretty printers
sys/linux/gdbpp/tdmuc_base.py: most of the code described in this article is in this file

Debugger selection

My first idea was to just use a debugger which supports natvis, since a few of them claim to have such a support. But I was quickly disappointed:

VS Code properly supports natvis only with Visual C++ debugger (reference). A limited portion of natvis is supported on Linux in MIEngine, implemented on top of GDB MI. It does not work well, and hangs up all the time.
CLion supports natvis only on Windows + MSVC toolchain (reference). Interestingly, natvis support has been implemented in a fork of LLDB. Some work is being done to extend it to Linux (issue), but it was not there when I looked into it.

So I started looking at alternative frameworks for customizing value visualization in the debugger:

GDB pretty printers is the Python framework native to GDB. So it is supported in CLion, VS Code, and many more IDEs, as long as they run on top of GDB.
LLDB data formatters is a similar Python framework native to LLDB. Supposedly supported by most IDEs as long as LLDB debugger is used.
QtCreator debugging helpers get an honorable mention. It is also a Python framework that works mainly in QtCreator. Since recently, it is also supported by CLion (reference).

I quickly discarded QtCreator options since it would tie us to a relatively unpopular IDE, and decided to pick either GDB or LLDB depending on which of them feels better. I gave up on LLDB data formatters after I realized that even enumerating the elements of an array requires non-trivial code with manual address computation and many interactions with the LLDB type system (see e.g. QVector formatter).

The main problems of all the natvis alternatives are: steep learning curve and high cost of error. Natvis is very compact and easy to use: after looking at a few examples you are ready to go. If you screw up, you'll see the raw data members in the Watch, and an error message in natvis diagnostics. But these alternative frameworks require you to know Python and have extensive API documentation. A lot of useful behavior which is provided out of the box in natvis is missing in GDB, so you have to implement it yourself in Python. And if you mess it up, you can easily get frozen GDB. And diagnostics are not always easy to get: exceptions and stdout messages can be silently swallowed or delayed.

Example

Here is an example of natvis for a typical std::vector-like container:

<Type Name="idList&lt;*&gt;">
    <DisplayString>[{num}] {list,[num]na}</DisplayString>
    <Expand>
        <ArrayItems>
            <Size>num</Size>
            <ValuePointer>list</ValuePointer>
        </ArrayItems>
    </Expand>
</Type>

And here is its equivalent in GDB pretty printers (without registration code):

class idListPrinter:
    def __init__(self, value):
        self._value = value

    def display_hint(self):
        return 'array'

    def to_string(self):
        n = int(self._value['num'])
        return '[%d]' % n

    def children(self):
        n = int(self._value['num'])
        pArr = self._value['list']
        for i in range(n):
            yield (str(i), pArr[i])

Here is how it looks in MSVC and CLion:

They are not strictly equivalent though, because the GDB version lacks some noticeable features:

The natvis version displays a few first elements straight in the display string, so often I see the contents without even having to expand it. And it does not hang while doing so.
The natvis version has a special [raw] child, which contains all the raw members of the class. It is useful on the rare occasions when I want to see capacity or address of the array.

After a lot of hard work, I have managed to implement both of these features on top on GDB pretty printers.

Inspection model

One thing in common between all the frameworks is that you basically customize two things for each type:

Display string or summary: a short plain text that you see instead of the value of the variable.
Children: the list of children inside variable, each children having a name and a value.

At the beginning, I hoped that correct GDB pretty printers would work in any debugger that runs on top of GDB, but I turned out to be wrong! In fact, most of the IDEs I tested don't work properly, which I believe is caused by the clash between two different value inspection models:

Lazily expanded tree. You see display string for the value. You can click the value to expand it and see its children. Then you can click on some children to expand them further, etc. Very similar to a typical filesystem browser. This is how GUI debuggers normally work.
Deep printing. You print out the display string of the value, then enumerate all of its children and recursively print them out. As the result, you see the complete contents of the value as one long text. This is how command line debuggers normally work, since they lack mouse interactions.

Deep printing model has one major problem: it can often traverse/print too much. If the children tree has cross-links, then the output can grow exponentially. If the children graph has loops, then deep-printing hangs completely. One can avoid such issue by forbidding to expand a child of pointer/reference type, but such a limitation makes value inspection useless in most applications. Even filesystems today are not trees because of all the directory links! GDB employs simple remedies like limiting print depth and width, but they are not reliable: imagine a large array and a large binary tree, where every element contains an expandable pointer to the root.

This becomes an ever-present pain with GDB pretty printers. Deep-printing model is so ingrained into the GDB mindset, it is very easy to invoke it accidentally, either by writing pretty printers incorrectly, or by doing something wrong on IDE side. Scrupulously following the lazy expansion model depends on the specific IDE debugger and is not guaranteed by the GDB itself.

Interestingly, this is specifically a GDB problem. Deep printing can be safe and fast if done properly. MSVC debugger does since forever: it deep-prints every value that misses a custom natvis DisplayString. I have also implemented proper deep-printing on top of GDB API, as you will see in the auto-summary chapter.

Fast Debug in Visual C++

2020-06-15T10:25:00+07:00

It is well-known that Debug build in Visual C++ is very slow compared to Release build, with the typical ratio about 10-20 times. Various reasons behind it are often stated, and some people even believe that it is inevitable because it is caused by lack of compiler optimizations.

If some issue happens only on a large dataset or in a real-time application, and it cannot be extracted into a smaller test case, then developer is forced to debug Release build. Which is rather painful experience, because debugger has problems showing control flow, values of local variables, sometimes even currently executed function, since all these concepts are messed up by optimizations. Ironically, the more optimizer-friendly your code is, the worse would be your experience of debugging Release build =)

Luckily, Visual C++ provides a lot of settings for tuning speed-vs-comfort ratio. It is entirely possible to create a "Fast Debug" configuration which works only a few times slower than Release yet is pretty easy to debug due to lack of optimizations.

Settings

Below you can see three pieces of advice which can greatly improve performance of Debug build. Of course, none of these improvements are free: with every change you lose some nice debug helper, so don't throw away the full Debug build yet.

Basic Runtime Checks

Disable either Basic Runtime Checks or Edit And Continue. If you want to know why it becomes slow, I recommend reading the full investigation by Bruce Dawson. But the bottom line is that having both of these settings causes a huge slowdown.

Unfortunately, both settings can be useful:

With Basic Runtime Checks enabled, debugger detects stack corruption. Such a hazard happens e.g. when a local array is modified out of bounds. There are other features included too, but this one seems to be the most useful, and you lose it if you disable the checks. Some stack errors will still be caught by Buffer Security Check, which is enabled even in Release, but without certainty.
Edit and Continue allows to pause program execution, edit the source code, update the build, and continue execution without restarting the application. This is especially useful in gamedev, as seen from the list of Live++ clients. The built-in MSVC implementation is not very reliable, but it nevertheless proved useful in TheDarkMod development.

In VC project, the Runtime Checks setting can be found in: C/C++ -> Code Generation -> Basic Runtime Checks. By default, the checks are enabled in Debug build, adding /RTC1 command line switch. Choose "Default" in drop-down list to disable it.

Speaking of "Edit and Continue", it is also enabled by default in Debug configuration. It can be found in: C/C++ -> General -> Debug Information Format. It generates /ZI compiler argument when enabled, and /Zi argument when disabled. Choose "Program Database" in drop-down list to disable it.

In CMake-generated project, the checks are also enabled by default. But as it often happens with CMake, the setting is not exposed. The simplest way to disable it is to remove it from compiler global parameters:

string(REPLACE "/RTC1" "" CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG}")

The "Edit and Continue" feature is disabled by default in CMake-generated projects. You have to do additional work if you want to enable it, but that's outside the scope of this article.

Inlining

Enable inlining level 1 (\Ob1). It allows VC compiler to inline calls to functions which are considered 'inline' according to C++ language rules. Function calls are cheap as far as I know, but inlining matters a lot for good performance because it allows to optimize the code much better (since optimizer operates on per-function basis). However, limited inlining helps without generic optimizations too. In most cases, only tiny functions and functions marked forceinline are inlined with /Ob1, so it does not harm debugging. But keep in mind that sometimes it can inline a function you want to debug.

The /Ob1 settings was enabled in "Debug with Inlines" configuration in Doom 3 source code. It is still used in TheDarkMod. It boosts FPS without noticeable drawbacks.

By default, both VC project and CMake completely disable inlining in Debug build (/Od). In VC project you can find the setting in: C/C++ -> Optimization -> Inline Function Expansion. Choose "Only __inline" from drop-down list to enable it, resulting in /Ob1 compiler argument. In CMake the easiest way to enable it is to add a global compiler parameter, although it can also be done on per-file basis:

# globally:
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /Ob1")
# for specified source files:
set_source_files_properties(${sources} PROPERTIES COMPILE_FLAGS "/Ob1")

By the way, CMake uses /Ob1 by default in its RelWithDebInfo configuration. In order to get full Release build with CMake, one has either to enable /Ob2 manually on RelWithDebInfo build or to enable debugging information in Release build.

Version of CRT

Use Release version of C Runtime Library. Doing so in debug build is rather questionable idea. Depending on CRT version, compiler sets _DEBUG define and enables/disables iterator debugging. Hence, ABI usually depends on CRT version, i.e. you cannot link together object files compiled against debug and release versions of CRT. Except if you write in pure C or avoid STL completely: in such case you can link debug and release code.

The immediate consequence of ABI change is that you are forced to use release builds of prebuilt libraries if you use release CRT. But this is not necessarily a problem, since in most cases you don't have to debug third-party libraries. Quite the opposite, using release libraries allows you to greatly accelerate your debug build if the program spends much time in them. Perhaps "Unoptimized Release" would be better name for such configuration than "Fast Debug" though.

Other downsides of using release CRT are:

No more iterator debugging. Most importantly, using std::vector<T>::operator[] with index out of range won't trigger any error. You are lucky if you use your own containers instead of STL, because you can still have an assert in your operator[] for a very low performance cost. You will lose even more debug checks if you love putting STL iterators everywhere (I don't care since I prefer indices to iterators).
Heap corruption check gone. This check allows to detect writes to heap block outside of its bounds. Without the check, the heap would probably break anyway, but it would happen later and diagnostic message would be less helpful.

Both in VC project and in CMake-generated project, debug CRT is used for Debug build by default. More precisely /MDd argument, which means "Multithreaded Debug DLL". In VC project settings, it can be found in C/C++ -> Code Generation -> Runtime Library. Choose "Multi-threaded DLL" in drop-down list to switch to release CRT (which is /MD).

Changing the setting in CMake is more difficult. One way is to replace global compiler argument:

string(REPLACE "/MDd" "/MD" CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG}")

More correct approach is to use the new MSVC_RUNTIME_LIBRARY property, but it did not work for me.

Benchmark

A piece of code working with small std::vector is shown below. Note that you should avoid making small local vectors for good performance. Better use local array, vector-like container which can be linked to local buffer, or custom allocator. This test is ill-written, and stresses low-level performance, heap operations, and methods inlining. All the performance issues are severely magnified, so don't expect the same gains on a real program.

//almost all the time is spent in this function (in.size() == 5)
static NOINLINE int OrderSum(const std::vector<int> &in) {
    std::vector<int> sorted;
    sorted = in;
    int n = sorted.size();
    for (int i = 0; i < n; i++)
        for (int j = 0; j < i; j++)
            if (sorted[i] < sorted[j])
                std::swap(sorted[i], sorted[j]);
    int res = 0;
    for (int i = 0; i < n; i++)
        res += sorted[i] * (i+1);
    return res;
}

int64_t GlobalSum;
void FullTest() {
    static const int SIZE = 5;
    std::vector<int> vec;
    for (int i = 0; i < SIZE; i++)
        vec.push_back((i*3)%5 + i);
    for (int q = 0; q < 1000; q++)
        GlobalSum += OrderSum(vec);
}

The call of FullTest is wrapped into a large loop executed for at least one second. The total time is divided by total number of OrderSum calls, obtaining the average time spent per one OrderSum call. Default project was created in VC2017, x64 Debug configuration is used with only the explicitly stated modifications. The program is run 3 times under devenv with debugger attached, median time is recorded. Ryzen 1600 CPU is used.

Time spent per OrderSum call, in nanoseconds
E&C disabled	/RTC disabled	/Ob1 enabled	release CRT	result
no	no	no	no	20700
yes	no	no	no	7321
no	yes	no	no	3565
yes	yes	no	no	3491
no	no	yes	no	4278
yes	no	yes	no	2455
no	yes	yes	no	1379
yes	yes	yes	no	1339
no	no	no	yes	10716
yes	no	no	yes	3596
no	yes	no	yes	1617
yes	yes	no	yes	1693
no	no	yes	yes	848
yes	no	yes	yes	506
no	yes	yes	yes	268
yes	yes	yes	yes	268
			[release]	86

Some conclusions from the raw data:

If you have already disabled Runtime Checks, then disabling Edit and Continue yields no additional performance improvement.
Disabling Edit and Continue is less helpful than disabling Runtime Checks (about 2x difference in time).
Aside from correlation between Edit and Continue and Runtime Checks, all settings are more or less independent.
Disabling Runtime Checks improves performance in 6x times without inlining, and in 3x times with inlining.
Inlining accelerates the code in 3x times without release CRT but with Runtime Checks disabled.
Release CRT makes the code faster in 2x times, given that inlining is disabled.
Release CRT and inlining have major synergy, providing impressive x14 boost together when Runtime Checks are disabled.
Default Debug build is x240 times slower than default Release build. With all the aforementioned settings enabled, Fast Debug build is only x3 times slower than Release build (and that's with optimization still disabled!). The total improvement of Fast Debug over default Debug is 77x times.

I hope this example ruins the myth that Debug build is slow because it lacks optimization, and that two-digit slowdown is inevitable. Indeed, you still need Debug build to simplify debugging all the nasty issues, but I think Fast Debug build should be the default one used for debugging.

UPDATE: Since Visual Studio 2019, functions are never inlined when Edit and Continue is enabled. The measurements above were done on VS 2017. There is no way to make /Ob1 inlining work with Edit and Continue in VS 2019 and 2022.

Localized tuning

If debug speed really matters to you, you shouldn't stop at tweaking global settings. In most cases there are few places in the program which waste considerable amount of time. Usually these places are well-written and are almost never touched with debugger. If a bit of code spends time but does not need to be debugged, it's better to enable optimization on it locally.

Third-party libraries

A common case is when performance-heavy code is inside a prebuilt library. You don't want to debug that library, so it would be great to use its release version in your debug build. There are several ways to achieve it:

If it is a pure C library or a C++ library without STL, then its release version can be linked with debug code statically. If it contains any of the STL, then you won't be able to do this due to settings mismatch (iterator debugging and CRT version). Otherwise you can link it even if it was compiled with release CRT and the rest of the code uses debug CRT. A warning about the mismatch can be muted.
You can build the library as DLL with CRT statically linked in. This DLL can be used in any application, as long as the exported interface contains no STL and does not pass heap allocations across DLL boundary. If any of this happens (it almost certainly happens for a heavy C++ library), then you cannot use this approach. Perhaps there are more restrictions here, e.g. you should not print to console both in the DLL and in your application. That's the typical pain from having several CRTs in a program.
Finally, you can switch your code to release CRT, as suggested above. Then you will have to link release versions of all third-party libraries, although this is not a problem in most cases. And you will lose the benefits of debug CRT of course.

Pragma optimize

The other case is a performance-heavy piece of your own code. Let's assume it is so heavy and polished, that it can be kept optimized even in debug. There are two ways to do this:

Enable optimization on a single translation unit, i.e. on cpp file. It can be done both in the VC project file and in CMake. This approach has limited flexibility.
Enable optimization on a few functions. This can be done with #pragma optimize directive.

Let's concentrate on the latter. An important thing to know is that #pragma directive can be used inside macros in its __pragma form, so it is possible to wrap optimization pragmas into macros which are enabled depending on compiler and configuration:

#if defined(_MSC_VER) && defined(OPTIMIZE_DEBUG)
  //force optimization of this code section in special debug configuration
  #define DEBUG_OPTIMIZE_ON __pragma(optimize("gt", on))    //enable optimizations
  #define DEBUG_OPTIMIZE_OFF __pragma(optimize("", on))     //reset optimization settings
#else
  #define DEBUG_OPTIMIZE_ON
  #define DEBUG_OPTIMIZE_OFF
#endif

Optimization can be force-enabled for a bunch of functions like this (excerpt from TheDarkMod):

DEBUG_OPTIMIZE_ON
void R_GlobalPlaneToLocal( const float modelMatrix[16], const idPlane &in, idPlane &out ) {
    out[0] = DotProduct( in, &modelMatrix[0] );
    out[1] = DotProduct( in, &modelMatrix[4] );
    out[2] = DotProduct( in, &modelMatrix[8] );
    out[3] = in[3] + modelMatrix[12] * in[0] + modelMatrix[13] * in[1] + modelMatrix[14] * in[2];
}
DEBUG_OPTIMIZE_OFF

There are several details to keep in mind:

I suggest creating a special macro flag OPTIMIZE_DEBUG for this tweak, since there is no predefined macro showing optimization level or build configuration. The macro _DEBUG is usually defined when debug CRT is used, so it will be missing if you switch your Debug build to release CRT. The macro NDEBUG only affects removal of asserts.
Be sure to close the section of force-optimized code with DEBUG_OPTIMIZE_OFF, otherwise the rest of cpp file will become surprisingly hard to debug.
The tweak has no effect on template and/or inline functions. One cannot just surround the body of such function and think it will be optimized. Because template functions are compiled when they are instantiated (this usually happens implicitly), and inline functions are compiled as the part of the function they are inlined into (at least optimization matters at this moment). It shouldn't confuse people who know how machine code is generated.
Function calls are rarely inlined in debug builds, even if /Ob1 is enabled (and absolutely never if it's disabled). One way to workaround the issue is to use macros instead of inline functions. The other way is to mark small accessors with __forceinline keyword (wrapped in a cross-platform macro of course).

Given that it is easy to make a mistake with this approach, I recommend generating assembly output with /FAs and checking it. Verify that optimization is applied as expected to functions, and that there are no non-inlined function calls in optimized sections. Optimized functions are preceded with the following line in VC assembly listing:

; Function compile flags: /Ogtp

Non-optimized functions have:

; Function compile flags: /Odtp /ZI

The key difference here is /Og against /Od.

This is not the most clean and easy-to-use technique, but it can be helpful sometimes. For instance, the original Doom 3 engine contains a set of SIMD routines written in assembly. The most performance-critical of these routines are used every frame to compute animations and stencil shadows of dynamic meshes. Yeah, that's CPU-side shadows and animation, greetings from year 2004 =) These routines had to be rewritten with intrinsics in order to accelerate 64-bit build of TheDarkMod, since assembly is locked in 32-bit. One weak point of intrinsics is that they are quite slow in Debug. So optimization was force-enabled on all the SIMD routines in the "Debug with Inlines" configuration.

Conclusion

All the materials used for benchmark are available in this repo.

I'd like to mention the article Fifty shades of debug by Mathieu Ropert here. It contains many ideas similar to the ones presented in this article.

utf8lut: Vectorized UTF-8 converter. Introduction

2019-08-04T15:31:00+07:00

Some time ago I wrote a surprising answer to stackoverflow question "Fastest way to get IPv4 address from string". At that very moment I discovered that _mm_shuffle_epi8 instruction combined with sufficiently large lookup-table can do wonders. Actually, this exact methodology was used to implement vectorized merging algorithm in one of my previous blog posts.

Inspired by the new discovery, I tried to apply this LUT + shuffle idea to some well-known and moderately generic problem. I tried to apply the idea to accelerate conversion from UTF-8 to UTF-16 and was successful. Initially, I had to postpone all efforts on the problem due to PhD-related activities. After that I thought that I would write a scientific article on the topic. When it became clear that I don't want to lose time on anything like that anymore, I decided to write about it in a blog. As a result, almost three years after the initial work (update: it is four years already), I finally managed to write this report, describing the algorithm and the utf8lut library.

Previous work

Converting UTF-8 to UTF-16 is a well-known problem. The most elegant and also quite efficient solution to UTF-8 decoding was presented by Bjoern Hoehrmann in his article. While there are plenty of approaches and libraries to do this simple conversion, there are not so many attempts at vectorizing this process. This is not surprising, because UTF-8 is essentially a variable length encoding, and dealing with dynamic data layout in SSE/AVX is a huge problem. I'll briefly describe here all the attempts at UTF-8 vectorization that I've heard of. In fact, utf8lut is based on many ideas which were already used in these solutions.

Rob D. Cameron: u8u16 (2007)

This library used to have a website (currently offline). It is best described in a technical article u8u16 - A High-Speed UTF-8 to UTF-16 Transcoder Using Parallel Bit Streams.

The whole idea of vectorization is based on the so-called "parallel bit streams". Imagine that each byte is a struct with eight fields, each field being a single bit. When programs receives an array of bytes, it efficiently receives array of eight-bit structs, e.g. data in AoS layout. It is well-known that this layout is not convenient for vectorization, the SoA layout is much more convenient. If we convert the data to SoA, we will get eight separate arrays of bits: one will have exactly all the 0-th bits of bytes, one will have exactly all the 1-st bits of bytes, ..., the last one will have all the 7-th (last) bits of the bytes. Each such array of bits is called a bit stream, and there are eight bit streams in total.

In each processing step, u8u16 first loads next 1024 bits of input data as eight __m128i values (AoS layout). Then it converts these values into eight bit streams (SoA layout), each bit stream has 128 bits exactly fitting into a __m128i value. So there is one __m128i containing 0-th bit of all the 128 bytes read, one __m128i containing 1-st bits of all the 128 bytes read, etc. The actual UTF-8 decoding operates on these bit streams by performing trivial bitwise operations on them, like: and, andnot, or, xor. For any arbitrary condition, you can obtain a bitmask showing which bytes satisfy it. Using this methodology, 16 bit streams (note: the number has doubled) are computed for the UTF-16 code units. In UTF-16 streams, not all the 128 positions have meaningful data: usually the 16-bit code unit is attached to the position of the last byte in its UTF-8 encoding. The unused positions must be removed from the bit streams, so stream compaction algorithm is run. Then the 16 compacted bit streams are converted back into sequence of byte pairs (AoS layout), the code units are written to output and stream pointers are advanced forward.

The good point of this approach is that the middle part of the algorithm is perfectly vectorized: 128 bits are processed in one instruction, using very cheap bitwise logical operations. Also, most of the algorithm is branchless, including even the stream compaction. However, converting into bit stream representation and back takes time. Also, the bit streams methodology does not solve the problem of data moving dynamically between positions. This problem is localized to the stream compaction of bit streams, which looks rather heavy.

Olivier Goffart: utf8sse4 (2012)

This algorithm was described in article UTF-8 processing using SIMD, which was posted on Woboq's website. This code is also available on GitHub. It was born due to an attempt to accelerate QString::fromUtf8 function in Qt5, but it turned out that in the real world negligible portion of input strings are long/hard enough to benefit from acceleration.

The algorithm uses direct approach, and processes 14-16 bytes simultaneously in one step. To avoid complexity of surrogate pairs, non-BMP characters are handled outside of vectorized code path (just like in utf8lut). Immediately after reading 16 bytes of input, the algorithm classifies all the bytes into the leading bytes and the continuation bytes, and for each leading byte, the length of the corresponding code point is determined (4 classes in total). After that, 16-bit unicode values are calculated. Similar to u8u16, each obtained unicode value is attached to position where the last byte of its UTF-8 encoding is located. One __m128i contains high bytes of these 16-bit values, and the other contains low bytes of them. Just like in u8u16, empty positions must be removed using stream compaction algorithm. But the stream compaction algorithm itself is closer to one used in utf8lut: single instruction _mm_shuffle_epi8 is used to do the compaction itself. But unlike utf8lut, considerable number of instructions is spent on computing the shuffle mask.

Since only 16 bytes of data are processed at once (not 128 bytes like in u8u16), the whole algorithm is much more compact. It is branchless, except for a bit of conditional at the very end for advancing the source pointer. However, this algorithm uses quite a lot of instructions. Often it has to compute some value 3-4 times (once per each class of bytes) and blend the results together based on the mask of classes. Also, it recklessly uses a few string instructions from SSE 4.2 in validation, which are known to be slow. On the bonus side, it does not use CPU cache for anything, unlike utf8lut.

Hiroshi Inoue (2008)

This algorithm was probably presented in a conference according to slides Accelerating UTF-8 Decoding using SIMD Instructions. Since this work was done in IBM research, it was applied only to PowerPC with its AltiVec instructions. IBM probably explains why the author follows the shitty practice of filing US patents "like hotcakes", including the US7394411 patent on the UTF-8 decoding algorithm.

The single step of the algorithm works as follows. First, information about the next 8 code points is gathered from the input byte array: the lengths of these eight code points are determined and combined into a single integer number. The information is gathered by scalar traversal of corresponding leading bytes without any vectorization, but the traversal is done in branchless fashion. The obtained integer number ranges from 0 to 6560, and it serves as index in 6561-entry lookup table. For each configuration of code points, the LUT contains some bitmasks, and most importantly, the shuffling mask. This data is enough to shuffle data into desired position and to produce the UTF-16 data for exactly the next 16 bytes.

The algorithm is very similar to utf8lut. It uses shuffling by mask generated during runtime in order to cope with the problem of dynamic movement of bytes in the vectors. The shuffling mask and the auxiliary data are also taken from lookup table. The main difference is how the LUT index is computed: utf8lut uses suitable SSE2 operations to get the index, while the algorithm by Inoue uses scalar instructions (quite a lot of them actually). Aside from that, the presentation is less detailed: it omits validation, encoding problem (UTF-16 to UTF-8). Finally, it uses AltiVec, while utf8lut is implemented exclusively for SSSE3. One positive side of the Inoue's algorithm is that its LUT is more compact than in utf8lut.

Improvements to SymbolSort tool for C++ code bloat analysis

2018-05-27T00:09:00+07:00

It is known that code bloat significantly increases build times in C++. The question is: how can we quantify code bloat? How can we see things which are compiled too many times or the things which generate too much object code? How can we know which places would benefit most from improvement efforts?

This article is about an excellent tool called SymbolSort, which produces useful code bloat statistics in a project of any size. In particular, the article describes a few improvements which I have implemented recently to provide more useful analysis.

Manual

The simplest approach to see what exactly is compiled is to enable generation of assembly listings (/FA or /FAs on MSVC) and look through them from time to time. In case of MSVC, all generated symbols are listed at the top of the .asm file as PUBLIC symbols, e.g.:

; Listing generated by Microsoft (R) Optimizing Compiler Version 18.00.40629.0

include listing.inc

INCLUDELIB LIBCMT
INCLUDELIB OLDNAMES

PUBLIC  hypot
PUBLIC  main
PUBLIC  ??$calcSum@H@@YAHPEBHH@Z                                            ; calcSum<int>
PUBLIC  ?run@?$Bar@H$06@@QEBAHXZ                                            ; Bar<int,7>::run
PUBLIC  ??$accumulate@PEBHH@std@@YAHPEBH0H@Z                                ; std::accumulate<int const * __ptr64,int>
PUBLIC  ??$accumulate@PEBHHU?$plus@X@std@@@std@@YAHPEBH0HU?$plus@X@0@@Z     ; std::accumulate<int const * __ptr64,int,std::plus<void> >
PUBLIC  ??$_Unchecked@PEBH@std@@YAPEBHPEBH@Z                                ; std::_Unchecked<int const * __ptr64>
PUBLIC  ??$_Accumulate@PEBHHU?$plus@X@std@@@std@@YAHPEBH0HU?$plus@X@0@@Z    ; std::_Accumulate<int const * __ptr64,int,std::plus<void> >
PUBLIC  ??$?RAEAHAEBH@?$plus@X@std@@QEBA?BHAEAHAEBH@Z                       ; std::plus<void>::operator()<int & __ptr64,int const & __ptr64>
PUBLIC  ??_C@_03PMGGPEJJ@?$CFd?6?$AA@                                       ; `string'
PUBLIC  __xmm@00000004000000030000000200000001
EXTRN   printf:PROC
EXTRN   _hypot:PROC
EXTRN   __GSHandlerCheck:PROC
EXTRN   __security_check_cookie:PROC
EXTRN   __security_cookie:QWORD
EXTRN   _fltused:DWORD
...

It is especially insightful to take a small .cpp file which has suspiciously large .obj file and look into the assembly generated. In the simplest case, it is enough to look through function symbols, although there is a lot of other stuff: constants of various types and exception handlers. Also note that EXTRN symbols are not generated in this file.

Anyway, this approach does not provide statistics over the whole project. It can help to gain understanding of what happens, but it requires a good guess to actually notice a problem. And without changing the code there is no way to see how big the problem is.

SymbolSort

SymbolSort is a tool created by Adrian Stone, originally announced in his blog and later released for public on GitHub. It is recommended to read the original blog article for detailed description of its options and results. In general, the tool parses a set of input files, extracts information about all symbols contained in them, and then prints out various types of statistics.

SymbolSort supports two major types of input files:

A dump of object file (or files), produced either by dumpbin.exe (Visual C++) or by nm (GNU binutils). This type of input is most useful for analyzing build times, since it counts symbols before linker deduplicates them. In the current article, only MSVC platform will be considered.
A module (either executable or dynamic library) or a .pdb file with debug symbols. As far as I understand, analyzing an .exe file is equivalent to analyzing its .pdb file. Since this is the final binary with all the symbols deduplicated, this type of analysis is most useful for reducing executable size and improving code cache performance, but not for improving build performance.

To demonstrate SymbolSort capabilities, I'll apply it to some real-world C++ project. I'll try to analyze The Dark Mod (TDM), which is an open source standalone game based on Doom 3 engine. It is very important to note that the Doom 3 engine originally does not use STL at all: it has its own library called "idLib" with greatly simplified custom alternatives to STL stuff. However, the dark mod coders used STL in many places over their code, so we will definitely see it in the analysis too.

Demo: exe

First let's run SymbolSort on the final TDM executable:

SymbolSort -in TheDarkModx64.exe -out TheDarkModx64.exe.report

After a few seconds, we can see the result in TheDarkModx64.exe.report (the full file is much longer):

Raw Symbols
Total Count  : 64112
Total Size   : 60215952
Unattributed : 2750964
--------------------------------------
Sorted by Size
        Size Section/Type  Name                     Source
    16777216          bss  optEdges                 c:\thedarkmod\darkmod_src\tools\compilers\dmap\optimize.cpp
    7834688          bss  sessLocal                c:\thedarkmod\darkmod_src\framework\session.cpp
    6887456          bss  gameLocal                c:\thedarkmod\darkmod_src\game\game_local.cpp
    6291456          bss  optVerts                 c:\thedarkmod\darkmod_src\tools\compilers\dmap\optimize.cpp
    3407872          bss  outputTris               c:\thedarkmod\darkmod_src\tools\compilers\dmap\shadowopt3.cpp
    2359296          bss  silQuads                 c:\thedarkmod\darkmod_src\tools\compilers\dmap\shadowopt3.cpp
    1572864          bss  shadowVerts              c:\thedarkmod\darkmod_src\renderer\tr_stencilshadow.cpp
    1572864          bss  silEdges                 c:\thedarkmod\darkmod_src\tools\compilers\dmap\shadowopt3.cpp
    786432          bss  rb_debugLines            c:\thedarkmod\darkmod_src\renderer\tr_rendertools.cpp
    589824          bss  EventPool                c:\thedarkmod\darkmod_src\game\gamesys\event.cpp
    576360         data  localConsole             c:\thedarkmod\darkmod_src\framework\console.cpp
    524288          bss  udpPorts                 c:\thedarkmod\darkmod_src\sys\win32\win_net.cpp
    458752          bss  rb_debugPolygons         c:\thedarkmod\darkmod_src\renderer\tr_rendertools.cpp
    393216          bss  shadowIndexes            c:\thedarkmod\darkmod_src\renderer\tr_stencilshadow.cpp
    ...             ...  ...                      ...

Note that the file TheDarkModx64.exe is only 10 MB, and here it says that there are 60 MB of symbols. Moreover, the largest symbols are attributed to .bss section, meaning that they are global or static variables. It seems that IdTech4 loves global variables too much. In order to workaround this problem, we can use the newly added -sections parameter to filter only code symbols:

SymbolSort -in TheDarkModx64.exe -out TheDarkModx64.exe.report -sections code

This gives us much more useful statistics, which includes only the machine code. The SymbolSort produces many different lists of symbols (or groups of symbols), sorted by total count, size or name. You can download this complete report and look into it yourself. Here I will pay close attention only to two lists.

The first is the list of file contributions, which shows how much code or symbols are generated for each source code file, also with information about directories:

File Contributions
--------------------------------------
Sorted by Size
        Size   Count  Source Path
    6028867   33730  c:
    5869935   31632  c:\thedarkmod\darkmod_src
    2957843   16774  c:\thedarkmod\darkmod_src\game
    1194434    7034  c:\thedarkmod\darkmod_src\tools
    631682    3243  c:\thedarkmod\darkmod_src\game\ai
    581988    3156  c:\thedarkmod\darkmod_src\tools\radiant
    541960    2149  c:\thedarkmod\darkmod_src\idlib
    509794    1878  c:\thedarkmod\darkmod_src\renderer
    338432    1156  c:\thedarkmod\darkmod_src\game\physics
    280497    1768  c:\thedarkmod\darkmod_src\framework
    249891     458  c:\thedarkmod\darkmod_src\idlib\math
    230854     665  c:\thedarkmod\darkmod_src\tools\compilers
    172726     499  c:\thedarkmod\darkmod_src\game\ai\ai.cpp
    168365    1477  c:\thedarkmod\darkmod_src\game\gamesys
    163355     814  c:\thedarkmod\darkmod_src\game\ai\states
    158932    2098  c:\program files (x86)\microsoft visual studio 12.0\vc
    156801    2002  c:\program files (x86)\microsoft visual studio 12.0\vc\include
    153308    1057  c:\thedarkmod\darkmod_src\game\entity.cpp
    138646     933  c:\thedarkmod\darkmod_src\ui
    ...        ...  ...
    76322     506  c:\program files (x86)\microsoft visual studio 12.0\vc\include\xtree
    ...        ...  ...

As you see, it is easy to learn how much code space is taken by each game subsystem. The whole code is 6 MB. The game logic itself (darkmod_src\game) occupies half of code space (3 MB), while various tools (darkmod_src\tools) take only about 20% (1.2 MB). Also, this view makes it most convenient to see how much code is generated for MSVC standard library: only 160 KB is taken by noninlined functions of standard library, with almost half of it (75KB) generated by xtree header. This header contains implementation of std::set and std::map, which often constitutes most of the machine code from STL.

Another useful statistics shows how much space is taken by all instances of a template function or method. Here it is (sorted by size):

Merged Template Symbols
Merged Count  : 26353
--------------------------------------
Sorted by Total Count
    ...

Sorted by Total Size
  Total Size  Total Count  Name
      5027         2160  `dynamic initializer for '...''
      2032         2030  `dynamic atexit destructor for '...''
      6417          220  public: void __cdecl idList<T>::Resize(int) __ptr64
      5410           67  ai::`dynamic initializer for '...''
      2087           17  protected: class std::_Tree_iterator<T> __cdecl std::_Tree<T>::_Insert_at<T>(bool,struct std::_Tree_node<T> * __ptr64,struct std::pair<T> && __ptr64,struct std::_Nil) __ptr64
      1261           26  public: class std::_Tree_iterator<T> __cdecl std::_Tree<T>::erase(class std::_Tree_const_iterator<T>) __ptr64
      8678           16  public: class A0xa82155d7::xpath_ast_node * __ptr64 __cdecl `anonymous namespace'
      8577           13  protected: class std::_Tree_iterator<T> __cdecl std::_Tree<T>::_Insert_hint<T>(class std::_Tree_const_iterator<T>,struct std::pair<T> & __ptr64,struct std::_Tree_node<T> * __ptr64) __ptr64
      7112           18  protected: struct std::pair<T> __cdecl std::_Tree<T>::_Insert_nohint<T>(bool,struct std::pair<T> && __ptr64,struct std::_Nil) __ptr64
      6563           19  void __cdecl `anonymous namespace'
      6286           12  private: class A0xa82155d7::xpath_node_set_raw __cdecl `anonymous namespace'
      5662          140  `std::shared_ptr<T>::_Resetp<T>'...'::catch$0
      5504           13  protected: struct std::pair<T> __cdecl std::_Tree<T>::_Insert_nohint<T>(bool,struct std::pair<T> & __ptr64,struct std::_Tree_node<T> * __ptr64) __ptr64
      5338            4  private: static bool __cdecl `anonymous namespace'
      4883           13  protected: class std::_Tree_iterator<T> __cdecl std::_Tree<T>::_Insert_at<T>(bool,struct std::_Tree_node<T> * __ptr64,struct std::pair<T> & __ptr64,struct std::_Tree_node<T> * __ptr64) __ptr64
      ...           ...  ...

Let's leave the problem of dynamic initializers and destructors --- those are most likely initialization code for global and static variables, and idTech4 has a lot of them. The first highlighted line shows that idList<T>::Resize method has 220 instances of total size 6.4 KB. The other highlighted lines show various methods of std::_Tree template (which is the implementation of std::set and std::map). Information like this is especially useful for generic template classes, which are used a lot in the project.

One question which easily comes to mind is: "how much code is generated for the whole std::_Tree class"? The stock version of SymbolSort cannot answer this question.

Demo: obj

As mentioned before, the analysis of executable may be useful for optimizing instruction caching, but it does not represent well where the build time is spent. Every function defined in header is compiled independently as many times as there are source files using it (read more here), with duplicates being merged by linker later. Writing code in headers may be necessary for inlining, and also for templates. And even though idTech4 was written long time ago by people mainly proficient with C, it still contains some templates.

To better estimate build time, we should analyze .obj files instead of executable. Every .obj file represents all the code generated by compiler for the corresponding translation unit, so if we sum up the stats across all .obj files, we will know how much code was compiled (including duplicates). And it is probably better to do so for debug build with inlining disabled, because even though inlined functions can be omitted in .obj files, they still take compiler's time.

In order to run SymbolSort on .obj files, we have to 1) run dumpbin.exe /headers on every .obj file and 2) list all the resulting symbol files as arguments to SymbolSort. It is also allowed concatenate all symbol files into one file all_obj.smb. Here is an example:

SymbolSort -in:comdat .\DarkModTools\all_obj.smb -in:comdat .\idLib\all_obj.smb -out object_files.report

To simplify invoking SymbolSort, I have created bloatinfo.py script. In order to get single report over several projects, just run the following command in the directory containing their .obj files:

bloatinfo --obj=. --dump --analyze

The script calls dumpbin internally, merges symbol files for each project and runs SymbolSort on them. The results are put into object_files.report in the current directory (you can download the full report):

Raw Symbols
Total Count  : 126533
Total Size   : 21001307
Unattributed : 0
--------------------------------------
Sorted by Size
        Size Section/Type  Name                                                                                                                      Source
    106356     .text$mn  public: bool __cdecl idClass::ProcessEventArgPtr(class idEventDef const *,__int64 *)                                      .\DarkModTools\Class.obj
    65536         .bss  char (* `char * __cdecl va(char const *,...)'::`2'::string)[16384]                                                        .\idLib\Str.obj
    65536         .bss  char (* `public: static char const * __cdecl idStr::FloatArrayToString(float const *,int,int)'::`2'::str)[16384]          .\idLib\Str.obj
    65536         .bss  char (* `public: static char const * __cdecl idTypeInfoTools::OutputString(char const *)'::`2'::buffers)[16384]           .\DarkModTools\TypeInfo.obj
    65536         .bss  class idClipModel * * `int __cdecl GetObstacles(class idPhysics const *,class idAAS const *,class idEntity const *,int,class idVec3 const &,class idVec3 const &,struct obstacle_s *,int,class idBounds &,struct obstaclePath_s &)'::`5'::clipModelList  .\DarkModTools\AI_pathing.obj
    65536         .bss  class idEntity * * `public: float __cdecl idPush::ClipRotationalPush(struct trace_s &,class idEntity *,int,class idMat3 const &,class idRotation const &)'::`2'::entityList  .\DarkModTools\Push.obj
    65536         .bss  class idEntity * * `public: void __cdecl idPlayer::PerformFrobCheck(void)'::`42'::frobRangeEnts                           .\DarkModTools\Player.obj
    64768         .bss  struct polyhedron * `struct polyhedron __cdecl make_sv(struct polyhedron const &,class idVec4)'::`2'::lut                 .\DarkModTools\tr_shadowbounds.obj
    62651     .text$mn  public: bool __cdecl idMat6::InverseSelf(void)                                                                            .\idLib\Matrix.obj
    ...            ...  ...

As usual, there are many .bss symbols here. The total size by sections is:

Merged Sections / Types
Merged Count  : 10
--------------------------------------
Sorted by Total Count
Total Size  Total Count  Name
    15530261        84799  .text$mn
     2288278        15786  .rdata
      780809         9632  .text$x
      211188         7545  .rdata$r
      321431         2609  .text$di
      104892         2289  .data$r
       86136         2206  .text$yd
     1629712         1258  .bss
       48392          383  .data
         208           26  .CRT$XCU

As you see, there are only 1.5 MB of such symbols, not 50 MB as we saw in the executable. I have no idea why the other global variables are not shown here.

Anyway, we have 15.5 MB of machine code compiled in total over all object files, contrasted with 11.5 MB of code in TheDarkModx64.exe when built in Debug configuration. This gives us nice 4/3 ratio of overcompilation, which is very good. On my initial run I even got a ratio less than one, which happened because function-level linking was not enabled, so some symbols were missing in dumpbin outputs. A rule of thumb is: enable function-level linking for .obj analysis, otherwise non-template non-inline stuff will be missing in the report!

I think such low overcompilation ratio was only possible due to the "C with classes" mindset. Modern C++ programming with templates abuse is much worse in this regard. For instance, one project of OpenCollada has 9/2 ratio of overcompilation. It means that each piece of code has to be independently compiled 4.5 times on average. Actually, most of this bloat in object files comes from a few templates, which have crazy ratio of overcompilation (near hundred).

But let's continue with TheDarkMod. The "file contributions" section in the object_files.report is not revealing:

File Contributions
--------------------------------------
Sorted by Size
        Size   Count  Source Path
    20976534  126384  .
    19193802  120624  .\DarkModTools
     1782732    5760  .\idLib
      477915    2320  .\DarkModTools\AI.obj
      439014    1926  .\DarkModTools\Player.obj
      432376    2643  .\DarkModTools\Entity.obj
      333805    2241  .\DarkModTools\Game_local.obj
      318057     399  .\idLib\Matrix.obj
      254341    1203  .\DarkModTools\pugixml.obj
      238372    1041  .\DarkModTools\Physics_AF.obj
      189301     510  .\DarkModTools\EditorBrush.obj
      184638     886  .\DarkModTools\MainFrm.obj
      174207    1138  .\DarkModTools\Actor.obj
      163010     641  .\DarkModTools\Window.obj
      ...        ...  ...

Here machine code is classified into object files. If some class like idList generates much bloat because its methods are defined in header, we won't see it here. Because every .obj file will have at most one duplicate of each of these methods. It turns out that the most nasty code bloaters are evenly spread across .obj files, making this "file contributions" section useless.

A better view is provided by "merged template symbols" section:

Merged Template Symbols
Merged Count  : 39841
--------------------------------------

Sorted by Total Size
  Total Size  Total Count  Name
      865280          104  protected: static struct ATL::CTrace::CategoryMap * ATL::CTrace::m_nMap
      482057         6611  `string'
      297459         2299  void __cdecl `dynamic initializer for '...''
      243853          107  public: class std::_Tree_iterator<T> __cdecl std::_Tree<T>::erase(class std::_Tree_const_iterator<T>)
      207875          523  public: void __cdecl idList<T>::Resize(int)
      139609           83  protected: struct std::pair<T> __cdecl std::_Tree<T>::_Insert_nohint<T>(bool,struct std::pair<T> &&,struct std::_Nil)
      124320           21  protected: class std::_Tree_iterator<T> __cdecl std::_Tree<T>::_Insert_hint<T>(class std::_Tree_const_iterator<T>,struct std::pair<T> &,struct std::_Tree_node<T> *)
       91184           82  protected: class std::_Tree_iterator<T> __cdecl std::_Tree<T>::_Insert_at<T>(bool,struct std::_Tree_node<T> *,struct std::pair<T> &&,struct std::_Nil)
       80271          679  public: void __cdecl idList<T>::Clear(void)
       79856         2045  void __cdecl `dynamic atexit destructor for '...''
       70246          103  public: class std::_Tree_iterator<T> __cdecl std::_Tree<T>::erase(class std::_Tree_const_iterator<T>,class std::_Tree_const_iterator<T>)
       59325          525  public: __cdecl idList<T>::idList<T>(int)
       50102           94  public: void __cdecl std::basic_string<T>::_Copy(unsigned __int64,unsigned __int64)
       43036          106  public: class std::_Tree_const_iterator<T> & __cdecl std::_Tree_const_iterator<T>::operator--(void)
       42840          126  protected: void __cdecl std::_Tree<T>::_Lrotate(struct std::_Tree_node<T> *)
       42840          126  protected: void __cdecl std::_Tree<T>::_Rrotate(struct std::_Tree_node<T> *)
       42284          341  protected: void __cdecl idStr::Init(void)
       41460           28  private: void __cdecl `anonymous namespace'
       37736          106  public: struct std::_Tree_node<T> * __cdecl std::_Tree_buy<T>::_Buynode0(void)
       35658          126  public: bool __cdecl std::_Tree_const_iterator<T>::operator==(class std::_Tree_const_iterator<T> const &)const 
       35406           42  public: bool __cdecl idBounds::AddPoint(class idVec3 const &)
       ...            ...  ...

Note that the very first symbol ATL::CTrace::m_nMap is a .bss symbol, so it does not take compiler's time. The second symbol `string' corresponds to all string constants compiled, so it is not code also. Among the rest, std::_Tree pieces are consistently in the top =) Some methods of idList<T> are also noticeable. It would be great to know how much code is generated for idList class in total, but the original SymbolSort does not provide such insight.

Improvements

As you see above, the original version of SymbolSort lacks one important feature on .obj input: you cannot learn how much code a template class is generating. This could be especially useful for widely used generic template classes, in particular for templates like idList and std::set. There are two approaches to provide this functionality:

PDB. Attribute each symbol to the source or header file where its implementation is physically located, instead of attributing it to the .obj file where it is compiled. As the result, the stats for the header file (where template is defined) in the "file contributions" section should give the wanted information.
classpath. Extract classpath from the name of each symbol, which shows the class name with all namespaces containing it, e.g. boost::container::deque. Then show amount of code with each classpath. In addition, statistics per namespaces (e.g. boost) can be shown just like statistics per directories are shows in "file contributions" section.

Both approaches are implemented in the updated version of SymbolSort (currently available in my fork). The classpath approach works without any additional input, as long as undname.exe tool is in PATH. The PDB approach (as the name suggests) requires .pdb files to be provided using -info parameters. For the bloatinfo.py script, it is enough to simply provide the root directory containing the .pdb files:

bloatinfo --obj=. --pdb=exeDir --dump --analyze

The script passes all the .pdb files present in this directory to SymbolSort. Here is the full list of command line arguments passed to SymbolSort in my case (seen in options.txt file):

-out object_files.report
-in:comdat .\DarkModTools\all_obj.smb -in:comdat .\idLib\all_obj.smb
-info exeDir\TheDarkModx64.pdb              # determine code location (source file) using this pdb

As usual, you can download the full report. It differs from the previous one in only two sections: "file contributions" and newly added "namespaces and classes contributions". Here you can see the first one, which shows results of the PDB approach:

File Contributions
--------------------------------------
Sorted by Size
        Size   Count  Source Path
    16996266   99982  c:
    13219968   65465  c:\thedarkmod\darkmod_src
     4986400   24553  c:\thedarkmod\darkmod_src\game
     4385995   22000  c:\thedarkmod\darkmod_src\idlib
     3776298   34517  c:\program files (x86)\microsoft visual studio 12.0\vc
     3748270   23115  [not_in_pdb]
     3646356   33365  c:\program files (x86)\microsoft visual studio 12.0\vc\include
     1965992    8770  c:\thedarkmod\darkmod_src\tools
     1594575    6939  c:\program files (x86)\microsoft visual studio 12.0\vc\include\xtree
     1348758    7184  c:\thedarkmod\darkmod_src\idlib\math
     1255015    7762  c:\thedarkmod\darkmod_src\game\ai
      952373    3052  c:\thedarkmod\darkmod_src\tools\radiant
      866746     424  c:\thedarkmod\darkmod_src\idlib\precompiled.cpp
      837893    6008  c:\thedarkmod\darkmod_src\idlib\containers
      716211    5007  c:\thedarkmod\darkmod_src\idlib\containers\list.h
      682448    1730  c:\thedarkmod\darkmod_src\renderer
      522322    3779  c:\thedarkmod\darkmod_src\framework
      456809    1828  c:\thedarkmod\darkmod_src\game\physics
      446605    7671  c:\program files (x86)\microsoft visual studio 12.0\vc\include\xmemory0
      392003    2832  c:\program files (x86)\microsoft visual studio 12.0\vc\include\xstring
      371915    4260  c:\thedarkmod\darkmod_src\idlib\str.h
      340839    5005  c:\program files (x86)\microsoft visual studio 12.0\vc\include\memory
      337826    3349  c:\thedarkmod\darkmod_src\idlib\math\vector.h
      323912     659  c:\thedarkmod\darkmod_src\idlib\geometry
      322392    1728  c:\thedarkmod\darkmod_src\game\gamesys
      315030    1384  c:\thedarkmod\darkmod_src\game\ai\ai.cpp
      296359     693  c:\thedarkmod\darkmod_src\tools\compilers
      288751     809  c:\thedarkmod\darkmod_src\idlib\bv
      280758     129  c:\thedarkmod\darkmod_src\idlib\math\matrix.cpp
      280446    1479  c:\thedarkmod\darkmod_src\game\ai\states
      262804    1280  c:\thedarkmod\darkmod_src\ui
      256771    3436  [unclear_source]
        ...      ...  ...

We can learn a lot of things from this list:

The whole C/C++ standard library generates 3.7 MB of object code.
Implementation of std::set / std::map (xtree) generates at least 1.5 MB of code. 800 KB is generated by STL memory management, perhaps also for set/map. Despite being rarely used, std::string seems to gen 400 KB --- a bit more than widely used idStr from str.h.
idList from list.h generates only 700 KB of code --- that's a very little. idStr generates 370 KB, and all vector math generates 330 KB.
Among different components: game logic (game) generates 5 MB, the ID's generic library (idlib) generates 4.4 MB, in-game tools (tools) generates 2 MB, framework (framework) generates 500 KB, and the renderer core (renderer --- written in C) generates 700 KB. Probably the results would be more precise if we remove bss and data sections from here.

A minor nuisance of this report is that we see two special filenames: [not_in_pdb] and [unclear_source]. These categories contain all symbols for which SymbolSort failed to find proper location. A symbol gets into [unclear_source] when it has some code and is present in PDB, but it does not have any source code location. Mainly, these are all the implicitly generated class methods: automatically generated constructors, destructors and assignment operators. Here are some examples:

public: virtual void * __cdecl idAASLocal::`scalar deleting destructor'(unsigned int)
public: void * __cdecl eas::tdmEAS::`scalar deleting destructor'(unsigned int)
public: void * __cdecl idList<struct SBoolParseNode>::`vector deleting destructor'(unsigned int)
public: class idAASSettings & __cdecl idAASSettings::operator=(class idAASSettings const &)
public: __cdecl idDrawVert::idDrawVert(void)
public: class idSurface & __cdecl idSurface::operator=(class idSurface const &)
void __cdecl `vector constructor iterator'(void *,unsigned __int64,int,void * (__cdecl*)(void *))
void __cdecl `public: static class Library<class ai::State>::Instance & __cdecl ai::Library<class ai::State>::Instance(void)'::`2'::`dynamic atexit destructor for '_instance''(void)
public: __cdecl std::_Tree_buy<struct std::pair<int const ,class CFrobDoor *>,class std::allocator<struct std::pair<int const ,class CFrobDoor *> > >::~_Tree_buy<struct std::pair<int const ,class CFrobDoor *>,class std::allocator<struct std::pair<int const ,class CFrobDoor *> > >(void)
public: __cdecl std::_Iterator012<struct std::bidirectional_iterator_tag,struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class std::function<class std::shared_ptr<class ai::Task> __cdecl(void)> >,__int64,struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class std::function<class std::shared_ptr<class ai::Task> __cdecl(void)> > const *,struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class std::function<class std::shared_ptr<class ai::Task> __cdecl(void)> > const &,struct std::_Iterator_base12>::_Iterator012<struct std::bidirectional_iterator_tag,struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class std::function<class std::shared_ptr<class ai::Task> __cdecl(void)> >,__int64,struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class std::function<class std::shared_ptr<class ai::Task> __cdecl(void)> > const *,struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class std::function<class std::shared_ptr<class ai::Task> __cdecl(void)> > const &,struct std::_Iterator_base12>(struct std::_Iterator012<struct std::bidirectional_iterator_tag,struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class std::function<class std::shared_ptr<class ai::Task> __cdecl(void)> >,__int64,struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class std::function<class std::shared_ptr<class ai::Task> __cdecl(void)> > const *,struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class std::function<class std::shared_ptr<class ai::Task> __cdecl(void)> > const &,struct std::_Iterator_base12> const &)

The larger [not_in_pdb] category is comprised of symbols which are not present in PDB. They are: initialization/finalization code for global and static variables, static functions, exception handling, runtime check cookies. Here are some examples:

void __cdecl `dynamic initializer for 'engineVersion''(void)
void __cdecl std::`dynamic initializer for 'piecewise_construct''(void)
void __cdecl `dynamic initializer for 'public: static class idTypeInfo CAbsenceMarker::Type''(void)
void __cdecl `dynamic atexit destructor for 'public: static class idTypeInfo CAbsenceMarker::Type''(void)
int `public: static class Alloc * __cdecl idAAS::Alloc(void)'::`1'::dtor$0
int `public: virtual void __cdecl idAASLocal::DeReferenceDoor(class CFrobDoor *,int)'::`1'::dtor$0
int `public: class dtor$0 & __cdecl idList<class idDrawVert>::operator=(class dtor$0 const &)'::`1'::dtor$0
int `public: void __cdecl idAASLocal::ShowArea(class idVec3 const &)const '::`2'::lastAreaNum
class idPlane `public: virtual class idPlane const & __cdecl idAASLocal::GetPlane(int)const '::`5'::dummy
unsigned int `public: virtual class idPlane const & __cdecl idAASLocal::GetPlane(int)const '::`5'::$S5
void __cdecl Com_EditSounds_f(class idCmdArgs const &)
int `public: static class idClass * __cdecl CAbsenceMarker::CreateInstance(void)'::`1'::catch$0
public: static void (__cdecl* std::_Error_objects<int>::_Iostream_object$initializer$)(void)
S12<`template-parameter-2',idAFEntity_VehicleSimple::wn,unsigned int, ?? &>
int `public: void __cdecl std::allocator<struct std::_Tree_node<struct std::pair<enum ai::EAlertState const ,class idStr>,void *> >::construct<struct std::_Tree_node<struct std::pair<enum ai::EAlertState const ,class idStr>,void *> *,struct std::_Tree_node<struct std::pair<enum ai::EAlertState const ,class idStr>,void *> * &>(struct allocator<struct std::_Tree_node<struct std::pair<enum ai::EAlertState const ,class idStr>,void *> >::_Tree_node<struct std::pair<enum ai::EAlertState const ,class idStr>,void *> * *,struct allocator<struct std::_Tree_node<struct std::pair<enum ai::EAlertState const ,class idStr>,void *> >::std * &)'::`1'::dtor$0
?ToString@idBox@@QEBAPEBDH@Z$rtcName$0

It is indeed very enlightening to look at all this crazy stuff and try to understand what it is about =) Still have no idea what the hell the highlighted line is. It looks like even the undecorating code did not work properly for that symbol.

And now let's look at the new section called "Namespaces and classes contributions", which is the outcome of the "classpath" approach discussed above. Note that the individual methods are not shown here, because the last component of classpath (which is usually method name) is stripped off. The leading .:: is added for implementation simplicity.

Namespaces and classes Contributions
--------------------------------------
Sorted by Size
        Size   Count  Source Path
    20763966  125021  .
    4482084   45792  .::std
    1210159    3772  .::std::_Tree<T>
    1037109    2296  .::ATL
    954888     738  .::ATL::CTrace
    731544    5265  .::idList<T>
    654808    3972  .::ai
    417844    4085  .::idStr
    375015     829  .::idAI
    349640    2411  .::std::basic_string<T>
    289220    2598  .::idVec3
    281714     502  .::idPlayer
    237341    1512  [unknown]
    203972    1032  .::idEntity
    190771    1284  .::std::_Tree_const_iterator<T>
    189928     738  .::idBounds
    175113    3761  .::std::allocator<T>
    171348     232  .::idCollisionModelManagerLocal
    166539    1258  .::std::_Tree_alloc<T>
    165286    2140  .::std::shared_ptr<T>
    153105     293  .::idGameLocal
    152834    1480  .::std::_Iterator_base12
    152768    2514  .::std::_Wrap_alloc<T>
    150042     757  .::idMat3
    145294      62  .::idRenderMatrix
    131388     283  .::idMatX
    128163     453  .::idClass
    126683    2332  .::std::_Ref_count<T>
    122005    1541  .::std::_Func_impl<T>
    121407     294  .::idWindow
    112618     396  .::ai::State
    109591     746  .::std::_Tree_buy<T>
    105087    1042  .::idDict
    102964     827  .::std::_Func_class<T>
       ...     ...  ...

Here are some notes about these results:

With all the non-PDB symbols, STL generates 4.5 MB of code. Out of which at least 1.6 MB are done by highlighted classes, which comprise set/map implementation.
It looks like ATL::CTrace generates 1 MB of code, but judging from the other parts of the report, most of this size consists of .bss symbols, i.e. not code at all.
We confirm that idList template class generates 730 KB of code, idStr generates 420 KB.
Among vector math, idVec3 is obviously the most popular: it generates 290 KB of code. Classes idBounds and idMat3 are also rather widely used.
The whole ai namespace (AI of TDM) generates 650 KB of object code.

As with PDB approach shown previously, here we also see some symbols with [unknown] classpath. It is hard to say exactly why are they unclassified, but given that classpath extraction is done (partly) with manual regex-based recognition, there is no surprise that some crazy cases are not working. The typical problematic cases are: anonymous namespaces and lambda functions. The other fails are complex examples, where single pattern is not enough to deduce classpath. Here are some examples:

class idStr __cdecl `anonymous namespace'::NormalisePath(class idStr const &)
void __cdecl VertexMinMax<class <lambda_e15de9cdacbee52611b15ce13c2dbb01> >(class idVec3 &,class idVec3 &,class idDrawVert const *,int,class <lambda_e15de9cdacbee52611b15ce13c2dbb01>)
bool __cdecl `anonymous namespace'::convert_buffer_utf16<struct `anonymous namespace'::opt_false>(char * &,unsigned __int64 &,void const *,unsigned __int64,struct A0xa82155d7::opt_false)
?parse@xml_parser@?A0xa82155d7@@SA?AUxml_parse_result@pugi@@PEAD_KPEAUxml_node_struct@4@I@Z$rtcName$0
int `class catch$0::basic_ostream<char,struct std::char_traits<char> > & __cdecl std::operator<<<struct std::char_traits<char> >(class catch$0::std &,char const *)'::`1'::catch$0
int `void __cdecl `dynamic initializer for 'sLootTypeName''(void)'::`1'::dtor$0
void __cdecl `dynamic initializer for 'public: static class idCVar <unnamed-tag>::in_mouse''(void)
class <lambda_e261b9d4d68e91887cf921d793e3e07c> `RTTI Type Descriptor'
[thunk]: __cdecl CBinaryFrobMover::`vcall'{1032,{flat}}' }'

As you see, both approaches are not perfect. Both of them sometimes misinterpret or fail to interpret some symbols. But note that it is very hard to even define exactly how to determine source file and owning class for every symbol. The whole analysis is approximate anyway.

Implementation details

PDB approach

It is impossible to detect source code locations of functions using .obj files only, but this information is available in .pdb file. That's why .pdb files must be specified in order to attribute symbols from .obj files to the headers where they are defined. The new parameter -info is introduced to specify such .pdb files: these files are read as usual, but their contents are not counted in the reported statistics. The only way how the contents of these .pdb files influence the report is that the source filenames of all symbols are deduced from them.

If no parameters are specified with -info key, then the analysis is performed the old way, i.e. each symbol is attributed to the .obj file where it is compiled. If at least one -info parameter is specified, then the filename replacing is performed. Aside from that, the analysis continues as usual.

Every pdb file specified as -info is read, and all the symbols are put into special storage. These symbols do not get into the main storage, where the symbols read from .obj files are located. Now we establish correspondence between the symbols of the two storages. For each symbol from the main storage, we find a symbol in special pdb storage with same name, and take its filename to replace the original one.

Console.WriteLine("Connecting symbols to PDB info...");
int connectedCnt = 0, allCnt = symbols.Count;
foreach (Symbol s in symbols)       //"symbols"  --- main storage
{                                   //"infoDict" --- special pdb storage
    Symbol info;
    if (infoDict.TryGetValue(s.raw_name, out info))
    {
        connectedCnt++;
        s.source_filename = info.source_filename;
    }
    else
        s.source_filename = "[not_in_pdb]";
}
Console.WriteLine("Connected {0}% symbols ({1}/{2})", (uint)(100.0 * connectedCnt / allCnt), connectedCnt, allCnt);

The source filename replacement happens in the highlighted line. Here is an example for one symbol:

s.name = "public: __cdecl idList<class idVec4>::~idList<class idVec4>(void)"
s.raw_name = "??1?$idList@VidVec4@@@@QEAA@XZ"
s.source_filename = ".\DarkModTools\AAS.obj"                                // the original source filename (replaced)
info.source_filename = "c:\thedarkmod\darkmod_src\idlib\containers\list.h"  // the source filename from PDB (taken)

Correspondence between symbols is established based on their raw names, i.e. the decorated/mangled names. The original version of SymbolSort did not read these names neither from comdat nor from PDB inputs, so the code must have been extended for it. Let's start with comdat output of .obj file. Here is how it looks like:

SECTION HEADER #30
.text$mn name
       0 physical address
       0 virtual address
      2E size of raw data
    87CC file pointer to raw data (000087CC to 000087F9)
    87FA file pointer to relocation table
       0 file pointer to line numbers
       1 number of relocations
       0 number of line numbers
60501020 flags
         Code
         COMDAT; sym= "public: __cdecl idList<class idVec4>::~idList<class idVec4>(void)" (??1?$idList@VidVec4@@@@QEAA@XZ)
         16 byte align
         Execute Read

The undecorated name is extracted from the highlighted line using simple regex, and it is trivial to extend the regex to extract the raw name as well.

For the PDB file, extracting decorated name is a bit harder. It turns out that there is a major distinction between public and private symbols in PDB. In particular, all public symbols have raw symbol name stored in IDiaSymbol.name, while private symbols have human-readable name stored in it. Unfortunately, I did not know about this distinction, and SymbolSort extracts only private symbols by default, so I spent a lot of time trying to get raw name out of private IDiaSymbol-s. I even found one way which works for most private symbols for no apparent reason (and fails on some of them, also for no apparent reason). Later I switched to using public symbols.

symbol.short_name = diaSymbol.name == null ? "" : diaSymbol.name;
symbol.name = diaSymbol.undecoratedName == null ? symbol.short_name : diaSymbol.undecoratedName;
symbol.flags = additionalFlags;

if (type == SymTagEnum.SymTagPublicSymbol)
{
    symbol.raw_name = symbol.short_name;
}
else
{
    //there is no reason this can work, but it often works...
    string rawName;
    IDiaSymbolUndecoratedNameExFlags flags =
        IDiaSymbolUndecoratedNameExFlags.UNDNAME_32_BIT_DECODE |
        IDiaSymbolUndecoratedNameExFlags.UNDNAME_TYPE_ONLY;
    diaSymbol.get_undecoratedNameEx((uint)flags, out rawName);
    if (rawName != null)
    {
        //ignore trashy names like " ?? :: ?? ::Z::_NPEBDI_N * __ptr64 volatile "
        if (!rawName.Contains(' '))
            symbol.raw_name = rawName;
    }
}

The first highlighted line simply takes IDiaSymbol.name as raw name, provided that symbol is public. Also I enabled a flag so that public symbols are always extracted and they take precedence, so probably the case of private symbols is not necessary. For a private symbol, IDiaSymbol.name contains partly undecorated name, but calling get_undecoratedNameEx method with weird flag often returns correct result, e.g.:

diaSymbol.name = "idList<idVec4>::~idList<idVec4>"
diaSymbol.get_undecoratedNameEx(UNDNAME_TYPE_ONLY) = "??1?$idList@VidVec4@@@@QEAA@XZ"

The last problem which I faced with PDB approach was how source filenames were determined for symbols from pdb. Normally, SymbolSort asks DIA framework about the lines of code which correspond to symbol's address, and if it finds any, then it saves the file path of the first such line. So it seems that the filename of a pdb symbol is exactly the source file where debugger would stop if you put a breakpoint into the symbol. However, if a pdb symbol has no corresponding lines in code, then SymbolSort detects the filename by looking which translation unit provided the code at its address. This is exactly what we tried to avoid in the first place, ignoring the fact that with such mechanism all the duplicates of one symbol get attributed randomly to one of the many source files where it is used. That's why every symbol which has no lines of code in PDB is marked accordingly, so that its filename is later replaced with [unclear_source] to avoid confusion.

Classpath approach

In a basic case, every code symbol is a method of some class, which is probably contained in other classes or namespaces. This whole construction (e.g. boost::container::map::insert) can be called "classpath", which is similar to file path, but with namespaces and classes instead of directories and files. The idea is to determine classpath of each symbol from its name, then strip its last component (it is usually the method's name), and attribute the symbol to the remaining classpath. Note that we remove the method name for two reasons: 1) we want to obtain per-class statistics, per-method report can be seen elsewhere, and 2) there are too many methods to list them all in the report.

As simple as it may sound, extracting classpath is actually quite hard. Direct processing of fully undecorated symbol name is problematic because it has a lot of excessive info which can get in the way. While it is possible to remove template arguments by simply detecting outer-level angle brackets (which SymbolSort already does), simply splitting the symbol name by "::" won't work even in simple cases. Consider the following examples:

class std::_Ref_count_base * && __cdecl std::_Move<class std::_Ref_count_base * &>(class std::_Ref_count_base * &)
public: __cdecl std::_Wrap_alloc<class std::allocator<char> >::_Wrap_alloc<class std::allocator<char> >(class std::allocator<char> const &)

As you see, return values, function parameters and template arguments can all have internal "::" separators, which must not be used to split the classpath. Luckily, some of the routines for undecorating symbols support partial undecoration. Calling WinAPI function UnDecorateSymbolName with flag UNDNAME_NAME_ONLY results in undecorated name without return value and function parameters:

UnDecorateSymbolName("?Warning@idLib@@SAXPEBDZZ", 0):
  public: static void __cdecl idLib::Warning(char const * __ptr64,...)"
UnDecorateSymbolName("?Warning@idLib@@SAXPEBDZZ", UNDNAME_NAME_ONLY):
  idLib::Warning

The partially undecorated symbol names are much easier to use: in the considered example the result is already a ready-to-use classpath. However, not all the cases are as simple as this one. Here is a hacky regex-based code which handles most of the cases (with tricky symbol names seen in comments):

private static string allowedSpecials = @"<=>,\[\]()!~^&|+\-*\/%" + "$";
private static string reClassWord = @"[\w " + allowedSpecials + "]+";
private static string reClassPath = String.Format(@"({0}::)*{0}", reClassWord);
private static Regex regexClassPath = new Regex("^" + reClassPath + "$", RegexOptions.Compiled);
private static string reLocalEnd = @".*";  //@"(`.+'|[\w]+(\$0)?)";
private static Regex regexFuncLocalVar = new Regex(String.Format(@"^`({0})'::`[\d]+'::{1}$", reClassPath, reLocalEnd));

public static string[] Run(string short_name) {
    //(all string constaints)
    if (short_name == "`string'")
        return new string[] { short_name };

    // Array<SharedPtr<Curve>, Allocator<SharedPtr<Curve>>>::Buffer::capacity
    // std::_Error_objects<int>::_System_object$initializer$
    if (regexClassPath.IsMatch(short_name))
        return splitByColons(short_name);

    // std::bad_alloc `RTTI Type Descriptor'
    const string rttiDescr = " `RTTI Type Descriptor'";
    if (short_name.EndsWith(rttiDescr)) {
        string[] res = Run(short_name.Substring(0, short_name.Length - rttiDescr.Length));
        if (res == null) return null;
        return res.Concat(new string[] { rttiDescr.Substring(1) }).ToArray();
    }

    // `CustomHeap::~CustomHeap'::`1'::dtor$0
    // `std::basic_string<char,std::char_traits<char>,std::allocator<char> >::_Copy'::`1'::catch$0
    // `CustomHeap<ShapeImpl>::instance'::`2'::some_var
    // `HeapWrap < ShapeImpl >::Stub::get'::`7'::`local static guard'
    // `HeapWrap<ShapeImpl>::Stub::get'::`7'::`dynamic atexit destructor for 'g_myHeap''
    // `Mesh::projectPoints'::`13'::$S1
    // `GroupElement::getNumElements'::`2'::MyCounter::`vftable'
    if (regexFuncLocalVar.IsMatch(short_name))
        return Run(regexFuncLocalVar.Match(short_name).Groups[1].Value);

    // `dynamic initializer for 'BoundingBox::Invalid''
    // `dynamic initializer for 'std::_Error_objects<int>::_System_object''
    // std::`dynamic initializer for '_Tuple_alloc''
    // UniquePtr<Service>::`scalar deleting destructor'
    if (short_name.EndsWith("'")) {
        int backtickPos = short_name.IndexOf('`');
        if (backtickPos >= 0) {
            string prefix = short_name.Substring(0, backtickPos);
            string quoted = short_name.Substring(backtickPos + 1, short_name.Length - backtickPos - 2);
            if (quoted.Count(c => c == '\'') == 2) {
                int left = quoted.IndexOf('\'');
                int right = quoted.LastIndexOf('\'');
                quoted = quoted.Substring(left + 1, right - left - 1);
            }
            string[] quotedWords = Run(quoted);
            if (quotedWords == null)
                return null;
            string[] prefixWords = splitByColons(prefix);
            return prefixWords.Take(prefixWords.Length - 1).Concat(quotedWords).ToArray();
        }
    }

    return null;
}

Unfortunately, we are not yet over with classpath approach =) It turned out that UnDecorateSymbolName function is no longer maintained. This function belongs to Dbghelp.dll library, the latest version of which was released in February 2010. It was even before C++11 was out, so this function fails to work on symbols containing rvalue references. There is an updated version called _unDName, which is not exposed to public, and even newer version called __unDNameEx, also for internal usage only. The latter version is used in undname.exe program which is a tool distributed with Visual C++.

In the end I decided to use undname.exe directly by creating a temporary file and spawning it with CreateProcess. The tool allows to put many symbol names into file, in which case it undecorates them all, so there are no performance problems like creating process thousand times. Also, it is safe to assume that user has undname.exe in PATH, since he anyway has to run dumpbin.exe to decode .obj files, which is located in the same directory.

Afterword

I hope the two improvements will provide a better vision of how much bloat headers and templates produce. In my opinion, both approaches are useful, despite being hacky and not 100% precise. Right now they are available in my fork, but I will prepare pull requests in the nearest future.

SymbolSort has two more features which I never tried: analyzing nm dumps and producing diff reports. The latter feature may be very useful in continuous integration if you really care about build times and code bloat.

On C++ code bloat

2018-05-13T19:15:00+07:00

The C++ language is known for its slow build times. This problem is not present in the world of pure C, which might give a hint that the problem is caused by some C++ feature. In fact, it is caused by bad habit or writing code in headers, severely worsened by C++ templates and inspired by STL itself.

Headers

The first reason of code bloat is the linking model, which is inherited directly from C. Suppose that a function or a method is defined in header file like this:

// ===== utils.h =====
#pragma once
#include <numeric>
#include <vector>

inline int calcSum(const int *arr, int num) {
    int sum = 0;
    for (int i = 0; i < num; i++)
        sum += arr[i];
    return sum;
}

struct Foo {
    std::vector<int> arr;
    int run() const {
        return std::accumulate(arr.begin(), arr.end(), 0);
    }
};

Both calcSum and Foo::run are functions defined in header file. So each of them will be compiled separately in every source file where it is used (directly or indirectly). In some cases (dllexport, virtual method, or just old MSVC) such functions are compiled in every source file which includes this header, even if they are not used! One particularly nasty type of header-defined functions is comprised of methods generated implicitly by the compiler, i.e. destructor, copying, constructors. Last but not least, a dynamic class also needs virtual functions table and RTTI data, and every source file where at least one header-defined constructor is called (including implicitly generated one) also forces compiler to generate all this data.

When the project is large, it contains hundreds of source files, so every popular function defined in header may easily be compiled hundreds of times. One may hope that compiler is clever, that it knows that these functions are same in every source file, and that it manages to do all the hard work only once. But I have never heard of anything like this. Even though linker will merge all the duplicate symbols across .obj files into one, you will get enough time for some swordsplay before it happens =)

Templates

And now we come to C++ templates. Almost every modern language supports generic programming of some sort, but C++ calls this feature "templates" instead of "generics". C++ templates work like a very advanced code generator integrated into the language: every instance of a template is a completely separate piece of code generated by compiler (although linker can merge some methods if their machine code is exactly the same). Consider an example:

// ===== utils2.h =====
#pragma once
#include <numeric>

template<class T> T calcSum(const T *arr, int num) {
    return std::accumulate(arr, arr + num, T(0));
}

template<class T, int N> struct Bar {
    T arr[N];
    T run() const {
        T res = 0;
        for (int i = 0; i < N; i++)
            res += arr[i];
        return res;
    }
};

Suppose that the program has only one source file. Then function calcSum is compiled once per each used type of template parameter T. The function Bar::run is compiled as many times as there are different <T, N> pairs for which the method run is called. This particular case can be quite disastrous because size N can vary a lot. There are some techniques to diminish the problem (e.g. use type-erased base class), but they are not simple.

Also, the aforementioned issue with headers also applies to templates, because templates are almost always defined in header files. In fact, there are two only ways to avoid duplication of template functions across translation units:

Define template functions in source file and explicitly instantiate them there. Unfortunately, it is possible only if you can enumerate all the wanted instances of the template in this source file. Surely, this approach cannot be applied to container classes like std::vector.
(C++11) Use extern template declarations in header, combined with explicit template instantiations in source files. This can work for class templates only if methods are defined outside of the class body. Needless to say, today everyone has a habit of defining methods inside class body. In particular, this is how STL is usually implemented, so extern templates won't help with STL.

Although both reasons of template code bloat can be addressed to greatly reduce the issue, almost nobody does so. Because the code written without concern works perfectly well! It is smaller, simpler, faster to write and easier to read, and easier to maintain. Why bother wasting time on something when it is not necessary? And when the project grows large enough that build times start causing pain, it may be too late already to change your mind.

Questionable solutions

Some people think that precompiled headers is the way to reduce code bloat. This is not the case: precompiled headers are used to reduce the cost of parsing header files many times. This is yet another inefficiency in C++ linking model: each header has to be parsed once again for every translation unit it is included in. The precompiled header may remove some of code bloat in MSVC, but it definitely won't help with functions which are used only after PCH ends.

There are some hopes that not-yet-standardized C++ modules will fix the problem. As it seems now, upcoming C++ modules are simply modular precompiled headers. They will definitely solve problems with parsing, but I'm not so sure about code bloat. The latest draft (N4465, section 4.10) has some rules for instantiations of exported templates, which looks like an improvement from what we have know. The time will show, I guess.

The most widely used solution to fight code bloat (across translation units) is to use unity builds, i.e. merge all source files into one. In other words, instead of horrendously abusing linking model, people simply throw it away. Of course, this solution backfires at the speed of incremental builds.

Binary size

Often when people hear "code bloat", they think only about the size of the resulting binary. Things are much better in this regard: linker merges all occurences of the same symbol across the whole module, so at most one copy of function's machine code remains, even if the function is defined in header. Moreover, as mentioned above, different functions with bytewise equal machine code are often merged together too. So the trashpile of a gigabyte of object files often links into 10 MB binary, and it looks like there is no problem at all. However, things are not so perfect.

First of all, modern programs are often split into many dynamic libraries. The funniest reason for doing this is that sometimes linker simply cannot swallow all the object files in one batch due to their insane size. Each DLL is a separate module and is linked separately, so the merging of symbols does not work across modules. Therefore, in a program with 10 DLLs you can easily find 10 duplicates of std::vector<int> methods in total.

The second problem is the inlining optimization itself. When inlining, compiler takes the code of a function and copies it into the call site. In the worst case, the size of the resulting binary could increase by the size of the function inlined. Compilers are very careful about this, since careless inlining could lead to insane code size growth. All compilers weight carefully potential performance benefits versus code size increase. But while many C++ programmers prefer to think of the optimizer as an omnipotent being which will always do everything in the best way possible, optimizer has very little information and has to resort to simple heuristics when it comes to inlining decision. In a world where half of the code lives in headers and is inlinable, compiler is likely to inline more that necessary. For instance, look at the code of this function:

void doit(std::unordered_map<int, int> &mymap) { mymap[16] = 13; }

Among three popular compilers, MSVC 2017 generates 286 bytes of code, GCC 8.1.0 generates 182 bytes, and only Clang 8.0.1 generates 43 bytes and calls non-inlined _Try_emplace method directly (all checked on Windows with optimization enabled). This does not sound too bad, just 100 more bytes, but when this situation happens on every step, who knows how much bloat is generated in total.

(section added on March 2020).

Conclusion

The problem of C++ code bloat is so severe today, that vast majority of build time is wasted on unnecessary duplicates instead of any useful compiling. And the duplication is caused by C++ linking model not being respected. Any code present in a source file won't cause problems, but every function defined in header is a candidate for unlimited duplication. The templates are especially harmful, since they generate code bloat across two dimensions: across template parameters and across translation units.

In my opinion, the overuse of templates today is caused by modern trends in C++. Moreover, these trends have organically grown up from the Standard Template Library itself, which by the way typically causes a lot of code bloat in any project. Given that rare project is brave enough to throw STL away, it means that code bloat will almost certainly stay with us. Gamedev is perhaps one of the largest areas with enough distrust towards STL, let's hope they will not fall to the trend.

As of now, we can only learn how compilation works and try to avoid too much code bloat. Define stuff in source files as much as possible and prefer virtual methods or type erasure over templates when performance is not critical. If you are still not convinced that the issue is serious, please review the C++ recommendations in Chromium project, the first half of which is solely dedicated to the "don't define stuff in headers" mantra.

Blog migrated to Pelican and GitHub pages

2018-05-02T20:07:00+07:00

This article is a sort of report about blog migration.

On wordpress.com

Initially, I started this blog on wordpress.com. I chose wordpress.com thinking that I only need a small personal blog and I can easily live without any advanced features. I was pretty sure that I will post only technical articles, and do so quite rarely, so I searched for a free solution and wordpress.com was a good fit. And the experience was quite good in general: I did not have to configure anything, just choose a classic theme and go on writing.

A problem which I noticed rather quickly is that the width of the text in my blog was way too small. The amount of text fitting into one line was small, and it was especially ugly for source code fragments. When I wanted to post a piece of code from somewhere, I had to add split lines into pieces, so that the whole code fits and no scroll bar appears.

I even had to simplify my own code because of this problem. The following piece of code from Doom 3 is a good example (the new version has plenty of space):

Fixing this problem is rather easy with CSS, but cannot edit CSS unless you switch to premium plan and pay monthly. I tried to search for new themes, but to no avail. At the end I decided that I should search for another hosting.

One way to get away from this limitations is to use wordpress.org on self-hosted site with custom domain. But it costs money again. Given that I do not need a full-blown website, I do not want to pay. In the age when I can freely share one gigabyte file or video with everyone, having to pay money for serving a few megabytes of website data is plain stupid.

Static website

Finally, I decided to move the blog to GitHub pages. It is free, and the concept of static blog is simple, fast, secure. Of course, there is a serious disadvantage: you have to install, configure, fine-tune a lot of stuff, fix some css styles, html templates. I would definitely advise against creating static blogs to all people who are either not familiar with website internals or not willing to spend some time to hack things together. Also, dynamic stuff (like comments) can be implemented only with ugly workarounds.

First of all, I had to choose a static blog generator. There are several such generators. The full list can be seen on netlify's staticgen.com site. All of them allow you to write articles in Markdown and apply various themes to the blog. Here are the options I chose from:

Jekyll in Ruby. This option is clearly the most popular, and it is the only one integrated into GitHub (builds are simpler). Even though I have no experience with Ruby, I wanted to choose this option initially. But then I discovered that Jekyll is not supported on Windows. Moreover, you cannot even use Ruby packages without downloading and installing 1GB Linux subsystem. It became apparent that this ecosystem does not care for Windows users (which I am one of).
Hexo in javascript. This option is less popular, but will probably rise in the nearest future. Using javascript for generating websites is a great idea, because any client-side code has to be in javascript anyway, so it is more convenient to use one language in both areas. I have suffered enough from javascript on my daily job, so I decided to avoid this too.
Pelican in Python. This option is probably even less popular than Hexo. Python is a great language for a variety of tasks, and my experience with it is strongly positive. It is very user-friendly, truly cross-platform, does not have tons of stupid legacy stuff like javascript.

I decided to use Pelican at the end.

Quick-start

First of all, I created a pipenv environment, which works similar to how npm in node.js does. Although I never used pipenv before, it seems that it would be much easier to run Pelican on another machine if I do so.

I ran pelican-quickstart command to generate some initial files. Then I deleted the generated makefile and shell script, since there are of no use for me on Windows. It seems that they are duplicated by fabfile.py, which is also generated. To simplify running fabfile tasks, I created fab.cmd file:

pipenv run fab.exe %*

After I got familiar to existing fabfile tasks, I realized that I want some sort of watch & live reload option similar to what wevpack-dev-server provides. Following the article by Merlijn van Deen, I have added a livereload task to fabfile.py. It is based on livereload Python package and requires installing browser extension. Unlike the cited article, I later extended the livereload task to also watch over theme and config files:

def livereload():
    """Serve site at http://localhost:5500/ automatically livereload in browser"""
    # adapted from: https://merlijn.vandeen.nl/2015/pelican-livereload.html
    p = None
    def build_content():
        try:
            p.run()
        except SystemExit as e:
            pass
    def build_all():
        nonlocal p
        p = Pelican(read_settings('pelicanconf.py'))
        build_content()
    build_all()
    server = livereload_Server()
    server.watch('content/', build_content)
    server.watch('theme/', build_content)
    server.watch('pelicanconf.py', build_all)
    server.serve(root='output')

As a result, I can run editor on the left half of the screen, and browser on the right shows the current look of the article being edited:

Converting articles

I had to convert five articles from my wordpress.com blog, which are available only as html files. It turned out that Markdown processor carefully copies all html tags into the output, so even putting html article into markdown file is enough to get it rendered. Here is what I had to do for every article:

Copy article source as html into markdown file. Article source is shown on wordpress.com if you edit your article in html tab.

Add meta information at the beginning of the .md file:

Title: Blog migrated to GitHub pages and pelican
Date: 2018-04-19 23:26
Category: Off-topic
Tags: blog, pelican, gh-pages

Fix all code blocks: use indentation and :::language instead of [code] tags, e.g.:

The code starts by iterating over the elements to be moved:

    :::cpp
    for (size_t i = 0; i < Count; i += 4) {

(Optional) Download all the images used in the article from wordpress.com, then fix all links to them in markdown files. Without doing so, the images will still be loaded from wordpress.com.
```
<img src="{filename}/images/0_doom3_typeinfo/uninitialized_warning_console.png" ... />
```
(Optional) Replace all html header tags with markdown headers. This was necessary because markdown toc extension does not understand html tags. So I had to do it later to make all headers referenceable.
Manually improve each and every image in the blog. First of all, I had some wordpress-specific images ([gallery] tag) which had to be converted. Second, every image had to be resized and repositioned to look nice. In most cases, now I use something like:
```
<img
    src="{filename}/images/0_doom3_typeinfo/typeinfo_scheme1.png"
    class="centered w500"
/>
```

Choosing a theme

Pelican has a set of themes which define the visual look of the blog and provide some features. Each theme consists of html templates for the pages, css styles, and sometimes a bit of javascript code. In fact, there are two criteria when choosing a theme:

How nice it looks: layout, fonts, colors, etc.
Which features it supports: responsiveness, categories and tags pages, integration of disqus comments and tipue search.

With livereload enabled, I started iterating over all themes in the repository of themes. But no theme perfectly matched my wishes. In fact, later I have learnt two things about choosing a theme:

Many themes by default turn off considerable part of their features. So, a brief visual look at the theme is not enough: you have to read through the theme's readme to see which features and additional customization are supported.
Taking an existing theme "as is" is unlikely to be enough anyway. Most likely you'll have to tweak a lot of stuff to your liking, which will require css + html knowledge and some time. With this point in mind, it is worth looking into theme's source code and see if you are ready to hack with it.

First I thought about using elegant or pelican-twitchy theme, since they have most features from the very beginning. But when I started investigating the topic of comments and search, it became clear that 1) search is easy to integrate, 2) I dislike out-of-the-box solutions for commenting. So I decided to concentrate more on the visual look of a theme and chose blueidea theme. As an additional benefit, this theme is also rather simple: all css styles (except code highlighting) are written in single css file, no javascript is used.

Customizing the theme

My initial idea was to list all the tweaks and customizations which I did to the blueidea theme. But while I spent more time tweaking html and css, I did more and more changes all over the code. At the end, the tweaking went out of control =) So I'll try to provide an overview of the changes and only describe the moments which I consider interesting.

Responsive

I made the theme responsive and mobile-friendly (originally this theme was fixed-width) with CSS3. To my surprise, it was very easy to do.

First, I added meta to base.html to disable scaling on mobiles:

<meta name="viewport" content="width=device-width, initial-scale=1.0"/>

Then I refactored main.css so that all width-related sizes were computed from a single variable. I added a main-width variable on body (which is sort of global):

body {
    --main-width: 1200px;
    background: #f2f3f3;
    /*...*/
}

And fixed all places where the width of the text area was used implicitly. Now it is deduced via calc function, e.g.:

#content {
    /*...*/
    padding: 20px 20px;
    width: calc(var(--main-width) - 40px);
    /*...*/
}

Finally, I added logic of choosing smaller values of main-width if browser screen width is not wide enough (using media queries):

@media screen and (max-width: 1400px) {
    body {
        --main-width: 1000px;
    }
}
@media screen and (max-width: 1100px) {
    body {
        --main-width: 800px;
/* ... more cases follow */

Another problem with blueidea theme is how it displays menu buttons on a small screen. Below you can see the old look on the left, and the new on the right (I marked the issue in red):

To ensure that buttons are correctly moved to the next line on overflow, it is enough to do the following:

#banner nav {
    overflow: auto;
    min-height: 40px;  /* instead of "height: 40px;" */
    /*...*/
}

Finally, I remember having to insert clear: both; somewhere to fix layout after right-aligned buttons.

Reference headers

I enabled markdown toc extension with anchorlink=True to make headers referenceable. Here is the corresponding section from pelicanconf.py:

MARKDOWN = {
    'extension_configs': {
        'markdown.extensions.toc': {'anchorlink': True},  #referenceable headers
        'markdown.extensions.codehilite': {'css_class': 'highlight'},
        'markdown.extensions.extra': {},
        'markdown.extensions.meta': {},
    },
    'output_format': 'html5',
}

It took me quite some time to realize that it does not apply to <h3>-like tags (i.e. html headers). So I had to replace html headers with their markdown equivalents in all the articles.

Menus and pages

I changed the set of items displayed on the top menu (in pelicanconf.py):

# hide categories in the main menu (little space there)
DISPLAY_CATEGORIES_ON_MENU = False
# display categories in a small submenu instead
DISPLAY_CATEGORIES_ON_SUBMENU = True

# buttons displayed in the main menu
MENUITEMS = (
    ('ARCHIVES', '/archives.html'),         # list of all articles (compact)
    ('CATEGORIES', '/categories.html'),     # list of categories (with counts)
    ('TAGS', '/tags.html'),                 # list of tags (with counts)
    ('CONTACT', '/pages/contact.html'),     # ordinary page with email address
)

# global html templates which are always rendered (even if unreferenced)
DIRECT_TEMPLATES = [
    'index', 'archives',
    'categories', 'tags',
    # 'authors',   --- don't need the list of authors
]
# do not generate per-author index pages
AUTHOR_SAVE_AS = ''

Featured article

I disabled featured article on all the index pages (in index.html template). My articles are usually too long, and having a featured article makes index pages indistinguishable from article pages.

    :::jinja
    {# note: DISPLAY_FEATURED_ARTICLE = False in pelicanconf.py #}
    {# First item #}
    {% if DISPLAY_FEATURED_ARTICLE and loop.first and not articles_page.has_previous() %}
        <aside id="featured" class="body">

Plugins

I enabled the following plugins:

render_math plugin to render LaTeX formulas via MathJax.
sitemap to aid indexing by search engines.

summary for configurable summaries on index pages. I configured summary plugin to detect  tag as end of summary, exactly as on wordpress.com:

PLUGIN_PATHS = ['plugins']
PLUGINS = [
    'summary',
    # ...
]

SUMMARY_BEGIN_MARKER = '<!--summary-->'
SUMMARY_END_MARKER = '<!--more-->'  #as on wordpress.com

share_post for buttons like "Share on Twitter". I spent huge amount of time on styling these buttons! I even had to draw a few missing icons myself=( This is because blueidea theme does not support share_post plugin out-of-the-box: it is much easier to choose a theme which supports all the features you want.
tipue_search for searching over the website (discussed in more detail later).
pelican_comment_system for user comments (discussed in more detail later).

Tipue search

The blueidea theme has builtin integration with duckduckgo search engine. The whole integration is trivial: when you enter some text into search box, trivial javascript callback redirects you to duckduckgo.com address with entered text and your site in http parameters:

{% if DISPLAY_SEARCH_FORM -%}
    <form id="search" action"#" onsubmit="javascript:window.open('https://duckduckgo.com/?q='+document.getElementById('keywords').value+'+site:{{ SITEURL }}');">
        <input id="keywords" type="text" />
    </form>
{% endif %}

This relies on external search engine and has its benefits and drawbacks. The main benefit is that google is much more intelligent and surely can do some stemming on your query. The drawback is that this search will only work properly after the up-to-date version of site gets visited by search engine crawlers.

So I decided to go the full static route and integrate full-javascript search instead. Luckily, tipue search is already available for this purpose. It is integrated into several pelican themes, including the aforementioned elegant theme.

The way it works is very simple: during pelican build, it produces file tipuesearch_content.json, which contains the plain text of each article. When user enters some keywords into the search box, he is redirected to search.html page (with entered text passed as http parameter). Here tipuesearch.js runs. It checks how much keywords are present as substrings in each article and sorts articles by this number decreasing (well, title and tags also influence the score). The articles which contain none of the keywords are excluded from the results list. Finally, it splits results into pages and generates their html representation into search-content element.

The integration of tipue search into pelican site is very well covered in the mygeekdaddy's article. I won't repeat everything, just mention a few important things:

I had to change tipue_search.py file in the source code of tipue_search plugin, as described here. Due to some weird reason, the plugin puts addresses into url property, while the javascript code expects them in loc property. This is easily fixed by updating two places in the python code:
```
node = {'title': page_title,
        'text': page_text,
        'tags': page_category,
        'loc': page_url,  #note: write address to 'loc' too
        'url': page_url}
```
Since search.html template is not referenced directly in any html files, pelican will not render it by default. To force rendering of this file, it must be marked as direct template in pelicanconf.py:
```
DIRECT_TEMPLATES = [
    'index', 'categories', 'authors', 'archives',   #(default)
    'search', #tipue search results
]
```

Comments

Integrating comments is a big problem for all static blogs. Mainly because whenever reader leaves a comment, it must somehow alter the website, which contradicts with the fact that it is static. A static blog has no server which could receive the comments from user and update the blog contents. On the other hand, a static blog can communicate with foreign servers all over the web.

Overview

There are several known ways to integrate comments into static blog:

Integration with Disqus. This is the simplest and the most common choice, it is integrated in most pelican themes, including blueidea theme. In this case all the comments are stored on Disqus servers, the blog only displays them and works as a frontend for adding more comments to those servers. While Disqus is a good option in general, there are some negative views on it over the web: ads and sponsored comments, ownership privacy and openness questions, slowness and privacy again.
All the comments are stored in the blog's repository and they are compiled by blog generator just as articles do. The pelican_comment_system plugin does this exactly: it translates comments from markdown files into the html code, just as pelican does for all articles by default. So the main question is how can reader comments get from the static website into the repository. Here are some options:

2.a. Reader sends his comment to the author of blog via email. He can either do it manually, or click on a button with mailto: link. The default template comments.html of pelican_comment_system plugin works this way.

2.b. Reader sends his comment to staticman server, which commits it into predefined GitHub repository. It either pushes changes directly or submits pull requests which blog owner has to review and approve. It is possible to configure so that comments appear on the site in almost real time soon after reader posts it. To achieve it, some continuous integration is needed (or simply using Jekyll and GitHub pages). You can read more about staticman integration e.g. in this article.

2.c. When reader decides to leave a comment, he is redirected to GitHub page of an issue in blog owner's repository. This issue is created in advance by blog owner specifically for the article being commented. Reader posts his message as a comment on this GitHub issue, and it is saved there. The blog owner uses GitHub public API to fetch comments on the issues at any time. This workflow is described in this article and is implemented in gitment project for Hexo.

Choice

While I think Disqus is perfectly suitable for comments, I once again decided to go the "purist's way" and treat all the comments as part of my blog. The option 2.b is very nice, but it looks quite complicated and it relies on the third-party server (api.staticman.net). Also it ties your blog repository to GitHub, which btw does not allow free private repositories. The option 2.c is rather simple. But it requires user to have GitHub login (unlikely a problem though) and it redirects user to GitHub site to even start writing his comment (maybe this is fixable).

I chose the option 2.a with sending comments over email. In my opinion, it is the most proper way of commenting static sites. It returns us back to roots: if you want to say something, just send an email to the author. No intermediary is needed, embrace direct communication.

The email-based system does not depend on any third party. Since everyone uses emails for a long time, it is much more reliable than any custom server. And even if e.g. Gmail becomes unusable, I can easily switch to another email provider. Everyone who reads technical blogs has an email box (probably even several of them), so no special login is required. Commenting is as secure as emails are, since no information leaves reader's machine except for the email, which he sends himself.

Implementation

The first step was to enable the plugin itself. In order to play with it a bit, I manually converted all comments from the wordpress.com site into markdown files. It did not take much time, since I don't have many comments yet.

The html template and javascript code was initially taken from the plugin's official example (html and js). However, I had to change it quite heavily. Also, I had to write all the styles myself. The blueidea theme supports only Disqus comments, and I could not even see how these comments were originally supposed to look. The major points are listed below.

Collapsible. The whole commenting section is collapsible. In fact, it is collapsed by default (unless javascript is disabled). This is one of the arguments used to popularize elegant theme =)

The collapsing and expanding is done by javascript code, animation is done via css transition. I was quite unhappy that there is no way to animate max-height to the actual height of the content, so I had to do some hacks:

//getting height of potentially hidden element: https://stackoverflow.com/a/2345813/556899
var previousStyle = content.attr("style");
content.css({position: 'absolute', visibility: 'hidden', display: 'block', maxHeight: 'none'});
fullHeight = content.height() + 100;
content.attr("style", previousStyle || "");

And another hack was needed to trigger transition when I change max-height property:

content.css({maxHeight: (isExpanding ? 0 : fullHeight) + 'px'});
content.css('maxHeight');   //why is this needed?
content.css({maxHeight: (isExpanding ? fullHeight : 0) + 'px'});

View email source. Reader can click a button to see the generated email contents before sending it. This is especially helpful if reader's machine is not properly configured to open mailto links: then he can copy/paste the message into his web client and send it manually (avoiding mailto link).

Move form on reply. When reader clicks a "reply" button near existing comment, the comment form moves to the place immediately after it. Also, dynamic content showing that currently a reply is being composed is moved to the header (legend).

Spam protection. The sample code has some protection of blog author's email, but not very good, so I decided to obfuscate it a bit further. Plus, added a minor nuisance for a potential bot filling the form.

Mailto config help. I added a link to page where it is explained how to configure Chrome to open mailto links in Gmail. I think it is important, because a lot of people use web mail client but don't know how to open mailto links with it. To be honest, I plan to create a special page behind this button, which would thoroughly explain how commenting system works and how to use it easily.

Even-odd coloring. To distinguish consecutive comments from each other, every even comment has gray background. Astonishingly, css even has a rule nth-child(2n) for it, which is used in blueidea styles for Disqus comments. However, it works incorrectly with nested enumerations. The problem is fixed by assigning even/odd classes to comments during html generation. Notice the namespace hack to circumvent scoping rules of jinja:

<ul>
{% set recursive_counter = namespace(a=0) %}
{% for comment in article.comments recursive %}
    {% set recursive_counter.a = recursive_counter.a + 1 %}
    <li id="comment-{{comment.slug}}">
        <div class="comment-left">
            <!-- ... -->
        </div>
        <div class="comment-body">
            <div class="comment-self {{'even' if recursive_counter.a % 2 else 'odd'}}">
                <!-- ... -->
                <div class="pcs-comment-content" {# class used as id in comments.js#}>
                    {{ comment.content }}
                </div>
            </div>
            {% if comment.replies %}
                <ul>
                    {{ loop(comment.replies) }}
                </ul>
            {% endif %}
        </div>
    </li>
{% endfor %}
</ul>

Android issue

I stumbled upon a nasty issue in Chrome browser on Android. When I open an article from my blog in the browser, it loads the page, and soon after that it hangs for noticeable time. The whole process is completely frozen. It becomes unresponsive and OS soon suggests to kill it. Chrome profiler (via remote debugging) shows nothing, so the hang happens somewhere in the internals.

I noticed the following:

If I remove all images from the page, the issue still happens. If I remove all javascript code, the issue still happens. If I remove all css files, the issue still happens. In fact, even if I remove everything except the html file itself, the issue still happens.
The issue does not happen if I remove all code fragments from the page.
The length of the freeze is dependent on the page, but on this article it takes literally minutes.
The freeze happens on Chrome for Android recent version (66.0.3359.126). It does not happen on stock version of Chrome (55.0.2883.91), and it does not happen on Firefox.

So it looks like a bug caused by bulky html of the code fragments. I'll probably try to submit a bug report about it.

Publish

The very last thing for a new static website is to publish the result.

All the differences between local development environment and deployment environment are usually described in publishconf.py file. By default, command fab preview can be used to build website with this publishing config. Here is what I have in my publishconf.py:

from pelicanconf import *

SITEURL = 'https://dirtyhandscoding.github.io'
RELATIVE_URLS = False

OUTPUT_PATH = 'publish_output/'
DELETE_OUTPUT_DIRECTORY = True
OUTPUT_RETENTION = [".hg", ".git", ".nojekyll"]

As you see, the differences are minimal:

The final URL of the site is specified, so all the internal links are expressed as absolute paths using it.
The result is generated to a different publish_output subdirectory. The contents of this directory are cleaned before every build, except for .hg and .git subdirectories (mercurial/git repository) and .nojekyll file (disables Jekyll build on GitHub pages), which are saved between builds.

I honestly dislike git and prefer using mercurial (TortoiseHG) to do things. Also, I want to have full control over the publishing operation, performing it manually. That's why I cannot use the popular ghp-import tool.

Instead, I simply maintain a second mercurial repository inside publish_output subdirectory specifically to track the final website data. Whenever I want to publish new version, I run fab preview, then do a full commit in this mercurial repo and review changes. Finally, I push the current state to GitHub using hg-git extension.

UPDATE: Unfortunately, TortoiseHG failed to push my blog output to GitHub. Most likely due to some restrictions on the GitHub side, because smaller commits are pushed successfully. Did not manage to find out what the problem was. Unhappily switched to TortoiseGit for pushing the output =(

Now that it is complete, I feel tired of all the hacking done. On the positive side, I have learnt a lot about html and css. Let's hope that this blog won't need much care in the nearest future =)

Vectorizing small fixed-size sort

2018-01-05T23:26:00+07:00

After a long break, I can finally return to the topic which was started in the previous blog post.

Imagine that we have a small array of compile-time constant size with integers. For instance, N = 32. And we want to sort it as fast as possible. What is the best solution for this problem?

The wisest of you would suggest simply using std::sort, because every modern implementation of it is well-optimized and contains special handling of small subarrays to accelerate the generic quicksort algorithm. The ones who don't trust libraries would suggest using insertion sort: this is the algorithm usually used to handle small cases in std::sort. The performance geeks and regular stackoverflow visitors would definitely point to sorting networks: every question like "the fastest way to sort K integers" ends up with a solution based on them (N=6, N=10, what, why). I'm going to present a much less known way to sort small arrays of 32-bit keys, with performance comparable to sorting networks.

In the previous blog post I wrote a list of things to look for if you want to implement an algorithm working fast on small data:

Avoid branches whenever possible: unpredictable ones are very slow.
Reduce data dependency: this allows to fully utilize processing units in CPU pipeline.
Prefer simple data access and manipulation patterns: this allows to vectorize the algorithm.
Avoid complicated algorithms: they almost always fail on one of the previous points, and they sometimes do too much work for small inputs.

Comparison network

Consider the sample implementation. The best known sorting network for N = 16 was invented by Green in 1969, it contains 60 comparators which can be processed in 10 batches.

#ifdef _MSC_VER
    #define MIN(x, y) (x < y ? x : y)
    #define MAX(x, y) (x < y ? y : x) 
    #define COMPARATOR(i, j) {  \
        auto &x = dstKeys[i];   \
        auto &y = dstKeys[j];   \
        auto a = MIN(x, y);     \
        auto b = MAX(x, y);     \
        x = a;                  \
        y = b;                  \
    }
#else
    #define COMPARATOR(x,y) { int tmp; asm(                     \
        "mov %0, %2 ; cmp %1, %0 ; cmovg %1, %0 ; cmovg %2, %1" \
        : "=r" (dstKeys[x]), "=r" (dstKeys[y]), "=r" (tmp)      \
        : "0" (dstKeys[x]), "1" (dstKeys[y]) : "cc"             \
    ); }
#endif

COMPARATOR(0, 1);
COMPARATOR(2, 3);
COMPARATOR(4, 5);
COMPARATOR(6, 7);
COMPARATOR(8, 9);
COMPARATOR(10, 11);
COMPARATOR(12, 13);
COMPARATOR(14, 15);
COMPARATOR(0, 2);
// ... (48 more comparators) ... 
COMPARATOR(11, 12);
COMPARATOR(6, 7);
COMPARATOR(8, 9);

Notice all the hard work to implement comparators in efficient way. For MSVC, ordinary min/max implementation via ternary operator is OK (note: do not try to use std::min and std::max, they are not optimizer-friendly). The same works well on ICC and Clang, according to gcc.godbolt. But on GCC it does not generate cmov, so I had to copy/paste some piece of inline assembly from somewhere. Note that the version with branches is 2.7 times slower than branchess version with cmovs on my machine.

A sorting algorithm based on comparison network satisfies points 1, 2, and 4 of the list above. It can be implemented in branchless way, although you have be extremely careful and check assembly output to make sure that cmov instructions are actually generated. Comparison network was initially designed as a parallel algorithm, so a lot of comparisons can be done in parallel. Given that for a fixed and small size the whole network is usually fully unrolled, the CPU is free to schedule as many instructions as it can at every moment. Finally, the algorithm is as simple as "do the given list of compare-and-swap operations", so there is no overhead which could waste performance.

Only the point 3 is missing, because data access patterns are defined by the sorting network. As a result, sorting networks are hard to vectorize, although people still do it. For instance, here is in-register sort of six byte values, and similarly, in-register merging of eight 32-bit integers is possible.

Position-counting sort

Luckily, there is a way which fits all the four points of the above list.

Imagine that all elements are distinct. For each element, we can easily determine its final location in the sorted array: it is simply the number of elements less than it. We can iterate over all the elements and compare them with the element under attention, and count number of "less" outcomes. This is very similar to the idea driving the linear search in the previous article. After destination position is computed for all elements, we can iterate over them and copy them exactly to the desired location.

Here is the illustration of counting less elements to compute final position:

If equal elements are possible in the input array, then counting algorithm becomes more complicated. Let's try to make our sorting algorithm stable. Then the final position of element A[i] = X is equal to the number of elements less than X plus the number of elements equal to X located to the left of position i. It means that in addition to less elements, we also have to count number of equal elements, although only in about half of the array. This additional computation is inevitable, unless you are absolutely sure the input array has no duplicates.

The simple implementation of the suggested algorithm looks like this:

for (size_t i = 0; i < Count; i++) {
    int elem = inKeys[i];
    size_t cnt = 0;
    for (size_t j = 0; j < Count; j++)
        cnt += (inKeys[j] < elem);
    for (size_t j = 0; j < i; j++)
        cnt += (inKeys[j] == elem);
    dstKeys[cnt] = elem;
}

How should this sorting algorithm be called? I have never seen anything like this before. However, while looking for Green's sorting network at Morwenn's repository, I stumbled upon a page which contains an interesting link at the very end. It points to the so-called "Exact sort", which is exactly what I described just above. I don't like this name, because it gives too much attention to the very last step of the algorithm (i.e. moving elements to their places). In my opinion, the element-counting mechanic is the much more important part of the algorithm, which will be evident in the next section. That's why I prefer to call it "position-counting sort" ("counting sort" is already taken).

Let's recall the sacred four points. The algorithm is very simple in its core. It has no branches if fully unrolled (recall: array size is small and compile-time constaint). Note that the comparison does not generate a branch in x86. Unrolling this code is not recommended, even without unrolling branches should be well-predicted. As for data dependencies: outer iterations are completely independent, the inner loops behave like the reduce algorithm, so their iterations can be made quite independent from each other too. Finally, the data access patterns are very simple: inner loops access all elements sequentally (order does not even matter). This means that we can attempt to vectorize this algorithm.

Vectorization

As just noted, data dependencies are rather weak across iterations of both loops. So it makes sense to vectorize both by iterations of inner and outer loops. Vectorization of inner iterations should produce something like:

__m128i cnt = _mm_setzero_si128();
for (size_t j = 0; j < Count; j += 4) {
    __m128i data = _mm_load_si128((__m128i*)&inKeys[j]);
    cnt = _mm_sub_epi32(cnt, _mm_cmplt_epi32(data, elem_x4));
}

At the end, horizontal sum of cnt register has to be computed to obtain the sought-for final position of the considered element. In order to reduce number of loads, make horizontal sum faster, and in general reduce the overhead of loops, we can also process the elements to be moved in packs of four. So in the outer loop we load four elements at once too.

Here is the plan (bottom half shows horizontal sum at the end):

The code starts by iterating over the elements to be moved:

for (size_t i = 0; i < Count; i += 4) {

Here is the body of the outer loop:

//load four elements (to be moved into their locations)
__m128i reg = _mm_load_si128((__m128i*)&inKeys[i]);
//prepare a register with each value broadcasted
__m128i reg0 = _mm_shuffle_epi32(reg, SHUF(0, 0, 0, 0));
__m128i reg1 = _mm_shuffle_epi32(reg, SHUF(1, 1, 1, 1));
__m128i reg2 = _mm_shuffle_epi32(reg, SHUF(2, 2, 2, 2));
__m128i reg3 = _mm_shuffle_epi32(reg, SHUF(3, 3, 3, 3));

//init one register with counters per each element considered
__m128i cnt0 = _mm_setzero_si128();
__m128i cnt1 = _mm_setzero_si128();
__m128i cnt2 = _mm_setzero_si128();
__m128i cnt3 = _mm_setzero_si128();
for (size_t j = 0; j < Count; j += 4) {
    __m128i data = _mm_load_si128((__m128i*)&inKeys[j]);
    //update counters for all four elements of *reg*
    cnt0 = _mm_sub_epi32(cnt0, _mm_cmplt_epi32(data, reg0));
    cnt1 = _mm_sub_epi32(cnt1, _mm_cmplt_epi32(data, reg1));
    cnt2 = _mm_sub_epi32(cnt2, _mm_cmplt_epi32(data, reg2));
    cnt3 = _mm_sub_epi32(cnt3, _mm_cmplt_epi32(data, reg3));
}

Now we need to take equal elements into account. They are processed in very similar way, but only elements to the left of i are compared:

for (size_t j = 0; j < i; j += 4) {
    __m128i data = _mm_load_si128((__m128i*)&inKeys[j]);
    cnt0 = _mm_sub_epi32(cnt0, _mm_cmpeq_epi32(data, reg0));
    cnt1 = _mm_sub_epi32(cnt1, _mm_cmpeq_epi32(data, reg1));
    cnt2 = _mm_sub_epi32(cnt2, _mm_cmpeq_epi32(data, reg2));
    cnt3 = _mm_sub_epi32(cnt3, _mm_cmpeq_epi32(data, reg3));
}

Finally, there may be equal elements across elements i, i+1, i+2, i+3. We need to compare all elements in reg with each other, while carefully masking out the comparisons where operands go in wrong order:

//cnt0 = _mm_sub_epi32(cnt0, _mm_and_si128(_mm_cmplt_epi32(reg, reg0), _mm_setr_epi32( 0,  0,  0,  0)));
cnt1 = _mm_sub_epi32(cnt1, _mm_and_si128(_mm_cmpeq_epi32(reg, reg1), _mm_setr_epi32(-1,  0,  0,  0)));
cnt2 = _mm_sub_epi32(cnt2, _mm_and_si128(_mm_cmpeq_epi32(reg, reg2), _mm_setr_epi32(-1, -1,  0,  0)));
cnt3 = _mm_sub_epi32(cnt3, _mm_and_si128(_mm_cmpeq_epi32(reg, reg3), _mm_setr_epi32(-1, -1, -1,  0)));

Now all counts are gathered in XMM registers, and we have to calculate horizontal sum of each register. Since there are four of them, it makes sense to fully transpose them and then do a simple vertical sum:

//4x4 matrix transpose in SSE
__m128i c01L = _mm_unpacklo_epi32(cnt0, cnt1);
__m128i c01H = _mm_unpackhi_epi32(cnt0, cnt1);
__m128i c23L = _mm_unpacklo_epi32(cnt2, cnt3);
__m128i c23H = _mm_unpackhi_epi32(cnt2, cnt3);
__m128i cntX = _mm_unpacklo_epi64(c01L, c23L);
__m128i cntY = _mm_unpackhi_epi64(c01L, c23L);
__m128i cntZ = _mm_unpacklo_epi64(c01H, c23H);
__m128i cntW = _mm_unpackhi_epi64(c01H, c23H);
//obtain four resulting counts
__m128i cnt = _mm_add_epi32(_mm_add_epi32(cntX, cntY), _mm_add_epi32(cntZ, cntW));

Now we have final locations of all the four considered elements. We need only to extract them and move the elements to their places:

unsigned k0 = _mm_cvtsi128_si32(cnt   );
unsigned k1 = _mm_extract_epi32(cnt, 1);
unsigned k2 = _mm_extract_epi32(cnt, 2);
unsigned k3 = _mm_extract_epi32(cnt, 3);
dstKeys[k0] = inKeys[i+0];
dstKeys[k1] = inKeys[i+1];
dstKeys[k2] = inKeys[i+2];
dstKeys[k3] = inKeys[i+3];

That's all!

Comparison

Below you can see a barchart with performance comparison of all major implementations (on Ryzen 5 1600 with MSVC 2013 x64). Text label above each bar in format "1 / X" means than that the corresponding sort implementation takes X nanoseconds per call on average. In other words, its throughput is (1/X) sorts per nanosecond. The height of each bar is set as its throughput divided by throughput of std::sort (higher = faster).

For instance, implementation PCSort_Main for N = 16 (dark green bar on the left) takes 41 nanoseconds per call, i.e. processes (1/41) calls per nanosecond. It is about 3.5 times faster than std::sort implementation.

The dashed bars correspond to implementations which work properly only on input arrays with all elements distinct.

Here is the explanation of different implementations:

SortingNetwork: branchless sorting networks. For N = 16 it is Green's network of size 6 (as described above), for N = 32 it is something built on top of it (taken from here).
PCSort_Main: the standard vectorized implementation of position-counting sort, exactly as described above.
PCSort_Optimized: combines counting less elements and equal elements to the left of considered position into counting less-or-equal elements. This optimization allows to process each element of array only once per outer iteration (while in PCSort_Main the elements to the left are processed twice).
PCSort_Trans: somewhat transposed implementation. A local array of N counters is created. The outer loop goes through the elements to compare with. The inner loop goes through the elements for which the final location is to be computed. During each outer loop, we have to pass through the array of counters and increase some of them. Note that optimization from PCSort_Optimized is already applied: each element is processed once per outer iteration.
PCSort_WideOuter: this implementation uses wider vectorization in the outer loop compared to PCSort_Main, it computes final location for 16 elements during one outer iteration. In the inner loop elements are loaded in packs of four, and each element is broadcasted to the whole register to make comparison. This implementation works only for distinct elements: supporting equal elements in it is a major pain.
PCSort_WideOuter: same as PCSort_WideOuter, but all loops are fully unrolled (works only for N = 16). To my surprise, it works slower than the version with loops.

According to the data, the standard position-counting sort (PCSort_Main) is 20-25% faster than sorting network for N = 16 and N = 32. The optimized implementations are even better for larger N (+17% for N = 32, and +30% for N = 64). If you are sure that all elements are surely distinct, you can get additional +30% performance boost. It is very important to note that the results differ a lot depending on compiler. 64-bit MSVC turned out to be the best compiler for position-counting sort, while network sort is much faster on 64-bit GCC (details in table below).

Below you can see performance plotted by N increasing. For each N and each method, the ratio of its speed to speed of std::sort is shown. For example, PCSort_Main is 3.5 times faster than std::sort for N = 16.

As you see, the proposed sorting method is always faster than insertion sort, and the std::sort implementation catches up with proposed method only for N = 256 (or a bit later for optimized/distinct-only versions).

I also mesured performance across several compilers and CPUs. The difference between compilers is surprisingly noticeable! Here is the full table:

Time of one N=32 sort, in ns
Platform	PC_Main	PC_Opt	Network	Insertion
Ryzen5 + VC2013 x64	127.8	108.7	153.5	334.7
Ryzen5 + VC2013 x32	154.5	128.7	174.5	341.4
Ryzen5 + GCC7 x64 (VM)	119.4	127.3	100.8	347.4
Ryzen5 + GCC5 x64 (VM)	120.0	118.6	104.0	350.3
Ryzen5 + GCC7 x32 (VM)	140.4	118.7	154.7	430.7
Ryzen5 + GCC5 x32 (VM)	139.9	126.4	150.6	427.0
Ryzen5 + TDM GCC5 x32	137.3	123.0	145.0	356.7
i3-5005U + VC2017 x64	293.7	240.3	324.2	644.7
i3-5005U + VC2017 x32	272.8	213.6	392.0	651.4
i3-5005U + VC2013 x32	282.3	224.1	387.2	670.4
i3-5005U + VC2015 x32	276.6	213.6	392.0	649.5

The list of CPUs used:

Ryzen5: Ryzen 5 1600, 3.2 GHz
i3-5005U: Intel i3-5005U, 2.0 GHz (Broadwell)

The list of compilers used:

VC20xx: Microsoft Visual C++ of year 20xx (compiled with cl.exe with /O2 parameter).
GCC7: GCC 7.1.0 inside VirtualBox VM with Ubuntu (with -O3 -msse4).
GCC5: GCC 5.4.0 inside VirtualBox VM with Ubuntu (with -O3 -msse4).
TDM GCC5: TDM GCC 5.1.0 on Windows (with -O3 -msse4).

As you see, 64-bit GCC manages to get 1.5 times faster performance from the sorting network. I suppose it succeeds in fitting everything into registers, while 32-bit GCC and MSVC cannot avoid register stalls. The position-counting sort also benefits from 64-bit case, most likely also due to having more XMM registers available.

Discussion

In practice, sorting only keys is rarely enough. In most cases each element consists of a key and an attached value, and an element must be moved as a whole during sort. There are two ways to store array of elements: having single array of key-value pairs (array-of-structures, AoS) and having two separate arrays, one with keys and the other one with values (structure-of-arrays, SoA).

It is very easy to extend the proposed algorithm to process keys with attached values, given that elements are stored in SoA layout, i.e. values are stored in a separate array. Since the vectorized part determines the final position (index) of every element, it is enough to send the values to same positions as their keys:

unsigned k0 = _mm_cvtsi128_si32(cnt   );
unsigned k1 = _mm_extract_epi32(cnt, 1);
unsigned k2 = _mm_extract_epi32(cnt, 2);
unsigned k3 = _mm_extract_epi32(cnt, 3);
dstKeys[k0] = inKeys[i+0];
dstKeys[k1] = inKeys[i+1];
dstKeys[k2] = inKeys[i+2];
dstKeys[k3] = inKeys[i+3];
//add this code to support values array:
dstVals[k0] = inVals[i+0];
dstVals[k1] = inVals[i+1];
dstVals[k2] = inVals[i+2];
dstVals[k3] = inVals[i+3];

No computations are added for values at all! Only a minimal amount of additional time is spent for moving every value directly to its final location. It takes O(N) time, which is dominated by O(N^2) part for counting positions. Although the keys are restricted to be 32-bit integers (32-bit floats can be supported the same way), the values can be of any type.

As for performance measurements, adding support for 32-bit values increases time per one sort for PCSort_Main:

from 41 ns to 47.7 ns (+16.3%) for N = 16
from 127.8 ns to 136.4 ns (+6.7%) for N = 32)
from 465.4 to 486.4 ns (+4.5%) for N = 64.

Now compare it with sorting networks. Supporting values in a separate array (SoA) is more expensive: two additional CMOV operations must be added per one comparator (for swapping values conditionally). And it only works as long as the values are simple enough to be CMOV-ed. If the values are 32-bit, then it is much easier to work with array-of-structures layout. Store value first, key then, and treat each key-value pair as 64-bit integer. Compare them as 64-bit integers and CMOV them as such, and you'll get values support for free on 64-bit platform.

Compiler issue

I stumbled against a tiny compiler inefficiency. Visual C++ 64-bit compiler (and I guess other compilers too) generates unnecessary instruction for _mm_extract_epi32 intrinsic. Look at the assembly code generated by compiler for extracting single position and moving a key to its final location:

; 113  :  unsigned k2 = _mm_extract_epi32(cnt, 2);
; 117  :  dstKeys[k2] = inKeys[i+2];

;load inKeys[i+2] to eax:
mov     eax, DWORD PTR [rdx-16]
;extract k2 to ecx:
pextrd  ecx, xmm2, 2
;zero-fill upper half of ecx
mov     ecx, ecx
;store eax to dstKeys[k2]:
mov     DWORD PTR [r8+rcx*4], eax

For the first glance, the third instruction seems to do nothing. However, it was specifically inserted by compiler here, in order to clear the upper 32 bits of rcx register. What compiler does not know is that pextrd instruction already clears the upper 32 bits of 64-bit destination register, so the instruction is excessive. The performance impact is most likely minimal, but it is still strange to see such things. The same issue happens for mm_movemask* intrinsics, see this question.

The code

As usual, all the materials and all the C++ code is available in position-counting-sort repo on GitHub.

Conclusion

The position-counting sorting algorithm is rather unknown, although it yields performance comparable to that of optimal sorting networks (which are well-known for their speed). It works in plain O(N^2) time, while sorting networks have to do somewhere between O(N log N) and O(N log^2 N) work. Having worse asymptotic complexity, it manages to be so fast only because it is vectorized very efficiently. So it can serve as a good alternative for sorting networks in some cases.

Addendum to Performance comparison: linear search vs binary search

2017-08-29T02:21:00+07:00

The previous blog post got some attention and several good questions and suggestions. So I feel that I should sum up the main points noted by readers. Since the main article is already too long, I decided to keep all of this as a separate post.

Large arrays

Let's make it clear: the whole blog post was only about small arrays!

Since I was interested in linear search more than in binary search, I was mainly interested in cases like N <= 128. It is clear that for larger arrays linear search is out of consideration anyway. All the performance results (except for one pair of plots) were measured in such a way that everything important fits into L1D cache. All the complicated things about memory access like using L2/L3/TLB/RAM were not considered at all. That's why the provided results may mislead if you want to implement binary search over 1 MB array.

Speaking of really large arrays, there are several points to consider:

The provided implementation of branchless binary search always has power-of-two step, which causes ill cache utilization for large arrays (due to limited cache associativity). One of the solutions would be using ternary search instead of binary one. You can read more about it in Paul Khuong's blog post.
If you can reorder allements beforehand, better rearrange them so that frequently used elements go first. The resulting layout is equivalent to storing perfectly balanced binary search tree in breadth-first order. This order is much more cache-friendly. Read more in bannalia's blog post.
If you can rearrange/preprocess large array, probably you should use some data structure for your searches. Various variations of B-tree are the most promising. You can read a bit about it e.g. in this article.

Branchless without ternary operator

Regarding branchless binary search, some wondered if this can really be called branchless:

//binary_search_branchless
pos = (arr[pos + step] < key ? pos + step : pos);

There is no point in arguing about terminology, but it is quite interesting to see how other versions perform. For instance, it is possible to use the result of comparison as 0/1 integer value and do some math on it:

//binary_search_branchlessM
pos += (arr[pos + step] < key) * step;

//binary_search_branchlessA
pos += (-(arr[pos + step] < key)) & step;

Also, it is possible to avoid comparison completely, using arithmetic shift on difference to get empty or full bitmask:

//binary_search_branchlessS
pos += ((arr[pos + step] - key) >> 31) & step;

Just remember that this code is not perfectly portable: it relies on undefined behavior according to C/C++ standards.

I included these three cases into extended comparison (see plots below). They are noticeably slower of course, but still much faster than branching binary search.

Preload pivot elements

One really smart idea about binary search was from seba, who suggested preloading both possible pivots for the next iteration while still performing the current iteration. This means doing two memory loads per iteration instead of one, but since they are started one iteration earlier, their latency is hidden behind other logical instructions going on.

Note that after both potential pivots are loaded, we need to do additional work to choose the real future pivot among them, and it can be done with cmov with after the comparison already performed. Here is the resulting implementation:

int binary_search_branchless_pre (const int *arr, int n, int key) {
    intptr_t pos = MINUS_ONE;
    intptr_t logstep = bsr(n);
    intptr_t step = intptr_t(1) << logstep;
    int pivot = arr[pos + step];
    while (step > 1) {
        intptr_t nextstep = step >> 1;
        int pivotL = arr[pos + nextstep];
        int pivotR = arr[pos + step + nextstep];
        pos = (pivot < key ? pos + step : pos);
        pivot = (pivot < key ? pivotR : pivotL);
        step = nextstep;
    }
    pos = (pivot < key ? pos + step : pos);
    return pos + 1;
}

The idea turns binary search into something similar to quaternary search:

int quaternary_search_branchless (const int *arr, int n, int key) {
    assert((n & (n+1)) == 0); //n = 2^k - 1
    intptr_t pos = MINUS_ONE;
    intptr_t logstep = bsr(n) - 1;
    intptr_t step = intptr_t(1) << logstep;
    while (step > 0) {
        int pivotL = arr[pos + step * 1];
        int pivotM = arr[pos + step * 2];
        int pivotR = arr[pos + step * 3];
        pos = (pivotL < key ? pos + step : pos);
        pos = (pivotM < key ? pos + step : pos);
        pos = (pivotR < key ? pos + step : pos);
        step >>= 2;
    }
    pos = (arr[pos + 1] < key ? pos + 1 : pos);
    return pos + 1;
}

Both of these implementations are included into extended performance comparison given below. The binary search with preloading (binary_search_branchless_pre) actually becomes significantly faster in latency, sacrificing some throughput. The quaternary search is not so good.

Hybrid search

I tried to perform branchless binary search until the segment of interest becomes small enough, then use linear search within it. It seems that making several iterations of linear search is not a good idea though: in fact, the best way is to make the length of linear search fixed, so that it could be fully unrolled. That's why I stopped on doing 16-element linear search. Using movemask instead of counting seems to be a bit faster in this case:

int hybridX_search (const int *arr, int n, int key) {
    intptr_t pos = MINUS_ONE;
    intptr_t logstep = bsr(n);
    intptr_t step = intptr_t(1) << logstep;
    while (step > 8) {
        pos = (arr[pos + step] < key ? pos + step : pos);
        step >>= 1;
    }
    pos++;
    step <<= 1;

    __m128i vkey = _mm_set1_epi32(key);
    __m128i cnt = _mm_setzero_si128();
    __m128i cmp0 = _mm_cmpgt_epi32 (vkey, _mm_load_si128((__m128i *)&arr[pos+0]));
    __m128i cmp1 = _mm_cmpgt_epi32 (vkey, _mm_load_si128((__m128i *)&arr[pos+4]));
    __m128i cmp2 = _mm_cmpgt_epi32 (vkey, _mm_load_si128((__m128i *)&arr[pos+8]));
    __m128i cmp3 = _mm_cmpgt_epi32 (vkey, _mm_load_si128((__m128i *)&arr[pos+12]));
    __m128i pack01 = _mm_packs_epi32 (cmp0, cmp1);
    __m128i pack23 = _mm_packs_epi32 (cmp2, cmp3);
    __m128i pack0123 = _mm_packs_epi16 (pack01, pack23);
    uint32_t res = _mm_movemask_epi8 (pack0123);

    return pos + bsf(~res);
}

It actually helped to reduce both throughput and latency time a bit (see plots below).

Compiler issues

mttd pointed out that there is intrinsic __builtin_unpredictable in Clang, which helps a lot in getting cmov instructions generated. I guess it is not present in GCC and MSVC...
I created an issue for Visual C++ about that stupid "or r9, -1" instruction generated. UPDATE(Nov.2017): this transformation is disabled when compiling for speed, the fix is targeted for version 15.6.

Extended performance results

Since all the added implementations are for binary search, the best plot would be the one which draws time per search divided by log2(N), which is something like "time per iteration of simple binary search":

Here is the plain plot also:

Sorry for the mess: there are too many implementations being compared already. Note that linear searches now use cross style of markers, while binary searches use solid figures or asterisks for markers.

The updated code including all new implementations is in the same repository as before.

Raw results are also available there in a branch. For instance, you can see all the text logs for throughput here and all the text logs for latency here.

Performance comparison: linear search vs binary search

2017-08-25T21:02:00+07:00

While working on an implementation of merge sort promised in the previous article, I realized that I'd like to use one neat little thing, which is worth its own post. It is a simple strategy for sorting or doing comparison-based tasks, which works wonderfully when input data is small enough.

Suppose that we have a very small array and we want to sort it as fast as possible. Indeed, applying some fancy O(N log N) algorithm is not a good idea: although it has optimal asymptotic performance, its logic is too complicated to outperform simple bubble-sort-like algorithms which take O(N^2) time instead. That's why every well-optimized sorting algorithm based on quicksort (e.g. std::sort) or mergesort includes some simple quadratic algorithm which is run for sufficiently small subarrays like N <= 32.

What exactly should we strive for to get an algorithm efficient for small N? Here is the list of things to look for:

Avoid branches whenever possible: unpredictable ones are very slow.
Reduce data dependency: this allows to fully utilize processing units in CPU pipeline.
Prefer simple data access and manipulation patterns: this allows to vectorize the algorithm.
Avoid complicated algorithms: they almost always fail on one of the previous points, and they sometimes do too much work for small inputs.

I decided to start investigating a simpler problem first, which is solved by std::lower_bound: given a sorted array of elements and a key, find index of the first array element greater or equal than the key. And this investigation soon developed into a full-length standalone article.

Binary search

The problem is typically solved with binary search in O(log N) time, but it might easily happen so that for small N simple linear algorithm would be faster. Of course, not all implementations of binary search are created equal: for small arrays branchless implementation is preferred. You can read about it for instance in this demofox's blog post. Here is a branchless implementation that we will use in the comparison later:

int binary_search_branchless (const int *arr, int n, int key) {
    intptr_t pos = -1;
    intptr_t logstep = bsr(n);
    intptr_t step = intptr_t(1) << logstep;
    while (step > 0) {
        pos = (arr[pos + step] < key ? pos + step : pos);
        step >>= 1;
    }
    return pos + 1;
}

This code only works properly when N+1 is power of two. In order to support arbitrary size of input array, some sort of modification is required. The best approach for it is described in this blog post from Paul Khuong, which is: make the very first iteration special by using step = n+1 - 2^logstep instead of just 2^logstep, so that regardless of comparison result the next search interval would have length 2^logstep, including either the beginning of array or the end of array (note that two such possible intervals overlap, and this is not a problem).

The comparison itself is included in the ternary operator, which is supposed to compile into cmovXX instruction. And it really happens (unless your compiler thinks that you compile for 486), according to assembly listing of the innermost loop:

$LL491@main:
    lea rax, QWORD PTR [rcx+rdx]
    cmp DWORD PTR [rdi+rax*4], r8d
    cmovl rdx, rax
    sar rcx, 1
    test rcx, rcx
    jg SHORT $LL491@main

How good is this binary search implementation? It has no branches (point 1 from the above list), which is great. But it is inherently sequental (point 2): you cannot know which element to load on the next step until the element on the current step has been loaded and compared completely. Data access is scalar (point 3): you cannot vectorize it even if several keys are searched simultaneously.

We can take closer look by using Intel Architecture Code Analyzer: IACA is a small static analysis tool, which analyzes a snippet of code at instruction level. It assumes that the code piece is run in the infinite loop, that all branches are well-predicted and not taken, and that all memory accesses hit L1 cache (plus maybe some other assumptions). It is somewhat tricky to use with Visual C++ 64-bit compiler: you have to copy-paste the piece of code from assembly listing, surround it with markers (which are a bit wrong in iacaMarks.h by the way), then compile it with ml64.exe, and pass the resulting object file into iaca.exe.

Here is what IACA says about the innermost loop:

Throughput Analysis Report
--------------------------
Block Throughput: 2.15 Cycles       Throughput Bottleneck: Dependency chains (possibly between iterations)

Port Binding In Cycles Per Iteration:
---------------------------------------------------------------------------------------
|  Port  |  0   -  DV  |  1   |  2   -  D   |  3   -  D   |  4   |  5   |  6   |  7   |
---------------------------------------------------------------------------------------
| Cycles | 1.5    0.0  | 0.9  | 0.5    0.5  | 0.5    0.5  | 0.0  | 1.1  | 1.5  | 0.0  |
---------------------------------------------------------------------------------------

| Num Of |                    Ports pressure in cycles                     |    |
|  Uops  |  0  - DV  |  1  |  2  -  D  |  3  -  D  |  4  |  5  |  6  |  7  |    |
---------------------------------------------------------------------------------
|   1    |           | 0.3 |           |           |     | 0.6 |     |     |    | lea rax, ptr [rdx+rcx*1]
|   2    |           | 0.6 | 0.5   0.5 | 0.5   0.5 |     | 0.4 |     |     |    | cmp dword ptr [rdi+rax*4], r8d
|   1    | 0.4       |     |           |           |     |     | 0.6 |     |    | cmovl rdx, rax
|   1    | 0.5       |     |           |           |     |     | 0.5 |     | CP | sar rcx, 0x1
|   1    | 0.6       |     |           |           |     |     | 0.4 |     | CP | test rcx, rcx
|   0F   |           |     |           |           |     |     |     |     |    | jnle 0xffffffffffffffee
Total Num Of Uops: 6

So it says that bottleneck is "dependency chains" and it estimates single iteration of the loop in 2.15 cycles. The estimate is clearly wrong: it is easy to see that the first three instructions must be executed sequentally, and the first of them cannot start before that last of them from the previous iterations has finished. A load from L1 costs 4 cycles latency-wise, address generation and other three instructions take at least 1 cycle each. So the innermost loop should take 8 cycles per iteration. If we look how performance measurements increase from doubling N, we can estimate one iteration in 4-5 cycles throughput and 10 cycles latency: both are much greater that the estimate given by IACA.

Since the loop in the binary search is very small (4-6 microops) and it goes for tiny number of iterations (e.g. 6 iterations for N = 64), one might suspect that the loop itself can take considerable time. That's why I have also implemented fully unrolled version of branchess binary search:

template<intptr_t MAXN> int binary_search_branchless_UR(const int *arr, int n, int key) {
    assert(n+1 == MAXN);
    intptr_t pos = -1;
    #define STEP(logstep) \
        if ((1<<logstep) < MAXN) \
            pos = (arr[pos + (1<<logstep)] < key ? pos + (1<<logstep) : pos);
    STEP(9)
    STEP(8)
    STEP(7)
    STEP(6)
    STEP(5)
    STEP(4)
    STEP(3)
    STEP(2)
    STEP(1)
    STEP(0)
    #undef STEP
    return pos + 1;
}

This is of course quite far from the code usable in real programming, but it allows us to ignore all the overhead from the binary search and look directly at its core. IACA provides the following information about the code of the function (N = 64):

Throughput Analysis Report
--------------------------
Block Throughput: 29.00 Cycles       Throughput Bottleneck: Dependency chains (possibly between iterations)

Port Binding In Cycles Per Iteration:
---------------------------------------------------------------------------------------
|  Port  |  0   -  DV  |  1   |  2   -  D   |  3   -  D   |  4   |  5   |  6   |  7   |
---------------------------------------------------------------------------------------
| Cycles | 5.0    0.0  | 5.0  | 3.0    3.0  | 3.0    3.0  | 0.0  | 5.0  | 5.0  | 0.0  |
---------------------------------------------------------------------------------------

| Num Of |                    Ports pressure in cycles                     |    |
|  Uops  |  0  - DV  |  1  |  2  -  D  |  3  -  D  |  4  |  5  |  6  |  7  |    |
---------------------------------------------------------------------------------
|   1*   |           |     |           |           |     |     |     |     |    | mov r9, rcx
|   1    |           | 1.0 |           |           |     |     |     |     |    | mov eax, 0x1f
|   1    |           |     |           |           |     | 1.0 |     |     |    | or rcx, 0xffffffffffffffff
|   2^   |           |     | 1.0   1.0 |           |     |     | 1.0 |     | CP | cmp dword ptr [r9+0x7c], r8d
|   1    | 1.0       |     |           |           |     |     |     |     | CP | cmovnl rax, rcx
|   2    |           | 1.0 |           | 1.0   1.0 |     |     |     |     | CP | cmp dword ptr [r9+rax*4+0x40], r8d
|   1    |           |     |           |           |     | 1.0 |     |     |    | lea rdx, ptr [rax+0x10]
|   1    | 1.0       |     |           |           |     |     |     |     | CP | cmovnl rdx, rax
|   2    |           |     | 1.0   1.0 |           |     |     | 1.0 |     | CP | cmp dword ptr [r9+rdx*4+0x20], r8d
|   1    |           | 1.0 |           |           |     |     |     |     |    | lea rax, ptr [rdx+0x8]
|   1    | 1.0       |     |           |           |     |     |     |     | CP | cmovnl rax, rdx
|   2    |           |     |           | 1.0   1.0 |     | 1.0 |     |     | CP | cmp dword ptr [r9+rax*4+0x10], r8d
|   1    |           | 1.0 |           |           |     |     |     |     |    | lea rcx, ptr [rax+0x4]
|   1    |           |     |           |           |     |     | 1.0 |     | CP | cmovnl rcx, rax
|   2    |           |     | 1.0   1.0 |           |     | 1.0 |     |     | CP | cmp dword ptr [r9+rcx*4+0x8], r8d
|   1    |           | 1.0 |           |           |     |     |     |     |    | lea rdx, ptr [rcx+0x2]
|   1    | 1.0       |     |           |           |     |     |     |     | CP | cmovnl rdx, rcx
|   2    |           |     |           | 1.0   1.0 |     |     | 1.0 |     | CP | cmp dword ptr [r9+rdx*4+0x4], r8d
|   1    |           |     |           |           |     | 1.0 |     |     |    | lea rax, ptr [rdx+0x1]
|   1    | 1.0       |     |           |           |     |     |     |     | CP | cmovnl rax, rdx
|   1    |           |     |           |           |     |     | 1.0 |     | CP | inc rax
Total Num Of Uops: 27

It is hard to say how IACA got 29 cycles per search. The critical path is marked properly: groups of cmov and cmp must go sequentally after each other, each group taking 7 cycles latency-wise, so latency of the whole code should be about 42 cycles. In the benchmark, the whole search takes 21 cycles throughput and 59 cycles latency, while doubling N adds 4-5 cycles more throughput and 8-9 cycles more latency. It seems that CPU manages to execute several consecutive searches in parallel thanks to pipelining, that's why throughput time is lower than latency time for branchless binary searches.

I believe this is the best you can get from a binary search.

Linear search

Things start to get more interesting when we try to invent a simple search algorithm working in O(N) time. Because there are actually several ways to do so. If you look the question "How fast can you make linear search?" on stackoverflow, you will see a basic code like this:

int linearX_search_scalar (const int *arr, int n, int key) {
    intptr_t i = 0;
    while (i < n) {
        if (arr[i] >= key)
            break;
        ++i;
    }
    return i;
}

Most of the answers for the question are just optimized versions of this algorithm: even if they compare more than one element at a time (with SSE), they still break out when the desired element is found. For instance, the author of the question (Schani) suggests the following code in his blog post:

int linearX_search_sse (const int *arr, int n, int key) {
    __m128i *in_data = (__m128i*)arr;
    __m128i key4 = _mm_set1_epi32(key);
    intptr_t i = 0;
    int res;
    for (;;) {
        __m128i cmp0 = _mm_cmpgt_epi32(key4, in_data [i + 0]);
        __m128i cmp1 = _mm_cmpgt_epi32(key4, in_data [i + 1]);
        __m128i cmp2 = _mm_cmpgt_epi32(key4, in_data [i + 2]);
        __m128i cmp3 = _mm_cmpgt_epi32(key4, in_data [i + 3]);
        __m128i pack01 = _mm_packs_epi32(cmp0, cmp1);
        __m128i pack23 = _mm_packs_epi32(cmp2, cmp3);
        __m128i pack0123 = _mm_packs_epi16 (pack01, pack23);
        res = _mm_movemask_epi8 (pack0123);
        if (res != 0xFFFF)
            break;
        i += 4;
    }
    return i * 4 + bsf(~res);
}

This code has no issues from the list of things that may slow down the algorithm, except for one mispredicted branch. The branch which checks if the answer is found is never taken except for the very last iteration when the loop terminates (which may happen at random moment). Of course, single misprediction is not a problem in a long loop, but it may cause problems when we want to optimize a tiny loop. In fact, Paul Khuong's blog post claims that linear search is always slower than branchless binary search because of this single mispredicted branch. Luckily, there is a way to avoid branches completely.

The idea is very simple: the sought-for index is precisely the number of elements in the array which are less than the key you search for. In fact, this criterion would work the same way even if you shuffle the input array randomly =) Here is the scalar implementation based on this criterion:

int linear_search_scalar (const int *arr, int n, int key) {
    int cnt = 0;
    for (int i = 0; i < n; i++)
        cnt += (arr[i] < key);
    return cnt;
}

For a vectorized implementation, we can compare a pack of four elements with the key, and then subtract the resulting masks (recall that they are -1 when comparison result is true) from the common accumulator. At the end we have to compute horizontal sum of the accumulator to get the total count. Finally, it is better to process several packs during single iteration (similar to unrolling) to reduce loop overhead. Here is the code:

int linear_search_sse (const int *arr, int n, int key) {
    __m128i vkey = _mm_set1_epi32(key);
    __m128i cnt = _mm_setzero_si128();
    for (int i = 0; i < n; i += 16) {
        __m128i mask0 = _mm_cmplt_epi32(_mm_load_si128((__m128i *)&arr[i+0]), vkey);
        __m128i mask1 = _mm_cmplt_epi32(_mm_load_si128((__m128i *)&arr[i+4]), vkey);
        __m128i mask2 = _mm_cmplt_epi32(_mm_load_si128((__m128i *)&arr[i+8]), vkey);
        __m128i mask3 = _mm_cmplt_epi32(_mm_load_si128((__m128i *)&arr[i+12]), vkey);
        __m128i sum = _mm_add_epi32(_mm_add_epi32(mask0, mask1), _mm_add_epi32(mask2, mask3));
        cnt = _mm_sub_epi32(cnt, sum);
    }
    cnt = _mm_add_epi32(cnt, _mm_shuffle_epi32(cnt, SHUF(2, 3, 0, 1)));
    cnt = _mm_add_epi32(cnt, _mm_shuffle_epi32(cnt, SHUF(1, 0, 3, 2)));
    return _mm_cvtsi128_si32(cnt);
}

It requires the input array to be padded with sentinel elements INT_MAX until its size becomes divisible by 16. This requirement may be lifted by processing at most 15 last elements with additional loop(s), but I suppose such a version would work somewhat slower than the version with sentinels.

IACA provides the following info for the innermost loop of the code:

Throughput Analysis Report
--------------------------
Block Throughput: 5.05 Cycles       Throughput Bottleneck: Dependency chains (possibly between iterations)

Port Binding In Cycles Per Iteration:
---------------------------------------------------------------------------------------
|  Port  |  0   -  DV  |  1   |  2   -  D   |  3   -  D   |  4   |  5   |  6   |  7   |
---------------------------------------------------------------------------------------
| Cycles | 0.5    0.0  | 4.5  | 2.0    2.0  | 2.0    2.0  | 0.0  | 4.5  | 0.5  | 0.0  |
---------------------------------------------------------------------------------------

| Num Of |                    Ports pressure in cycles                     |    |
|  Uops  |  0  - DV  |  1  |  2  -  D  |  3  -  D  |  4  |  5  |  6  |  7  |    |
---------------------------------------------------------------------------------
|   2^   |           | 0.5 | 1.0   1.0 |           |     | 0.5 |     |     |    | vpcmpgtd xmm1, xmm3, xmmword ptr [rcx-0x20]
|   2^   |           | 0.5 |           | 1.0   1.0 |     | 0.5 |     |     |    | vpcmpgtd xmm0, xmm3, xmmword ptr [rcx-0x10]
|   1    |           | 0.5 |           |           |     | 0.5 |     |     | CP | lea rcx, ptr [rcx+0x40]
|   1    |           | 0.5 |           |           |     | 0.5 |     |     |    | vpaddd xmm2, xmm1, xmm0
|   2^   |           | 0.5 | 1.0   1.0 |           |     | 0.5 |     |     | CP | vpcmpgtd xmm1, xmm3, xmmword ptr [rcx-0x30]
|   2^   |           | 0.5 |           | 1.0   1.0 |     | 0.5 |     |     | CP | vpcmpgtd xmm0, xmm3, xmmword ptr [rcx-0x40]
|   1    |           | 0.5 |           |           |     | 0.5 |     |     | CP | vpaddd xmm1, xmm1, xmm0
|   1    |           | 0.5 |           |           |     | 0.5 |     |     | CP | vpaddd xmm2, xmm2, xmm1
|   1    |           | 0.5 |           |           |     | 0.5 |     |     | CP | vpsubd xmm4, xmm4, xmm2
|   1    | 0.5       |     |           |           |     |     | 0.5 |     |    | sub rdx, 0x1
|   0F   |           |     |           |           |     |     |     |     |    | jnz 0xffffffffffffffd4
Total Num Of Uops: 14

And again: I'm not sure how IACA got 5.05 cycles per iteration. Most of the work done each iteration is clearly independent between iterations, so in a theoretical infinite loop several consecutive iterations can be executed in parallel, thus hiding latency of memory loads and dependencies. As for performance measurements, going from 64 elements to 128 elements increases time by 12.4 ns, which is 6.2 cycles per iteration. The cost per iteration decreases to 5.5 cycles for larger N. Note that both latency and throughput measurements increase equally with increase of N, which confirms the fact that iterations are processed independently. Given that execution ports pressure is 4.5 cycles per iteration, spending 5.5 cycles is pretty efficient usage of hardware (compared to binary search).

To make sure loop overhead is not critical, I have also implemented fully unrolled version of this vectorized implementation. For N = 64 it has four iterations unrolled, and here is its assembly code analyzed by IACA:

Throughput Analysis Report
--------------------------
Block Throughput: 19.25 Cycles       Throughput Bottleneck: FrontEnd, Port1, Port5

Port Binding In Cycles Per Iteration:
---------------------------------------------------------------------------------------
|  Port  |  0   -  DV  |  1   |  2   -  D   |  3   -  D   |  4   |  5   |  6   |  7   |
---------------------------------------------------------------------------------------
| Cycles | 1.0    0.0  | 18.8 | 8.0    8.0  | 8.0    8.0  | 0.0  | 19.3 | 0.0  | 0.0  |
---------------------------------------------------------------------------------------

| Num Of |                    Ports pressure in cycles                     |    |
|  Uops  |  0  - DV  |  1  |  2  -  D  |  3  -  D  |  4  |  5  |  6  |  7  |    |
---------------------------------------------------------------------------------
|   1*   |           |     |           |           |     |     |     |     |    | vpxor xmm3, xmm3, xmm3
|   1    |           |     |           |           |     | 1.0 |     |     | CP | vmovd xmm5, r8d
|   1    |           |     |           |           |     | 1.0 |     |     | CP | vpbroadcastd xmm5, xmm5
|   2^   |           | 0.9 | 1.0   1.0 |           |     | 0.1 |     |     | CP | vpcmpgtd xmm1, xmm5, xmmword ptr [rcx+0x30]
|   2^   |           | 0.8 |           | 1.0   1.0 |     | 0.3 |     |     | CP | vpcmpgtd xmm0, xmm5, xmmword ptr [rcx+0x20]
|   1    |           | 0.5 |           |           |     | 0.5 |     |     | CP | vpaddd xmm2, xmm1, xmm0
|   2^   |           | 0.6 | 1.0   1.0 |           |     | 0.4 |     |     | CP | vpcmpgtd xmm0, xmm5, xmmword ptr [rcx]
|   2^   |           | 0.4 |           | 1.0   1.0 |     | 0.6 |     |     | CP | vpcmpgtd xmm1, xmm5, xmmword ptr [rcx+0x10]
|   1    |           | 0.6 |           |           |     | 0.4 |     |     | CP | vpaddd xmm1, xmm1, xmm0
|   2^   |           | 0.4 | 1.0   1.0 |           |     | 0.6 |     |     | CP | vpcmpgtd xmm0, xmm5, xmmword ptr [rcx+0x60]
|   1    |           | 0.6 |           |           |     | 0.4 |     |     | CP | vpaddd xmm2, xmm2, xmm1
|   2^   |           | 0.4 |           | 1.0   1.0 |     | 0.6 |     |     | CP | vpcmpgtd xmm1, xmm5, xmmword ptr [rcx+0x70]
|   1    |           | 0.6 |           |           |     | 0.4 |     |     | CP | vpsubd xmm4, xmm3, xmm2
|   1    |           | 0.4 |           |           |     | 0.6 |     |     | CP | vpaddd xmm2, xmm1, xmm0
|   2^   |           | 0.6 | 1.0   1.0 |           |     | 0.4 |     |     | CP | vpcmpgtd xmm0, xmm5, xmmword ptr [rcx+0x40]
|   2^   |           | 0.4 |           | 1.0   1.0 |     | 0.6 |     |     | CP | vpcmpgtd xmm1, xmm5, xmmword ptr [rcx+0x50]
|   1    |           | 0.6 |           |           |     | 0.4 |     |     | CP | vpaddd xmm1, xmm1, xmm0
|   2^   |           | 0.4 | 1.0   1.0 |           |     | 0.6 |     |     | CP | vpcmpgtd xmm0, xmm5, xmmword ptr [rcx+0xa0]
|   1    |           | 0.6 |           |           |     | 0.4 |     |     | CP | vpaddd xmm2, xmm2, xmm1
|   2^   |           | 0.4 |           | 1.0   1.0 |     | 0.6 |     |     | CP | vpcmpgtd xmm1, xmm5, xmmword ptr [rcx+0xb0]
|   1    |           | 0.6 |           |           |     | 0.4 |     |     | CP | vpsubd xmm3, xmm4, xmm2
|   1    |           | 0.4 |           |           |     | 0.6 |     |     | CP | vpaddd xmm2, xmm1, xmm0
|   2^   |           | 0.6 | 1.0   1.0 |           |     | 0.4 |     |     | CP | vpcmpgtd xmm0, xmm5, xmmword ptr [rcx+0x80]
|   2^   |           | 0.4 |           | 1.0   1.0 |     | 0.6 |     |     | CP | vpcmpgtd xmm1, xmm5, xmmword ptr [rcx+0x90]
|   1    |           | 0.6 |           |           |     | 0.4 |     |     | CP | vpaddd xmm1, xmm1, xmm0
|   2^   |           | 0.4 | 1.0   1.0 |           |     | 0.6 |     |     | CP | vpcmpgtd xmm0, xmm5, xmmword ptr [rcx+0xe0]
|   1    |           | 0.6 |           |           |     | 0.4 |     |     | CP | vpaddd xmm2, xmm2, xmm1
|   2^   |           | 0.4 |           | 1.0   1.0 |     | 0.6 |     |     | CP | vpcmpgtd xmm1, xmm5, xmmword ptr [rcx+0xf0]
|   1    |           | 0.6 |           |           |     | 0.4 |     |     | CP | vpsubd xmm4, xmm3, xmm2
|   1    |           | 0.4 |           |           |     | 0.6 |     |     | CP | vpaddd xmm2, xmm1, xmm0
|   2^   |           | 0.6 | 1.0   1.0 |           |     | 0.4 |     |     | CP | vpcmpgtd xmm0, xmm5, xmmword ptr [rcx+0xc0]
|   2^   |           | 0.4 |           | 1.0   1.0 |     | 0.6 |     |     | CP | vpcmpgtd xmm1, xmm5, xmmword ptr [rcx+0xd0]
|   1    |           | 0.6 |           |           |     | 0.4 |     |     | CP | vpaddd xmm1, xmm1, xmm0
|   1    |           | 0.4 |           |           |     | 0.6 |     |     | CP | vpaddd xmm2, xmm2, xmm1
|   1    |           | 0.6 |           |           |     | 0.4 |     |     | CP | vpsubd xmm3, xmm4, xmm2
|   1    |           |     |           |           |     | 1.0 |     |     | CP | vpshufd xmm0, xmm3, 0x4e
|   1    |           | 1.0 |           |           |     |     |     |     | CP | vpaddd xmm2, xmm0, xmm3
|   1    |           |     |           |           |     | 1.0 |     |     | CP | vpshufd xmm1, xmm2, 0xb1
|   1    |           | 1.0 |           |           |     |     |     |     | CP | vpaddd xmm0, xmm1, xmm2
|   1    | 1.0       |     |           |           |     |     |     |     |    | vmovd eax, xmm0
Total Num Of Uops: 56

The broadwell execution ports 1 and 5 are fully saturated here, the beginning and the ending of the function take about 3 cycles throughput, so there is about 16 cycles spent on the middle part, which gives about 4 cycles per loop iteration (i.e. per 16 elements). Performance measurements fully confirm this estimate: going from 64 to 128 elements increases time by 8.3 ns, which is 4.15 cycles per iteration. For larger N, cost per loop iteration perfectly converges to 4 cycles. These statements are true both for throughput and latency performance.

Finally, I have implemented AVX versions of the same algorithm (both in a loop and fully unrolled). They would also be present in comparison.

Comparison

The testing code works as follows. Both keys and input elements are generated randomly and uniformly between 0 and N+1. The input array always has size N in form 2^k-1, not including one sentinel element INT_MAX. Several input arrays and several keys for search are generated before performance measurement. The search function is called in a loop (without inlining), different input array and key from the pre-generated sets are chosen on each iteration. The cumulative sum of all the answers is printed to console to make sure nothing is thrown away by optimizer. Here is the main loop:

int start = clock();
int check = 0;
static const int TRIES = (1<<30) / SIZE;
for (int t = 0; t < TRIES; t++) {
    int i = (t * DARR + (MEASURE_LATENCY ? check&1 : 0));
    int j = (t * DKEY + (MEASURE_LATENCY ? check&1 : 0));
    i &= (ARR_SAMPLES - 1);
    j &= (KEY_SAMPLES - 1);
    const int *arr = input[i];
    int key = keys[j];
    int res = search_function(arr, n, key);
    check += res;
}
double elapsed = double(clock() - start) / CLOCKS_PER_SEC;
printf("%8.1lf ns : %40s   (%d)\n", 1e+9 * elapsed / TRIES, search_name, check);

Originally I wanted to measure only throughput performance, but after having compared the results of MSVC and GCC compilers I realized that I need a mode to measure latency performance too, which is achieved by making input array and key indices dependent on the result of the previous search. The constants ARR_SAMPLES and KEY_SAMPLES are chosen in such a way that the total size of all pre-generated input arrays is equal to the total size of all pre-generated search keys.

There are several reasons for using such a testing environment. First of all, I don't want to include random number generation into performance measurement, that's why input data is generated beforehand. Secondly, I want to avoid branch predictor memorizing the course of execution completely, that's why there are many input arrays and many keys, which are rotated in the loop. It is important that the number of pre-generated keys is large enough. Lastly, I want to be able to emulate reading from prescribed cache level (L1D, L2), and this is approximated by setting the total size of all pre-generated input data appropriately. Unless said otherwise, all the results and plots are given for the fastest L1D cache: input data takes 64 KB of memory, of which input arrays take 32 KB and fit into cache precisely.

All the measurements were done on Intel Core i3-5005U CPU (Broadwell, 2 GHz, no turbo boost). The code was compiled using VC++2017 x64 compiler with /O2 and /arch:AVX2 (unless noted otherwise). For all the plots below, the points on plot lines for binary search implementations have cross-like style, while the points for linear search implementations are marked with bold filled shapes. The plot lines are also grouped by color into: black = branching binary search, blue = branchless binary search, yellow = linear search with break, red/magenta = counting linear search (SSE/AVX).

Here are the names of search implementations:

binary_search_std: direct call to std::lower_bound
binary_search_simple: basic binary search with branches
binary_search_branchless: branchless binary search
binary_search_branchless_UR: branchless binary search - fully unrolled
linearX_search_sse: linear search with break (SSE)
linear_search_sse: counting linear search (SSE)
linear_search_sse_UR: counting linear search (SSE) - fully unrolled
linear_search_avx: counting linear search (AVX)
linear_search_avx_UR: counting linear search (AVX) - fully unrolled

Initial attempt

Here is the plot I got initially with results of throughput performance measurement:

I spent considerable effort analyzing this plot and making conclusions out of it. I wrote a page of text about it =) Of course, I did notice how much performance differs for unrolled and looped versions of branchless binary search. But after looking into assembly output, I did not try to further investigate it. And then I tried to compile the code with GCC (just in case), which changed everything...

With TDM GCC 5.1 x64, I got significantly different throughput performance for several search implementations. Linear searches work more or less in the same time (GCC being slightly slower), but the binary searches are tremendously different:

binary_search_std improved from 60 ns to 47 ns
binary_search_simple improved from 49 ns to 16 ns
binary_search_branchless improved 25 ns to 11 ns
binary_search_branchless_UR worsened from 8 to 26 ns

The point 1 is explained by different implementations of STL. The points 2 and 4 are caused by differences in cmovXX instructions generation: binary_search_simple uses cmov on GCC instead of branches (quite unexpectedly: branches were intended), and the assembly code of binary_search_branchless_UR is some terrible mess on GCC, including both cmov-s and branches mixed (that's really bad of GCC). The point 3 cannot be explained so easily: assembly outputs are very similar, and cmov-s are used in both of them.

After having spent some time on blending one assembly code into the other one, I found the critical instruction which spoils everything for MSVC-generated assembly code:

?binary_search_branchless@@YAHPEBHHH@Z PROC
bsr edx, edx         ; intptr_t logstep = bsr(n);
mov eax, 1
or r9, -1            ; intptr_t pos = -1;
shlx rax, rax, rdx   ; intptr_t step = intptr_t(1) << logstep;

test rax, rax        ; while (step > 0) {
jle SHORT $LN13
npad 10
$LL2:
    ; pos = (arr[pos + step] < key ? pos + step : pos);
    ; step >>= 1;

    lea rdx, QWORD PTR [rax+r9]
    sar rax, 1
    cmp DWORD PTR [rcx+rdx*4], r8d
    cmovge rdx, r9
    mov r9, rdx
    test rax, rax
jg SHORT $LL2

lea rax, QWORD PTR [rdx+1]   ; return pos + 1;
ret 0
$LN13:
lea rax, QWORD PTR [r9+1]      ; return pos + 1;
ret 0
?binary_search_branchless@@YAHPEBHHH@Z ENDP

The instruction "or r9, -1" on 4-th line is a cunning way of putting -1 into register: it takes only 4 bytes of machine code, while the obvious "mov r9, -1" takes 7 bytes of machine code. It is very similar to the "xor eax, eax" hack for zeroing a register, which every CPU understands today and handles during register renaming. Unfortunately, there is no special handling of instruction "or r9, -1" in Broadwell or other x86 architectures, so false dependency is created: CPU assumes that the new value of r9 register depends on its previous value, which is not set before that in the function. Since the function is called over and over again, most likely the previous value is assigned in line 17 of the previous function call, and it is set to the answer of search. As a result, CPU cannot start executing the next search without knowing the result of the previous one, hence latency is actually measured on MSVC instead of throughput.

This explains why MSVC version is so much slower than that of GCC. If we look into latency measurements (provided below), we'll see the values for binary_search_branchless there equal to those on the plot above. The issue can be circumvented by assigning value -1 from non-const global variable, then MSVC has to load it from memory (hopefully from L1D) with mov instruction:

intptr_t MINUS_ONE = -1;
int binary_search_branchless (const int *arr, int n, int key) {
    //intptr_t pos = -1;            //generates "or r9, -1" on MSVC -- false dependency harms throughput
    intptr_t pos = MINUS_ONE;       //workaround for MSVC: generates mov without dependency

Revised code

After I resolved the issue with MSVC code gen, I decided that I should measure both throughput and latency time. Here are the results for revised code (throughput and latency performance):

Here are some facts about the results:

For a binary search, branchless implementation is much faster than branching one: almost four times faster in throughput sense and two times faster by latency. I think it would remain so even if support for non-power-of-two array lengths was added to branchless implementation.
Full unrolling of branchless binary search (with exactly known number of comparisons to do) makes search faster by about 30% in throughput sense. The difference in latency is negligible: 3-4 cycles.
It is hard to ensure that binary search written in C++ is branchless on every platform, every compiler and every set of settings. I realized this after looking at GCC's work: it managed to miracously emit cmov in once case and stupidly screw cmov-s in the other place. Also, MSVC screws the last cmov in binary_search_branchless_UR if it is inlined into the benchmark code. There is no way to get cmov-s reliably except for using assembly.
Latency time of branchless binary search is twice as large as throughput time. I suppose it means that CPU runs two searches in parallel when they are independent. However, it can be harder to ensure such independence in real workflow, when binary search is not the only computation done in the hot loop.
Counting linear search is faster than linear search with break for small N. This is because linear search with break executes unpredictable branch (which does not happen for N = 16 because it always needs one iteration). There is about 60% improvement in throughput for N = 32, and about 35% improvement for N = 64 (less improvement in latency). It seems that the branch costs about 10 cycles in both throughput and latency.
The linear search with break becomes faster than counting linear search shortly after N = 128. For N = 1024 it is 80% faster, and I guess the performance ratio should converge to two at infinity. This happens because linear search with break processes only half of input array on average, while counting linear search always goes through the whole array. It is questionable whether it is a huge benefit, given that for N > 128 branchless binary search is faster and reads substantially less data anyway.
For counting linear search, AVX version is noticeably faster only for N = 64, and for larger N it is about 50% faster. Perhaps more efficient AVX implementation would achieve better ratio. As for full unrolling, it gives another almost 50% boost for large N (like N = 256 or more), althrough for small N its benefit is again much less noticeable.
As for binary vs linear search competition, the situation is different for throughput and latency. For throughput performance, branchless binary search is slower for N < 64 (by at most 30%) and faster for N > 64, although the fully unrolled version of it is as fast as SSE linear search even for N = 16 and N = 32. For latency performance, branchless binary search catches up with SSE linear search only for N between 128 and 256 (double that for AVX), and it is slower by up to 50% for N = 32 and N = 64.

Additional plots

We can also estimate how much time per element is required for each search. Here are the plot where elapsed time is reported per array element (recall that you have to multiply by two to convert from ns to cycles):

Notice how all the lines are converging to each other on the left: for N = 16 all searches have very similar performance. I think this is the overhead of benchmarking environment which makes them so similar: I simply cannot see e.g. how much faster AVX version is (compared to SSE version) with my crude measuring tool. The linear searches are very compact, for instance unrolled SSE version for N = 16 executes only 16 instructions without any branches at all.

And here is yet another view of the results where time per search is divided by the logarithm of N (i.e. by the number of comparisons in binary search). It shows the constant in O(log N) estimate for binary searches, just like the previous plot shows constant in O(N) estimate for linear searches. These images can be helpful to estimate the cost of single step of a binary search.

Finally, below are plots recorded while using 512KB of input data, which tries to simulate L2 memory access. The CPU has 256 KB of L2 cache per core, which means that all input arrays fit into it perfectly. It seems that adding L1 cache misses into benchmark makes things closer to each other, but does not change the situation much.

Note that not all memory loads are forced to access L2 cache, sometimes faster L1 cache is also used. If you consider counting linear search, it always accesses all the array elements during search. Because of that the whole 256 KB of input array data is accessed over and over again, which makes sure that it is read from L2 cache. In case of binary search, different elements have different probability of being accessed: for N = 1023, 511-st element is always accessed, elements 255 and 767 are accessed with probability 50%, even-indexed elements have probability 0.2% of being accessed (see more about it in this bannalia blog post) So it is quite likely that the most popular elements get into smaller L1 cache, accelerating several first comparisons in the search. For instance, for N = 512 there are 128 input arrays and 32 KB of L1D cache, so each input array gets 4 cache lines in L1D, enough to accelerate first two iterations. This is a natural benefit of binary search, which works in real world too.

The code

All the code and materials are available in GitHub repository linear-vs-binary-search.

Conclusion

The proposed "counting" linear search is noticeably faster for small array lengths than linear search with break, because branch misprediction is removed from it. So it makes sense to always use counting version of linear search instead of the breaking one, since for larger array lengths binary search becomes faster anyway.

Binary search is surprisingly good to stand against linear search, given that it fully utilizes conditional move instructions instead of branches. In my opinion, there is no reason to prefer linear search over binary search, better make sure that your compiler does not generate branches for the binary search. Counting linear search is worth using only in a rare case when it is known that array length is very small and the search performance is really critically important. Also, the idea of counting linear search is easy to embed into some vectorized computation, unlike the binary search.

Addendum

I decided to keep all the additional information about this article in a separate blog post. You may find more information in the addendum, including some more implementations and extended performance comparison.

This article is discussed on reddit here.

Vectorizing std::merge with vpermd from AVX2 and lookup table

2017-08-02T23:20:00+07:00

Recently I stumbled upon a question on stackoverflow, which asked how to vectorize computing symmetric difference of two sorted int32 arrays using AVX2. I decided to try doing it myself, and this post is about what I achieved. Of course, the best way to compute symmetric difference of sorted sets is by running ordinary merge algorithm (e.g. with std::merge) for the arrays plus some simple postprocessing. I'll concentrate only on the generic merging algorithm here.

I'll handle 32-bit integer keys only. Having keys of larger size would reduce significantly the efficiency of the algorithm (as usual with vectorization). Using 32-bit floating point keys is not much different from using integer keys; moreover, sorting 32-bit floats can be easily reduced to sorting 32-bit integers, which is often used to run radix sort on floating point data (see this and that). Also I'll briefly discuss the case when 32-bit values are attached to 32-bit keys (sorting without values is pretty useless in practice).

Better scalar code

Before trying to vectorize the algorithm, let's try to see how fast scalar implementation can be. Here is a simple implementation of merging algorithm:

int Merge_ScalarTrivial(const int *aArr, int aCnt, const int *bArr, int bCnt, int *dst) {
    int i = 0, j = 0, k = 0;
    while (i < aCnt && j < bCnt) {
        if (aArr[i] < bArr[j])
            dst[k++] = aArr[i++];
        else
            dst[k++] = bArr[j++];
    }

    memcpy(dst + k, aArr + i, (aCnt - i) * sizeof(dst[0]));
    k += (aCnt - i);
    memcpy(dst + k, bArr + j, (bCnt - j) * sizeof(dst[0]));
    k += (bCnt - j);
    return k;
}

The first possible improvement is about termination criterion. Since the inner loop is tiny, checking i < aCnt and j < bCnt can take noticeable time (note also that both i and j grow unpredictably). This can be improved in one of three ways:

Add sentinel element (INT_MAX) immediately after each input array, so that we can iterate by k in range [0 .. aCnt + bCnt). This approach is great, and the main drawback is that it forces us to copy the whole input if we cannot freely overwrite such after-the-end elements.
Do iterations in blocks of size 32 (for instance). Just replace the condition of while with i <= aCnt - 32 && j <= bCnt - 32, and add second loop for (int t = 0; t < 32; t++) just inside the first one. This way the innermost loop has simple stop criterion t < 32, and the outer loop terminates when it cannot be guaranteed that the inner loop won't get out of any input array. The remaining iterations are processed in the usual way, which is not very nice of course (there can be many iterations remaining).
(taken). Carefully detect when exactly the loop finishes. In order to achieve this: find which input array has greater last element, then iterate back from that element to detect all elements which get into memcpy-ed tail. Given that number of such elements is X, we can freely replace the loop with for (int k = 0; k < aCnt + bCnt - X;). In future with vectorization, we'll have to generalize this approach so that at least four elements are always available in the inner loop: this is achieved by doing merge in reversed order (from the ends of both arrays) until both arrays have at least four elements processed.

The second improvement is removal of branch if (aArr[i] < bArr[j]). This branch is completely unpredictable if the input arrays are randomly interleaved, which makes CPU do excessive work and increases latency of each iteration. In order to determine which element must be stored into dst[k], it is enough to use ternary operator, which would compile into cmov instruction. In order to increment i and j counters, we can add boolean results of some comparisons to both of them, which would compile into setXX instructions.

These two changes reduce average time per element from 6.5 ns to 4.75 ns (on random inputs, on my laptop). Here is the resulting code:

int Merge_ScalarBranchless_3(const int *aArr, int aCnt, const int *bArr, int bCnt, int *dst) {
    //let's assume values = INT_MIN are forbidden
    int lastA = (aCnt > 0 ? aArr[aCnt-1] : INT_MIN);
    int lastB = (bCnt > 0 ? bArr[bCnt-1] : INT_MIN);
    if (lastA < lastB) //ensure that lastA >= lastB
        return Merge_ScalarBranchless_3(bArr, bCnt, aArr, aCnt, dst);
    int aCap = aCnt;
    while (aCap > 0 && aArr[aCap-1] >= lastB)
        aCap--;

    size_t i = 0, j = 0, k = 0;
    for (; k < aCap + bCnt; k++) {
        int aX = aArr[i], bX = bArr[j];
        dst[k] = (aX < bX ? aX : bX); //cmov
        i += (aX < bX);               //setXX
        j += 1 - (aX < bX);           //setXX
    }
    assert(i == aCnt || j == bCnt);

    memcpy(dst + k, aArr + i, (aCnt - i) * sizeof(dst[0]));
    k += (aCnt - i);
    memcpy(dst + k, bArr + j, (bCnt - j) * sizeof(dst[0]));
    k += (bCnt - j);
    return k;
}

Idea of vectorization

The main problem with the merge algorithm is that it is inherently sequental. Before you finish comparison of two head elements, you cannot compare the next ones, you don't even know which two elements you have to compare next. This issue makes both scalar and vectorized code run slower, because instructions have to wait for each other to complete, and the compute units in CPU stay idle during this time.

Another problem is that the data is moved and shuffled in complex and dynamic ways during merging process. This is not noticeable in scalar implementation, because you operate on only two elements at a time. But if you want to vectorize merge operation, you have to work with more elements at once. If you take four elements from each array, you soon realize that these elements must be shuffled in one of 70 ways to get them into single sorted sequence. This is a big issue for vectorization, because it was mostly designed to do some simple r[i] = a[i] + b[i] things:

Yes, SSE and AVX have many shuffling operations. But most of them accept only immediate operand (i.e. compile-time constant) as control mask, so they can only shuffle data in a static way. The rare exceptions are: pshufb (_mm_shuffle_epi8) from SSSE3, vpermilpd and vpermilps (_mm256_permutevar_p?) from AVX, and vpermd and vpermps (_mm256_permutevar8x32_*) from AVX2. The first two can only shuffle data within 128-bit blocks, but the last one can shuffle eight 32-bit integers within 256-bit register in arbitrary way, which should allow us to merge 4 + 4 elements in one iteration.

Constructing shuffle control mask directly in the code is possible, but this is often too complex, and thus too slow. This is where lookup table (LUT) comes into play. The generic approach looks like this:

Obtain enough information about input (f.i. by comparing elements), so that the needed shuffle control mask could be uniquely determined from this information.
Convert information into compact bitmask in general purpose register (usually done by some shuffling and movemask intrinsic).
(optional) Apply perfect hash function (e.g. some multiplicative hash) to bitmask to obtain number of smaller size.
Use obtained number as index in a lookup table. The corresponding element of the table must contain the desired control mask for shuffling (precomputed beforehand).

If this sounds complicated, just wait a bit: in the next section all of these steps will be thoroughly explained on example.

Speaking of our particular merging problem, we will use this approach to process four input elements per iteration. Suppose that from each input array, four minimal elements are loaded into xmm register. First of all, we merge these eight elements into a single sorted sequence. First half of the resulting sequence (4 elements) can be stored to the destination array immediately. The second half should remain in processing, because some future input elements can still be less than them.

An important question here is: how to move pointers in the input arrays? What are we going to load and process on the next iteration? Two approaches come into mind (see pictures below):

Move each input pointer by number of its elements written to destination array (i.e. by how many elements from input array got into the four minimal elements of the sorted eight-element sequence). It means one of: move each pointer by two elements, move only one pointer by four elements, or move one pointer by three and the other pointer by one. On the next iteration, we'll simply load four elements again from each array using corresponding moved pointer. This approach has many drawbacks: we have to use unaligned loads for input arrays, we have to load each input element twice on average, and we have to spend additional time on computing the pointer movements. Moreover, it is hard to learn what exactly we must load for the next iteration until we have fully sorted the eight elements on the current iteration: this makes dependency chain quite long. Here is an illustration:
(taken). Move one input pointer by exactly four elements, leave the other one unchanged. When choosing which pointer to move, look where the last element considered is less: it should be moved. Let it be e.g. from the input array B; then on the next iteration the next four elements from B must be loaded and processed. We cannot use the same four elements from A on the next iteration, because some of them may already be dumped to the destination array. So we take the second half of the sorted eight-element sequence from the current iteration to be used as four elements from A on the next iteration (these are exactly the elements loaded from A and B which are not yet dumped to the destination array). The core advantage here is that we can determine which pointer to move without waiting for the eight-element sorted sequence to be computed. It means that CPU will load the data for the next iteration and compare/sort the current data in parallel.

Implementation

Assume for simplicity that all input elements are distinct: then there is unique order in which the elements must go in the merged sequence. Let's suppose that we already have four elements from A loaded into srcA, and four elements of B loaded into srcB (both are xmm registers).

Here is an illustration for steps 1-4 and step A:

Step 1. It is enough to compare all elements from srcA with all elements from srcB to fully determine their order in the common sorted sequence. In fact, the position of element srcA[i] in the final sorted array is equal to number of elements from srcB less than it plus its index i. If AVX2 had shuffling instruction which accepts scatter-style indices, we could even construct shuffle control mask directly without LUT using this property =) Unfortunately, SSE/AVX shuffles accept only gather-style indices.

This means we need to perform 16 comparisons (4 x 4) in total. Clearly, we can do four comparisons in single instruction, then rotate one of the registers by one element and compare again, and then repeat it two more times --- as the result, we'll get four xmm registers with comparison results bitmasks.

[1]  __m128i mask0 = _mm_cmplt_epi32(srcA, srcB);
[2]  __m128i mask1 = _mm_cmplt_epi32(srcA, _mm_shuffle_epi32(srcB, SHUF(1, 2, 3, 0)));
[3]  __m128i mask2 = _mm_cmplt_epi32(srcA, _mm_shuffle_epi32(srcB, SHUF(2, 3, 0, 1)));
[4]  __m128i mask3 = _mm_cmplt_epi32(srcA, _mm_shuffle_epi32(srcB, SHUF(3, 0, 1, 2)));

Step 2. Sixteen comparisons means that we should be able to get 16-bit mask, where each bit corresponds to the result of single comparison. In order to achieve this, we pack all the four bitmasks obtained on the previous step into single xmm register, where each byte contains result of one comparison. Using intrinsic _mm_packs_epi16 three times does this exactly. Then apply _mm_movemask_epi8 to the result: it gets sign bits of all 16 bytes and puts them into single 16-bit mask in general purpose register.

[5]  __m128i mask01 = _mm_packs_epi16(mask0, mask1);
[6]  __m128i mask23 = _mm_packs_epi16(mask2, mask3);
[7]  __m128i mask = _mm_packs_epi16(mask01, mask23);
[8]  uint32_t bits = _mm_movemask_epi8(mask);

Step 3. The obtained 16-bit bitmask can be used directly to load control mask from a lookup table. However, such a lookup table would take 65536 x 32 B = 2 MB of space, and the algorithm would read randomly some of its entries. This would waste precious TLB entries, so it would be great to compress this LUT somehow.

First we need to understand how many entries are really used in the table. Notice that each 16-bit mask determines precisely the ordering of all eight elements and vice versa (recall: all elements are distinct for now). Given that each set of four elements is sorted, each global ordering corresponds to a coloring of eight things into black and white, so that four things are colored in each color. This is binomial coefficient C(8,4) = 70. So only 70 bitmasks of 65536 are really possible as values of bits variable.

Now we want to create a hash function which maps valid bitmasks into integers from 0 to K-1, so that K is quite small (a bit greater than 70), hash function is perfect (i.e. gives no collisions) and blazingly fast to compute. Multiplicative hash function fits well under these criteria:

$f(x) = \left\lceil \frac{A \cdot x}{2^{32-b}} \right\rceil;$ where $x \in [0 \ldots 2^{32}), A \in (0 \ldots 2^{32}), b \in [0 \ldots 32)$.

For the problem considered, there is such a hash function for b = 7 and A = 0x256150A9. The code for its usage is:

[9]   uint32_t hash = ((bits * 0x256150A9U) >> 25);
[10]  __m256i perm = ((__m256i*)lookupTable)[hash];

This also finishes step 4 from the generic description above.

LUT generation. This part can easily take more time and more code than the whole merging algorithm =) But in its essence, it is just a small exercise in algorithmic programming. I'll only outline how it is done, you can look at the full code at the end of the article if you want.

Iterate through all 8-bit bitmasks with exactly 4 bits set: the i-th bit says from which input array i-th element in the sorted sequence comes (in fact this is an even coloring of eight elements into two colors). For each of them:
1. Assign numbers e.g. from 0 to 7 to the elements, so that they go in proper order according to the coloring. Note that exact values of the elements do not matter, only their ordering does.
2. Generate 16-bit bitmask by doing all pairwise comparisons between the elements from each input array exactly as you do it in the merging algorithm (4 x 4 comparisons, one bit generated by each of them). This is the key for LUT.
3. Generate the desired shuffle control mask which you need to use in this exact case to get the sorted sequence. This is the value for LUT.
4. Dump all 70 key-value pairs into an array.
Generate perfect hash function of the form shown above for the keys. For a fixed value of b (for instance, b = 7), iterate over all values of A. For each value, apply hash function to all the 70 keys and check for collisions. If there are no collisions, then you are done; otherwise, try the next value of A. This step took about a minute of computing time on my laptop.
Apply the generated hash function to each key, and save corresponding value into the entry of lookup table with the obtained index.
Dump the obtained lookup table into a text file in C++-compilable format. Then copy/paste its contents into your C++ file, so that you don't have to generate this data each time during program startup.

Step A. Now we have to apply this magic shuffling intrinsic _mm256_permutevar8x32_epi32. Since we have input elements in two separate xmm registers, we have to combine them first. Also, we have to split the result into two halves afterwards.

[11]  __m256i allSrc = _mm256_setr_m128i(srcA, srcB);
[12]  __m256i sorted = _mm256_permutevar8x32_epi32(allSrc, perm);
[13]  __m128i sortedLow = _mm256_castsi256_si128(sorted);  //nop
[14]  __m128i sortedHigh = _mm256_extractf128_si256(sorted, 1);

Note that the elements from the lower half can be stored immediately into the destination array:

[15]  _mm_store_si128((__m128i*)(dst + k), sortedLow);
[16]  k += 4;

Step B. Finally, we have to move the pointers in the input arrays. More precisely, we want to move only one pointer by 4 elements, depending on how the last elements in srcA and srcB compare to each other. We want to avoid branches (not to waste CPU time) and memory accesses (to keep latency low).

I must admit that there are a lot of ways to achieve it, and tons of possible code snippets that do it. Moreover, each snippet may compile to different assembly code, depending on exact choice of intrinsics and bitness/signedness of integer variables. I'll just describe the approach which worked best for me among numerous tries I did.

There are two pointers ptrA and ptrB: each of them moves along one of the input arrays. To simplify code, we want to sometimes swap these two pointers, so that one specific pointer (e.g. ptrB) is always moved. Unfortunately, it is quite hard to efficiently swap two registers on condition in branchless fashion, but it becomes easier if we maintain address difference abDiff = ptrB - ptrA instead of the pointer ptrB. In such case swapping is equivalent to ptrA = ptrA + abDiff and abDiff = -abDiff.

First of all, recall that we have already compared 3-rd elements of srcA and srcB, and the result is saved in mask0 register. Let's extract it from there into GP register:

[17]  __m128i mask0last = _mm_shuffle_epi32(mask0, SHUF(3, 3, 3, 3));
[18]  size_t srcAless = _mm_cvtsi128_sz(mask0last);

_mm_cvtsi128_sz is a macro for _mm_cvtsi128_si32 or _mm_cvtsi128_si64, depending on bitness.

Then we want to swap pointers if srcAless = -1, and retain their values if srcAless = 0. The following ugly piece of code achieves that in 4 instructions:

[19]  aPtr = (const int *)(size_t(aPtr) + (abDiff & srcAless));
[20]  abDiff ^= srcAless;  abDiff -= srcAless;

Now the 3-rd element from ptrA is surely greater, so we have to move the ptrB pointer. Since we do not maintain ptrB, we have to increase abDiff instead:

[21]  abDiff += 4 * sizeof(int);

After that we load four new elements from ptrB (do not forget that the pointer must be computed):

[22]  bPtr = (const int *)(size_t(aPtr) + abDiff);
[23]  srcB = _mm_load_si128((__m128i*)bPtr);

According to the plan (see above), we must use higher half of the sorted sequence in place of the elements from array A on the next step, so:

[24]  srcA = sortedHigh;

The description of the vectorized implementation ends here.

Full code. All the code snippets gathered above can be combined to get a working merge function:

int Merge_Simd_KeysOnly(const int *aArr, int aCnt, const int *bArr, int bCnt, int *dst) {
    //merge input arrays in reverse direction (from ends)
    intptr_t i = aCnt - 1, j = bCnt - 1, k = aCnt + bCnt - 1;
    while (i >= 0 && j >= 0 && (i >= aCnt - 8 || j >= bCnt - 8)) {
        if (aArr[i] < bArr[j])
            dst[k--] = bArr[j--];
        else
            dst[k--] = aArr[i--];
        //TODO: any vectorized code if loop does not finish soon?
    }
    i++, j++, k++;

    //these cases are necessary to ensure that arrays are long enough
    if (i == 0) {
        memcpy(dst, bArr, j * sizeof(dst[0]));
        return aCnt + bCnt;
    }
    if (j == 0) {
        memcpy(dst, aArr, i * sizeof(dst[0]));
        return aCnt + bCnt;
    }

    //prepare for the first iteration
    size_t maxK = k, tail = aCnt + bCnt - k;
    const int *aPtr = aArr;
    const int *bPtr = bArr;
    intptr_t abDiff = intptr_t(bArr) - intptr_t(aArr);
    __m128i srcA = _mm_load_si128((__m128i*)aPtr);
    __m128i srcB = _mm_load_si128((__m128i*)bPtr);

    for (k = 0; k < maxK; ) {
        //inner-loop: combined from all the snippets
        __m128i mask0 = _mm_cmplt_epi32(srcA, srcB);
        __m128i mask1 = _mm_cmplt_epi32(srcA, _mm_shuffle_epi32(srcB, SHUF(1, 2, 3, 0)));
        __m128i mask2 = _mm_cmplt_epi32(srcA, _mm_shuffle_epi32(srcB, SHUF(2, 3, 0, 1)));
        __m128i mask3 = _mm_cmplt_epi32(srcA, _mm_shuffle_epi32(srcB, SHUF(3, 0, 1, 2)));
        __m128i mask = _mm_packs_epi16(_mm_packs_epi16(mask0, mask1), _mm_packs_epi16(mask2, mask3));
        uint32_t bits = _mm_movemask_epi8(mask);

        uint32_t hash = ((bits * 0x256150A9U) >> 25);
        __m256i perm = ((__m256i*)lookupTable)[hash];

        __m256i allSrc = _mm256_setr_m128i(srcA, srcB);
        __m256i sorted = _mm256_permutevar8x32_epi32(allSrc, perm);
        __m128i sortedLow = _mm256_castsi256_si128(sorted);
        __m128i sortedHigh = _mm256_extractf128_si256(sorted, 1);

        size_t srcAless = _mm_cvtsi128_sz(_mm_shuffle_epi32(mask0, SHUF(3, 3, 3, 3)));
        //conditionally swap ptrA and ptrB (abDiff)
        aPtr = (const int *)(size_t(aPtr) + (abDiff & srcAless));
        abDiff ^= srcAless;  abDiff -= srcAless;

        abDiff += 4 * sizeof(int);
        bPtr = (const int *)(size_t(aPtr) + abDiff);
        srcA = sortedHigh;
        srcB = _mm_load_si128((__m128i*)bPtr);

        _mm_store_si128((__m128i*)(dst + k), sortedLow);
        k += 4;
    }
    return aCnt + bCnt;
}

Generality

Equal elements

The algorithm (and the code) works properly even if equal elements are present. It can be checked using the "perturbation method", which is sometimes used to resolve singular cases in computational geometry.

Imagine that eps is a small positive number much less than 1. Consider packs srcA and srcB with four integers in each. Add i*eps to srcA[i] and subtract j*eps from srcB[j]. This perturbation does not change the masks generated during step 1 of the algorithm, because we do only comparisons of style "A[i] < B[j]" there. The code works properly for perturbed elements (because they are distinct), and it will shuffle the original elements in exactly the same way, so after step 4 we'll get properly sorted eight-element sequence.

The last thing to note is that when we choose which pointer to move, we can choose any of the two if their last elements are equal (just as we do in the ordinary scalar merge if elements are equal).

Key-value pairs

It is rather easy to extend the code to work with 32-bit values attached to 32-bit keys. The idea is to store four values in register valuesA for the four keys stored in register srcA, and do the same for array B. All the data movement happening on the keys must be exactly reproduced on the values too.

The lines 1-10 of the code listing are only gathering information about keys, so they can be left untouched. The lines 11-14 must be duplicated for the values with the same shuffle control mask.

If keys and values are provided in separate arrays (SoA layout), then lines 19-22 must also be duplicated, since values would have their own pointer into array A and difference between pointer addresses. And this is rather unfortunate, because it is a lot of instructions. On the other hand, lines 15 and 23 (load/store) can be simply duplicated without any changes.

If keys and values are interleaved (AoS layout), then it becomes a bit simpler. Lines 11-14 can be left untouched, but loads and stores in lines 15 and 23 must be accompanied with some shuffling to interleave/deinterleave keys and values (in addition to duplication, of course).

Performance

For testing performance, two integer sequences of length N were generated, each element is random with uniform distribution from 0 to 3*N. After that both of the sequences were sorted. Then all the methods were run on these two input arrays, each method was run many times in a row. Note that each run was done on the same input, and it is very important especially for small N, where branch predictor can simply memorize everything. Total elapsed time was measured, and average time in nanoseconds per one output element was computed. This time is shown in this table: Five implementations were tested:

Simd: vectorized version based on SSE/AVX2 (Merge_Simd_KeysOnly function described in the article)
Br.-less: scalar version without branches (see Merge_ScalarBranchless_3 function listing)
Opt.: optimized scalar version with branches (not shown in the article)
Triv.: baseline scalar version with branches (see Merge_ScalarTrivial function listing)
std::: direct call to std::merge (scalar, with branches)

Note that only the first two implementations were thoroughly optimized. That's why e.g. "Triv." performance is so different on 32-bit vs 64-bit. Most likely I did not inspect the assembly output, so there are some excessive 32/64-bit conversions, which normally does not affect performance, but in such tiny loops it becomes very important.

Although results for N = 1K are provided, they have little sense actually. I'm pretty sure that branch predictor simply memorized how the elements compare to each other, that's why scalar implementations with branches become so fast. In the real world, the input sequences would be different each time, so the performance results would be different too.

I was testing GCC in VirtualBox VM, and I hope that it does not affect the results too much. Speaking of MSVC vs GCC comparison, the first two implementations work equally well on them.

Brand new AMD Ryzen CPU and mobile Intel CPU (broadwell) were used for performance measurements. The two main implementations are branchless, and they work almost the same way on both processors. The one really interesting thing here is that the solutions with branches for N = 1K work a lot better on Ryzen then on Broadwell. I suppose the well-marketed branch predictor based on neural net does its job well and memorizes all the branches, unlike its Intel counterpart =)

All the code is available in this gist (both merge implementations and lookup table generator).

Conclusion

The vectorized version is 1.8 times faster than the scalar branchless version, and 2.3 times faster than std::merge. Is such a speedup worth all the trouble?! I think not =)

A related but simpler problem is set intersection problem: find all common elements in two strictly increasing arrays. A vectorized solution for it (very similar to the algorithm described in this article) is described in this blog post, and its optimized version is available here.

You might ask: why is the performance improvement from vectorization so low? is it because of lookup table or something else? I think I'll save these questions for another post =) Aside from some analysis of the code shown here, I'll try to implement merge sort on top of it. I wonder if I could beat std::sort with it...

Doom 3: C++ enhanced RTTI and memory debugging

2017-07-19T10:49:00+07:00

Although Doom 3 was released 13 years ago, there is still of lot of interesting stuff to find in it. The main reason for that is: it was developed in the time of changes. The ID software team had just moved from C to C++ (using Visual C++ 6). Graphic cards were moving from fixed pipeline to programming pipeline, CPUs were getting SIMD extensions. All of this caused a lot of diversity in the code: crazy mix of C++/C/asm code, several renderer backends, code acceleration for MMX/SSE/AltiVec (yeah, Macs used PowerPC at that time). Not to mention tons of scripts and defs, including C++-like scripting language, and in-game GUI system.

I have to deal with Doom 3 engine in the context of The Dark Mod (TDM for short).

This article is about memory debugging configuration in Doom 3 SDK (more precisely, "Debug with inlines and memory log").

Example

To get started, here are some messages dumped to game console in the memory debugging configuration (current version of TDM):

Also, after exiting game, a file "tdm_main_leak_size.txt" appears with this data:

size: 40 KB, allocs: 3: idlib/containers/HashIndex.cpp, line: 53
size: 24 KB, allocs: 3: idlib/containers/HashIndex.cpp, line: 50
size: 22 KB, allocs: 212: game/Game_local.cpp, line: 7116
size: 18 KB, allocs: 523: idlib/Str.cpp, line: 90
size: 8 KB, allocs: 38: c:/thedarkmod/tdm/idlib/containers/List.h, line: 377
size: 6 KB, allocs: 138: game/DarkModGlobals.cpp, line: 467
size: 3 KB, allocs: 150: , line: 0
size: 3 KB, allocs: 35: renderer/Model_md5.cpp, line: 743
size: 2 KB, allocs: 110: game/darkModLAS.cpp, line: 1282
size: 1 KB, allocs: 8: game/Objectives/ObjectiveLocation.cpp, line: 72
size: 0 KB, allocs: 39: game/SndPropLoader.cpp, line: 723
size: 0 KB, allocs: 39: game/SndProp.cpp, line: 339
size: 0 KB, allocs: 13: game/SEED.cpp, line: 3449
size: 0 KB, allocs: 8: renderer/draw_arb2.cpp, line: 841
size: 0 KB, allocs: 8: c:/thedarkmod/tdm/idlib/math/Polynomial.h, line: 601
size: 0 KB, allocs: 2: framework/I18N.cpp, line: 428
size: 0 KB, allocs: 1: framework/FileSystem.cpp, line: 2756
size: 0 KB, allocs: 3: game/EscapePointManager.cpp, line: 181
1333 total memory blocks allocated
134 KB memory allocated

Memory leak detection does not feel new, but the fact that each message about uninitialized member contains the exact name of the member is quite intriguing.

Memory debugging

First let's deal with the simpler part: memory heap debugging. Looking at Heap.h, we see that ID_DEBUG_MEMORY enables a special piece of code:

#else /* ID_DEBUG_MEMORY */

void * Mem_Alloc( const int size, const char *fileName, const int lineNumber );
void * Mem_ClearedAlloc( const int size, const char *fileName, const int lineNumber );
void Mem_Free( void *ptr, const char *fileName, const int lineNumber );
char * Mem_CopyString( const char *in, const char *fileName, const int lineNumber );
void * Mem_Alloc16( const int size, const char *fileName, const int lineNumber );
void Mem_Free16( void *ptr, const char *fileName, const int lineNumber );

#ifdef ID_REDIRECT_NEWDELETE

__inline void *operator new( size_t s, int t1, int t2, char *fileName, int lineNumber ) {
    return Mem_Alloc( s, fileName, lineNumber );
}
__inline void operator delete( void *p, int t1, int t2, char *fileName, int lineNumber ) {
    Mem_Free( p, fileName, lineNumber );
}
__inline void *operator new[]( size_t s, int t1, int t2, char *fileName, int lineNumber ) {
    return Mem_Alloc( s, fileName, lineNumber );
}
__inline void operator delete[]( void *p, int t1, int t2, char *fileName, int lineNumber ) {
    Mem_Free( p, fileName, lineNumber );
}
__inline void *operator new( size_t s ) {
    return Mem_Alloc( s, "", 0 );
}
__inline void operator delete( void *p ) {
    Mem_Free( p, "", 0 );
}
__inline void *operator new[]( size_t s ) {
    return Mem_Alloc( s, "", 0 );
}
__inline void operator delete[]( void *p ) {
    Mem_Free( p, "", 0 );
}

#define ID_DEBUG_NEW new( 0, 0, __FILE__, __LINE__ )
#undef new
#define new ID_DEBUG_NEW

#endif

#define Mem_Alloc( size ) Mem_Alloc( size, __FILE__, __LINE__ )
#define Mem_ClearedAlloc( size ) Mem_ClearedAlloc( size, __FILE__, __LINE__ )
#define Mem_Free( ptr ) Mem_Free( ptr, __FILE__, __LINE__ )
#define Mem_CopyString( s ) Mem_CopyString( s, __FILE__, __LINE__ )
#define Mem_Alloc16( size ) Mem_Alloc16( size, __FILE__, __LINE__ )
#define Mem_Free16( ptr ) Mem_Free16( ptr, __FILE__, __LINE__ )

#endif /* ID_DEBUG_MEMORY */

The first part declares debug versions of ID's memory allocation functions, each of them having additional parameters filename and lineNumber. Then global operator new and operator delete are overriden to capture all allocations done with standard new/delete. Again, they have additional parameters. After that the main source of issues comes: new keyword is redefined with a macro, which allows to capture file and line of each call automatically.

The implementation of debug memory allocation looks like:

void *Mem_AllocDebugMemory( const int size, const char *fileName,
                    const int lineNumber, const bool align16 ) {
    //...

    void *p = mem_heap->Allocate( size + sizeof( debugMemory_t ) );
    Mem_UpdateAllocStats( size );

    debugMemory_t *m = (debugMemory_t *) p;
    m->fileName = fileName;
    m->lineNumber = lineNumber;
    m->frameNumber = idLib::frameNumber;
    m->size = size;
    m->next = mem_debugMemory;
    m->prev = NULL;
    if ( mem_debugMemory )
        mem_debugMemory->prev = m;
    mem_debugMemory = m;

    return ( ( (byte *) p ) + sizeof( debugMemory_t ) );
}

Each user-returned block of memory is prepended by 32 bytes of data, stored in debugMemory_t struct. It captures some information about allocation. When this memory is freed, size data member is used to update allocation stats and to provide some limited double-free detection. Since all the blocks are always linked in a doubly linked list, it is straight-forward to dump all used dynamic memory blocks at any moment, including doing so at program termination to detect memory leaks.

In order to see memory stats constantly during playing, you can type com_showMemoryUsage 1 into the game console, and you'll see something like this:

The main benefit of implementing memory profiling system yourself versus taking a ready third-party solution is that you can see exactly the data you need. In this case, per-frame allocation data is recorded and shown (which works thanks to Mem_ClearFrameStats being called each frame).

As for dumping memory blocks, there are three ways to do it:

Exit game and look into file [tdm_game]_leak_location.txt (contents were shown above): this way is used to detect memory leaks.
Write memoryDump into game console and look into created file memoryDump.txt. This file contains all blocks alive, and can be used to perform additional offline analysis (sample).
Write memoryDumpCompressed -s into game console and look into file memoryDump.txt. Unlike the previous command, this one shows per-allocation-site statistics sorted according to parameter (-s = by size), which is rather compact (300 lines):

memoryDump.txt contents (click to display)

:::cpp size: 256912 KB, allocs: 354: c:/thedarkmod/tdm/idlib/Allocators.h, line: 613 size: 133217 KB, allocs: 4206: renderer/tr_main.cpp, line: 297 size: 27157 KB, allocs: 34927: c:/thedarkmod/tdm/idlib/containers/List.h, line: 377 size: 10240 KB, allocs: 10: c:/thedarkmod/tdm/idlib/Allocators.h, line: 378 size: 5494 KB, allocs: 11682: framework/DeclManager.cpp, line: 1907 size: 4466 KB, allocs: 2358: game/gamesys/Class.cpp, line: 468 size: 2813 KB, allocs: 41174: cm/CollisionModel_load.cpp, line: 660 size: 2693 KB, allocs: 103: c:/thedarkmod/tdm/idlib/Allocators.h, line: 80 size: 2684 KB, allocs: 337: cm/CollisionModel_load.cpp, line: 2913 size: 2449 KB, allocs: 46008: idlib/Str.cpp, line: 90 size: 2086 KB, allocs: 309: renderer/Model_md5.cpp, line: 195 size: 2048 KB, allocs: 2: renderer/CinematicFFMpeg.cpp, line: 607 size: 1983 KB, allocs: 769: cm/CollisionModel_files.cpp, line: 337 size: 1950 KB, allocs: 960: game/BrittleFracture.cpp, line: 331 size: 1872 KB, allocs: 1546: renderer/Image_init.cpp, line: 1361 size: 1762 KB, allocs: 764: cm/CollisionModel_files.cpp, line: 390 size: 1679 KB, allocs: 1819: cm/CollisionModel_load.cpp, line: 596 size: 1677 KB, allocs: 6173: idlib/containers/HashIndex.cpp, line: 50 size: 1487 KB, allocs: 528: game/physics/Clip.cpp, line: 102 size: 1131 KB, allocs: 13795: idlib/MapFile.cpp, line: 316 size: 1121 KB, allocs: 12488: framework/DeclManager.cpp, line: 1701 size: 1043 KB, allocs: 309: renderer/Model_md5.cpp, line: 196 size: 1024 KB, allocs: 1: renderer/tr_main.cpp, line: 245 size: 902 KB, allocs: 1200: cm/CollisionModel_load.cpp, line: 566 size: 814 KB, allocs: 337: cm/CollisionModel_load.cpp, line: 2905 size: 810 KB, allocs: 1572: renderer/RenderWorld.cpp, line: 254 size: 759 KB, allocs: 3471: c:/thedarkmod/tdmframework/DeclManager.h, line: 228 size: 691 KB, allocs: 680: cm/CollisionModel_files.cpp, line: 316 size: 683 KB, allocs: 5212: idlib/containers/HashIndex.cpp, line: 53 size: 669 KB, allocs: 27325: , line: 0 size: 651 KB, allocs: 704: ui/Window.cpp, line: 2188 size: 625 KB, allocs: 1: game/Game_local.cpp, line: 554 size: 620 KB, allocs: 980: idlib/containers/HashIndex.cpp, line: 101 size: 594 KB, allocs: 15207: c:/thedarkmod/tdm/idlib/containers/StrPool.h, line: 106 size: 590 KB, allocs: 2200: c:/thedarkmod/tdm/idlib/containers/HashIndex.h, line: 159 size: 512 KB, allocs: 1: renderer/CinematicID.cpp, line: 71 size: 458 KB, allocs: 3912: game/anim/Anim_Blend.cpp, line: 3073 size: 401 KB, allocs: 123: game/gamesys/Class.cpp, line: 161 size: 367 KB, allocs: 7238: tools/compilers/aas/AASFile.cpp, line: 913 size: 354 KB, allocs: 3776: sound/snd_cache.cpp, line: 98 size: 309 KB, allocs: 769: cm/CollisionModel_files.cpp, line: 433 size: 285 KB, allocs: 4876: idlib/MapFile.cpp, line: 540 size: 283 KB, allocs: 1339: c:/thedarkmod/tdm/idlib/containers/List.h, line: 536 size: 247 KB, allocs: 2181: c:/thedarkmod/tdm/idlib/containers/HashIndex.h, line: 166 size: 215 KB, allocs: 4598: ui/GuiScript.cpp, line: 375 size: 205 KB, allocs: 6590: game/OverlaySys.cpp, line: 245 size: 205 KB, allocs: 6560: game/script/Script_Program.cpp, line: 1256 size: 199 KB, allocs: 379: ui/Window.cpp, line: 2196 size: 178 KB, allocs: 251: game/AF.cpp, line: 676 size: 175 KB, allocs: 1000: idlib/MapFile.cpp, line: 125 size: 172 KB, allocs: 1107: cm/CollisionModel_load.cpp, line: 530 size: 164 KB, allocs: 860: framework/CVarSystem.cpp, line: 653 size: 161 KB, allocs: 2292: idlib/MapFile.cpp, line: 379 size: 159 KB, allocs: 1: game/physics/Clip.cpp, line: 714 size: 159 KB, allocs: 3143: ui/RegExp.cpp, line: 246 size: 159 KB, allocs: 1167: game/Entity.cpp, line: 5878 size: 148 KB, allocs: 73: sound/snd_world.cpp, line: 200 size: 142 KB, allocs: 111: renderer/RenderWorld.cpp, line: 408 size: 136 KB, allocs: 4368: ui/Window.cpp, line: 1625 size: 131 KB, allocs: 960: game/BrittleFracture.cpp, line: 1171 size: 128 KB, allocs: 1: renderer/CinematicID.cpp, line: 70 size: 121 KB, allocs: 3: game/ai/AAS_routing.cpp, line: 241 size: 114 KB, allocs: 4499: c:/thedarkmod/tdm/idlib/math/Vector.h, line: 1727 size: 107 KB, allocs: 3444: game/anim/Anim_Blend.cpp, line: 88 size: 105 KB, allocs: 406: cm/CollisionModel_load.cpp, line: 625 size: 91 KB, allocs: 835: renderer/ModelManager.cpp, line: 336 size: 87 KB, allocs: 40: game/anim/Anim_Blend.cpp, line: 3289 size: 83 KB, allocs: 1190: idlib/math/Ode.cpp, line: 32 size: 77 KB, allocs: 2198: game/script/Script_Program.cpp, line: 1223 size: 75 KB, allocs: 807: game/script/Script_Program.cpp, line: 1138 size: 74 KB, allocs: 2260: c:/thedarkmod/tdm/idlib/containers/List.h, line: 419 size: 69 KB, allocs: 1019: c:/thedarkmod/tdm/idlib/math/Matrix.h, line: 2308 size: 68 KB, allocs: 2197: game/Entity.cpp, line: 1006 size: 65 KB, allocs: 580: renderer/Material.cpp, line: 1540 size: 65 KB, allocs: 1392: ui/Window.cpp, line: 1862 size: 64 KB, allocs: 1: renderer/CinematicID.cpp, line: 68 size: 60 KB, allocs: 3: game/ai/AAS_routing.cpp, line: 128 size: 58 KB, allocs: 426: game/StimResponse/StimResponseCollection.cpp, line: 105 size: 57 KB, allocs: 733: idlib/geometry/Winding.cpp, line: 36 size: 53 KB, allocs: 55: ui/Window.cpp, line: 2253 size: 52 KB, allocs: 450: game/anim/Anim_Blend.cpp, line: 3359 size: 52 KB, allocs: 166: game/AF.cpp, line: 793 size: 52 KB, allocs: 215: game/StimResponse/StimResponseCollection.cpp, line: 91 size: 51 KB, allocs: 397: game/anim/Anim.cpp, line: 976 size: 49 KB, allocs: 38: ui/Window.cpp, line: 2220 size: 48 KB, allocs: 512: game/script/Script_Program.cpp, line: 1124 size: 45 KB, allocs: 162: game/physics/Physics_AF.cpp, line: 1061 size: 44 KB, allocs: 369: game/ai/AAS_routing.cpp, line: 52 size: 38 KB, allocs: 64: framework/DeclAF.cpp, line: 786 size: 37 KB, allocs: 599: framework/DeclManager.cpp, line: 998 size: 35 KB, allocs: 1148: ui/GuiScript.cpp, line: 520 size: 34 KB, allocs: 251: game/AF.cpp, line: 673 size: 32 KB, allocs: 299: renderer/ModelManager.cpp, line: 282 size: 32 KB, allocs: 1: cm/CollisionModel_load.cpp, line: 3385 size: 32 KB, allocs: 1: renderer/CinematicID.cpp, line: 69 size: 31 KB, allocs: 230: game/Moveable.cpp, line: 193 size: 28 KB, allocs: 1961: c:/thedarkmod/tdm/ui/Winvar.h, line: 44 size: 23 KB, allocs: 82: framework/DeclParticle.cpp, line: 213 size: 23 KB, allocs: 1190: game/physics/Physics_RigidBody.cpp, line: 1164 size: 22 KB, allocs: 369: game/ai/AAS_routing.cpp, line: 50 size: 22 KB, allocs: 212: game/Game_local.cpp, line: 7116 size: 20 KB, allocs: 39: renderer/RenderWorld_load.cpp, line: 623 size: 19 KB, allocs: 60: game/AF.cpp, line: 826 size: 18 KB, allocs: 30: framework/DeclAF.cpp, line: 1079 size: 17 KB, allocs: 25: game/AFEntity.cpp, line: 1524 size: 16 KB, allocs: 60: game/physics/Physics_AF.cpp, line: 1697 size: 16 KB, allocs: 26: framework/DeclAF.cpp, line: 1157 size: 15 KB, allocs: 397: c:/thedarkmod/tdm/idlib/containers/HashTable.h, line: 191 size: 14 KB, allocs: 208: game/script/Script_Program.cpp, line: 2076 size: 13 KB, allocs: 564: game/ai/EAS/EAS.cpp, line: 614 size: 13 KB, allocs: 564: game/ai/EAS/EAS.cpp, line: 616 size: 12 KB, allocs: 263: game/ai/AAS_routing.cpp, line: 894 size: 12 KB, allocs: 3: framework/FileSystem.cpp, line: 1253 size: 12 KB, allocs: 3: game/ai/AAS_routing.cpp, line: 238 size: 12 KB, allocs: 13: ui/UserInterface.cpp, line: 270 size: 11 KB, allocs: 86: game/Entity.cpp, line: 5868 size: 11 KB, allocs: 60: framework/CVarSystem.cpp, line: 578 size: 11 KB, allocs: 10: ui/Window.cpp, line: 2231 size: 11 KB, allocs: 1: framework/KeyInput.cpp, line: 709 size: 10 KB, allocs: 185: game/anim/Anim_Blend.cpp, line: 4637 size: 10 KB, allocs: 684: ui/Window.cpp, line: 1694 size: 10 KB, allocs: 649: renderer/tr_polytope.cpp, line: 91 size: 9 KB, allocs: 133: game/StimResponse/Response.cpp, line: 212 size: 9 KB, allocs: 615: ui/Window.cpp, line: 1628 size: 9 KB, allocs: 68: game/Mover.cpp, line: 380 size: 9 KB, allocs: 1158: game/PVSToAASMapping.cpp, line: 197 size: 8 KB, allocs: 19: game/ai/AI.cpp, line: 1652 size: 8 KB, allocs: 271: framework/CmdSystem.cpp, line: 371 size: 8 KB, allocs: 53: renderer/ModelManager.cpp, line: 288 size: 8 KB, allocs: 3: game/ai/AAS_routing.cpp, line: 218 size: 8 KB, allocs: 173: ui/GuiScript.cpp, line: 450 size: 7 KB, allocs: 1: game/Game_local.cpp, line: 585 size: 6 KB, allocs: 12: game/ModelGenerator.cpp, line: 296 size: 6 KB, allocs: 138: game/DarkModGlobals.cpp, line: 467 size: 6 KB, allocs: 271: framework/CmdSystem.cpp, line: 366 size: 6 KB, allocs: 3: game/ai/AAS_routing.cpp, line: 247 size: 5 KB, allocs: 256: framework/DeclManager.cpp, line: 408 size: 4 KB, allocs: 255: framework/DeclManager.cpp, line: 418 size: 4 KB, allocs: 106: game/ai/AAS_routing.cpp, line: 1033 size: 4 KB, allocs: 16: c:/thedarkmod/tdm/idlib/containers/List.h, line: 60 size: 4 KB, allocs: 4: ui/Window.cpp, line: 2209 size: 4 KB, allocs: 4: ui/EditWindow.cpp, line: 99 size: 4 KB, allocs: 294: c:/thedarkmod/tdm/ui/Window.h, line: 110 size: 4 KB, allocs: 141: game/anim/Anim_Blend.cpp, line: 343 size: 4 KB, allocs: 72: game/anim/Anim_Blend.cpp, line: 4597 size: 4 KB, allocs: 2: sound/snd_world.cpp, line: 81 size: 3 KB, allocs: 21: idlib/math/Lcp.cpp, line: 1603 size: 3 KB, allocs: 11: game/Player.cpp, line: 1325 size: 3 KB, allocs: 14: game/Missions/MissionDB.cpp, line: 74 size: 3 KB, allocs: 238: ui/Window.cpp, line: 1632 size: 3 KB, allocs: 34: renderer/Model_md5.cpp, line: 743 size: 3 KB, allocs: 105: ui/Window.cpp, line: 2305 size: 3 KB, allocs: 30: game/SndProp.cpp, line: 332 size: 3 KB, allocs: 294: ui/Window.cpp, line: 2335 size: 3 KB, allocs: 25: game/AFEntity.cpp, line: 1481 size: 3 KB, allocs: 271: framework/CmdSystem.cpp, line: 367 size: 3 KB, allocs: 23: renderer/ModelManager.cpp, line: 294 size: 3 KB, allocs: 3: framework/FileSystem.cpp, line: 1252 size: 2 KB, allocs: 13: ui/UserInterface.cpp, line: 157 size: 2 KB, allocs: 20: game/Entity.cpp, line: 5939 size: 2 KB, allocs: 20: game/Actor.cpp, line: 2427 size: 2 KB, allocs: 76: game/ai/EAS/EAS.cpp, line: 104 size: 2 KB, allocs: 13: renderer/Cinematic.cpp, line: 57 size: 2 KB, allocs: 19: game/ai/AI.cpp, line: 1986 size: 2 KB, allocs: 19: game/AFEntity.cpp, line: 702 size: 2 KB, allocs: 110: game/darkModLAS.cpp, line: 1282 size: 2 KB, allocs: 4: framework/DeclAF.cpp, line: 961 size: 2 KB, allocs: 19: game/ai/AI.cpp, line: 1655 size: 2 KB, allocs: 3: game/ai/AAS_routing.cpp, line: 244 size: 2 KB, allocs: 19: renderer/Model_prt.cpp, line: 95 size: 1 KB, allocs: 30: game/SndProp.cpp, line: 384 size: 1 KB, allocs: 14: game/Mover.cpp, line: 3391 size: 1 KB, allocs: 8: game/physics/Physics_AF.cpp, line: 1872 size: 1 KB, allocs: 1: ui/Window.cpp, line: 2286 size: 1 KB, allocs: 105: c:/thedarkmod/tdm/ui/Window.h, line: 129 size: 1 KB, allocs: 7: game/Inventory/Inventory.cpp, line: 656 size: 1 KB, allocs: 1: game/SndProp.cpp, line: 366 size: 1 KB, allocs: 20: game/StimResponse/Response.cpp, line: 189 size: 1 KB, allocs: 3: tools/compilers/aas/AASFileManager.cpp, line: 50 size: 1 KB, allocs: 1: ui/Window.cpp, line: 2264 size: 1 KB, allocs: 4: game/physics/Physics_AF.cpp, line: 1084 size: 1 KB, allocs: 81: game/darkModLAS.cpp, line: 1290 size: 1 KB, allocs: 16: game/ai/States/IdleState.cpp, line: 174 size: 1 KB, allocs: 1: game/SndProp.cpp, line: 314 size: 1 KB, allocs: 5: game/physics/Physics_AF.cpp, line: 3102 size: 1 KB, allocs: 1: cm/CollisionModel_load.cpp, line: 745 size: 1 KB, allocs: 1: ui/ListWindow.cpp, line: 46 size: 1 KB, allocs: 16: game/ai/Tasks/IdleAnimationTask.cpp, line: 437 size: 1 KB, allocs: 4: game/AF.cpp, line: 748 size: 1 KB, allocs: 8: game/Objectives/ObjectiveLocation.cpp, line: 72 size: 1 KB, allocs: 14: game/Missions/MissionDB.cpp, line: 71 size: 1 KB, allocs: 33: game/anim/Anim_Blend.cpp, line: 323 size: 1 KB, allocs: 1: c:/thedarkmod/tdm/idlib/containers/HashTable.h, line: 86 size: 0 KB, allocs: 1: game/SndProp.cpp, line: 319 size: 0 KB, allocs: 39: game/SndPropLoader.cpp, line: 723 size: 0 KB, allocs: 39: game/SndProp.cpp, line: 339 size: 0 KB, allocs: 43: ui/Window.cpp, line: 2433 size: 0 KB, allocs: 26: game/ai/Conversation/Conversation.cpp, line: 475 size: 0 KB, allocs: 6: game/ai/AAS.cpp, line: 30 size: 0 KB, allocs: 1: cm/CollisionModel_load.cpp, line: 742 size: 0 KB, allocs: 24: game/anim/Anim_Blend.cpp, line: 702 size: 0 KB, allocs: 19: game/ai/AI.cpp, line: 1656 size: 0 KB, allocs: 19: game/ai/AI.cpp, line: 1657 size: 0 KB, allocs: 19: game/ai/AI.cpp, line: 1658 size: 0 KB, allocs: 19: game/ai/AI.cpp, line: 1659 size: 0 KB, allocs: 23: game/anim/Anim_Blend.cpp, line: 826 size: 0 KB, allocs: 23: game/anim/Anim_Blend.cpp, line: 724 size: 0 KB, allocs: 3: framework/minizip/unzip.cpp, line: 792 size: 0 KB, allocs: 42: renderer/RenderWorld_load.cpp, line: 295 size: 0 KB, allocs: 42: idlib/geometry/Winding.cpp, line: 469 size: 0 KB, allocs: 5: framework/CVarSystem.cpp, line: 151 size: 0 KB, allocs: 2: sound/snd_system.cpp, line: 960 size: 0 KB, allocs: 20: game/anim/Anim_Blend.cpp, line: 686 size: 0 KB, allocs: 7: game/ai/Conversation/ConversationSystem.cpp, line: 242 size: 0 KB, allocs: 14: game/SEED.cpp, line: 972 size: 0 KB, allocs: 19: game/anim/Anim_Blend.cpp, line: 811 size: 0 KB, allocs: 13: game/SEED.cpp, line: 3449 size: 0 KB, allocs: 8: framework/DeclManager.cpp, line: 975 size: 0 KB, allocs: 13: framework/DeclManager.cpp, line: 943 size: 0 KB, allocs: 16: game/anim/Anim_Blend.cpp, line: 359 size: 0 KB, allocs: 10: ui/Window.cpp, line: 2107 size: 0 KB, allocs: 29: game/darkModLAS.cpp, line: 1299 size: 0 KB, allocs: 14: game/anim/Anim_Blend.cpp, line: 834 size: 0 KB, allocs: 14: game/anim/Anim_Blend.cpp, line: 849 size: 0 KB, allocs: 7: game/Inventory/Inventory.cpp, line: 393 size: 0 KB, allocs: 2: renderer/RenderSystem.cpp, line: 1067 size: 0 KB, allocs: 6: game/ai/AAS.cpp, line: 48 size: 0 KB, allocs: 21: game/physics/Physics_AF.cpp, line: 7366 size: 0 KB, allocs: 16: game/ai/States/IdleState.cpp, line: 383 size: 0 KB, allocs: 1: game/Pvs.cpp, line: 798 size: 0 KB, allocs: 1: game/SndProp.cpp, line: 363 size: 0 KB, allocs: 1: cm/CollisionModel_load.cpp, line: 682 size: 0 KB, allocs: 9: game/anim/Anim_Blend.cpp, line: 605 size: 0 KB, allocs: 2: game/IK.cpp, line: 539 size: 0 KB, allocs: 8: game/anim/Anim_Blend.cpp, line: 504 size: 0 KB, allocs: 3: game/ai/States/IdleSleepState.cpp, line: 106 size: 0 KB, allocs: 1: game/Player.cpp, line: 1887 size: 0 KB, allocs: 7: game/anim/Anim_Blend.cpp, line: 437 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 625 size: 0 KB, allocs: 8: game/ai/MovementSubsystem.cpp, line: 423 size: 0 KB, allocs: 16: game/ai/Tasks/RandomHeadturnTask.cpp, line: 138 size: 0 KB, allocs: 8: renderer/draw_arb2.cpp, line: 841 size: 0 KB, allocs: 2: framework/FileSystem.cpp, line: 2889 size: 0 KB, allocs: 2: framework/FileSystem.cpp, line: 2756 size: 0 KB, allocs: 8: c:/thedarkmod/tdm/idlib/math/Polynomial.h, line: 601 size: 0 KB, allocs: 1: game/darkModLAS.cpp, line: 1209 size: 0 KB, allocs: 1: game/Pvs.cpp, line: 793 size: 0 KB, allocs: 1: game/PVSToAASMapping.cpp, line: 115 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 6588 size: 0 KB, allocs: 3: game/ai/MovementSubsystem.cpp, line: 330 size: 0 KB, allocs: 1: game/Entity.cpp, line: 5823 size: 0 KB, allocs: 1: game/AFEntity.cpp, line: 1262 size: 0 KB, allocs: 1: game/Player.cpp, line: 5738 size: 0 KB, allocs: 1: game/Player.cpp, line: 8806 size: 0 KB, allocs: 2: framework/FileSystem.cpp, line: 2053 size: 0 KB, allocs: 16: game/Pvs.cpp, line: 809 size: 0 KB, allocs: 4: game/anim/Anim_Blend.cpp, line: 566 size: 0 KB, allocs: 8: game/ai/Memory.cpp, line: 446 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 607 size: 0 KB, allocs: 1: game/ai/Tasks/HandleDoorTask.cpp, line: 2963 size: 0 KB, allocs: 1: renderer/ModelManager.cpp, line: 206 size: 0 KB, allocs: 1: renderer/ModelManager.cpp, line: 214 size: 0 KB, allocs: 1: renderer/ModelManager.cpp, line: 220 size: 0 KB, allocs: 1: renderer/ModelManager.cpp, line: 309 size: 0 KB, allocs: 3: game/Inventory/Inventory.cpp, line: 904 size: 0 KB, allocs: 2: framework/I18N.cpp, line: 428 size: 0 KB, allocs: 3: game/anim/Anim_Blend.cpp, line: 754 size: 0 KB, allocs: 3: game/anim/Anim_Blend.cpp, line: 734 size: 0 KB, allocs: 1: ui/UserInterface.cpp, line: 210 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 1546 size: 0 KB, allocs: 1: game/Objectives/MissionData.cpp, line: 1697 size: 0 KB, allocs: 2: renderer/CinematicFFMpeg.cpp, line: 606 size: 0 KB, allocs: 2: game/anim/Anim_Blend.cpp, line: 744 size: 0 KB, allocs: 1: game/LightGem.cpp, line: 101 size: 0 KB, allocs: 3: game/ai/States/IdleSleepState.cpp, line: 143 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 623 size: 0 KB, allocs: 1: game/Entity.cpp, line: 10650 size: 0 KB, allocs: 2: game/ai/MovementSubsystem.cpp, line: 348 size: 0 KB, allocs: 1: renderer/RenderSystem_init.cpp, line: 2464 size: 0 KB, allocs: 1: renderer/RenderSystem_init.cpp, line: 2467 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 591 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 611 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 615 size: 0 KB, allocs: 3: game/EscapePointManager.cpp, line: 181 size: 0 KB, allocs: 1: game/Pvs.cpp, line: 792 size: 0 KB, allocs: 3: framework/FileSystem.cpp, line: 2079 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 589 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 619 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 1612 size: 0 KB, allocs: 1: game/anim/Anim_Blend.cpp, line: 776 size: 0 KB, allocs: 1: game/anim/Anim_Blend.cpp, line: 786 size: 0 KB, allocs: 1: game/anim/Anim_Blend.cpp, line: 525 size: 0 KB, allocs: 1: renderer/tr_main.cpp, line: 242 size: 0 KB, allocs: 2: framework/FileSystem.cpp, line: 2052 size: 0 KB, allocs: 1: sound/snd_system.cpp, line: 444 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 590 size: 0 KB, allocs: 1: ui/Window.cpp, line: 2090 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 8065 size: 0 KB, allocs: 1: game/Missions/MissionManager.cpp, line: 40 size: 0 KB, allocs: 1: renderer/CinematicFFMpeg.cpp, line: 249 size: 0 KB, allocs: 1: idlib/math/Simd.cpp, line: 43 size: 0 KB, allocs: 1: idlib/math/Simd.cpp, line: 161 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 588 size: 0 KB, allocs: 1: game/Game_local.cpp, line: 631 338574 total memory blocks allocated 486355 KB memory allocated

Uninitialized members

Leaving class members of primitive type uninitialized can cause annoying and non-reproducible bugs. That's why Visual C++ fills allocated memory with magic values like 0xCD or 0xCC in debug build, to increase the chances of program crashing due to such an error. For some reason, ID devs decided to implement their own version of this system for game objects.

In the game module of Doom 3, most of the classes inherit from idClass base, which allows to create object instance by type name and supports events for scripting system. All objects in this hierarchy are manually filled with 0xCD because idClass overloads operator new and operator delete:

void * idClass::operator new( size_t s ) {
    int *p;

    s += sizeof( int );
    p = (int *)Mem_Alloc( s );
    *p = s;
    memused += s;
    numobjects++;

#ifdef ID_DEBUG_UNINITIALIZED_MEMORY
    unsigned int *ptr = (unsigned int *)p;
    int size = s;
    assert( ( size & 3 ) == 0 );
    size >>= 2;
    for ( int i = 1; i < size; i++ ) {
        ptr[i] = 0xcdcdcdcd;
    }
#endif

    return p + 1;
}

Now the remaining question is: where are the messages about uninitialized class members coming from?

Quick search by the text of message gives the following bits of code:

#define CLASS_DECLARATION( nameofsuperclass, nameofclass ) \
    //...some more ...                                     \
    idClass *nameofclass::CreateInstance( void ) {         \
        try {                                              \
            nameofclass *ptr = new nameofclass;            \
            ptr->FindUninitializedMemory();                \
            return ptr;                                    \
        }                                                  \
        catch( idAllocError & ) {                          \
            return NULL;                                   \
        }                                                  \
    }                                                      \

void idClass::FindUninitializedMemory( void ) {
#ifdef ID_DEBUG_UNINITIALIZED_MEMORY
    unsigned int *ptr = ( ( unsigned int * )this ) - 1;
    int size = *ptr;
    assert( ( size & 3 ) == 0 );
    size >>= 2;
    for ( int i = 0; i < size; i++ ) {
        if ( ptr[i] == 0xcdcdcdcd ) {
            const char *varName = GetTypeVariableName( GetClassname(), i << 2 );
            gameLocal.Warning(
                "type '%s' has uninitialized variable %s (offset %d)",
                GetClassname(), varName, i << 2
            );
        }
    }
#endif
}

The method CreateInstance (wrapped into macro CLASS_DECLARATION) allows creating game objects by type, and it is always called when a game object is created. As you see, it calls FindUninitializedMemory method after creation, which checks which words still have values 0xCDCDCDCD, and reports them. The only thing remaining is the GetTypeVariableName function, and this is where the truly dark magic is.

TypeInfo

In order to obtain name of the class member from its offset, information about all the member offsets must be embedded into the code. Currently, C++ compilers do not do that (although there are proposals for something like that). Standard C++ RTTI only stores class name and inheritance hierarchy for polymorphic types, it does not store any info about members. There are several approaches for implementing reflection in C++. Preprocessor can generate the necessary information, given that you use macros for declaring members (example), or specialized parser can generate the necessary information (e.g. Qt MOC). Doom 3 uses the latter way: homebrew parser does that.

Recall that Doom 3 uses its own C++-like scripting language, which is parsed, compiled and interpreted by custom code (see review). The lexer and parser are reused for several different tasks. The parser also performs all the job of C preprocessor: it #includes files, checks #if-s and #ifdef-s, manages #define-s and performs macro substitution. This same parser is reused in the specialized TypeInfo compiler, whose only purpose is to generate a header with all the desired type information. It parses a single cpp file of the game module, which is supposed to include headers with all the declarations. So: ID's script parser is good enough to parse Doom 3 game sources and extract data from them! Well, it has its own issues (no pragma once support, some C++11 syntax not supported), but this is a remarkable achievement nonetheless.

After having parsed everything, TypeInfo compiler generates a header GameTypeInfo.h in game/gamesys/. This header contains the following information (you can download resulting GameTypeInfo.h for TDM):

constants: type, name, value (including enum values and static const members);
enums: list of all enums, list of name/value pairs for all enums;
class members: type, name, offset, size;
classes: full list, name, base class name, pointer to members list;

Here is an example of information for members of idStr class:

static classVariableInfo_t idStr_typeInfo[] = {
    { "int", "len", (int)(&((idStr *)0)->len), sizeof( ((idStr *)0)->len ) },
    { "char *", "data", (int)(&((idStr *)0)->data), sizeof( ((idStr *)0)->data ) },
    { "int", "alloced", (int)(&((idStr *)0)->alloced), sizeof( ((idStr *)0)->alloced ) },
    { "char[20]", "baseBuffer", (int)(&((idStr *)0)->baseBuffer), sizeof( ((idStr *)0)->baseBuffer ) },
    { NULL, 0 }
};

Clearly, it contains enough information to make it possible to get member name from offset. Note that sizes and offsets are written as C++ expressions evaluated to proper values, instead of plain integer constants. This allows to avoid messing with struct alignment rules of Visual C++ compiler. However, it also requires to do an ugly hack (called "Public Morozov" in Russia) to make sure that private and protected members are accessible in TypeInfo.cpp:

// This is real evil but allows the code
// to inspect arbitrary class variables.
#define private     public
#define protected   public

Here is the full diagram for building TypeInfo in memory debugging configuration (for the case of TDM):

Saving game state

Is it worth to implement the whole TypeInfo thing just for printing the name of an uninitialized member? I think not. But this is not the only thing which relies on TypeInfo. Another thing made possible by TypeInfo is printing the whole game state in a readable text format. Normally, all the savegame data is stored in binary format, because it is much faster to do so and the resulting file is smaller (although I added compression on top of it in TDM). But text representation may be very useful for debugging issues (e.g. forgot to save/restore some member), maybe even maintaining backwards compatibility of savegame files.

The code is located in the same TypeInfo.cpp file. It contains several methods which traverse all the game objects and do something with them. All of them rely on the methods idTypeInfoTools::WriteClass_r and idTypeInfoTools::WriteVariable_r, which implement the traversal itself. The latter method is responsible for interpreting every possible type of class member; for instance it contains special code which supports traversing all elements of idList container (which is ID's equivalent to std::vector):

int idTypeInfoTools::WriteVariable_r(
    const void *varPtr, const char *varName, const char *varType,
    const char *scope, const char *prefix, const int pointerDepth
) {
    //...(more code)...
    } else if ( token == "idList" ) {
        idList<int> *list = ((idList<int> *)varPtr);
        Write(varName, varType, scope, prefix, ".num", va( "%d", list->Num() ), NULL, 0);
        Write(varName, varType, scope, prefix, ".granularity", va( "%d", list->GetGranularity() ), NULL, 0);
        if ( list->Num() && ParseTemplateArguments( typeSrc, templateArgs ) ) {
            void *listVarPtr = list->Ptr();
            for ( i = 0; i < list->Num(); i++ ) {
                idStr listVarName = va( "%s[%d]", varName, i );
                int size = WriteVariable_r(listVarPtr, listVarName, templateArgs, scope, prefix, pointerDepth);
                if ( size == -1 )
                    break;
                listVarPtr = (void *)( ( (byte *) listVarPtr ) + size );
            }
        }

        typeSize = sizeof( idList<int> );
    } else if ( token == "idStaticList" ) {
    //...(more code)...
}

During traversal, method Write is called, as you can see on lines 5 and 8 of the last listing. This is actually not a method, but a function pointer, which is initialized differently depending on the current needs. This method accepts seven parameters:

varType | prefix | scope | varName | postfix | value | varSize
====================================================================================================
"float" | "" | "idEntity" | "m_MinLODBias" | "" | "0" | 4
"idBounds" | "idEntity::renderEntity." | "renderEntity_t" | "bounds" | "" | "(-6.75103283 -2.63098955 -20)-(7 2.5482502 49.51835632)" | 24
"idDict" | "" | "idEntity" | "spawnArgs" | "[7]" | "'noclipmodel'  '0'" | 0
"idEntity::entityFlags_s" | "" | "idEntity" | "fl" | ".invisible" | "false" | 0
"idList < idEntityPtr < idActor > >" | "idEntity::m_userManager." | "UserManager" | "m_users" | ".num" | "0" | 0

In this text, first line specifies order of seven parameters, and each of the next lines shows one example of their values, seven values per line, splitted with vertical dash. The first example corresponds to member idEntity::m_MinLODBias of float type, having zero value. The second one corresponds to member renderEntity_s::bounds of type idBounds (coordinates of two vertices are given), where renderEntity_s itself is a member idEntity::renderEntity. The third one shows 7-th entry of idEntity::spawnArgs dictionary (idDict) as 'key'-'value' pair. The forth example is a boolean value entityFlags_s::invisible in bitset structure idEntity::fl. The last one corresponds to number of elements in UserManager::m_Users, which has type idList (aggregated in idEntity::m_userManager).

Now let's return back to Doom 3 and game state saving. Whenever you save game in memory debugging configuration, a file named like "Quicksave_1_save.gameState.txt" is created near the ordinary savegame file. This is a text file of size about 40MB, where each member of each game entity is fully described. The full version of this file can be downloaded, and here is a small excerpt from it:

entity 36 idStaticEntity {
idEntity::entityNumber = "36"
idEntity::entityDefNumber = "28"
idEntity::spawnNode = "<unknown type 'idLinkList < idEntity >'>"
idEntity::activeNode = "<unknown type 'idLinkList < idEntity >'>"
idEntity::snapshotNode = "<unknown type 'idLinkList < idEntity >'>"
idEntity::snapshotSequence = "-1"
idEntity::snapshotBits = "0"
idEntity::name = "func_static_5"
idEntity::spawnArgs[0] = "'classname'  'func_static'"
idEntity::spawnArgs[1] = "'name'  'func_static_5'"
idEntity::spawnArgs[2] = "'model'  'models/darkmod/furniture/seating/chair_simple01.lwo'"
idEntity::spawnArgs[3] = "'origin'  '216.743 -47.89 20'"
idEntity::spawnArgs[4] = "'rotation'  '0.707107 -0.707107 0 0.707107 0.707107 0 0 0 1'"
idEntity::spawnArgs[5] = "'inline'  '0'"
idEntity::spawnArgs[6] = "'spawnclass'  'idStaticEntity'"
idEntity::spawnArgs[7] = "'solid'  '1'"
idEntity::spawnArgs[8] = "'noclipmodel'  '0'"
idEntity::scriptObject.idScriptObject::type = "<pointer type 'idTypeDef *' not listed>"
idEntity::scriptObject.idScriptObject::data = "<NULL>"
idEntity::thinkFlags = "0"
idEntity::dormantStart = "-2920"
idEntity::cinematic = "0"
idEntity::cameraTarget = "<NULL>"
idEntity::targets.num = "0"
idEntity::targets.granularity = "16"
idEntity::health = "0"
idEntity::maxHealth = "0"
idEntity::fl.notarget = "false"
idEntity::fl.noknockback = "false"

Whenever you restore game from a save in this configuration, full game state after loading is saved into file "Quicksave_1_restore.gameState.txt". Moreover, this restored state is compared field-by-field with the old one (which was written on save) using the same game state traversal code, and messages about mismatches (i.e. state diffs) are printed to the game console:

Conclusion

So all of this was about a tiny little part of Doom 3 source code: memory debugging and typeinfo processing. It is worth noting that typeinfo stuff was completely removed by ID from Doom3 BFG (it is missing in the sources). Most likely this is because of the maintenance burden it causes. You can read more about the issues caused by all this hackery in the corresponding TDM wiki article.

Speaking of TDM, now we have to understand how much value this system has. The first moves would be to fix all the issues detected currently, and to add support for some new data structures (include STL stuff). The TypeInfo system would either give some benefit or be removed, and it is not clear yet which path would be taken.