This article is part 3 of the Natvis-like experience with GDB pretty printers series:
One of the issues I have with pretty printers is that they look much more verbose than natvis: every printer has to be a separate class with 3 methods. And it gives a lot of room for error even in trivial cases. So I implemented something similar to natvis: a concise and partially declarative approach for creating pretty printers. Fully declarative way is not a good idea because it would be a new isolated world. I'd like to be able to interoperate with some Python code. And I definitely don't try to emulate natvis exactly in all the fine details: that would be too hard to achieve.
For example, creation of a pretty printers for a simple 3D vector with members x, y, z looks like this:
make_simple_printer('idVec3', '({$x}, {$y}, {$z})'),
In this case we don't specify children, so the raw members are displayed on expansion. The substring {$x} is a placeholder which is replaced with the display string of the member x.
Note that in MSVC we don't even need to have a natvis definition for such a struct, because the debugger by default enumerates all the members with their values in the display string. Such a behavior is reproduced with auto-summary. However, by default GDB leaves display string empty.
The 3D matrix internally has just one member idVec3 mat[3], but we want to immediately see three 3D vectors on expansion. So we can override children using the third argument:
make_simple_printer('idMat3', '3 x 3 matrix', lambda v: [('[%d]' % i, v['mat'][i]) for i in range(3)]),
The display string is just "3 x 3 matrix" here, and the third argument is a lambda producing the list of children for a given gdb.Value.
The 6D vector stores its coordinates in an array float p[6] too, but we want to see named coordinates:
make_simple_printer('idVec6', '({@x}, {@y}, {@z}, {@u}, {@v}, {@w})', lambda v: {
'x': v['p'][0], 'y': v['p'][1], 'z': v['p'][2],
'u': v['p'][3], 'v': v['p'][4], 'w': v['p'][5],
}),
Notice that placeholders look like {@x} here: in this case the display string of the child named x is taken, as produced by the lambda in the third argument.
Also, the children are described as a dictionary here instead of as a list of key-value tuples. Both alternatives are supported: list of tuples is more native for pretty-printers, but dictionary often looks cleaner.
Here is an example of an array-like structure:
make_simple_printer('idWinding', 'winding[{$numPoints}]', lambda v: [('numPoints', This)] + array_children_list(v['p'], v['numPoints'])),
The special value This is used to say "take the same-named member from the main gdb.Value". It does not add anything new, is just a minor shortcut for simple cases.
The same result can be achieved with the dictionary syntax and the caret character ^:
make_simple_printer('idWinding', 'winding[{$numPoints}]', lambda v: {
'numPoints': This,
'^': array_children_list(v['p'], v['numPoints']),
}),
If the name of a child starts with a caret, then it has special meaning: the corresponding value is inlined into the current list of children. In this case the elements of the array are inlined to become direct children of idWinding.
Inlining is most typically used to display all raw members of the class, similar to <ExpandedItem>this,!</ExpandedItem> in natvis:
make_simple_printer('idDecl', '{@base_}', class_attribs = {'allow_derived': True}, structure = lambda v: {
'^': raw_children_inline(v),
'@base_': v['base'].dereference(),
}),
The synthetic raw child raw_child_expandable can be inserted just the same way, despite being a single tuple. Notice also that it is possible to set pretty-printer class attributes like allow_derived (recall that it enables this printer for the types derived from idWinding).
A child is hidden if its name starts with email sign @: it can be used in display string, but is not shown when the value is expanded in debugger. It makes sense for idDecl printer because raw members already include base, so @base_ exists just to dereference the pointer and get rid of its address.
In natvis one can just write {base,na} in display string to drop address. Similarly, it is possible to add asterisk to deference the value:
make_simple_printer('idDecl', '{$base:*}', class_attribs = {'allow_derived': True}),
It is not necessary to inline everything though! It is possible to have nested children inside synthetics, meaning that the third argument can be a tree in the general case:
make_simple_printer('idMaterial', '{$base:*}', structure = lambda v: {
'^': raw_children_inline(v),
'[stages]': array_children_list(v['stages'], v['numStages']),
'[interactionGroups]': array_children_list(v['interactionGroupStarts'], int(v['numInteractionGroups']) + 1),
}),
This is a very typical example where we only want to add a few synthetic children in order to see some arrays and linked lists which are behind raw pointers and are otherwise not inspectable. Both synthetics are created automatically inside the implementation.
The only issue with idMaterial definition is that contents of the synthetics are evaluated when idMaterial value is expanded, even if we never expand the synthetics themselves. Most of the time it is undesirable. To make synthetic content lazily evaluated, we have to add yet another lambda and wrap it into make_synthetic. Here is one example where traversing the linked lists all the time would be too expensive:
make_simple_printer('idRenderEntityLocal', '{$parms}', lambda v: {
'^': raw_children_inline(v),
'[World Area]': idListPrinter(v['world'].dereference()['portalAreas']).get(int(v['index'])),
'[All Interactions]': make_synthetic(lambda: linked_list_children_list(
v['firstInteraction'],
lambda n: n.dereference()['entityNext']
)),
'[All RefAreas]': make_synthetic(lambda: linked_list_children_list(
v['entityRefs'],
lambda n: n.dereference()['next'],
lambda n: idListPrinter(v['world'].dereference()['portalAreas']).get(int(n.dereference()['areaIdx']))
)),
}),
Surely it does not look very clean with three-level lambdas =)
I guess this extra lambda is not that necessary after all. As we will discover later, CLion calls children method before a value is expanded anyway, and we should exploit generators to minimize the performance burden of this.
For the sake of comparison, here is the original natvis definition for idRenderEntityLocal:
<Type Name="idRenderEntityLocal">
<DisplayString IncludeView="simple">{parms,view(simple)}</DisplayString>
<DisplayString>idRenderEntity: {parms,view(simple)}</DisplayString>
<Expand>
<ExpandedItem>this,!</ExpandedItem>
<Item Condition="parms.entityNum==0" Name="[World Area]" ExcludeView="raw">world->portalAreas[index]</Item>
<Synthetic Name="[All Interactions]" ExcludeView="raw">
<Expand>
<LinkedListItems>
<HeadPointer>firstInteraction</HeadPointer>
<NextPointer>entityNext</NextPointer>
<ValueNode>this</ValueNode>
</LinkedListItems>
</Expand>
</Synthetic>
<Synthetic Name="[All refAreas]" ExcludeView="raw">
<Expand>
<LinkedListItems>
<HeadPointer>entityRefs</HeadPointer>
<NextPointer>next</NextPointer>
<ValueNode>((idRenderWorldLocal*)gameRenderWorld)->portalAreas[areaIdx]</ValueNode>
</LinkedListItems>
</Expand>
</Synthetic>
</Expand>
</Type>
The two are not exactly equivalent of course: there is some tweaking with simple view in natvis, and [World Area] synthetic is displayed only on condition.
Implementation
I think it is clear that implementing this make_simple_printer function is totally possible with all the building blocks provided in the previous chapter. This has little to do with GDB API and pretty printers, it is mostly an exercise in general Python programming. I provide the full code just below for completeness, you can skip it if you are not interested.
This = '$%#this#%$'
# Children tree is a json-like structure like this:
# {
# 'size': 15, # member 'size' = 15
# 'parent': This, # member 'parent' = self.value['parent'] (taken from 'this' object)
# 'duration': gdb.Value(...), # member with specified gdb.Value
# '@hidden': gdb.Value(...) # hidden member: not displayed, but can be referenced in display string
# 'extra': { # synthetic member: can be expanded to see its contents
# 'angle': 179.0
# 'analysed': gdb.Value(...),
# },
# '^dsfsd': raw_child_expandable(value), # add expandable synthetic with all raw members (key name ignored)
# '^oiwer': raw_children_inline(value), # list all raw members here (key name ignored)
# '^ivnfd': [('a': 7), ('b'): 15] # in general case, '^???' means "insert this child/children list here"
# }
#
# Formatted display string looks similar to Python format string, for example:
# 'From {@parent} under angle {@extra.angle}: {@hidden}'
# The placeholders are surrounded with {}, and can be of two types:
# {@path} --- use value from children tree under specified path
# {$name} --- use self.value[name], i.e. take from 'this' object
# You can also add format specification after colon:
# {$ptrToObject:*} --- take ptrToObject member and dereference it before display
# {$mystruct:a} --- generate auto-summary for mystruct member (not recommended)
# given children tree as described above, or a lambda that returns it,
# returns same tree with all the values resolved as gdb.Value, 'this' no longer needed
def preprocess_children_tree(tree, thisValue):
if tree is None:
return raw_children_inline(thisValue) # default if tree not specified
if callable(tree):
structure = tree(thisValue) # typically the tree is behind lambda
def preprocess_structure_recursive(tree):
if isinstance(tree, dict):
tree = list(tree.items()) # dict literal is alternative to list of tuples
result = []
for key, val in tree:
if key.startswith('^'): # insert value = list of key/value tuples (or one)
if isinstance(val, list):
result += val
else:
result.append(val)
continue
elif isinstance(val, str) and val == This: # use member of 'this' by name
val = thisValue[key]
elif isinstance(val, (dict, list)): # synthetic subobject
val = preprocess_structure_recursive(val)
elif isinstance(val, gdb.Value): # normal value
pass
else: # primitives like int, float, string
val = gdb.Value(val)
result.append((key, val))
return result
return preprocess_structure_recursive(structure)
# tracked against total memory limit of g_gdb_helper_table table
def estimate_size_of_preprocessed_tree(tree):
res = 0
for key, val in tree:
if isinstance(val, list):
res += estimate_size_of_preprocessed_tree(val)
res += len(val.bytes)
return res
# pretty-printer-like class used for synthetic objects
# note: unlike true PP, it accepts its contents instead of gdb.Value
# so it can only be used from inside convert_preprocessed_tree_into_children_list
class ContainerPrinter:
def __init__(self, tree):
self.tree = tree
def children(self):
return convert_preprocessed_tree_into_children_list(self.tree)
# converts the result of preprocess_children_tree to be returned from 'children' method of pretty printer
def convert_preprocessed_tree_into_children_list(tree):
result = []
for key, val in tree:
if key.startswith('@'): # hidden
continue
if isinstance(val, list):
val = embed_printer_for_value(val, ContainerPrinter, size = estimate_size_of_preprocessed_tree(val))
result.append((key, val))
return result
# find member by name (or even path) inside the preprocessed children tree
# this is used by the feature: {@mystuff.data.first} in display string
def fetch_from_preprocessed_tree(tree, path):
assert tree is not None
parts = path.split('.')
node = tree
for p in parts:
nnode = None
for k, v in node:
if k == p or k == '@' + p:
assert nnode is None
nnode = v
node = nnode
return node
# creates string from special 'format string', gdb value of 'this' and optionally preprocessed children tree
def compose_formatted_display_string(format, value, tree):
pieces = string.Formatter().parse(format)
res = ''
for text, placeholder, specs, _ in pieces:
res += text
if placeholder:
if placeholder.startswith('@'):
member = fetch_from_preprocessed_tree(tree, placeholder[1:])
elif placeholder.startswith('$'):
member = value[placeholder[1:]]
else:
assert False
try:
if '*' in specs:
member = member.dereference()
except:
member = 'err'
if ENABLE_AUTO_SUMMARY and specs.endswith('a'):
s = AutoDisplayStringPrinter(member).to_string()
else:
s = display_string(member)
res += str(s)
return res
# Makes pretty-printer using simply and mostly declarative format.
# The return value can be registered directly in TdmPrettyPrinterCollection.
# typename: C++ class name (can be wildcard)
# format: will be shown as 'display string', placeholders like {...} are replaced
# structure: json-like structure that describes 'children', possibly with synthetic subchildren
# See large comment above for syntax and possibilities of display string format and json-like structure.
# Note: if structure is None, then you can't use {@smth} placeholders in display string.
def make_simple_printer(typename, format, structure = None, *, class_attribs = {}):
class SimplePrinter:
def __init__(self, value):
self.value = value
self.structure = preprocess_children_tree(structure, value)
def to_string(self):
if ENABLE_AUTO_SUMMARY and getattr(SimplePrinter, 'auto_summary', False):
return get_auto_summary_as_string(self.value)
return compose_formatted_display_string(format, self.value, self.structure)
def children(self):
return convert_preprocessed_tree_into_children_list(self.structure)
if typename is not None:
setattr(SimplePrinter, 'wildcard', typename)
SimplePrinter.__name__ += '(%s)' % typename
for k, v in class_attribs.items():
setattr(SimplePrinter, k, v)
return SimplePrinter