C++ Test Framework

I had a couple of questions about my test framework, so I thought I'd come clean and show just how quickly you can get off the ground with a couple of macros in C++

With all of the code in Pioneer, my goal has been to do the minimum required to get the job done and my test framework is no exception. I rolled my own test framework as an exercise in understanding what a test framework does, is and should do. Needlessly reinventing the wheel is core to the development philosophy of pioneer and a great way to learn about systems.

I decided that I want to be able to tag tests at the end of the source file for the class they test, and that the syntax should be simple. I also wanted them to be easy to compile out so I knew macros were likely to be involved.

/////////////////////////////////
#include <unittest.h>
START_TEST( ExampleTest )
... test code ...
END_TEST


My minimum test for a class is a smoke test that instantiates an object and confirms that its non null. This is a sanity test for the constructor and checks for crashes, assertions, exceptions, errors etc.  I prefer to only use tested dependencies - test friendlys - and any other test on the object would require an instantiated object

/////////////////////////////////
#include <unittest.h>
START_TEST( World_SmokeTest )

{
WorldPtr world( new World() );
CheckNonNULL( world.get() );
}
CheckTrue( true );

END_TEST

This smoke test makes two checks, the first is that a world is instantiated without error and the second is that the object can be destroyed without error. I've used a scoped pointer, so this test just needs to report two successful checks and I'm happy. The default behaviour for my test log is to report the number of passed tests, the number of successful checks and a verbose list of all of the failures.

Within each test I can call on any of these Checks, which have so far been enough.
CheckNonNULL( a ) 
CheckNULL( a )
CheckEqual( a, b )
CheckNotEqual( a, b )
CheckTrue( a )
CheckFalse( a )

The START_TEST macro declares a class with the name of the test. It also registers it with a static instance of the test suite.  The test suite is the only global static object I use, and is compiled out in production code. Its the worst way I could configure the test runner except all of the other options.

With my START_TEST macro, the test case is registered and becomes part of the global test suite, which makes adding new tests super easy. There is no excuse for not adding a test after writing (most) public methods and its easy to write the test before the method too.

However, a few improvements could still be made. As the START_TEST macro declares a class with the test name, all test names have to be globally unique. This hasn't been a problem because I prefix the test name with the name of the class being tested but its a weakness in the system nonetheless.

I'd like to be able to declare tests per class. For example the test declaration might be:

START_TEST( World, SmokeTest )
... test code ...
END_TEST

So now I could report the classname of the system under test in the test report along with the test name, and potentially use it for test-coverage metrics as I could count the number of tested classes, tests per class and potentially add instrumentation for untested classes so I had a good idea of coverage.
I have an idea that a test-energy graph could be overlaid on a class graph so I could visualise test coverage as a heat map.

Secondly I might want a test suite to contain subsets of classes. Input/GUI tests, gameplay tests, network code tests, etc... This could be grouped with an ADD_TEST_TO_SUITE macro or with an extra parameter in START_TEST for the suite name.  I don't have a reason to run a subset of tests though - I really like having them all run every time and they are so fast that there is no cost to doing this. As soon as I fall into the practice of running a subset I might suffer from slow-to-execute test, or from breaking a test that I'm not running.
I've not got a good reason to add test suites, yet, but I've got a niggling feeling its a good idea.

Mesh Smooooothing

Mesh Smoothing in general terms is pretty straightforward, but automating is a little harder.

I'm trying to automate my unify rather than manually specifying for each mesh. The compromise I've come up with is to record the bounds of the mesh, and compare to the bounds of the unified normal tips. Whichever normal orientation has the larger bounds for its tips must be the mesh that is the right way around.

My Mathematical solution was to determine if the normals are divergent or convergent, which kinda works for platonic solids but probably not for any of my meshes. I'd love to investigate this further as it feels like fun, but remains the wrong solution to the problem so I'll be leaving that stone unturned for a long time.

Checking the bounds should work for non enclosed meshes, which means I can split my airtank into several submeshes and resolve them individually with reasonable confidence they will all be the same way around when I recombine them.

I've also written some code I really don't like for this.  My CalculateFaceNormals(...) method takes one parameter - that it operates on via side effects and has no return value.  Likewise for my UnifyNormals method. They both take a collection of triangles. Now that its in blog form, its really apparent that they should be methods of the MeshResource class.

In short I think I really prefer
myMesh->CalculateFaceNormals();
to
CalculateFaceNormals( triangleCollection );

So in the spirit of being a Good Boy Scout I think I'll move those over before I leave the mesh code alone.  I've found it reasonable easy to read, even though I've not touched mesh code forever, and the new importer dropped straight in with no fuss.

As a consumer of the Mesh code, the class you deal with is a MeshFactory.  I use a MeshFactory to make MeshInstances for me, and put them in a MeshRenderComponent that I can attach to the scene graph.  

The MeshFactory gets MeshResources from a MeshDatabase.

All of the new import and postprocessing works behind the scenes of even the MeshDatabase. As long as the importer derives from MeshImporter which returns a MeshResource then I can keep developing new loaders and mesh processing and remain seamless to the rest of the application.

By moving my CalculateNormals() UnifyNormals() and AutoSmoothNormals() code to the MeshResource, I can apply it to MeshResources loaded from any file or even procedurally generated meshes. I'm temped to make them mesh processing components of the MeshResource because they don't apply to *every* mesh but for now I think adding them directly to the MeshResource is fine.

I'd *kind* of like to be able to apply them to MeshInstances so I could include them in a mesh view tool, and toggle each on and off but really that''s not relevant to the task so it can wait.

Gasping for air

So my LDR mesh importer is far from complete, but its a good start. If I want to build meshes at runtime suitable for GPU skinning there is still more work to do. I've broken the back of the mesh pipeline but do have a couple of bugs to ponder over before pushing too far ahead.

The LDR format isn't really designed with run-time performance and games in mind. They models are generally high detail - in many cases higher detail than I need but not without flaws in the geometry. Model 3838, the classic space Air Tank, is a good example of a well detailed model that diverges from games requirements - Its hollow. Being hollow is great for injection molding but not so great for realtime 3D. I could cap the ends and save the internal geometry. Actually, a lot of LDR models are brick-accurate in ways like this that are irrelevant to me.
Secondly, the AirTank is clearly several submeshes, and I'd rather have one mesh per element.

The LDR format doesn't specify a winding order - worse it specifies that winding order is irrelevant. However, I'm using counter-clockwise triangles so have had to write a Unify Normals mesh post-processor which isn't too much work.

Couple this with the knowledge that the meshes are not contiguous, which means you can't walk the geometry and unify normals. This is something of a shame, and demonstrated here.
The AirTank element is recognizably made from three or four non-contiguous sub-meshes.
The crossbar at the top, the mounting harness and the two cylinder tanks.

The closeup demonstrates that the mounting harness contains a mixture of CW and CCW triangles by visualizing the vertex normals as red lines.

The lovely attention to detail is otherwise well appreciated.  Its just that in this instance it gets in the way of progress.

My solution is to reduce the model into its submesh components and unify the surface normals per submesh and then concatenate into the original mesh before saving out as a serialized bytestream.
My quick-fix, since silicon is free and infinite, is to disable back face culling and ignore the winding order. The increase in rendering is insignificant in the small scenes that I'm using.

By not occluding back faces, I can move straight on to building a skeleton, rig and skin out of my minifig mesh parts.

Spaced Out

After taking a bunch of time off I thought I'd write some mesh tools.
This takes a vertex list, generates face normals and then uses the face normals to generate vertex normals.

In short, its a mesh-auto-smoothinator which will be a keystone in my content pipeline. The example mesh shows a side-by-side of auto-normals and then a smoothinated mesh with smooth areas and hard edges maintained.
(Also, this is a super-secret clue on what the next feature is.)

The LDR mesh importer loads LDR primitive objects. It throws away the line type data although I'm seriously considering processing this at a later date, since I'd love to have good wireframes.
The Mesh Import was reasonably quick at one 200 line class. Although there is an obvious extraction refactor to split the code up I found it was manageable enough at its size so have let it be.

The Mesh Post Processing was aroud 500 lines of code, and rammed into a single class with about four or five responsibilities.

• Quantize the mesh so that similar vertices are identical
• Build a tringle list from the vertex data
• Unify normals by rewinding data
• Calculate face normals
• Autosmooth to generate vertex normals

The system relies on a dozen or so unit tests, although coverage isn't exhaustive.

Under Construction

After taking a couple of months out to play Pioneer, to work out what to do next - then a couple of months to figure out how to do it... I've finally moved back into development. As usual Life Got In The Way, which it has a habit of doing and overall I've taken close to a years break between sprints.

I've taken a little time out to rework the renderer a bit, I've gone for soft shadows which really helps ground the scene and make it look a lot more solid. The look and feel here is key, and while I'm a long way off I think shadows were a nice distraction to have worked on. There are a few other graphics and rendering tasks on the list, but they might be backshelved for a while if I move into serious development.

Viewing the work of LEGO artists and communities on-line has really helped define the first goal of Pioneer Development - Sharing models and scenes. While there are a number of pre-rendered LEGO images that artists have made, I've not seen much real-time LEGO building, and I think the Pioneer environment would be great to help people share their MOCs if they don't have access to CG rendering or CAD.

Pioneer should be about sharing images, models, scenes or worlds. Its about Playing Well, and Playing Together. I want to be able to share what I've built and see what other people are building. This is about social building, and brickmanship.

I also want to be able to view historical sets, particularly the out of production iconic sets like Classic Space that helped build the LEGO brand over all these years. This is a celebration of everything they have done, and my homage to them. I'm not sure how this is going to play out, it might be I make a gallery of images, or of models to view. I'm avoiding all modern sets, all recent copyrights, brands and licencee material and anything that overlaps the LEGO company, their business model, their on-line presence and their console games. 

Lastly, I've got my eye on building a world that Minifigs can explore. A dynamic, changing world made of colourful bricks. This is a log term concept for the project, and goes above and beyond anything I've got planned.  But I'm aware that building a castle, town or lunar-base for minifigs to inhabit - or for you to walk around among them - would be a lot of fun. I pioneered networked multi-player during early development and think there is some strength to the idea of collaborative play.

So, that's where I am, and what I've been thinking about.  If you were part of the Pioneer Alpha program, thanks for playing. There will be a beta soon - I'm putting together an improved building UI based on feedback and incorporating a couple of other features that were requested. 

Keep it square, and think inside the box!

New Code

A dozen classes later and I've injected a mock into my rendering thread so I can interrupt draw calls and simulate almost the entire process. This basically makes everything up to the draw call unit testable and turns my code into the most complicated of bagatelle machines.


Network packet arrives in one end, bounces from pillar to post, draw call comes out the other end!


This brings me a little close to Alpha, but more in a holiday-spirit-playing-with-code sense than the typical slaving over a hot keyboard.  As I don't get paid for developing Pioneer I've been putting it aside for my birthday and then Xmas and have only snatched a couple of half-days this month.


In general the classes are easier to work with, and places the last stepping stone before adding variable level of detail rendering modes and dynamic shadows.  Its also a little faster/smoother when playing and has fixed a really obscure visibility bug that would sometimes leave you with an invisible brick.


The large benefits have been the removal of a cyclic dependency which has meant I can straighten the code out into a directed acyclic graph. All elementary stuff but removing this particular spanner from the works has been kind of a personal quest. The new object graph (and a couple of mocks) has meant I can easily increase the test coverage, which is the next thing I'm going to fiddle with because I was a bit lazy on that front in the last couple of sprints. I'm probably going to see what I can do about cutting those mock objects out of the test harness too, since they add a potential maintenance cost.

Code Mines

This is another no update blogpost, but a real update is getting closer.  A previous code review left me pondering... What if I just did it right?
So I thought I'd practice what I preach and do all of those fancy things that smart programmers dao.

The Pioneer codebase has always been dominated by the Brick class which describes the size, shape, position and colour of a building brick, and the Chunk class which measures 16x16 studs, contains zero to many Bricks and is arranged spatially with other Chunks to form the World.  Almost everything was either in one of those classes, or operated on them.

So the code was largely a bit of a lump. The Brick and Chunk classes became tightly coupled, with the World becoming a resource locator for chunks. A simple shared pointer made it far too easy to pass ChunkPtrs around and hold on to them even though - theoretically - they were short lifespan objects.

While I stuck to reasonable guidelines and practices, it became easy to add lazy features and it wasn't until I wanted to add mesh LoD and runtime LoD switching and discovered that it was a bit tricky. The possible solutions
- expose more members as public, either through access specifiers or get accessors.
- bundle more responsibility onto the already bloated Chunk class
- refactor so that the object graph makes the new feature easy to add.

Both of the least-effort solutions felt pretty bad, but it looked like it should be fairly easy to extract a responsibility FROM the chunk and isolate the rendering in a new object.  Even if it was a RenderableChunk class, it could still steal all of the occlusion, rendering and become a container for that stuff where it should be easy to add that mesh LOD functionality too.

And so the project went through whatever the opposite of suffering is as I pruned back the almighty Chunk. It lost a lot of fat, it also became a Directed Acyclic Graph and I could strip out a now redundant cycle.

Compile times were a little slow so I retypedeffed my shared pointers to incomplete types and made all of their clients rely on the abstract interface instead of the implementation.  This reduced the amount of include files getting chained together and allowed me to hack away in small, fast iterations.

The new acyclic object graph made the test suite simpler and much easier to expand to be more comprehensive. I was never satisfied with my test coverage and having that easy to improve is a load off. Its almost as if the "better" code was also easier to unit test. Astounding!

Programming can be a bit like a new LEGO build. You know what its supposed to be when its finished but you haven't got any instructions so you add and remove bricks, and each iteration will get you incrementally closer to your goal. One brick removed, two bricks added.

There will need to be more time at the keyboard before anything appears, but the last of the big changes has happened. From here it looks like I'll just be putting bricks back together. Maybe the odd one or two will be shifter, but I've solved the problem I wanted to solve and a few more to boot.