Test and Build

The common problem faced by developers of C++ libraries is that, in spite of having a detailed standard supported by a very dedicated group of volunteers, C++ programs cannot be guarenteed to be portable. This is due to a couple of factors

  • There exist many C++ constructs which are legal syntax but whose semantics are undefined. Usually these aren't even flagged as warnings by the compiler. It's very easy to inadvertently and unknowingly write code which depends on this undefined behavior. Its very common for code which passes test on the developer's environment to fail when moved to another. Many developers test on multiple platforms for this reason. This helps, but it means that code only get's tested on the most commonly available platforms.

  • All code depends upon other code libraries. Given the size of these libraries - e.g. the C++ standard library - it's inevitable that there will be subtle variations in implementation and interface of these other libraries.

  • All compilers have bugs.

  • All compilers omit some feature of the latest C++ standard. Older but still widely used compilers omit many features of the latest standard. It's not at all easy to keep track of which compilers/libraries support which features.

One of the key features of Boost libraries is that they are extensively tested. This is done by volunteers who test all the libraries on particular platforms. From our perspective, this has a number of drawbacks:

  • It is very demanding on the volunteers. Usually takes all night to run tests on all of boost.

  • The environments being tested might or might not be the same as those platforms that users actually use.

  • It's not obvious how to test just one library.

  • It only includes libraries already accepted into boost. So it is of no help to target audience of this web site.

To address these considerations we will require that every library submission include the following:

  • A reasonably complete set of tests which can be run by anyone who downloads the library.

  • A test script for running these tests.

  • Logging and display of test results from other library users.

This will permit any users of a submitted library to run his own tests. It will also permit prospective library users to get some idea as to how the library might work in his particular environment before he even downloads it.

Elsewhere on this website, we'll describe the test system we recommend.

There are 0 comments and replies

Comment on This Page