| | 1 | = Object Model of My Scons-based Build System = |
| | 2 | Jan 20 2015 |
| | 3 | |
| | 4 | [[Image(htdocs:images/scons/SCons.png, align=center)]] |
| | 5 | |
| | 6 | We used [http://www.scons.org SCons] as our build system at Axham Games. SCons is implemented as a Python script and a set of modules. Configuration files in SCons are also Python scripts which are executed; this makes for a very potent build system for you have all of the power of Python, its modules and extensions available for use in your build system. |
| | 7 | |
| | 8 | In this post, I'll focus on the object model for our build system which essentially treats each group of libraries as a component. If warranted and there's sufficient interest, I'll get into details of the entire system in a future follow up post. |
| | 9 | |
| | 10 | == Primary Objectives == |
| | 11 | 1. modularity and ease of extension; primarily be able to add libraries with minimum addition to the build system |
| | 12 | 1. ability to build on multiple platforms without requiring additional steps |
| | 13 | 1. flexibility to be able to swap library versions with minimal alterations |
| | 14 | 1. override build parameters at the command line |
| | 15 | |
| | 16 | == Object Model == |
| | 17 | Each set of libraries in the build system is treated as a single component. One of the major issues I ran into early on when compiling for multiple platforms and architectures, particularly when using different compiler toolchains, was that they all followed vastly '''different and inconsistent naming conventions'''. I found this to be true even for the same set of libraries built using the same compiler toolchain where only the platform differed. The most common example is the prefixing of "lib" on POSIX systems. On Windows, some libraries get built out on POSIX emulation toolchains (like mingw-gcc) with library naming conventions similar to POSIX systems, while some libraries do not. |
| | 18 | |
| | 19 | All sets of libraries used typically have a fixed set of properties viz. include path, library path and libraries to statically or dynamically link to. So, in essence we can treat each set of libraries like boost or Ogre3d as a single component for the purposes of our build system. |
| | 20 | |
| | 21 | == Boost as an Example == |
| | 22 | Let us take boost as an example for the purposes of explaining the usage of the object model. Boost, has a rather convoluted naming convention (with good reason, probably) for its libraries, wherein the boost version is embedded in the middle of the library name with a mixture of underscores and hyphens. e.g. `libboost_thread-mgw48-mt-d-1_55.dll.a` |
| | 23 | |
| | 24 | Let's break this example name down into its constituent parts: |
| | 25 | |
| | 26 | [[Image(htdocs:images/scons/boost-lib-name-example.png, align=center)]] |
| | 27 | |
| | 28 | 1. '''lib''' - this indicates that it's a library of some sort, could be static, dynamic or a stub library you link to use a dynamic library. |
| | 29 | 1. '''boost''' - name of the library "group" it's a part of |
| | 30 | 1. '''thread''' - the actual library, in this case it's a thread library |
| | 31 | 1. '''mgw48''' - compiler toolchain and version, in this case, mingw gcc 4.8 |
| | 32 | 1. '''mt''' - multi-threaded |
| | 33 | 1. '''d''' - debug |
| | 34 | 1. '''1_55''' - boost version |
| | 35 | 1. '''dll.a''' - stub library to link to use the dynamic library |
| | 36 | This is the name of the stub multi-threaded debug thread library from boost ver 1.55 built using the |
| | 37 | mingw gcc toolchain ver 4.8. On Visual C++ 2013, this same library would be named |
| | 38 | `libboost_thread-vc120-mt-gd-1_55.dll.a`. |
| | 39 | |
| | 40 | The compiler toolchain used is project specific as is the version of boost. As long as the libraries used by the project are available, the developer will be able to use the libraries by minimally specifying the versions for the toolchains used in the project. |
| | 41 | |
| | 42 | The onerous task of building the specific library names per platform architecture and toolchain which are specific to the set of libraries is encapsulated within a simple Python class modeled as a component. All components have standard properties to provide include path(s), library path(s) and libraries to link to. These properties are assembled in the constructor of the component based on environment variables which can be overridden by the build environment and further by values specified as parameters on the command line when the build is fired off. |
| | 43 | |
| | 44 | So, in the SConstruct file of the project (which may have multiple sub-projects), the developer specifies the following: |
| | 45 | {{{#!python |
| | 46 | # externals_dir is the base directory for 3rd party libs |
| | 47 | # plat_arch is the platform-architecture |
| | 48 | # externals_variant is the build variant e.g. debug |
| | 49 | env['COMPONENT_NAMES'] = set(['boost']) |
| | 50 | env['BOOST_DIR'] = util.join_path( |
| | 51 | externals_dir, plat_arch, 'boost') |
| | 52 | env['BOOST_VARIANT'] = externals_variant |
| | 53 | env['BOOST_VERSION'] = '1-55' |
| | 54 | env['BOOST_INCLUDE_DIR_EPILOG'] = 'include' |
| | 55 | |
| | 56 | env['BOOST_LIBRARY_EPILOGS'] = { |
| | 57 | 'win.mingw.x86' : {'debug' : '-mgw48-mt-d-1_55', |
| | 58 | 'release' : '-mgw48-mt-1_55'}, |
| | 59 | 'win.msvc.x86' : {'debug' : '-vc120-mt-gd-1_55', |
| | 60 | 'release' : '-vc120-mt-1_55'}} |
| | 61 | env['BOOST_LIBRARIES'] = { |
| | 62 | 'win.mingw.x86' : set(['boost_system', |
| | 63 | 'boost_thread', |
| | 64 | 'boost_program_options']), |
| | 65 | #Since boost::thread depends on boost_chrono in msvc alone. |
| | 66 | 'win.msvc.x86' : set(['boost_system', |
| | 67 | 'boost_thread', |
| | 68 | 'boost_chrono', |
| | 69 | 'boost_program_options'])} |
| | 70 | }}} |
| | 71 | There are a few basic assumptions that the components of the object model makes which are documented rather than enforced via codifying rules: |
| | 72 | 1. the base directory for the component is specified as `<COMPONENT_NAME>_DIR` |
| | 73 | 1. libraries used by the component are specified in a nested dictionary with the platform-architecture as the outer dictionary key and the build-variant as the inner dictionary key |
| | 74 | 1. components may require additional parameters to be specified, which are documented |
| | 75 | The main reason for not enforcing the rules via code checks within the build system is that we are a small team of at most four engineers and the work of codification of rules has not been seen as required. If the team grows larger (most likely indicated by the need to open a build engineer position), it may make sense codify the rules. Any errors that we run into due to missing or incorrectly specified parameters has caused the build to fail, and fail fast. Over the years, I've become a strong believer in YAGNI (you aren't going to need it) and that's one of the foundational principles of any software I write these days, including but not limited to build systems. |
| | 76 | |
| | 77 | At this point, the developer is ready to start using and linking against boost libraries for all sub-projects, which can choose to override these values. |
