Organization of Build Environment Scripts

Jan 04 2016

Consistent organization is important when building code that uses multiple external and internal libraries. It all starts with scripts that are used to setup the build environment. The scripts need to accommodate customization on per project, per role (developer, scripter, artist, etc), per user and perhaps ever per computer basis.

I use ​mercurial for version control. There are multiple repositories, typically 1 per game or product. If a game or product is complex enough, it will warrant multiple repos. The layout of the "sub-repos" within a project is fixed and the developer needs to ensure adherence to the layout to get the project to build. The need for multiple repos within a single project stems from having teams with different needs and schedules working on various pieces. E.g. artists and animators work on assets on a different schedule and publish content to their own repo. Developers, QA and automated build & test systems can use placeholder assets initially and subsequently integrate the final assets when they are ready. This also provides the flexibility of using a different version control system for art and sound assets which is better suited to binary files e.g. ​Perforce rather than mercurial.

Coming to the actual layout, it looks something like this on Linux and MSYS2:

~vijay
    |-.bashrc [1]
    |-work
      |-repos
        |-0
        |-1
          |-private
            |-bin
              |-devenv.sh [3]
            |-developer
              |-bin
              | |-devenv.sh [4]
              |-project1
                |-project1.sh [5]
            |-artist
              |-bin
              | |-devenv.sh
              |-project1
                |-project1.sh
            |-vijay
              |-bin
                |-devenv.sh [2]

1. .bashrc [1] contains the startdev() function
2. devenv.sh [2] user specific overrides go here
3. devenv.sh [3] common to all projects, users and roles, like build / install directories
4. devenv.sh [4] role specific settings go here
5. project1.sh [5] project and sub-project (if any) specific settings go here

The way a build environment for a project is setup is to invoke a function, say startdev(), from .bashrc [1]. This function ensures that the local repository instance (1, 2, etc) and project are set. It then dot-sources the user's devenv.sh [2] script which internally dot-sources the main devenv.sh [3] script in private/bin and the role specific (in this case developer) devenv.sh [4] script. The role specific devenv.sh [4] dot-sources the project specific script, project1.sh [5]. At every stage, the sourcing script can override settings in the sourced script. This keeps the main line settings separated out from the overrides. Similarly, role and project specific settings are separated out into their own files.

There are a couple of things that are missing from this setup. The first is the lack of per machine overrides. These would be useful if, for example, a developer worked on 2 computers side-by-side or even virtual machines; one for debug builds and another for release builds or one for server code and one for client code. The second is that all projects have settings stored in sub-folders under each role; as you can see from the example above project1 has its build environment scripts split up, under artist and developer folders.

I've chosen not to address either of the issues at this time, because the per-machine overrides are easy to add when I'll actually need that feature in the future - it's only at that time I'll know the real requirements. Essentially, ​YAGNI. As for the second issue, while it would remove clutter from the per-role folders and consolidate the files under each project, I haven't felt a need to change it at this time since the number of projects is realistically rather small. Once again YAGNI applies, and I can revisit and easily rejigger the organization without too much hassle.