Shortcomings in tools that are used to build software products

This paper gives an overview of the problems and shortcomings in build tools that are encountered when building software products. The paper focuses on problems arising in large software projects. The status of this paper is preliminary, incomplete, ongoing.

Introduction - Motivation - Conceptual problems - Functionality limitations - Debugging limitations - Performance problems - Build data maintenance problems - Conclusions

Introduction

Building software components is still not an easy task. This fact stays true despite the enhancements done in software development environments. Conceptual and functional build problems will be discussed by analyzing the behavior of the now still commonly used Make tool.

Motivation

One of the motivations for this paper was to systematize the huge amount of problems that were encountered while integrating software build tools into software project management systems. The other was simply a strong interest in this topic.

Conceptual problems

Dynamic project data vs. build tools for static data

Make was designed to be used in a static manner. That means that the build description files (makefiles) are once written and remain rather constant throughout the software development process.

This does not correspond well to the concepts of modern object oriented software development. Certain kinds of project setups require frequent changes in build information. Samples are:

• object oriented software projects with one source file for each of the many classes (e.g. implemented in the Java programming language)
• libraries with lots of inherited classes, organized in subdirectories
• fast changing project structures (e.g. while prototyping)

Dependency specifications

Make lacks a comprehensive support of dependency notation.

Dependencies are either specified by

• exact file name
• file name suffix

Using suffix rules leads to the possible problem of matching an inappropriate source file. Pseudo targets need additional directives (.PHONY) or will pose a similar problem. Sample:

	>clean; make clean

Workspace support

Gmake offers the VPATH construct to support a workspace setup. However, the vpath is not very versatile:

• There is no distinction between source and object workspaces.
• There is no distinction between file types.
• Files can only be generated into the current directory.

Workspace support based on Make's vpath requires all tools (compiler, resource compilers, linkers, etc.) to offer options to specify input and output directories.

If input files are read implicitly (e.g. through C/C++'s #include), options to specify the search path for these are required as well. Although this is supported by C/C++ compilers as a standard (for other reasons than workspace support), the implicitly used current directory can't be excluded from search by some compilers (i.e. not all C/C++ compilers support gcc's -I- option).

It's a far better approach to use a workspace concept implementation outside Make. Workspace support based on a virtual filesystem, implemented by a configuration management tool is certainly the more powerful and faster solution.

Using Make as back end vs. not using Make

Creating makefiles dynamically from a development environment seems to be a paradox approach since the logic that must be already present to generate the according makefile can be used to do the build directly. Makefiles are bad interfaces to build systems.

Functionality limitations

Minimal control over the different levels of evaluation

Make doesn't offer much control over the evaluation of quotes, backslashes and the like. Quoting is especially a problem on (non-Unix) operating systems that use backslashes in path names.

It is tremendously difficult to preserve backslashes in paths; however, the operating system may also support the canonical slash path (these may pose problems to the compilers and other tools though).

Limited handling of blanks

Leading and trailing blanks in macros can't be preserved. They may be important when substitution patterns are applied on macros.

Blanks are also used as file name delimiters by Make; therefore handling paths with blanks in it is a difficult task.

No explicit source dependency with suffix rule

A target can't have an explicit source dependency when a suffix rule is used to build that target. The file space is searched for a (in Make's view) appropriate source file instead.

Shell script vs. other scripting languages

Shell scripting is powerful but yet does miss some programming concepts:

• strong typing for (immediate) error detection
• debugging facilities

Scripting language substitutes might be:

• Tcl
• Python

Make offers only very limited means of debugging. The fact that Make is rule based, not procedure based, makes it even more difficult to locate errors.

Make

• doesn't allow to set break points
• can't diagnose inconsistencies in the rules
• doesn't issue very usable verbose output

There are some particularities very bad suited for debugging:

• possible recursive build tool invocations (that are necessary in Make to achieve certain functionality)
• obscure features like the .SUFFIX directive required in addition to the according suffix rules (not only to set the precedence of the suffix rules)
• missing control over default rules

Make shows certain types of performance deficiencies. Due to the many build steps required to build large software products, these deficiencies cumulate and may become very severe.

Shell scripts used

Make uses the shell to execute more complex actions (e.g. with pipes in them, with redirection, etc.).

Shell scripts

• were not especially designed for performance
• are especially slow on (non-Unix) systems with expensive process creation

Interprocess communication (mostly in the form of pipes), that is deployed extensively in shell scripts

• is expensive on (non-Unix) systems that have slow task switches or poor process scheduling
• require more (expensive) processes

Recursive Make invocations

• Recursive build runs usually invoke Make at each directory level. This leads to a lot of processes which will be a performance problem.
• Build data that is generated on the fly (by the build system itself) requires Make to invoke itself recursively. This will also create additional processes.

Macro evaluation

Make macros are evaluated each time they are referenced. This is expensive if shell script fragments are used, even more expensive if also pipes are involved. Sample:

	IPATH = `echo $(VPATH) | sed -e 's/:/ -I/'`

No plugings deployable

Make requires all used software components to be applications.

E.g. invocations of the often used revision control tools will create additional processes. Revision control tools linked as library would bear much less overhead.

Use of application wrappers

Application wrappers that are used to establish common interfaces may bring performance problems since they require additional processes to be launched. The problem is worse if the wrappers are implemented as shell scripts.

Extended Make scripting capabilities would make such wrappers unnecessary.

Vpath searches

The file searches that are necessary to support the vpath workspace concept (in gmake) are expensive. Workspace implemented by a configuration management tool will be the faster solution.

Build data maintenance problems

Maintenance of build information may be one of the central points in software engineering. It usually consumes a large amount of working time and errors are often discovered late (e.g. missing dependency information, wrong build flags, etc.).

Make doesn't offer much help in controlling the build environment or diagnosing possible build problems. Logs must be created manually by redirection and scanned for possible anomalies. Checking build run output is even worse when parallel compiles are used (e.g. with gmake -j).

Consistency

Build information must be kept consistent. Changes in the project directory setup must be reflected in the build information. Dependencies must be correct.

Manual makefile maintenance leaves always the possibility of inconsistent build information. Missing dependencies may lead to mismatched linking modules with their according hard-to-track runtime errors. Typically,

	make clean; make

Spread build information

Single project setup changes often require changes in many parts of a makefile. Build information can't be capsulated and abstracted. This does not enhance the ease of use of Make based build systems; the robustness of Make based build systems remains poor.

Makefiles don't reflect much of a design; build information is scattered all over.

Workspace support

Workspace settings in the form of vpaths are cumbersome to specify and maintain.

Conclusions

• Make is not well suited as build back end in complex project structures.
• Makefiles are not well suited as interface between a development environment and the build tool.
• The built-in workspace support (gmake's VPATH) is poor.
• Make based build systems may show severe performance problems.
• Builds using Make are especially slow on some non-Unix systems.
• Maintaining and debugging Make based build solutions is very expensive.

Keywords: buildtool, buildtools, make, maketool, maketools