Care and Feeding of bld.proj
Mbld is a build system put together to avoid the inadequacies of traditional
make
, which, although it's great, does not have a way of scanning for
dependencies ahead of time, and would require additional tooling to generate
makefiles.
Mbld knows enough about typical project structures and input files to know how to build, install, uninstall, clean, and test a project. It is aware of paths within a project, and while the project looks like it is recursively structured, it can refer to files in sibling directories painlessly.
The build files are split into two different kinds: Project files, which are
named bld.proj
, and subfiles, named bld.sub
, which have identical
contents, but affect the paths referenced in the build files differently.
This will be explained in more depth later.
An Trivial Example
To start off, we will show a trivial example for mbld:
bin hello-world =
hello-world.myr
;;
This produces a binary target called 'hello-world', using the input file 'hello-world.myr'. You may notice that no libraries are listed. This is becasue 'mbld' will find installed libraries, and add them to the dependency list for the binaries. Local libraries will still need to be added manually.
To build, install, and test this binary, the following commands can be run:
mbld # builds everything
mbld install # installs
mbld test # runs tests
mbld clean # removes generated files
A More Complete Example
This example covers most of the useful features that are available in a single directory mbld project:
# This is a comment
bin demo =
main.myr
demosrc.myr
lib convenience
;;
# Attributes go in '{' '}'
lib convenience {noinst} =
convenience.myr
asmstuff+x64.s
portglue+linux-x64.myr
portglue+plan9.myr
portglue.myr
;;
test extratests =
test.myr
;;
Starting off at the top, we have the 'demo' program. This program is composed
of two source files: main.myr
and demosrc.myr
. It also pulls in the
library convenience
, or whatever your platform names it. This is
similar to the trivial example above.
Next, there is the definition for the library convenience
. This is a
library which is not installed, hence the noinst
attribute. It contains
a Myrddin source file, an assembly file for x86-64, and a bit of portability
glue. This showcases a few interesting features of the build system, which
were inspired a bit by Plan 9 and Go: System tags. Files which have an identical
stem, but end with a "+tag-list" will be filtered by the tags, and the one
most appropriate for the system will be selected.
File Layout
Mbld files, as mentioned before, come in two flavors: There is bld.proj
,
which defines a top level project, and bld.sub
, which defines a subproject.
We will call the most recent directory in the heirarchy containing a
'bld.proj' file the project root.
When mbld
is run, it will walk up the directory tree to the first bld.proj
file that it can find, and treat that as the root of the build. This means
that any references to local libraries listed in the bldfile may not escape
the project, and the absolute paths (ie, those which start with @/
) are
all relative to the project root.
Mbld will accept projects inside projects, but references to files may not escape a single project. So, for example:
bld.proj
foo/bld.sub
bar/bld.proj
bar/baz/bld.sub
The root bld.proj
may refer to any files in this heirarchy. So, for example,
it could have a target:
bin b =
main.myr
lib @/foo:foo
lib @/bar:bar
lib bar/baz:baz
;;
Similarly, foo.sub may refer to other libraries:
bin foo =
foo.myr
@/bar:bar
../bar/baz:baz
;;
However, any attempts to use members of foo/ as part of the build from within bar/ will raise an error, as you are attempting to reference a build from outside of the project.
File types
mbld supports several types of files as inputs:
- .myr
Myrddin source.
- .s
assembly source.
- .glue.c
C glue source. The
.glue.c
suffix indicates that the code follows some additional conventions over plain C- .o
Object files. These can be produced by a gen rule, for linking code that is generated by some compiler.
- .6
Object files. Just on plan 9.
Commands
mbld supports a number of commands to run builds. The comprehensive list is below:
- all
-
This is the default target, although it can be specified explicitly. It runs every command needed to generate a compiled build. It will run the gen and cmd code, but does not run tests or install the code.
- gen
This will run all 'gen' rules in the entire project. It will not run any other commands.
- clean
This will remove all files that were automatically created by mbld in the build process, including any files generated by 'gen' commands that do not have the 'durable' attribute. Generated files with the durable attribute are preserved.
- install
This copies the files of interest to their final homes. If the installed files are not yet built, it will run 'mbld all' to compile them. If the DESTDIR environment is set
- uninstall
This is the converse of install, removing all installed binaries from their final homes. It does not currently remove empty directories that were created.
- test
This command runs unit tests. By default, if the input list contain a file named `foo.myr`, and there exists a file `test/foo.myr`, then the latter will be assumed to be a unit test for the former. In addition, all of the explicit test targets will be run.
- bench
-
This command runs benchmarks. By default, if the input list contain a file named `foo.myr`, and there exists a file `bench/foo.myr`, then the latter will be assumed to be a benchmark for the former. In addition, all of the explicit benchmark targets will be run.
Currently, benchmarks must provide subtest output with bencmkark data, or they will not produce useful output
- targ/path:target
This is an explicit build target. It's equivalent to what 'all' does, only it does it for one specific target. I'm not sure it's useful.
Target Types
mbld supports a small number of top level targets. All of these targets may be tagged with any number of 'tag=' attributes to restrict them to a single system with those tags. Multiple targets of the same name may be defined with different tags, and the same selection algorithm as is applied to files will be used to determine which one to build.
bin
The bin
target represents a single binary. By default, on Unix-like systems,
this binary is installed to $prefix/bin
. For input files, it will accept
Myrddin or assembly sources. The binary name does not include any system
specific prefixes or suffixes. For example, building with a custom runtime:
bin foo {rt=custom_runtime.o} = input.myr list.s ;;
The bin target can include libraries, including libraries from siblings. It may not reach outside of the current project, though. In other words, all elements that are not installed must be part of the same project.
In addition to the common attributes, the bin
supports the attributes
enumerated below:
- ldscript script
When linking a binary, use the custom linker script `script` to link the binary.
- runtime rt
When linking a binary, use the custom runtime version `rt` instead of the default one shipped with Myrddin.
- inc dir
Append `dir` to the default package search path.
- inst
Marks that this file should be installed. For a 'bin' target, this is the default.
- noinst
Marks that this file should not be installed.
- test
Marks that this file should be run as a test.
- notest
Marks that this file should not be run as a test. This is the default for 'bin' targets.
lib
The lib
target is similar to the bin
target, accepting the exact same
inputs. By default, on Unix-like systems, this library is installed to
$prefix/lib/myr
. This prefix is chosen both to make listing and removing
Myrddin code easier, and to prevent conflicts with other system libraries.
The library name used does not include the system specific prefixes or
suffixes. For example, to produce libfoo.a
:
lib foo = input.myr list.s ;;
Lib targets accept the same set of attributes as bin targets, with the caveat that the ldscript and runtime options do nothing.
test
The test
target is an explicit test. For the most part, this target should
only be needed rarely: implicit tests should cover most needs.
It's equivalent to a bin
target, only it is not installed. Instead, it is
run from mbld as a unit test. A successful exit is interpreted as a successful
test run. For example:
test foo = testfoo.myr ;;
Test targets accept the same attributes as bin targets. In fact, they are
identical, with the exception that their attributes default to
noinst=true, test=true, inc="."
.
bench
The bench
target is an explicit benchamrk. The only two ways that this differs
from a test target is that it is run on mbld bench
, and that it is expected
to output subtests containing benchmark results.
It behaves nearly identically to a test target.
data
The data
target lists a number of blobs to install. The default install
location defaults to '$PREFIX/$SHAREPATH/blobname', unless it is
overridden with the 'path' attribute. On unix-like systems, $SHAREPATH
defaults to 'share', while on Plan 9, it defaults to 'lib'.
For example, the rule below will install the inputs to $PREFIX/share/prog-icons/{foo.png, bar.png}
data prog-icons = foo.png bar.png ;;
If these blobs should be installed to, eg, "$PREFIX/foo/bar" instead, then the following rule can be used:
data prog-icons {path=foo/bar} = foo.png bar.png ;;
The data installed may be generated with a 'gen' target.
man
The man
target is a list of man pages to install. Eventually it should
probably be deprecated, and replaced with a 'doc' target that can handle
generation from a number of sources. It takes a list of manpages, which
are named with the section that they go into. For example:
man = apiref.3 cmdref.1 ;;
gen
The gen
target generates a file or list of files from a command. It will
run the command every time gen
is run, or if the output files do not exist.
If there is a dep
attribute set, it will also rerun the gen command whenever
at least one of outputs is older than any of the inputs. The dep
attribute
is necessary for this behavior because the gen
command doesn't know anything
about the structure of the command that it runs.
A notable thing about the command is that it is not run in a shell. This is
done to maximize portability and avoid accidental environmental dependencies.
Commands are instead run using std.execvp
, and are therefore resolved using
$PATH
, or $path
on Plan 9. For example, if you had a configure script
which takes a --redo
option to rerun while preserving variables, you may run
it like this:
gen config.myr {durable,dep=configure} = ./configure --redo ;;
The following additional attributes are supported by gen targets:
- durable
Do not remove the output files when running 'mbld clean'
-
- dep
Regenerate the generated outputs if one of these files changes.
sub
The sub
target includes the subdirectories listed in the file to
System Tags
Mbld supports system specific file versions. These are selected via the tags in the file name: Given a set of files with the same stem, and 0 or more tags, then the file with the best match for the system will be selected for the build, and all others ignored.
Tags in the file name follow the first '+', and are a '-' separated group of words, which mbld will match against. So, for example, if I am building on x86-64 linux, and I have the following set of files listed in my build:
foo+osx-x64.myr
foo.myr
foo+posixy.myr
foo+linux.myr
foo+linux:2.6.1-x64.myr
foo+linux:2.6.10-x64.myr
the match will proceed as follows:
First,
foo+osx-x64.myr
will be rejected. The 'osx' tag does not match the current system.Second,
foo.myr
will be recorded as having zero matches.Third,
foo+posixy.myr
will be recorded as having one match.Fourth,
foo+linux.myr
will be recorded as having one matchFifth,
foo+linux:2.6.1-x64.myr
will be recorded as having two matches.Fifth,
foo+linux:2.6.10-x64.myr
will be recorded as having two matches.
Since there are two files with 2 matches, there is an ambiguity, and the tie is broken by the file which has the larger version number. foo+linux-2.6.10 is the winner, and foo+linux:2.6.10-x64.myr is selected for the build.
If the build did not list foo+linux-x64.myr, then there would be an ambiguity, and the build would fail, as both foo+posixy.myr and foo+linux.myr have one match. There is no weighting for tags.
And finally, if there were no matching files -- for example, a linux build with only foo+osx-x64.myr listed -- then the build would fail due to a lack of matching inputs.
Attributes
Various targets support different sets of attributes in mbld; This is a list of all attribtues and which build targets they apply to:
Explicitly adds a dependency for the target. When the file listed in the dependency attribute changes, the rule will be rerun. This attribute may be repeated to list multiple files as dependencies. Valid on gen and cmd targets.
durable
Makes the outputs of this rule exempt from cleaning on an 'mbld clean'. This is useful for outputs that are expensive to generate, or created on the output of user commands, such as 'configure' scripts. Valid on gen and cmd targets.
inc=path
Adds an entry to the include path when running 6m. This can be useful for
nonstandard include, although a lib
directive should already include the use
path for the library. This should only be rarely needed. Valid on bin, test,
and lib targets.
ldscript=script
Uses a linker script when linking binaries, instead of going for the default linker behavior. This is useful when linking binaries with strange linker requirements -- for example, when linking a kernel. Valid on bin and test targets.
noinst
Tells mbld that a file should not be installed. Valid on bin, test, and lib targets.
runtime=rtpath
Tells mbld to link with a custom runtime; This is the code that handles the startup of the binary, various low level abortions, and emulation of certain unsupported operations. If you need to build a kernel, you will probably want to write your own custom runtime. Valid on bin and test targets.
sys
This adds system tagging to targets, allowing them to be built or not on various platforms. Allowed on gen, cmd, bin, lib and test targets.
Semiformal Syntax
The full syntax is below, in pseudo-yacc:
A bldfile is a list of targets:
bldfile: target
| bldfile target
A target is a target type, followed by the target information.
target: "bin" myrtarget
| "lib" myrtarget
| "test" myrtarget
| "gen" cmdtarget
| "cmd" cmdtarget
| "man" anontarget
| "sub" anontarget
The target information generally either takes the form name {attrs} = inputs
;;
or = inputs ;;
. Command targets take a list of output files.
myrtarget: name attrs "=" inputlist ";;"
cmdtarget: wordlist attrs "=" cmd ";;"
subtarget: attrs "=" wordlist ";;"
inputlist: input
| inputlist input
The inputs are generally a list of names. Myrddin targets can also include
dependency tags, in the form of lib libraryname
.
input: name
| "lib" name
namelist: name
| namelist name
Rules also accept a set of attributes, in the form of a brace-enclosed list of key-value pairs, for example, "{noinst,inc=local-path}"
kvplist: name
| name "=" val
attrs : "{" kvplist "}"
All names are fairly liberal in what they accept. Any alphanumeric character and a large amount of punctuation is acceptable. A quoted string is also a word, in case the usual word characters are too restrictive.
name: wordchar*
| "(\"|[^"])*"
wordchar: any unicode alphanumeric character
| "." |"_" |"$" |"-" | "/" | "@" | "!"
| "~" | "+" |"%" |"&" |"(" | ")" | "["
| "]" | ":"
Options
To see the most current list of options for mbld, run it with the '-h' option. For the sake of completeness, here's a dump of the options that are accepted.
-? print this help message
-B base install into 'base'
-I inc add 'inc' to your include path
-R runsrc source to compile and run
-S generate assembly when building
-b bin compile binary 'bin' from inputs
-h print this help message
-j jobs build with at most 'jobs' jobs
-o dir output directory
-r rt link against runtime 'rt'
-t tag build with specified systag