NetBSD Uber-Make Automation Tools

---

What are they for?

These tools are intended to help automate the tracking of netbsd-current via SUP, using "sacrificial" machines that are expected to boot and run the new binaries once they are built. The tools were developed on:

1. A homebuilt system using a Iwill XA-100 motherboard and commodity PC peripherals, running NetBSD/i386
2. A homebuilt system using a CATS motherboard and commodity PC peripherals, running NetBSD/arm32
3. (future) A low-end DEC Multia (just arrived from CPU Micromart) which will be running NetBSD/Alpha

Most definitely they are not perfect, but they accomplish their primary goals:

• to provide near-100% automation in the ideal case where no errors occur
• to restart somewhat intelligently after a failure has been investigated
• to emphasize reproducibility of results (often at the expense of speed)

What are they made of?

The automation system drives builds inside /usr/src, but lives in its own directory, typically /usr/netbsd-current. These two directories are specified at the top of Makefile and also by the supfile. The /usr/netbsd-current directory contains:

1. make-based top level interface:
   1. Makefile.
   2. mybsdbuild.mk, a makefile fragment included by Makefile to allow for site customization.
   3. stamp/, a directory of empty timestamp files used to track completed stages.
2. machinery to automate the maintainance of local patches:
   1. makepatches, an sh script used to find, apply, and remove local diffs.
   2. patches/, a shadow directory tree of diff -u output files maintained by makepatches.
3. normal SUP configuration files:
   1. supfile (mine was originally cloned from /usr/share/examples/supfiles/sup.netbsd.org).
   2. sup/, maintained by sup.
4. log file directories and utilities:
   1. suplogs/, output from sup commands used to obtain updated sources.
   2. patchlogs/, output from makepatches script runs used to apply local patches.
   3. buildlogs/, output from make build commands used to build the tree.
   4. destdirlogs/, output from make build commands with DESTDIR set.
   5. distlogs/, output from sh maketars commands used to make binary distribution sets.
   6. srctarlogs/, output from sh makesrctars commands used to make source distribution sets.
   7. TAILME, an sh script to assist in choosing log files for monitoring with tail -f.
5. output directories which will be overwritten:
   1. destdir/, populated by a make build with DESTDIR set.
   2. releasedir/, populated with distribution sets and genetic material (sup logs, local diffs).

How do they work?

Basic use of the system involves only two Makefile targets, current and world. Use current to obtain new sources from the SUP server, and world to build, install, and snapshot them. The full list of convenience targets is:

• cleandir: removes all log files and timestamps, but not the output directories, as those are automatically removed before they are used. Use this target to completely reset the system.
• clean: removes all timestamps. Use this to force a restart without updating the sources.
• patches: searches for all .orig files in the source directory and diffs their basename against them, saving the output in patches/. This operation does not interact with the stamp/* timestamps.
• current: reverts source files to their .orig version for all diffs contained in patches/, and runs sup to obtain new sources. This operation does not interact with the stamp/* timestamps, but it usually has the same effect as clean because the timestamps all ultimately depend on the "when" file that is updated by sup.
• local: applies (using patch -f) all diffs in patches/ to the source directory and fails if any hunks fail. If run again it will skip files that look patched (a .orig exists), except to reapply .rej files if it finds them. It is not too smart but should handle the common cases well enough.
• kernel: depends on local. This builds and installs config, removes all kernel compile directories, configures and builds a preferred kernel (such as GENERIC), and copies it up to /netbsd (which is first renamed to /netbsd.old). It then prints a message asking to be rebooted, updates its target, and then deliberately halts make by executing false.
• world: depends on kernel. This performs a make build, does another with DESTDIR set, makes distribution sets and (optionally) boot floppies, and puts genetic material (sup logs, local diffs) into releasedir/ to complete the snapshot.

Where are they?

Broken out files are located in ftp://ftp.toddpw.org/ubermake/.

A gzipped tar file of the same is at ftp://ftp.toddpw.org/ubermake.tgz.

Who maintains them?

Todd Whitesel (i.e. me). Send comments specifically about ubermake to ubermake@toddpw.org