Contributed by John Polstra
<jdp@FreeBSD.ORG>.
CVSup is a software package for distributing and updating source trees from a master CVS repository on a remote server host. The FreeBSD sources are maintained in a CVS repository on a central development machine in California. With CVSup, FreeBSD users can easily keep their own source trees up to date.
CVSup uses the so-called pull model of updating. Under the pull model, each client asks the server for updates, if and when they are wanted. The server waits passively for update requests from its clients. Thus all updates are instigated by the client. The server never sends unsolicited updates. Users must either run the CVSup client manually to get an update, or they must set up a cron job to run it automatically on a regular basis.
The term "CVSup", capitalized just so, refers to the entire software package. Its main components are the client "cvsup" which runs on each user's machine, and the server "cvsupd" which runs at each of the FreeBSD mirror sites.
As you read the FreeBSD documentation and mailing lists, you may see references to sup. Sup was the predecessor of CVSup, and it served a similar purpose. CVSup is in used in much the same way as sup and, in fact, uses configuration files which are backward-compatible with sup's. Sup is no longer used in the FreeBSD project, because CVSup is both faster and more flexible.
The easiest way to install CVSup if you are running FreeBSD 2.2 or later is to use either the port from the FreeBSD ports collection or the corresponding binary package, depending on whether you prefer to roll your own or not.
If you are running FreeBSD-2.1.6 or 2.1.7, you unfortunately cannot use the binary package versions due to the fact that they require a version of the C library that does not yet exist in FreeBSD-2.1.{6,7}. You can easily use the port, however, just as with FreeBSD 2.2. Simply unpack the tar file, cd to the cvsup subdirectory and type "make install".
Because CVSup is written in Modula-3, both the package and the port require that the Modula-3 runtime libraries be installed. These are available as the lang/modula-3-lib port and the lang/modula-3-lib-3.6 package. If you follow the same directions as for cvsup, these libraries will be compiled and/or installed automatically when you install the CVSup port or package.
The Modula-3 libraries are rather large, and fetching and compiling them is not an instantaneous process. For that reason, a third option is provided. You can get statically linked FreeBSD executables for CVSup from the USA distribution site:
as well as from the many FreeBSD FTP mirror sites around the world.
Most users will need only the client. These executables are entirely self-contained, and they will run on any version of FreeBSD from FreeBSD-2.1.0 to FreeBSD-current.
In summary, your options for installing CVSup are:
CVSup's operation is controlled by a configuration file called the "supfile". Beginning with FreeBSD-2.2, there are some sample supfiles in the directory /usr/share/examples/cvsup. These examples are also available from ftp://ftp.freebsd.org/pub/FreeBSD/FreeBSD-current/src/share/examples/cvsup/ if you are on a pre-2.2 system.
The information in a supfile answers the following questions for cvsup:
In the following sections, we will construct a typical supfile by answering each of these questions in turn. First, we describe the overall structure of a supfile.
A supfile is a text file. Comments begin with "#" and extend to the end of the line. Lines that are blank and lines that contain only comments are ignored.
Each remaining line describes a set of files that the user wishes to receive. The line begins with the name of a "collection", a logical grouping of files defined by the server. The name of the collection tells the server which files you want. After the collection name come zero or more fields, separated by white space. These fields answer the questions listed above. There are two types of fields: flag fields and value fields. A flag field consists of a keyword standing alone, e.g., "delete" or "compress". A value field also begins with a keyword, but the keyword is followed without intervening white space by "=" and a second word. For example, "release=cvs" is a value field.
A supfile typically specifies more than one collection to receive. One way to structure a supfile is to specify all of the relevant fields explicitly for each collection. However, that tends to make the supfile lines quite long, and it is inconvenient because most fields are the same for all of the collections in a supfile. CVSup provides a defaulting mechanism to avoid these problems. Lines beginning with the special pseudo-collection name "*default" can be used to set flags and values which will be used as defaults for the subsequent collections in the supfile. A default value can be overridden for an individual collection, by specifying a different value with the collection itself. Defaults can also be changed or augmented in mid-supfile by additional "*default" lines.
With this background, we will now proceed to construct a supfile for receiving and updating the main source tree of FreeBSD-current.
The files available via CVSup are organized into named groups called "collections". The collections that are available are described here. In this example, we wish to receive the entire main source tree for the FreeBSD system. There is a single large collection "src-all" which will give us all of that, except the export-controlled cryptography support. Let us assume for this example that we are in the USA or Canada. Then we can get the cryptography code with one additional collection, "cvs-crypto". As a first step toward constructing our supfile, we simply list these collections, one per line:
src-all cvs-crypto
With CVSup, you can receive virtually any version of the sources that ever existed. That is possible because the cvsupd server works directly from the CVS repository, which contains all of the versions. You specify which one of them you want using the "tag=" and "date=" value fields.
WARNING: Be very careful to specify any "tag=" fields
correctly. Some tags are valid only for certain collections of
files. If you specify an incorrect or misspelled tag, CVSup will
delete files which you probably do not want deleted.
In particular, use only "tag=." for the "ports-*"
collections.
The "tag=" field names a symbolic tag in the repository. There are two kinds of tags, revision tags and branch tags. A revision tag refers to a specific revision. Its meaning stays the same from day to day. A branch tag, on the other hand, refers to the latest revision on a given line of development, at any given time. Because a branch tag does not refer to a specific revision, it may mean something different tomorrow than it means today.
Here are the branch tags that users might be interested in:
The main line of development, also known as FreeBSD-current. Note: the "." is not punctuation; it is the name of the tag. Valid for all collections.
The line of development for FreeBSD-3.x, also known as FreeBSD-stable. Not valid for the ports-* collections.
The line of development for FreeBSD-2.2.x, also known as 2.2-stable. Not valid for the ports-* collections.
The line of development for FreeBSD-2.1.x - this branch is largely obsolete. Not valid for the ports-* collections.
Here are the revision tags that users might be interested in:
FreeBSD-3.0. Not valid for the ports-* collections.
FreeBSD-2.2.8. Not valid for the ports-* collections.
FreeBSD-2.2.7. Not valid for the ports-* collections.
FreeBSD-2.2.6. Not valid for the ports-* collections.
FreeBSD-2.2.5. Not valid for the ports-* collections.
FreeBSD-2.2.2. Not valid for the ports-* collections.
FreeBSD-2.2.1. Not valid for the ports-* collections.
FreeBSD-2.2.0. Not valid for the ports-* collections.
FreeBSD-2.1.7. Not valid for the ports-* collections.
FreeBSD-2.1.6.1. Not valid for the ports-* collections.
FreeBSD-2.1.6. Not valid for the ports-* collections.
FreeBSD-2.1.5. Not valid for the ports-* collections.
FreeBSD-2.1.0. Not valid for the ports-* collections.
WARNING: Be very careful to type the tag name exactly as shown. CVSup cannot distinguish between valid and invalid tags. If you misspell the tag, CVSup will behave as though you had specified a valid tag which happens to refer to no files at all. It will delete your existing sources in that case.
When you specify a branch tag, you normally receive the latest versions of the files on that line of development. If you wish to receive some past version, you can do so by specifying a date with the "date=" value field. The cvsup(1) manual page explains how to do that.
For our example, we wish to receive FreeBSD-current. We add this line at the beginning of our supfile:
*default tag=.
There is an important special case that comes into play if you specify neither a "tag=" field nor a "date=" field. In that case, you receive the actual RCS files directly from the server's CVS repository, rather than receiving a particular version. Developers generally prefer this mode of operation. By maintaining a copy of the repository itself on their systems, they gain the ability to browse the revision histories and examine past versions of files. This gain is achieved at a large cost in terms of disk space, however.
We use the "host=" field to tell cvsup where to obtain its updates. Any of the CVSup mirror sites will do, though you should try to select one that's close to you in cyberspace. In this example, we'll use a fictional FreeBSD distribution site, "cvsup666.FreeBSD.org":
*default host=cvsup666.FreeBSD.org
You'll need to change the host to one that actually exists before running CVSup. Note, on any particular run, you can override the host setting on the command line, with "-h hostname".
The "prefix=" field tells cvsup where to put the files it receives. In this example, we will put the source files directly into our main source tree, "/usr/src". The "src" directory is already implicit in the collections we have chosen to receive, so this is the correct specification:
*default prefix=/usr
The cvsup client maintains certain status files in what is called the "base" directory. These files help CVSup to work more efficiently, by keeping track of which updates you have already received. We will use the standard base directory, "/usr/local/etc/cvsup":
*default base=/usr/local/etc/cvsup
This setting is used by default if it is not specified in the supfile, so we actually do not need the above line.
If your base directory does not already exist, now would be a good time to create it. The cvsup client will refuse to run if the base directory does not exist.
There is one more line of boiler plate that normally needs to be present in the supfile:
*default release=cvs delete use-rel-suffix compress
"release=cvs" indicates that the server should get its information out of the main FreeBSD CVS repository. This is virtually always the case, but there are other possibilities which are beyond the scope of this discussion.
"delete" gives CVSup permission to delete files. You should always specify this, so that CVSup can keep your source tree fully up to date. CVSup is careful to delete only those files for which it is responsible. Any extra files you happen to have will be left strictly alone.
"use-rel-suffix" is ... arcane. If you really want to know about it, see the cvsup(1) manual page. Otherwise, just specify it and do not worry about it.
"compress" enables the use of gzip-style compression on the communication channel. If your network link is T1 speed or faster, you probably should not use compression. Otherwise, it helps substantially.
Here is the entire supfile for our example:
*default tag=. *default host=cvsup666.FreeBSD.org *default prefix=/usr *default base=/usr/local/etc/cvsup *default release=cvs delete use-rel-suffix compress src-all cvs-crypto
You are now ready to try an update. The command line for doing this is quite simple:
cvsup supfile
where "supfile" is of course the name of the supfile you have just created. Assuming you are running under X11, cvsup will display a GUI window with some buttons to do the usual things. Press the "go" button, and watch it run.
Since you are updating your actual "/usr/src" tree in this example, you will need to run the program as root so that cvsup has the permissions it needs to update your files. Having just created your configuration file, and having never used this program before, that might understandably make you nervous. There is an easy way to do a trial run without touching your precious files. Just create an empty directory somewhere convenient, and name it as an extra argument on the command line:
mkdir /var/tmp/dest cvsup supfile /var/tmp/dest
The directory you specify will be used as the destination directory for all file updates. CVSup will examine your usual files in "/usr/src", but it will not modify or delete any of them. Any file updates will instead land in "/var/tmp/dest/usr/src". CVSup will also leave its base directory status files untouched when run this way. The new versions of those files will be written into the specified directory. As long as you have read access to "/usr/src", you do not even need to be root to perform this kind of trial run.
If you are not running X11 or if you just do not like GUIs, you should add a couple of options to the command line when you run cvsup:
cvsup -g -L 2 supfile
The "-g" tells cvsup not to use its GUI. This is automatic if you are not running X11, but otherwise you have to specify it.
The "-L 2" tells cvsup to print out the details of all the file updates it is doing. There are three levels of verbosity, from "-L 0" to "-L 2". The default is 0, which means total silence except for error messages.
There are plenty of other options available. For a brief list of them, type "cvsup -H". For more detailed descriptions, see the manual page.
Once you are satisfied with the way updates are working, you can arrange for regular runs of cvsup using cron(8). Obviously, you should not let cvsup use its GUI when running it from cron.
The file collections available via CVSup are organized hierarchically. There are a few large collections, and they are divided into smaller sub-collections. Receiving a large collection is equivalent to receiving each of its sub-collections. The hierarchical relationships among collections are reflected by the use of indentation in the list below.
The most commonly used collections are src-all,
cvs-crypto, and ports-all. The other collections are used
only by small groups of people for specialized purposes, and some mirror
sites may not carry all of them.
cvs-all release=cvsThe main FreeBSD CVS repository, excluding the export-restricted cryptography code.
distrib release=cvsFiles related to the distribution and mirroring of FreeBSD.
doc-all release=cvsSources for the FreeBSD handbook and other documentation.
ports-all release=cvsThe FreeBSD ports collection.
ports-archivers release=cvsArchiving tools.
ports-astro release=cvsAstronomical ports.
ports-audio release=cvsSound support.
ports-base release=cvsMiscellaneous files at the top of /usr/ports.
ports-benchmarks release=cvsBenchmarks.
ports-biology release=cvsBiology.
ports-cad release=cvsComputer aided design tools.
ports-chinese release=cvsChinese language support.
ports-comms release=cvsCommunication software.
ports-converters release=cvscharacter code converters.
ports-databases release=cvsDatabases.
ports-deskutils release=cvsThings that used to be on the desktop before computers were invented.
ports-devel release=cvsDevelopment utilities.
ports-editors release=cvsEditors.
ports-emulators release=cvsEmulators for other operating systems.
ports-games release=cvsGames.
ports-german release=cvsGerman language support.
ports-graphics release=cvsGraphics utilities.
ports-japanese release=cvsJapanese language support.
ports-korean release=cvsKorean language support.
ports-lang release=cvsProgramming languages.
ports-mail release=cvsMail software.
ports-math release=cvsNumerical computation software.
ports-mbone release=cvsMBone applications.
ports-misc release=cvsMiscellaneous utilities.
ports-net release=cvsNetworking software.
ports-news release=cvsUSENET news software.
ports-plan9 release=cvsVarious programs from Plan9.
ports-print release=cvsPrinting software.
ports-russian release=cvsRussian language support.
ports-security release=cvsSecurity utilities.
ports-shells release=cvsCommand line shells.
ports-sysutils release=cvsSystem utilities.
ports-textproc release=cvstext processing utilities (does not include desktop publishing).
ports-vietnamese release=cvsVietnamese language support.
ports-www release=cvsSoftware related to the World Wide Web.
ports-x11 release=cvsPorts to support the X window system.
ports-x11-clocks release=cvsX11 clocks.
ports-x11-fm release=cvsX11 file managers.
ports-x11-fonts release=cvsX11 fonts and font utilities.
ports-x11-toolkits release=cvsX11 toolkits.
ports-x11-wm release=cvsX11 window managers.
src-all release=cvsThe main FreeBSD sources, excluding the export-restricted cryptography code.
src-base release=cvsMiscellaneous files at the top of /usr/src.
src-bin release=cvsUser utilities that may be needed in single-user mode
(/usr/src/bin).
src-contrib release=cvsUtilities and libraries from outside the FreeBSD project, used
relatively unmodified (/usr/src/contrib).
src-etc release=cvsSystem configuration files (/usr/src/etc).
src-games release=cvsGames (/usr/src/games).
src-gnu release=cvsUtilities covered by the GNU Public License (/usr/src/gnu).
src-include release=cvsHeader files (/usr/src/include).
src-kerberosIV release=cvsKerberosIV security package (/usr/src/kerberosIV).
src-lib release=cvsLibraries (/usr/src/lib).
src-libexec release=cvsSystem programs normally executed by other programs
(/usr/src/libexec).
src-release release=cvsFiles required to produce a FreeBSD release (/usr/src/release).
src-sbin release=cvsSystem utilities for single-user mode (/usr/src/sbin).
src-share release=cvsFiles that can be shared across multiple systems (/usr/src/share).
src-sys release=cvsThe kernel (/usr/src/sys).
src-tools release=cvsVarious tools for the maintenance of FreeBSD (/usr/src/tools).
src-usrbin release=cvsUser utilities (/usr/src/usr.bin).
src-usrsbin release=cvsSystem utilities (/usr/src/usr.sbin).
www release=cvsThe sources for the World Wide Web data.
cvs-crypto release=cvsThe export-restricted cryptography code.
src-crypto release=cvsExport-restricted utilities and libraries from outside the FreeBSD
project, used relatively unmodified (/usr/src/crypto).
src-eBones release=cvsKerberos and DES (/usr/src/eBones).
src-secure release=cvsDES (/usr/src/secure).
distrib release=selfThe CVSup server's own configuration files. Used by CVSup mirror sites.
gnats release=currentThe GNATS bug-tracking database.
mail-archive release=currentFreeBSD mailing list archive.
www release=currentThe installed World Wide Web data. Used by WWW mirror sites.
Most FreeBSD-related discussion of CVSup takes place on the
FreeBSD technical discussions mailing list
<freebsd-hackers@FreeBSD.ORG>. New versions of the software are announced there, as
well as on the FreeBSD announcements mailing list
<freebsd-announce@FreeBSD.ORG>.
Questions and bug reports should be addressed to the author of the program at cvsup-bugs@polstra.com.