CVSup Frequently Asked Questions
Frequently Asked Questions
John D. Polstra
This is the FAQ for CVSup.
Thanks to Sergey Osokin, there is now a Russian version too!
Contents
The Basics
What is CVSup?
What makes CVSup different from other network update packages such as rdist
and sup?
What kinds of files can be updated using CVSup?
Where can I get CVSup?
How can I report bugs or ask questions?
How should one pronounce the word "CVSup"?
Some Terminology
What is an RCS file?
What is a CVS repository?
Recipes
I just want to get the latest version of the main FreeBSD source tree. What do
I have to do?
When I installed FreeBSD months ago, I installed the sources too. Now I want
to begin using CVSup to keep my sources completely up to date. Do I have to
throw out my existing sources and transfer everything over the network again
using CVSup?
But you said above that CVSup won't delete any files if I don't have a
checkouts file. If I adopt my existing files as you describe, don't I run a
risk that some files which should be deleted won't be deleted?
How can I adopt my FreeBSD-2.2.5 sources, and update them directly to
FreeBSD-current?
Understanding cvsupfiles
Which things belong on "*default" lines in the cvsupfile, and which belong on
the lines for individual collections?
What does the cvsupfile's "delete" keyword do?
If I should always specify "delete", then why isn't it the default?
Refuse Files
Help! I can't get "refuse" files to work.
How do I specify the patterns in a refuse file?
How about an example of a refuse file?
My refuse file works for directories, but I can't seem to make it block
individual files.
Why does my refuse file block some files which don't match any of the patterns
in it?
Where should I put my refuse files?
Is there a way to create a global refuse file that will apply to all
collections?
CVSup and Firewalls
How can I make CVSup work through my TIS FWTK firewall?
Problems Using CVSup
Why has CVSup suddenly started giving me lots of messages saying, "Checksum
mismatch -- will transfer entire file"? Every file gets a "fixup" and that is
really slowing down my updates.
When I try to run CVSup, it says it has connected to the server, but then it
just hangs. What's wrong?
Whenever I run CVSup under FreeBSD, I get zillions of messages like this:
fatal process exception: page fault, fault VA = 0x11a610.
I tried to run the FreeBSD binary of CVSup under BSD/OS, but it dumped core
right away. Isn't that supposed to work?
The CVSup client dies with a segmentation violation when I try to use the GUI.
The Linux binary from the CVSup FTP site can't do DNS lookups on my Linux
system.
CVSup (client or server) dies with the message "gc: Could not extend the
traced heap".
CVSup client dies at random times with a subscript out of range in
DragonInt.m3.
I moved a CVSup mirror site which I manage to a different OS, and now CVSup
(client and/or server) keeps crashing.
Sometimes when I run the CVSup client, it spews a "SetAttrs" message for every
file.
I tried to update my files with CVSup, but all I got were a bunch of strange
looking files whose names all ended with ",v". Why?
Suddenly CVSup has started setting the timestamps of updated files to the year
1970!
Checkouts Files
What are these "checkouts" files I hear about occasionally?
This sounds dangerous. What if I accidentally delete one of my checkouts
files?
What if my checkouts file gets corrupted somehow?
Is there anything at all that can go wrong if I lose a checkouts file?
Local Modifications in your CVS Repository
Can I check my own local changes into a CVS repository that I update with
CVSup from a master site?
How can I keep CVSup from deleting the revisions I have checked in locally?
How can I keep the revision numbers of my local check-ins from conflicting
with revision numbers originating at the master site?
Setting Up a CVSup Server
How can I set up a simple collection to test the CVSup server?
Where can I find a set of CVSup server configuration files to use as an
example?
Building CVSup from the Sources
I've heard that it's not so easy to build CVSup from the sources. Why is that?
What steps are involved in building CVSup from the sources?
Has CVSup been ported to Windows 95 or NT?
Compatibility
When a new version of CVSup comes out, do the clients and the server have to
be upgraded at the same time?
Can I use the CVSup client to get updates from a SUP server?
Bidirectional Mirroring
How can I use CVSup to mirror changes in both directions between two CVS
repositories?
What other tools are available for bidirectional mirroring of CVS
repositories?
The Basics
What is CVSup?
CVSup is a software package for transferring and updating collections of files
across a network. It consists of a server called cvsupd and a client called
cvsup.
CVSup was written by John Polstra, a software consultant who lives in Seattle.
What makes CVSup different from other network update packages such as rdist
and sup?
CVSup is faster (often by an order of magnitude) and more flexible than
traditional network update packages.
What kinds of files can be updated using CVSup?
CVSup can efficiently update any kind of file. It can even update Unix device
nodes, symbolic links, and hard links. CVSup supports several different
algorithms for updating various kinds of files. It tries to use the most
efficient method for each file. For example, RCS files are updated using a
specialized algorithm that takes advantage of their structure to greatly
reduce the update time. Log files (which are changed only by appending new
text at the end) are likewise updated by a special algorithm that transmits
only the new text. Other text and binary files can be updated efficiently by
the rsync algorithm, which is built into CVSup.
Where can I get CVSup?
CVSup is free software, distributed under a BSD-style license. You can obtain
CVSup from ftp://ftp.FreeBSD.org/pub/FreeBSD/development/CVSup/, and from
FreeBSD's FTP mirror sites all over the world.
The sources subdirectory contains the full sources for CVSup. Be sure to read
this before fetching the source distribution.
We no longer provide binary distributions of CVSup, but other people are
welcome to do so as long as they satisfy the conditions set forth in the
License file included in the CVSup source distribution.
How can I report bugs or ask questions?
The e-mail address for bug reports and questions changes occcasionally to keep
the spammers at bay. To find out the current address, visit
http://www.cvsup.org/contact.html.
If you are reporting a problem with CVSup, provide as much detail as possible.
Specifically:
Include the output from running cvsup -v.
Include the output from running uname -a on the client machine, and on the
server too, if possible.
Include a copy of your supfile.
Include copies of any "refuse" files you are using.
Provide the exact command line you used to run CVSup.
If CVSup produced any error messages, include them verbatim.
Describe how you obtained your CVSup installation. Did you build it from
sources, or did you get a pre-built binary from somewhere? If you are using
a binary distribution, where did it come from?
Include the cvsupd server configuration files, if possible.
How should one pronounce the word "CVSup"?
The author of the program pronounces it see vee supp. It rhymes (more or less)
with "beam me up."
Some Terminology
What is an RCS file?
An RCS file stores many versions of a single source file, all in one place.
During the lifetime of a project, its source files evolve. Each source file
starts with an initial version. Over time, changes are made to the source
files to fix bugs and add features. At key moments, a programmer might wish to
save the current version of a source file. He can do this by checking it into
its corresponding RCS file. From a given RCS file, one can later extract any
desired version of the source file. This makes it easy to undo changes that
turned out to be ill-advised, or to recreate an earlier release of the
software.
What is a CVS repository?
A CVS repository is a collection of RCS files that are managed together. For
example, the RCS files for all of the sources in a project would typically be
stored together in a CVS repository.
Recipes
I just want to get the latest version of the main FreeBSD source tree. What do
I have to do?
Make a cvsupfile that looks something like this:
*default host=cvsup3.freebsd.org
*default base=/usr
*default prefix=/usr
*default release=cvs
*default delete use-rel-suffix
*default tag=.
src-all
This cvsupfile will create a directory tree "/usr/src" and populate it with
all of the main FreeBSD sources. It will also create a tree "/usr/sup"
containing the checkouts files that CVSup uses to maintain state between
updates. If you want your source tree to be somewhere other than "/usr/src",
change the "prefix=" clause. If you want the checkouts files to be located
somewhere other than "/usr/sup", change the "base=" clause.
Be sure not to leave out the "tag=." clause. That tells CVSup to use checkout
mode to send you the most recent version of each source file.
When I installed FreeBSD months ago, I installed the sources too. Now I want
to begin using CVSup to keep my sources completely up to date. Do I have to
throw out my existing sources and transfer everything over the network again
using CVSup?
No, you don't. CVSup is capable of "adopting" your existing sources and
bringing them up to date. If you think about it, your situation is the same as
if you had been updating with CVSup all along but had lost your checkouts
files. And CVSup uses the same technique to deal with both situations. It uses
checksums to determine which revisions of the files you have currently, and
then updates them in the appropriate ways to transform them into the latest
versions.
Look here for the safest way to do this.
But you said above that CVSup won't delete any files if I don't have a
checkouts file. If I adopt my existing files as you describe, don't I run a
risk that some files which should be deleted won't be deleted?
Yes, you do. The greater the distance between the revisions you have and the
revisions you want to get, the greater the chance that you'll miss some
important file deletions. Luckily, if you know approximately or exactly which
versions of the files you are starting with, you can reduce or eliminate this
risk. You do this by updating twice. First, you tell CVSup to "update" you to
the versions that you already have. This won't change any of your files, but
it will create a checkouts file that precisely reflects what you have
currently. Second, you update again, this time telling CVSup which version you
really want. On the second update, CVSup will have all the information it
needs in order to know which files to delete.
This is a little tricky and there are a couple of important details we haven't
mentioned yet. So we'd better give you an example. Suppose you installed
FreeBSD-2.2.5, including sources, from the CD-ROM. Now you decide you want to
use CVSup to track the 2.2-stable sources. For your first update only, use a
cvsupfile like this:
*default host=cvsup2.freebsd.org
*default base=/usr
*default prefix=/usr
*default release=cvs
*default delete use-rel-suffix
src-all tag=RELENG_2_2_5_RELEASE list=cvs:RELENG_2_2
For subsequent updates, change the last line to:
src-all tag=RELENG_2_2
The unmentioned details are in this last line. First of all, how do you know
what to use for the tag? The answer is, you have to consult a list. See the
"Configuration" section of the CVSup chapter in the FreeBSD Handbook for a
list of tags that are valid for the FreeBSD sources. The important point is
that the tag for your first update should correspond to the version of the
sources that you already have. And the tag for your second and subsequent
updates should correspond to the version of the sources that you want to
receive.
Second, what is this business with the "list" keyword? It is rarely used, but
this is a situation where it is necessary. When CVSup creates or consults one
of its checkouts files, it uses a filename which by default is based on the
"release" and "tag" values for the collection. Specifically, the usual name of
a checkouts file is "checkouts.RELEASE:TAG", where RELEASE and TAG are the
"release" and "tag" settings in your cvsupfile.
In the rather special situation we are addressing here, the naming conventions
for the checkouts files cause a problem. By default, our first update would
produce a file named "checkouts.cvs:RELENG_2_2_5_RELEASE", while the second
update would look for a file named "checkouts.cvs:RELENG_2_2". In order for
the second update to benefit from the information garnered in the first, both
updates must use the same checkouts file. The "list" specification in the
first cvsupfile allows us to accomplish this. It overrides the default suffix
in the name of the checkouts file, and forces it to have the same name as will
be used in subsequent updates.
Admittedly, this is arcane. But you only have to do it once.
How can I adopt my FreeBSD-2.2.5 sources, and update them directly to
FreeBSD-current?
Follow the previous instructions, except change the src-all lines as follows.
In the first update, use:
src-all tag=RELENG_2_2_5_RELEASE list=cvs:.
In subsequent updates, use:
src-all tag=.
Note carefully: that's a period following the tag= part. Don't leave it out.
Understanding cvsupfiles
Which things belong on "*default" lines in the cvsupfile, and which belong on
the lines for individual collections?
It doesn't make any difference. This cvsupfile:
*default host=cvsup3.freebsd.org
*default base=/usr
*default prefix=/usr
*default release=cvs
*default delete use-rel-suffix
*default tag=.
src-all
could just as well be expressed as a single line, like this:
src-all host=cvsup3.freebsd.org base=/usr prefix=/usr release=cvs delete use-rel-suffix tag=.
Using the "*default" lines can make your cvsupfile easier to read, by
shortening the lines. They can also make your cvsupfile more concise if you
are receiving several collections.
What does the cvsupfile's "delete" keyword do?
It gives CVSup permission to remove files on your machine. For example,
suppose you have a file "foo" which you originally received using CVSup. Now
the maintainer of the server host deletes "foo". When you next run CVSup, if
"delete" is specified in your cvsupfile then CVSup will delete "foo" on your
machine. Otherwise it will leave it alone.
Except for a few unusual applications, you should always specify "delete" in
your cvsupfile.
If I should always specify "delete", then why isn't it the default?
Originally CVSup was designed to be a drop-in replacement for sup. Because of
that, the defaults had to be the same whether they made sense or not.
Refuse Files
Help! I can't get "refuse" files to work.
Don't feel bad. Many people find them confusing. But they really do work.
The most common errors people make with refuse files are:
Specifying the patterns incorrectly.
Putting the refuse file in the wrong directory.
Giving the refuse file the wrong name.
We cover these problems in the following items.
How do I specify the patterns in a refuse file?
The most important thing to remember is that the patterns in refuse files are
relative to the prefix, which is often not what you think of as the logical
root of a particular collection. To determine the prefix, look at your
cvsupfile. Does it contain something like this?
*default prefix=/some/directory
Usually it will. In that case /some/directory is the prefix. Otherwise, the
prefix is the same as the base. To determine the base, see this.
Once you have determined your prefix, put yourself in that directory. Then
figure out the relative paths of the files and/or directories you want to
block, and make patterns that match them. Put these patterns into your refuse
file separated by whitespace. You can put each pattern on a separate line, or
put several on each line. Either way works.
How about an example of a refuse file?
OK. Suppose you use CVSup to receive the FreeBSD documentation files (the
"doc-all" collection), using the doc-supfile example from
/usr/share/examples/cvsup. Here's what that cvsupfile looks like, stripped of
comments:
*default host=CHANGE_THIS.FreeBSD.org
*default base=/usr
*default prefix=/usr
*default release=cvs tag=.
*default delete use-rel-suffix
*default compress
doc-all
As you can see, the prefix is /usr. Relative to there, the entire "doc-all"
collection is placed into a subdirectory named doc, which itself contains
these files and subdirectories:
FAQ/ handbook/ ru_SU.KOI8-R/
Makefile ja/ sgml/
en/ ja_JP.EUC/ share/
en_US.ISO_8859-1/ ja_JP.eucJP/ zh/
es/ ru/ zh_TW.Big5/
es_ES.ISO_8859-1/ ru_RU.KOI8-R/
Now let's suppose you're not interested in the Spanish, Japanese, Russian, or
Taiwanese versions of the documentation. So you want to refuse the directories
whose names begin with "es", "ja", "ru", and "zh". The correct patterns
relative to the prefix are:
doc/es*
doc/ja*
doc/ru*
doc/zh*
My refuse file works for directories, but I can't seem to make it block
individual files.
The patterns you specify must match the names of the files on the server. If
the files are coming from a CVS repository (the usual case), then on the
server they are RCS files. And RCS files always have names that end in ",v".
Your patterns must take that into account.
For example, suppose you want to block the Makefile in the example above. The
pattern "doc/Makefile" won't work, because on the server the file's name has a
",v" appended to it. The correct pattern to use is
doc/Makefile,v
or better still
doc/Makefile*
which will match the file on the server whether it is an RCS file or not.
Why does my refuse file block some files which don't match any of the patterns
in it?
You probably tried to put a comment into your refuse file. But (surprise!)
refuse files have no provision for comments. Thus your "comment" is actually
interpreted as a bunch of patterns, some of which may match files that you
want to receive.
Consider this incorrect refuse file:
# We accept the entire src and ports trees except for the games
src/games
That innocuous-looking first line is not a comment! It is 13 separate
patterns, including "#", "src", and "ports". That is almost certainly not what
you intended.
Some day it will probably be possible to put comments into refuse files. But
for now they are not allowed.
Where should I put my refuse files?
First you must determine your base directory.
When you run the "cvsup" program, do you use the "-b pathname" option? (Most
users don't.) If so, that pathname is your base. Otherwise ...
Does your cvsupfile contain a "base=pathname" specification? (Many of them
do.) If so, that pathname is your base. Otherwise ...
Your base is the built-in default, "/usr/local/etc/cvsup".
Next you must determine your collDir, the subdirectory of base where CVSup
keeps track of your collections.
When you run the "cvsup" program, do you use the "-c directory" option?
(Must users don't.) If so, that directory is your collDir. Otherwise ...
Your collDir is the built-in default, "sup".
Finally, you must know the name of the collection you want to restrict, for
example, "doc-all".
Combine those three items with a slash after each one, and then tack "refuse"
on the end. That's where to put your refuse file. For the example we've been
using, it works out to
/usr/sup/doc-all/refuse
assuming you don't use the "-b" or "-c" option when you run CVSup.
Is there a way to create a global refuse file that will apply to all
collections?
Yes. Simply leave out the name of the collection and the slash that follows it
when you formulate the name of the file. In the previous example, the global
refuse file would be named
/usr/sup/refuse
CVSup and Firewalls
How can I make CVSup work through my TIS FWTK firewall?
The following instructions were kindly submitted by Alan Strassberg:
Add this line to /etc/services:
cvsup 5999/tcp # CVSup
Add the following to /etc/inetd.conf:
cvsup stream tcp nowait root /usr/local/etc/plug-gw plug-gw cvsup
and send a SIGHUP to inetd.
Add the following to /usr/local/etc/netperm-table (or whatever the file is
named on your system):
cvsup: port cvsup A.B.C.D -plug-to W.X.Y.Z -port cvsup
where A.B.C.D is the IP address of the internal machine, and W.X.Y.Z is the
IP address of the CVSup server.
In your cvsupfile, set the CVSup server to be your firewall:
*default host=gatekeeper.foo.com
When you invoke the cvsup client, include "-P m" on the command line.
Troubleshooting: you should be able to telnet from the internal machine to the
firewall on port 5999 and see the CVS server greeting:
% telnet gatekeeper.foo.com 5999
OK 16 1 REL_16_1 CVSup server ready
If the greeting does not appear, run netstat -na on the firewall and verify
that it is listening on port 5999:
% netstat -na | grep 5999
...
tcp 0 0 *.5999 *.* LISTEN
Problems Using CVSup
Why has CVSup suddenly started giving me lots of messages saying, "Checksum
mismatch -- will transfer entire file"? Every file gets a "fixup" and that is
really slowing down my updates.
Both you and your friendly server administrator need to upgrade to CVSup 15.4
or later. That will solve this problem.
CVSup upgrades an RCS file by deconstructing it on the server, sending only
the pieces that have changed to the client, and reconstructing the file on the
client. (It's roughly patterned after the transporter on the starship
Enterprise.) CVSup then compares the MD5 checksum of the reconstructed file
with that of the original on the server, to make sure that the process worked
correctly.
Unfortunately, recent releases of CVS have introduced some gratuitous changes
in the format of the RCS files that they write. These are simply changes in
white space, which have no bearing on the logical meaning of the file.
However, the result is that an RCS file constructed by the CVSup client no
longer matches the original file byte-by-byte as before. Even though the
reconstructed file is logically identical to the original, it does not have
the same checksum. This causes older versions of CVSup to reject the updated
file and use a fixup to re-transfer the entire file.
To solve this problem, CVSup 15.4 introduces a logical checksum which is used
only for RCS files. Instead of blindly computing a byte-by-byte checksum over
the entire file, the new checksum algorithm carefully canonicalizes the file
so that irrelevant white space differences are ignored. The logical checksum
should make CVSup immune to any future problems of this nature.
When I try to run CVSup, it says it has connected to the server, but then it
just hangs. What's wrong?
You are behind a firewall which is blocking attempts by the CVSup server to
establish a second TCP connection to your client. Add the option "-P m" to
your cvsup command line, and everything should work fine.
Whenever I run CVSup under FreeBSD, I get zillions of messages like this:
fatal process exception: page fault, fault VA = 0x11a610.
When you built your FreeBSD kernel, you included the undocumented "options
DEBUG" in the kernel config file. Don't do that.
I tried to run the FreeBSD binary of CVSup under BSD/OS, but it dumped core
right away. Isn't that supposed to work?
Yes, the statically linked FreeBSD binaries work fine under other BSD-derived
operating systems. But for some of them, including BSD/OS, you have to add
"@M3novm" to the command line.
CVSup is written in Modula-3, and its runtime system uses a sophisticated
garbage collector which exploits hooks into the VM subsystem of the operating
system to gain better interactive performance. This feature stumbles upon an
incompatibility between BSD/OS and FreeBSD, causing the core dumps. The
cryptic argument "@M3novm" disables the VM hooks and makes it possible to run
FreeBSD binaries under other BSD-derived operating systems.
Also, recent versions (4.0 and later) of BSD/OS cannot run ELF format FreeBSD
executables. But they can run the older a.out executables without problems.
The CVSup client dies with a segmentation violation when I try to use the GUI.
The message looks like this, right?
***
*** runtime error:
*** Segmentation violation - possible attempt to dereference NIL
*** pc = 0x81f0708 = Cat + 0x18 in /b/jdp/pm3/pm3/libs/m3core/src/text/Text.m3
***
use option @M3stackdump to get a stack trace
Abort trap (core dumped)
There is a bug in the Modula-3 graphics library that causes this failure if
your DNS setup isn't correct. In particular, the failure happens if there is
no reverse DNS record for your system's IP address. Try commands like these to
check your DNS setup. (If your system doesn't have the "host" command, look
for "dig" or "nslookup" instead.)
$ hostname
bogus.example.com
$ host bogus.example.com
bogus.example.com has address 192.168.1.1
$host 192.168.1.1
Host not found, try again.
That last line shows the problem. If your DNS setup were correct, the output
would say
1.1.168.192.IN-ADDR.ARPA domain name pointer bogus.example.com
The best way to fix this problem is to correct your DNS setup. (You might need
to get some help from your system administrator.) If that is not possible, you
might be able to work around the problem by adding a line to your "/etc/hosts"
file. Finally, if all else fails, you can run the CVSup client without the GUI
by adding "-g" to the command line.
Someday I will fix the bug in the graphics library so that this won't be a
problem any more.
The Linux binary from the CVSup FTP site can't do DNS lookups on my Linux
system.
Check the FTP site again for a newer binary. This problem was caused by binary
incompatibilities between certain Linux kernels and certain versions of glibc.
The newer binary (distributed since 20 Febuary 2000) links the C library
dynamically, so it should work on any recent version of Linux.
CVSup (client or server) dies with the message "gc: Could not extend the
traced heap".
That's its quaint way of telling you it is out of memory. Make sure your
resource limits are set high enough. Under a sh-like shell, use "ulimit -Sa"
to check the "datasize" and "memoryuse" limits. Under a csh-like shell, use
the "limit" command to check "data seg size" and "max memory size".
CVSup client dies at random times with a subscript out of range in
DragonInt.m3.
That is caused by a bug in the SRC Modula-3 runtime system. Grab an up-to-date
CVSup binary from the CVSup FTP site. Or apply this patch to your SRC Modula-3
sources, and rebuild and reinstall them.
For a quick work-around, just run the CVSup client without the GUI by adding
"-g" to the command line.
I moved a CVSup mirror site which I manage to a different OS, and now CVSup
(client and/or server) keeps crashing.
If you moved your site from FreeBSD, NetBSD, OpenBSD, or BSD/OS to a non-BSD
system, the problem is in the "checkouts" files which you brought along. They
contain some information which non-BSD versions of CVSup don't understand.
This tickles a bug which is present in versions 16.1 and earlier. The bug has
been fixed, and this problem will disappear starting with version 16.2. Until
then, you can easily eliminate the problem by deleting the "checkouts*" files
under your base directory and letting cvsup regenerate them.
Sometimes when I run the CVSup client, it spews a "SetAttrs" message for every
file.
This generally happens when CVSup thinks it needs to update one or more file
attributes (owner, group, permissions, etc.) but isn't able to do so. The
actual file attributes may be incorrect, or their recorded values in the
"checkouts" file may be incorrect. Check for any of the following situations:
You run some updates as root (e.g., from a cron job) and other updates as a
different user. The root updates may cause some files to be owned by root.
Later, your normal user updates won't be able to modify the files.
You have different umask settings at various times when you run CVSup. This
is easy to fix in modern versions of CVSup. Just add "*default umask=2"
(pick your desired value) at the top of your supfile.
Your "checkouts" files under the "base" directory have permissions that make
it impossible for CVSup to write them.
I tried to update my files with CVSup, but all I got were a bunch of strange
looking files whose names all ended with ",v". Why?
Those strange files are RCS files, and CVSup sent them to you because your
cvsupfile told it to. When you ask CVSup to send you updates from a CVS
repository, there are two different things you can ask for. First, you can ask
for the source files to be extracted from the repository and sent to you. That
is apparently what you wanted in this case. Second, you can ask for the raw
RCS files (containing all versions of the sources) to be sent to you.
These two modes of operation are fundamentally different. Both work from RCS
files in a CVS repository on the server host. But they use the RCS files in
different ways. The first mode, called "checkout mode," extracts a particular
version of the sources from the RCS files, and sends you that version. The
second mode, called "CVS mode," sends you the RCS files themselves, in the
same form as on the server. In the cvsupfile, the "tag" and "date" keywords
control which mode is used. If either of these keywords is present, then CVSup
uses checkout mode to send you a set of source files. If neither "tag" nor
"date" is present, then CVSup uses CVS mode to send you the RCS files.
Assuming you want the most recent version of the sources, you simply need to
add "tag=." to your supfile.
Suddenly CVSup has started setting the timestamps of updated files to the year
1970!
This is caused by a long-dormant bug in CVSup which has only now made itself
apparent. The bug is present in all versions prior to SNAP_16_1d. You will
need to update both your client and your server to SNAP_16_1d in order to fix
this problem. (Future versions released after SNAP_16_1d will of course also
contain the bug fix.) For more information and for distributions of SNAP_16_1d
in source and binary forms, please visit http://people.freebsd.org/~jdp/s1g/.
Checkouts Files
What are these "checkouts" files I hear about occasionally?
In order to update your files efficiently, CVSup needs to know what you've
already got. It stores this information in files called "checkouts" files.
Each time you run cvsup, it reads your checkouts files to see which files (and
which revisions of them) you have. As it updates your files, it also updates
the information in the checkouts files.
Confusingly, checkouts files are also sometimes referred to as "list" files.
This sounds dangerous. What if I accidentally delete one of my checkouts
files?
It is not a big problem. If CVSup can't find a checkouts file that it needs,
it falls back on other methods of determining which files you have. One such
method is to compute checksums (MD5 file signatures) for each of your files,
and use those to figure out which file revisions you have. This is perfectly
safe, but it is inefficient. It slows down your update and also puts a heavier
load on the server.
What if my checkouts file gets corrupted somehow?
CVSup will detect the problem and quit, with a message suggesting that you
delete the corrupted file and try again. If you follow the suggestion, it will
complete your update using the fallback methods mentioned above, and recreate
your checkouts file for next time.
Is there anything at all that can go wrong if I lose a checkouts file?
There is just one thing that can go wrong, and it's not very serious. CVSup
will only delete files that are listed in its checkouts file. Thus if you lose
your checkouts file, and then a file "foo" is deleted on the server host, and
then you run CVSup, your file "foo" will not be deleted.
Files are never supposed to be deleted from a CVS repository, so this isn't
much of a problem in real life. But to be safe, you should run CVSup sooner
rather than later, if you find that you've lost a checkouts file.
Local Modifications in your CVS Repository
Can I check my own local changes into a CVS repository that I update with
CVSup from a master site?
If you are careful and if you understand what is going on behind the scenes,
you can make this work. CVSup was designed to allow extra locally checked-in
revisions to coexist in the RCS files along with the revisions taken from the
master repository on the server. Because CVSup understands the structure of
RCS files, it is able to bring in new revisions from the server without
disturbing your own revisions that you have checked in locally. Unfortunately,
certain logistical issues make this capability awkward to use in practical
situations.
To keep local revisions in your copy of the CVS repository, you have to:
Instruct CVSup not to delete them.
Ensure that revision numbers of your local revisions don't conflict with the
numbers of current or future revisions in the master copy of each RCS file
at the server.
We cover these topics separately below.
How can I keep CVSup from deleting the revisions I have checked in locally?
Simply remove the "delete" keyword from your cvsupfile. When present, this
keyword gives CVSup permission to delete extra revisions from the RCS files.
It also permits CVSup to delete entire RCS files, provided that it created
them in the first place. By removing the "delete" keyword, you can prevent
CVSup from removing extra revisions as well as entire RCS files. However, it
is very important to understand the consequences of removing the "delete"
keyword.
First, suppose that one or more RCS files are intentionally removed from the
CVS repository at the master site. Without the "delete" keyword, CVSup will
not propagate these file removals to your site. Thus you will have some RCS
files in your repository that otherwise shouldn't be there.
In an ideal world, this would not be a problem. That is because, in an ideal
world, one never deletes an RCS file from a CVS repository. Thus, the
situation should never arise.
Unfortunately, most administrators don't manage their repositories quite so
idealistically. Committers make mistakes and check files into the wrong
locations. Most administrators will manually repair such errors by moving the
files in the repository, rather than live with the mistake forever. As another
example, in any large repository, some files eventually become completely
obsolete. Eventually, developers will complain about the wasted disk space and
the clutter in the repository, and the repository administrator will respond
by deleting the files.
Usually, undeleted RCS files don't cause any problems, and they can be safely
ignored. This is especially true if the repository administrator at the master
site has taken care to mark the files "dead" on all branches with "cvs remove"
a few weeks before completely removing the unwanted RCS files. Nevertheless,
extra RCS files can cause problems in some cases, and you need to be aware of
that.
How can I keep the revision numbers of my local check-ins from conflicting
with revision numbers originating at the master site?
It is difficult, because CVS chooses the revision numbers itself. One way
around this is to modify CVS slightly. Before discussing that, though, the
first rule of thumb is to create a new branch to hold your local changes.
Don't try to check your local changes into the main branch or a branch that
exists in the master repository. Doing so would make revision number
collisions very likely, sooner or later.
If your local revisions are on their own branch, then the problem is reduced
to ensuring that your branch has a unique revision number which will never be
duplicated in the master repository. The easiest way to accomplish this is to
modify CVS. The version of CVS released with FreeBSD includes such a
modification. In that version, you can influence the revision numbers of
branches by setting the environment variable CVS_LOCAL_BRANCH_NUM. This
variable should be set to an integer value, and CVS will use that as the
starting point when choosing revision numbers for new branches. By default,
CVS allocates branch numbers starting with 1. So a high value such as 1000
makes a good choice. New branches at the master site will receive low revision
numbers, while your own local branches will receive high revision numbers.
Thus the two won't conflict.
When using this method, it is of the utmost importance that committers do not
set CVS_LOCAL_BRANCH_NUM when creating branches in the master repository.
If you don't have a version of CVS that supports CVS_LOCAL_BRANCH_NUM, it is
still possible to avoid conflicts with little cooperation from the master
site. Simply create a branch in the master repository, declare it to be
reserved for local modifications, and leave it unused in the master
repository.
Setting Up a CVSup Server
How can I set up a simple collection to test the CVSup server?
Make an empty directory somewhere. We'll call that directory base. Cd into
base and do a mkdir -p sup/test.
Next, cd into sup/test and create a file named releases with one line in it
like this:
cvs list=list.cvs prefix=prefix
In the above, replace prefix with the absolute path to some CVS repository on
your machine, e.g., /usr/cvs. If you don't have a CVS repository, you can use
any directory that has some files in it. But RCS files are the best for
testing purposes.
In the same directory as the releases file, make a file named list.cvs. It
should contain a single line:
upgrade src/bin
Here, replace src/bin with the path to some reasonably-sized subtree of your
CVS repository, relative to prefix. For example, if prefix is /usr/cvs in your
releases file, and the line in list.cvs is as above, then the transferred
subtree will be /usr/cvs/src/bin.
You have just created a CVSup collection named "test" with a release named
"cvs". You can run the server like this:
cvsupd -b base
replacing base appropriately with the pathname of the base directory that you
created above. If you run it this way, cvsupd will print its log messages to
stdout. It will serve exactly one client and then exit. To run multiple tests,
you have to restart the server each time. Alternatively, you can run the
server like this:
cvsupd -b base -C 1 -l /dev/stdout
and it will become a daemon and serve clients indefinitely until you kill it
manually. Note, it doesn't matter what your working directory is when you
start the server.
Now, create a separate empty directory where you'll run the client to receive
the updates from the server. We'll call that directory dest. In dest, make a
file supfile that looks like this:
*default host=localhost
*default base=.
*default release=cvs
*default delete use-rel-suffix
test
(Notice that there is a "." in "base=.".) Make sure the server is running, and
then, still in dest, run cvsup in the usual way. The simplest command is this:
cvsup supfile
but you can add -g -L 2 to disable the GUI if you wish.
If your CVSup client was built with the GUI, press the start button. The
non-GUI client starts automatically. At this point, your disk light will turn
into a 50-watt beacon and your hard drive will make that noise that says, "I'm
really really really busy." When it's done, you should find the updated files
under your dest directory. There will also be a directory sup which is used by
cvsup to record its state.
Assuming that worked, you can get arbitrarily thorough in testing the rest of
it. By manipulating the source RCS files, you can add deltas and/or tags, and
then make sure they propagate when you do another update. You can also try
deleting RCS files entirely, and adding new ones. On the client side, you can
add tag= and/or date= specifications to the supfile. Generally, if the
software works at all on a given platform, it's likely to work completely.
Almost all of the functionality is contained in OS-independent code.
One caveat: Doing an update all on one machine via localhost really hammers a
system, particularly the disk subsystem and the network stack. It can expose
OS bugs that nobody knew existed before. If you run into networking-related
problems, it might help to use -P m on the cvsup command line when you do your
updates.
Where can I find a set of CVSup server configuration files to use as an
example?
The server configuration files used by the FreeBSD project are available via
CVSup from any FreeBSD mirror site. To fetch a copy, get into an empty
directory and make a file named supfile containing this text:
*default host=a.mirror.site compress
*default release=cvs tag=.
*default base=.
*default delete use-rel-suffix norsync
distrib
Replace a.mirror.site with any CVSup site listed in the FreeBSD Handbook. Run
the CVSup client using this supfile. When it finishes, you'll find the
configuration files in the subdirectory distrib/cvsup. If you wanted to use
these configuration files directly, you'd use that subdirectory as your base
directory, like this:
cvsupd -b distrib/cvsup ...
In a real setup, you'd specify the base directory as an absolute pathname, of
course.
Building CVSup from the Sources
I've heard that it's not so easy to build CVSup from the sources. Why is that?
These days, CVSup is fairly easy to build on most Unix-like platforms. The
minor difficulties that still exist stem from the fact that CVSup is written
in the programming language NotC. What is NotC? Well, to most people, it
really doesn't matter. The important thing is that NotC is not C.
Consequently, it takes a little extra work to use it.
In the case of CVSup, NotC means Modula-3. Modula-3 is an efficient compiled
programming language with excellent built-in support for important features
such as garbage collection and threads. For more information about Modula-3,
see the Modula-3 Resource Page.
Several versions of Modula-3 are available. If your goal is simply to build
CVSup, I recommend using Ezm3, a relatively small distribution that is easy to
build on many popular platforms.
What steps are involved in building CVSup from the sources?
Build and install the Modula-3 compiler and runtime. See the Ezm3 Home Page
for details.
Build and install the "zlib" compression library. See the zlib home page for
details.
Build and install CVSup.
Has CVSup been ported to Windows 95 or NT?
No, CVSup does not currently work on Windows platforms. It relies on a number
of features which are specific to POSIX-like operating systems.
Compatibility
When a new version of CVSup comes out, do the clients and the server have to
be upgraded at the same time?
Traditionally, all publicly released versions of CVSup have been able to
interoperate with each other. However, that became undesirable with the
discovery of the unfortunate s1g bug, which was present in all versions of
CVSup prior to SNAP_16_1d. CVSup clients which contain the s1g bug create very
heavy loads on CVSup servers.
To protect the servers from this problem, checks were added to them beginning
with version SNAP_16_1e. These newer servers refuse to provide service to any
CVSup client which contains the s1g bug. When such a client connects to a
SNAP_16_1e or later server, the server terminates the connection immediately
and causes the client to display a message explaining the problem. The message
includes information about obtaining updated versions of CVSup.
CVSup clients also need protection from old servers which would deliver files
with incorrect timestamps. Therefore, similar checks were added to the
clients, beginning with version SNAP_16_1e. If one of these newer clients
connects to a CVSup server which has the s1g bug, the client displays an error
message and quits immediately. The error message explains the problem, and
urges the user to notify the maintainer of the site which is still running the
obsolete server.
Can I use the CVSup client to get updates from a SUP server?
No. Although the CVSup client can use supfiles written for SUP, the two
packages are otherwise incompatible.
Bidirectional Mirroring
How can I use CVSup to mirror changes in both directions between two CVS
repositories?
I'm sorry, but CVSup is the wrong tool for that. CVSup is designed for
unidirectional mirroring only. If you try to use it bidirectionally, you will
encounter some difficult problems. For example, if conflicting changes are
committed to a file, how shall the conflicts be resolved? There may be an ad
hoc solution that fits your particular situation, but you will have to
discover it on your own.
However, there is a modified and extended version of CVSup available in DCVS
(Distributed Concurrent Versions System) that will enable you to setup a
network of DCVS servers (using the extended CVSup version), each serving only
dedicated ranges of revision numbers which correspond to certain lines of
development (CVS branches). See the next answer for details.
What other tools are available for bidirectional mirroring of CVS
repositories?
It is not possible to mirror CVS repositories bidirectionally without special
provisions for protection and uniqueness of elements. There is a project
called DCVS (Distributed Concurrent Versions System) which consists of
modified and extended versions of CVS and CVSup that enable you to synchronize
a network of DCVS servers with dedicated lines of development. DCVS is
available in source and binary packages for several platforms from elego
Software Solutions GmbH, Berlin.
You can also download DCVS via CVSup or read an overview of DCVS.
$Id: faq.faq,v 1.29 2004/01/11 18:39:39 jdp Exp $
Copyright © 1998-2003 John D. Polstra (
geocities.com/hackermuda)