I have been spending a lot of time lately thinking about how we
can extend our current package system to allow the use of system
packages (and sets thereof) rather than tarballs as our primary
way of installing NetBSD systems. What follows is a random jumble
of thoughts on where we can go with this. Based on people's response
to this, I will put together a more concrete proposal soon.
1. Package Granularity
People would like to be able to install/remove system software
with a high level of granularity. (Do I want Kerberos, but
not UUCP? C but not Fortran? Text formatting but not printing?)
This could be best addressed by providing our system software
as a series of packages which could be installed individually.
Dependencies would then exist among these packages, which would
insure that a working system resulted from various combinations
users might choose. For example, users could choose a C package
and/or a Fortran package, and both would depend on a binutils
package.
At the same time, users should not need to choose between a
thousand little packages when installing the system.
This probably means that there should be the concept of `sets'
of packages to be installed all at once, roughly analogous to
our current breakdown into base, comp, etc, games and so on.
A set would contain a number of packages, and a contents file
indicating which packages were part of the set. An install
program could then default to let users pick among whole sets,
but could have an advanced mode which would let users pick
individual packages instead.
Very possibly, this would open the possibility of keeping set,
information, in addition to package information around, and
offering the ability to remove all installed packages from a
given set at a later date.
Alternately, a system similar to our existing pkg-src/meta-pkgs
mechanism could be used -- users would choose meta-pkgs more
or less equivalent to the current sets, and these would trigger
dependencies on the packages which make up the set.
2. Package Registration
Currently, packages are installed in /usr/pkg and /usr/X11R7,
and all package registration is done by creating directories
in /var/db/pkg.
This is generally agreed to have a couple of problems, especially
when you want to share /usr/pkg between multiple machines.
Unless /var/db/pkg is the same on both machines, you very
quickly end up with at least one machine not knowing what
packages it has installed.
Alistair Crooks has recently put together changes to use
${LOCALBASE}/etc/pkg and ${X11BASE}/etc/pkg to store package
registration information, which is a big step toward solving
this problem (yahoo!). Presumably, system packages will use
/etc/pkg, although this leaves open the question of what to do
if you share /usr between multiple client machines.
3. Package Creation
Right now, the Makefile in .../src/etc can be used to build
distribution sets, which are tarballs to be unpacked relative
to / by the installer. Sets are defined by lists in
.../src/distrib/sets/lists, and consist of MI and MD components.
Thus, where sources are installed in /usr/src,
/usr/src/distrib/sets/lists/comp/mi contains a list of files
in the compiler set which are available on all architectures,
while /usr/src/distrib/sets/lists/comp/md.i386 contains a list
of those files which are on the i386, but not on all other
architectures. All shared libraries, for example, are in the
md set, since not all ports have shared libraries.
In order to move to using system packages, we will need to
provide a way to generate binary packages from our source tree.
It should be noted that the granularity of packages does not
really match the granularity of source subdirectories in
/usr/src/*, so barring a source tree re-organization (which
seems undesirable), we will need a way to do this which (like
the current mechanism) is outside of the normal `make clean ;
make depend; make' recursion through the source tree. A system
will also be needed to allow building a set out of a group of
packages. Both of these systems will need to be simple to use
(one make command line), and should be possible to run on a
single-package or single-set level.
4. Package Versioning and Upgrades
One of the neatest features offered by having independent
packages for different parts of the system is the ability to
upgrade a single package independently of the whole system.
Although this should not be abused (we don't want to become
like Linux where on an average system nothing quite matches
anything else), it would provide an ideal way to distribute
important patches and security fixes.
For example, the security patch for NetBSD-SA1998-004 (at(1)
being usable to view any file) was distributed as a patch to
the NetBSD-1.3.2 source tree. This requires anyone who wishes
to install the fix to have a source tree available, download
and apply the patch, and then rebuild at(1).
In a system based on install systems, an updated version of
the specific package containing at(1) could be released, which
users could download and pkg_add in a matter of moments. It
would also be easy for an admin to tell if a system had been
upgraded by looking at pkg_info output.
