Chapter 6. Maintenance and Updates: The APT Tools

What makes Debian so popular with administrators is how easily software can be installed and how easily the whole system can be updated. This unique advantage is largely due to the APT program, that Falcot Corp administrators studied with enthusiasm.

APT is the abbreviation for Advanced Package Tool. What makes this program “advanced” is its approach to packages. It doesn't simply evaluate them individually, but it considers them as a whole and produces the best possible combination of packages depending on what is available and compatible (according to dependencies).

APT needs to be given a “list of package sources”: the file /etc/apt/sources.list will list the different repositories (or “sources”) that publish Debian packages. APT will then import the list of packages published by each of these sources. This operation is achieved by downloading Packages.{gz,bz2,lzma,xz} files (in case of a source of binary packages) and Sources.{gz,bz2,lzma,xz} (in case of a source of source packages) and by analyzing their contents. When an old copy of these files is already present, APT can update it by only downloading the differences (see sidebar TIP Incremental upgrade).

6.1. Filling in the `sources.list` File

6.1.1. Syntax

Each active line of the /etc/apt/sources.list file contains the description of a source, made of 3 parts separated by spaces.

The first field indicates the source type:

“deb” for binary packages,
“deb-src” for source packages.

The second field gives the base URL of the source (combined with the filenames present in the Packages.gz files, it must give a full and valid URL): this can consist in a Debian mirror or in any other package archive set up by a third party. The URL can start with file:// to indicate a local source installed in the system's file hierarchy, with http:// to indicate a source accessible from a web server, or with ftp:// for a source available on an FTP server. The URL can also start with cdrom:// for CD-ROM/DVD-ROM/Blu-ray disc based installations, although this is less frequent, since network-based installation methods are more and more common.

The syntax of the last field depends on the structure of the repository. In the simplest cases, you can simply indicate a subdirectory (with a required trailing slash) of the desired source (this is often a simple “./” which refers to the absence of a subdirectory — the packages are then directly at the specified URL). But in the most common case, the repositories will be structured like a Debian mirror, with multiple distributions each having multiple components. In those cases, name the chosen distribution (by its “codename” — see the list in sidebar COMMUNITY Bruce Perens, a controversial leader — or by the corresponding “suites” — stable, testing, unstable), then the components (or sections) to enable (chosen between main, contrib, and non-free in a typical Debian mirror).

VOCABULARY The main, contrib and non-free archives

Debian uses three sections to differentiate packages according to the licenses chosen by the authors of each work. Main gathers all packages which fully comply with the Debian Free Software Guidelines.

The non-free archive is different because it contains software which does not (entirely) conform to these principles but which can nevertheless be distributed without restrictions. This archive, which is not officially part of Debian, is a service for users who could need some of those programs — however Debian always recommends giving priority to free software. The existence of this section represents a considerable problem for Richard M. Stallman and keeps the Free Software Foundation from recommending Debian to users.

Contrib (contributions) is a set of open source software which cannot function without some non-free elements. These elements can be software from the non-free section, or non-free files such as game ROMs, BIOS of consoles, etc. Contrib also includes free software whose compilation requires proprietary elements. This was initially the case for the OpenOffice.org office suite, which used to require a proprietary Java environment.

The cdrom entries describe the CD/DVD-ROMs you have. Contrary to other entries, a CD-ROM is not always available since it has to be inserted into the drive and since only one disc can be read at a time. For those reasons, these sources are managed in a slightly different way, and need to be added with the apt-cdrom program, usually executed with the add parameter. The latter will then request the disc to be inserted in the drive and will browse its contents looking for Packages files. It will use these files to update its database of available packages (this operation is usually done by the apt-get update command). From then on, APT can require the disc to be inserted if it needs one of its packages.

6.1.2. Repositories for Stable Users

Here is a standard sources.list for a system running the Stable version of Debian:

Example 6.1. /etc/apt/sources.list file for users of Debian Stable

# Security updates
deb http://security.debian.org/ wheezy/updates main contrib non-free
deb-src http://security.debian.org/ wheezy/updates main contrib non-free

## Debian mirror

# Base repository
deb http://ftp.debian.org/debian wheezy main contrib non-free
deb-src http://ftp.debian.org/debian wheezy main contrib non-free

# Stable updates
deb http://ftp.debian.org/debian wheezy-updates main contrib non-free
deb-src http://ftp.debian.org/debian wheezy-updates main contrib non-free

# Stable backports
deb http://ftp.debian.org/debian wheezy-backports main contrib non-free
deb-src http://ftp.debian.org/debian wheezy-backports main contrib non-free

This file lists all sources of packages associated with the Wheezy version of Debian (the current Stable as of this writing). We opted to name “wheezy” explicitly instead of using the corresponding “stable“ alias (stable, stable-updates, stable-backports) because we don't want to have the underlying distribution changed outside of our control when the next stable release comes out.

Most packages will come from the “base repository” which contains all packages but is seldom updated (about once every 2 months for a “point release”). The other repositories are partial (they do not contain all packages) and can host updates (packages with newer version) that APT might install. The following sections will explain the purpose and the rules governing each of those repositories.

Note that when the desired version of a package is available on several repositories, the first one listed in the sources.list file will be used. For this reason, non-official sources are usually added at the end of the file.

As a side note, most of what this section says about Stable applies equally well to Oldstable since the latter is just an older Stable that is maintained in parallel.

6.1.2.1. Security Updates

The security updates are not hosted on the usual network of Debian mirrors but on security.debian.org (on a small set of machines maintained by the Debian System Administrators). This archive contains security updates (prepared by the Debian Security Team and/or by package maintainers) for the Stable distribution.

The server can also host security updates for Testing but this doesn't happen very often since those updates tend to reach Testing via the regular flow of updates coming from Unstable.

6.1.2.2. Stable Updates

Stable updates are not security sensitive but are deemed important enough to be pushed to users before the next stable point release.

This repository will typically contain fixes for critical bugs which could not be fixed before release or which have been introduced by subsequent updates. Depending on the urgency, it can also contain updates for packages that have to evolve over time… like spamassassin's spam detection rules, clamav's virus database, or the daylight-saving rules of all timezones (tzdata).

In practice, this repository is a subset of the proposed-updates repository, carefully selected by the Stable Release Managers.

6.1.2.3. Proposed Updates

Once published, the Stable distribution is only updated about once every 2 months. The proposed-updates repository is where the expected updates are prepared (under the supervision of the Stable Release Managers).

The security and stable updates documented in the former sections are always included in this repository, but there is more too, because package maintainers also have the opportunity to fix important bugs that do not deserve an immediate release.

Anyone can use this repository to test those updates before their official publication. The extract below uses the wheezy-proposed-updates alias which is both more explicit and more consistent since squeeze-proposed-updates also exists (for the Oldstable updates):

deb http://ftp.debian.org/debian wheezy-proposed-updates main contrib non-free

6.1.2.4. Stable Backports

The stable-backports repository hosts “package backports”. The term refers to a package of some recent software which has been recompiled for an older distribution, generally for Stable.

When the distribution becomes a little dated, numerous software projects have released new versions that are not integrated into the current Stable (which is only modified to address the most critical problems, such as security problems). Since the Testing and Unstable distributions can be more risky, package maintainers sometimes offer recompilations of recent software applications for Stable, which has the advantage to limit potential instability to a small number of chosen packages.

→ http://backports.debian.org

The stable-backports repository is now available on the usual Debian mirrors. But backports for Squeeze are still hosted on a dedicated server (backports.debian.org), and requires the following sources.list entry:

deb http://backports.debian.org/debian-backports squeeze-backports main contrib non-free

Backports from stable-backports are always created from packages available in Testing. This ensures that all installed backports will be upgradable to the corresponding stable version once the next stable release of Debian is available.

Even though this repository provides newer versions of packages, APT will not install them unless you give explicit instructions to do so (or unless you have already done so with a former version of the given backport):

$ sudo apt-get install package/wheezy-backports
$ sudo apt-get install -t wheezy-backports package

6.1.3. Repositories for Testing/Unstable Users

Here is a standard sources.list for a system running the Testing or Unstable version of Debian:

Example 6.2. /etc/apt/sources.list file for users of Debian Testing/Unstable

# Unstable
deb http://ftp.debian.org/debian unstable main contrib non-free
deb-src http://ftp.debian.org/debian unstable main contrib non-free

# Testing
deb http://ftp.debian.org/debian testing main contrib non-free
deb-src http://ftp.debian.org/debian testing main contrib non-free

# Stable
deb http://ftp.debian.org/debian stable main contrib non-free
deb-src http://ftp.debian.org/debian stable main contrib non-free

# Security updates
deb http://security.debian.org/ stable/updates main contrib non-free
deb http://security.debian.org/ testing/updates main contrib non-free
deb-src http://security.debian.org/ stable/updates main contrib non-free
deb-src http://security.debian.org/ testing/updates main contrib non-free

With this sources.list file APT will install packages from Unstable. If that is not desired, use the APT::Default-Release setting (see Section 6.2.3, “System Upgrade”) to instruct APT to pick packages from another distribution (most likely Testing in this case).

There are good reasons to include all those repositories, even though a single one should be enough. Testing users will appreciate the possibility to cherry-pick a fixed package from Unstable when the version in Testing is affected by an annoying bug. On the opposite, Unstable users bitten by unexpected regressions have the possibility to downgrade packages to their (supposedly working) Testing version.

The inclusion of Stable is more debatable but it often gives access to some packages which have been removed from the development versions. It also ensures that you get the latest updates for packages which have not been modified since the last stable release.

6.1.3.1. The Experimental Repository

The archive of Experimental packages is present on all Debian mirrors, and contains packages which are not in the Unstable version yet because of their substandard quality — they are often software development versions or pre-versions (alpha, beta, release candidate…). A package can also be sent there after undergoing subsequent changes which can generate problems. The maintainer then tries to uncover them thanks to advanced users who can manage important issues. After this first stage, the package is moved into Unstable, where it reaches a much larger audience and where it will be tested in much more detail.

Experimental is generally used by users who do not mind breaking their system and then repairing it. This distribution gives the possibility to import a package which a user wants to try or use as the need arises. That is exactly how Debian approaches it, since adding it in APT's sources.list file does not lead to the systematic use of its packages. The line to be added is:

deb http://ftp.debian.org/debian experimental main contrib non-free

6.1.4. Non-Official Resources: `apt-get.org` and `mentors.debian.net`

There are numerous non-official sources of Debian packages set up by advanced users who have recompiled some software, by programmers who make their creation available to all, and even by Debian developers who offer pre-versions of their package online. A web site was set up to find these alternative sources more easily. It contains an impressive amount of Debian package sources which can immediately be integrated into sources.list files. However, be careful not to add random packages. Each source is designed for a particular version of Debian (the one used to compile the packages in question); each user should maintain a certain coherence in what they choose to install.

→ http://www.apt-get.org/

The mentors.debian.net site is also interesting, since it gathers source packages created by candidates to the status of official Debian developer or by volunteers who wish to create Debian packages without going through that process of integration. These packages are made available without any guarantee regarding their quality; make sure that you check their origin and integrity and then test them before you consider using them in production.

COMMUNITY The debian.net sites

The debian.net domain is not an official resource of the Debian project. Each Debian developer may use that domain name for their own use. These websites can contain unofficial services (sometimes personal sites) hosted on a machine which does not belong to the project and set up by Debian developers, or even prototypes about to be moved on to debian.org. Two reasons can explain why some of these prototypes remain on debian.net: either no one has made the necessary effort to transform it into an official service (hosted on the debian.org domain, and with a certain guarantee of maintenance), or the service is too controversial to be officialized.

Installing a package means giving root rights to its creator, because they decide on the contents of the initialization scripts which are run under that identity. Official Debian packages are created by volunteers who have been co-opted and reviewed and who can seal their packages so that their origin and integrity can be checked.

In general, be wary of a package whose origin you don't know and which isn't hosted on one of the official Debian servers: evaluate the degree to which you can trust the creator, and check the integrity of the package.

→ http://mentors.debian.net/

6.1.5. Caching Proxy for Debian Packages

When an entire network of machines is configured to use the same remote server to download the same updated packages, any administrator knows that it would be beneficial to have an intermediate proxy acting as a network-local cache (see sidebar VOCABULARY Cache).

You can configure APT to use a "standard" proxy (see Section 6.2.4, “Configuration Options” for the APT side, and Section 11.6, “HTTP/FTP Proxy” for the proxy side), but the Debian ecosystem offers better options to solve this problem. The dedicated software presented in this section are smarter than a plain proxy cache because they can rely on the specific structure of APT repositories (for instance they know when individual files are obsolete or not, and thus adjust the time during which they are kept).

apt-cacher and apt-cacher-ng work like usual proxy cache servers. APT's sources.list is left unchanged, but APT is configured to use them as proxy for outgoing requests.

approx, on the other hand, acts like an HTTP server that “mirrors” any number of remote repositories in its top-level URLs. The mapping between those top-level directories and the remote URLs of the repositories is stored in /etc/approx/approx.conf:

# <name> <repository-base-url>
debian   http://ftp.debian.org/debian
security http://security.debian.org

approx runs by default on port 9999 via inetd (see Section 9.6, “The inetd Super-Server”) and requires the users to adjust their sources.list file to point to the approx server:

# Sample sources.list pointing to a local approx server
deb http://apt.falcot.com:9999/security wheezy/updates main contrib non-free
deb http://apt.falcot.com:9999/debian wheezy main contrib non-free