ZMailer Overview
[ This is mostly from 1988-1990 by Rayan Zachariassen,
some additional notes by Matti Aarnio, 1994-1995 ]
- What is this?
This is a package that implements an internet message transfer agent called
ZMailer. It is intended for gateways or mail servers or other large site
environments that have extreme demands on the abilities of the mailer. It
was motivated by the problems of the Sendmail design in such situations.
ZMailer is intended and designed as a multi-protocol mailer. The only
protocol supported in this distribution is RFC822 (and variations).
- What good will it do me?
Different people and sites differ in priorities. Keep that in mind
when reading this ``features'' list:
Code and Design features:
+ Strong limits on host impact, system loading is deterministic.
+ Secure design (and hopefully implementation).
+ Natural fit for client/server environments.
+ Extremely customizable configuration mechanism.
+ Flexible database interface with support for: sorted files, unsorted
files, dbm, ndbm, gdbm, nis (yellow pages), dns (BIND resolver),
/etc/hosts file, and in-core data.
+ Efficient message queue management.
+ Low-technology implementation.
+ Fast binary-transparent SMTP server and client.
[ It is still binary transparent, but MIME support has slowed
it slightly. Most propably it will still be able to flood
your Ethernet with a 10 MB SMTP transfer, so beware.. ]
Default configuration file features:
+ Default configuration will work for most sites.
+ Network protocol support for: smtp, uucp, bitnet, mail to news.
+ An easy way of overriding any external routing information.
+ Automatic handling of mailing lists.
- What experience shows that it can do?
Original developement systems had loads of 1000-2000 messages
a day (somewhat more route decissions, like 2-3 times that),
but multi-router mode has enabled serious processing to happen
which sends out 20-50 THOUSAND messages a day
On one burst-load test the system did show up to handle about
12500 messages a day per router process (three route lookups
on each message from DNS across the Ethernet).
If system can take more route processes, it is most likely
possible to increase system performance to hundreds of thousands
messages per day.. There is no conclusive evidence that next
possible bottle-neck, scheduler, won't clog, but there appears
to be ample power (if your host has it..) -- 50 000 messages
a day is no problem. ( Sun SS-10/41 ) [ Apr-1994 ]
Another burst-load test routed 1000 messages to "nobody" (via
local alias db to "/dev/null") on an Sun SS-10/50 MHz/Solaris 2.4,
gave speed of 120 000 messages per day per router process (the
test was run with 4 parallel routers)
On same machine with scheduler from zmailer-2.99.15, scheduling
the messages took a bit under two minutes (1:55), which indicates
speed of roughly 750 000 messages per day. Very likely it can
exceed million messages per day (not million recipients, like
when expanding lists, but million individual recipients) [ Aug-1995 ]
The ZMailer is used on various multi-domain machines, including
ones running Internet, ( and BITNET, and/or UUCP ).
There are ways on how to bloat router process memory requirements,
however the exactly same list of addresses in different format
doesn't bloat them much, if at all...
( Using new internal $(listexpand ... ) -function will not bloat
the memory -- see docs/guides/lists )
- How does it work?
ZMailer is a multi-process mailer, using two daemon processes to
manipulate messages. One of these processes is a router (which may
consist of N parallel processes), and makes all decisions about what
should happen to a message. The other daemon is a message queue
manager, used to schedule delivery of messages.
The Router uses a configuration file that closely follows Bourne-
shell script syntax and semantics with minimal magic. Message files
are moved around in a series of directories, and the Scheduler and
its Transport Agents run off of control files created by the Router.
The Router will process messages one at a time, as it finds them in a
directory where User Agents submit their outgoing messages. Envelope
and Message Header information is all kept in the same message file
along with the message body, and this file is never modified by any
ZMailer program. After parsing the envelope and RFC822 header
information, the Router validates the information extracted, and
calls functions defined in the configuration file to decide exactly
how to deliver the message and how to transform the embedded addresses.
The algorithms that do this are easily reconfigurable, since the control
flow and address manipulation is specified by familiar shell script
statements. When the Router is finished, it will produce a message
control file for use by the delivery processing stage of ZMailer, and
move the original message file to another location.
Once the Router has decided what to do with each of the addresses in
a message, the Scheduler builds a summary of this information by reading
the control file created by the Router. This knowledge is merged with
a data structure it maintains that stores which messages are supposed
to be sent where, and how. According to a pre-arranged agenda, the
Scheduler will execute delivery programs to properly move the message
envelope, header, and body, to the immediate destination. These delivery
programs are called Transport Agents, and communicate with the Scheduler
using a simple protocol that tells them which messages to process and
returns status reports to the Scheduler. The Scheduler also manages
status reports, taking appropriate action on delivery errors and when
all delivery instructions for a message have been processed.
There are several standard Transport Agents included with the ZMailer
distribution. The collection currently includes a local delivery program,
an SMTP client implementation, and a Transport Agent that can run
Sendmail-compatible delivery programs.
A separate utility allows querying the Scheduler for the state of its
mail queues. For existing Sendmail installations, a Sendmail replacement
program is included that simulates most of the Sendmail functionality
in the ZMailer environment. This allows ZMailer to replace a working
Sendmail installation without requiring changes in standard User Agents.
If you have a client/server host environment, mail clients need not run
any daemons. They only need two programs (the Sendmail replacement, and
a mail queue querying program) and a view of the mail server's spool area.
- How will it affect my current system?
Two programs will be replaced: /usr/lib/sendmail (with a functionality
emulator), and /bin/rmail (for completeness). Nothing else will be
changed, so it is very easy to back out if you want to.
- What problems can I expect?
This software is in use on several major mail gateways. The development
machine is a departmental system that processes 2000 messages a day.
This mailer has been running on it in some form for 5 years. This means
that you are unlikely to see crippling problems in the software.
[940829: Rayan was optimist.. Or, well, Serious Hacking to get new
MIME-features has caused much bugs to surface.. /mea]
System performance has been proven to handle 50 000 - 100 000 messages
per day, assuming there is enough CPU power and memory. A high-
performance feature developement has a goal of a MILLION messages
per day.
You may well encounter problems due to your local Operating System,
libraries, compiler (this software makes a good compiler test suite),
or other things about your environment that I have not considered.
The ZMailer Router RFC822 implementation is rather picky about adherence
to the protocol standards. You may run into cases where an error that
has been ignorable or unknown so far, will generate automatic complaints
from the mailer to users, and from users to you. Much of this pickyness
is required. Some is not, and is deselectable.
Due to the security mechanism, in particular the propagation and use of
privilege levels associated with addresses, the default setup requires that
the .forward and mailing list files be publically readable. Otherwise,
unprivileged addresses (e.g., those specified from outside the host), will
not be able to use the contents of such files. Similarly, if the source
of an address is not ``secure'', the address will have no privileges.
In particular, for an address to take on the privileges of whomever
specified it (as in a .forward file), its source file must be owner-only
writable, and in a directory which has the same ownership or is owned by
root and is also owner-only writable. These checks may be disabled.
- What level of support can I expect?
I will promptly fix problems that I consider serious if they are reproducible
on a system I have easy access to. Any comments or reports you send in will
be saved and considered. Because of my other activities, I cannot commit
to answering non-critical items in a timely manner, or at all.
[940829: 'I' has latter turned out to be <
[email protected]> ]
... and if you want to pay for support, ask <
[email protected]>
There is a mailing list you can join if you would like the ear of
a community. Send subscription requests to:
<
[email protected]>
- How do I install this?
If you are an SMTP (and possibly UUCP) site running a supported OS,
you can use a supplied host environment description and the default
Router configuration files.
Read the file "INSTALL" for detailed installation information.
Read the file doc/guides/aliases for information on local address
interpretation, and doc/guides/lists for a quick introduction to
setting up mailing lists.
