Path: senator-bedfellow.mit.edu!dreaderd!not-for-mail

Path: senator-bedfellow.mit.edu!dreaderd!not-for-mail
Message-ID: <mail/procmail/[email protected]>
Supersedes: <mail/procmail/[email protected]>
Expires: 2 Jun 2004 12:58:04 GMT
X-Last-Updated: 1999/04/28
Organization: none
From: <[email protected]> (Jari Aalto+mail.procmail)
Subject: Procmail pointers
Summary: This post contains excerpts from Procmail Tips page at
which contains lot of infomation about Procmail, ube (aka spam),
email headers, RFC headers etc. Athor's homepage is at
http://poboxes.com/jari.aalto/
Newsgroups: comp.mail.misc,comp.answers,news.answers
Approved: [email protected]
Followup-To: poster
Precedence: bulk
Originator: [email protected]
Date: 04 May 2004 13:01:16 GMT
Lines: 535
NNTP-Posting-Host: penguin-lust.mit.edu
X-Trace: 1083675676 senator-bedfellow.mit.edu 567 18.181.0.29
Xref: senator-bedfellow.mit.edu comp.mail.misc:79466 comp.answers:57062 news.answers:270895

Archive-name: mail/procmail/pointers
Posting-Frequency: 2 times a month
Maintainer: Jari Aalto <[email protected]>

Announcement: "Procmail resources"

Last updated: 1999-04-21 v1.72 pm-tips.txt

Faq archive is at

http://www.faqs.org/faqs/

The following list below is an excerpt from a bigger document.

http://www.procmail.org/jari/pm-tips.html

My Procmail module library page is at

http://www.procmail.org/jari/pm-tips.html

Era's exellent procmail link page (including procmail FAQ) is at

http://www.iki.fi/~era/procmail/links.html

4.0 Procmail pointers

4.1 Where to get procmail binary

ftp://ftp.informatik.rwth-aachen.de/pub/packages/procmail/

4.2 Where is procmail developed

Philip Guenther <[email protected]> is currently taking care of and
coordinating procmail bug fixes. Please send any procmail bugs to
the mailing list or to <[email protected]>. The development mailing
list is running SmarList at <[email protected]>. Furher
patch and bug info can be found at:

http://www.gac.edu/~guenther/procmail/todo.html
http://www.gac.edu/~guenther/procmail/warts.html

Newest Procmail code:

http://www.procmail.org/
ftp://ftp.procmail.org/

4.3 About procmail's Y2K compliance

Please consult Philip Guenther <[email protected]> for more up to date
details. Philip is the Procmail maintainer currently.

[1998-09-23 Bennett Todd <[email protected]> in Message-Id:
<[email protected]>] Well, from a simple ogle of the
grep over the sources, it looks like there may be a Y2038 problem
in the autoconf test code: unsigned otimet = time(). And another,
possibly less likely to express itself, in formail.c: unsigned long
h1 = time(). Those could express themselves when 32-bit signed
time_t wraps; long before then the time_t define should have been
changed to something that is bigger, even it's "long long". The
above type-mixes may fail to profit from a suitably redefined
time_t, and so may overflow on 2038.

I don't see any Y2K problems, though. And email headers use
four-digit years pretty consistently, so that should all be cool.
This estimation doesn't constitute an in-depth Y2k audit of
procmail, but the source code to procmail is ... kinda dense for
in-depth auditing.

[1998-09-25 Bennett Todd Message-Id:
<[email protected]>] As I see it there are at least
three measures that a whole email system, taken in aggregate, could
use for Y2K checking. First, capture a vast cross-section of
traffic and make sure no email software is using 2-digit years. I
don't recall having seen any, but it's still worth checking.
Second, generate a load of traffic with 2000 and 2001 dates and
shove it through all the channels. And third, run all the systems
end-to-end with their system clocks rolling over the millenium.

4.4 Procmail mailing lists

Traffic in this list is about 5-20 messages per day. Do not join
if you can't handle that much traffic. The list is run by SmartList,
which is a procmail based list software.

._MailingList_: questions/answers <[email protected]>
.subscription requests <[email protected]>
.digest request <[email protected]>

To get off the procmail mailing list

To get off the list: send a message to *procmail-request* with:

unsubscribe user@domain in the subject line
unsubscribe first line in the body

If that fails, try email to
<[email protected]> (purportedly that should
go to a person). See also the original subscriptions message that
you will received http://www.iki.fi/~era/procmail/welcome.txt

4.5 Procmail recipe modules and faqs

Procmail is discussed in usenet newsgroup *comp.mail.misc*.

"Procmail archive"
ftp://ftp.informatik.rwth-aachen.de:/pub/packages/procmail/
Articles from procmail mailing list: covers from 1994-08 to 1995-05
(A .gz file: ~2Meg when uncompressed)

And Latest articles can be found here, hosted by Achim Bohnet
Covers from 1995-10 to the present day.
<[email protected]>. The www page has nice search capabilities.
http://www.rosat.mpe-garching.mpg.de/mailing-lists/procmail/
http://www.rosat.mpe-garching.mpg.de/~ach/exmh/archive/procmail/

"Era's Procmail faq"
http://www.iki.fi/~era/procmail/mini-faq.html
http://www.dcs.ed.ac.uk/~procmail/faq/ [mirror]
Also available by email, the ITEM can be: links.html, mini-faq.html,
procmail-faq

To: <[email protected]>
Subject: send ITEM

"Era's Procmail Link collections"
http://www.iki.fi/~era/procmail/links.html
...A page with full of good links to the world of procmail

"Catherine's Getting Started With Procmail"
http://shell3.ba.best.com/~ariel/nospam/proctut.shtml
This is a quick tutorial intended to get a procmail neophyte
started using procmail with as little trouble and fuss as possible.

"Joe Gross's short Procmail tutorial"
http://www.procmail.net/ <[email protected]> ...Using procmail and a
feature of ph you can set up your own mailing list without
neededing root on your own machine.

"Unix manpages"
http://www.xs4all.nl/~pater/manpages/
...If you don't have procmail manpages at hand, check this site.
It contains wealth of Unix related manpages online.
<[email protected]> Jeroen Paternostre

4.6 Procmail Lint service (code check)

_NOTE_: This service has closed 1999-04-20. It has been mentioned
here for historical reasons only.

If you have and can use Emacs, please download the Procmail
programming mode, `tinypm.el', that [jari] has written. Lint is
included in there and it can auto-correct mistakes on the fly. You
can get it from the mentioned _uta_ ftp site (get tgz kit).

Because not all people know how to use Emacs, how to use Emacs lisp
packages or are otherwise clueless about Unix tools, I put up
a procmail based Lint service where you can send your code.

To: [email protected]
Subject: send pm-lint.hlp

This service is highly experimental and if traffic starts to get
too high, I have to close it because every message to the lint
starts a background Emacs process and it consumes server
resources. The preferred way is that you get your own Emacs package
and Lint your code locally. When you send a message to the Lint
it will respond to you with a message similar to this one:

*** 1997-11-24 22:13 (pm.lint) 3.11pre7 tinypm.el 1.80
cd /users/jaalto/junk/
pm.lint:010: Warning, no right hand variable found. ([$`']
pm.lint:055: Pedantic, flag orer style is not standard `hW:'
pm.lint:060: Warning, message dropped to folder, you need lock.
pm.lint:062: Warning, recipe with "|" may need `w' flag.
pm.lint:073: Warning, Formail used but no `f' flag found.

4.7 Procmail module list

Where to get the modules

The UBE stop procmail modules are not listed here. See pointers in
"procmail code" section later.

o All pm-ja*.rc modules are in Jari's procmail kit. Also
available ay Procmail code library page is at
http://www.netforward.com/poboxes/?jari.aalto --> pm-code.html

o Other modules are by Alan Stbbens http://reality.sgi.com/aks/

o 1998-12-08 Eli the Bearded <*@qz.to> announced in
comp.mail.misc that he had put his own procmail modules to
visible at http://www.qz.to/eli/src/procmail/. You may find
interesting procmail code there but the modules itself are not
general purpose *plug-in* modules that you could use right
away. Some functionality included:

Inline decoding of MIME text attachments (rc.mime-decode)
Cleansing of obscure "Re:" formats in subject (rc.pre-list)
Nifty autoresponder (rc.qz-2)
Sophisticated dupicate email catching (rc.dupes)
Example of using my mail bouncer (rc.lists-out)
Detection of some classes of autoreplies (rc.daemon)
Various junk mail filtering (rc.filter)
Daily log files (rc.vars)

Terminology

*subroutine* = A piece of code that gets something in `INPUT' and
responds with `OUTPUT'. Subroutine is not message specific.

*recipe* = A piece of code, that is somewhat self containing:
It reads something from the message or does something
according to matches in message. Recipe may be message specific.

Foreword to using modules

In the module listing, some of the modules are recipes and some can
be considered subroutines. Let's take the address explode module
that was discussed a while ago. First, visualise following familiar
programming language pseudo code:

(ret-val1, ret-val2 ...) = Function( arg1, arg2, arg3 ...)

*Function* may return multiple arguments and multiple arguments can
be passed to it. Clear so far. Let's show how this applies to
procmail modules:

RC_FUNCTION = $PMSRC/pm-xxx.rc # name the subroutine/module
RC_FUNCTION2 = ...

INPUT = "value" # Set the arg1 for module
INCLUDERC = $RC_FUNCTION # Call Function( $arg1 )

:0 # Examine function ret val
* ERROR ?? yes
...

This should be pretty clear too. You just have to look into the
subroutine/module, which you intend to use, to find out what
arguments it wants which you _need_ _to_ set (INPUT) before calling
it. The documentation also tells you what values are returned, eg.
one of them was ERROR.

If it were recipe/module, the call would be almost the same, but
instead of returning values, the recipe/module most likely does
something to your message or writes something to the data files or
etc. *Recipe/module* is much higher level hierarchy, because it may
call multiple subroutine/modules. The distinction between
subroutine and recipe module type is not crystal clear, but I hope
the above will clarify a bit the Procmail module/subroutine/recipe
concept.

Header file modules

These are like #include .h files in C, they define some common
variables, but do not contain actual code.

o pm-javar.rc -- Defines standard variables: SPC WSPC NSPC SPCL and
perl styled \s \d \D \w \W and \a \A (alphabet only)
o headers.rc -- From Alan's procmail-lib. Define standard regexp
and macros: address, from, to, cc, list_precedence

General modules

o *pm-jafrom.rc* -- Derive FROM field without calling `formail'
unnecessarily. If all fails, use formail.
o *get-from.rc* -- From Alan's procmail-lib. get the "best" from
address. Sets FROM and FRIENDLY, the latter being the "friendly"
user name sans any address.
o *pm-jaaddr.rc* -- Subroutine to extract various email components
from INPUT. Like [email protected], net=com, account=foo...
o *pm-jastore.rc* -- Subroutine for general mailbox delivery.
Define MBOX as the folder where to drop
message and this subroutine will store it appropriately.
Supports single mboxes, ".gz" mbox files, directory files and
MH folders with rcvstore.

Date and time handling

For these, you extract the date from somewhere first and then feed
the string to some of these subroutines:

o *pm-jatime.rc* -- a lowlevel subroutine. Parse time "hh:mm:ss"
from variable INPUT
o *pm-jadate1.rc* -- a lowlevel subroutine. Parse date
"Tue, 31 Dec 1997 19:32:57" from variable INPUT
o *pm-jadate2.rc* -- a lowlevel subroutine. Parse ISO standard date
"1997-11-01 19:32:57" from variable INPUT
o *pm-jadate3.rc* -- a lowlevel subroutine. Parse date
Tue Nov 25 19:32:57 from variable INPUT
o *pm-jadate4.rc* -- Call shell command "date" once to construct RFC
"Tue, 31 Dec 1997 19:32:57" and parse the YY MM HH and other
values. You usually use this subroutine if you can't get the date
anywhere else.

Date and time handling

You use these recipes to get the date directly from the message:

o *pm-jadate.rc* -- higher level recipe. Read date from message's
headers: From_ Received, or call shell `date' if none succeeds.
o *date.rc* -- higher level recipe.
From Alan's procmail-lib: parse date or from headers
Resent-Date:, Date, and From

Forwarding and account modules

o *pm-japop3.rc* -- Pop3 movemail implemented with procmail. You can
send a "pop3" request to move your messages from account X to
account Y. Each message is send separately. This recipe listens
"pop3" requests.
o *pm-jafwd.rc* -- controlling Forward remotedly. You can change the
forward address with a "control message" or turn on/off the
forwarding with a "control message"
o *pm-japing.rc* -- Send short reply to subject containing word
"ping" to show that the account is up and that email address is
valid.
o *correct-addr.rc* -- From alan's procmail lib. To help forward mail
from an OLD address to a NEW address, and do some mailing list
mail management. This recipe file is intended to make it easy
for users to forward their mail from their old address to a new
address, and, at the same time, educate their correspondents
about it by CC'ing them with the mail.

Vacation modules

o *pm-javac.rc* -- A framework for your vacation replies. This
recipe will handle the vacation cache and compose an initial
reply; which you only need to fill in. (Like putting vacation
message to the body)
o *ackmail.rc* -- From Alan's procmail lib. procmail rc to
acknowledge mail (with either a vacation message, or an
acknowledgement)

Message-id based modules

o *pm-jadup.rc* -- Handle duplicate messages by Message-Id.
Store duplicate message to separate folder.
o *dupcheck.rc* -- From Alan's procmail-lib. If the current mail has
a "Message-Id:" header, run the mail through "formail -D",
causing duplicate messages to be dropped. Can use md5 cache.

Cron modules

o *pm-jacron.rc* -- A framework for your daily cron tasks. This
recipe contains all the needed checks to ensure that your
includerc is called whenever a day changes. (Day change is
subject to messages you receive). Your own cron includerc is
run once a day.

Backup modules

o *pm-jabup.rc* -- Save messages to backup directory and keep only N
messages per day. Idea by John Gianni, packaged by Jari. Note:
The implementation will always call shell for each message you
receive; so using this module is not recommended if you get
many messages per day. Instead, use the cron module to clean
the messages' backup directory only once a day, and not everytime
a message arrives.

Confirmation modules

o *pm-jacookie.rc* -- Handle cookie (unique id) confirmations.
Also known as Procmail authentication service (PAS).
This simple procmail module will accept messages only from
user's who have returned a "cookie" key. You can use this to
to protect your mailing list from false "subscribe" messages
or from getting mail from unknown people, typically spammers
who won't send the cookie back to you to "validate" themselves.
Uses subroutine pm-jacookie1.rc, which generates the unique
cookie; CRC 32 by default.
o See also Michelle's confirmation module for SmartList

File Servers

o *pm-jasrv.rc* -- A Mime Procmail file server (MPFS) It contains
all the instructions and supports several MIME encoding types:
text/plain and gzip. The keyword SEND is configurable. You
can set up as many files servers as you need to different
directories by changing the SEND keyword. MPFS supports
password for file access.
o *commands.rc* -- From Alan's procmail-lib, check for commands
in the subject line. Handles commands (send|get)
[help|info|procmail info|procmail lib|procmailrc] and few
others.
o *send-file.rc* is a very simplistic piece of procmail code
to send file (non-MIME support) requested in subject line.
http://www.universe.digex.net/~mbr/unix/send-file.html

Mime modules

o *pm-jamime.rc* -- Subroutine to read MIME headers and put the
mime version, boundary string, content-type information to
variables.
o *pm-jamime-decode.rc* -- recipe to decode quoted-printable
or base64 encoding in the body.
o *pm-jamime-kill.rc* -- Recipe for attachment killing: wipes out the
extra mime cruft leaving only the plain text. Applications for
killing: ms-tnef attachement (MS Explorer 7k),
html attachements (netscape, MS Express) vcard (Netscape),
PCX attachement (Lotus Notes).
o *pm-jamime-save.rc* -- Recipe for saving simple file attachement.
When you receive _ONE_ file attachement in a message, this
recipe can save it to separate directory. The content is
also decoded (base64,qp) while saving to file.

Filtering message body or headers

o *pm-jadaemon.rc* -- Handle DAEMON messages by changing subject to
reflect the a) error reason b) to whom the message was originally
sent c) original subject sent and what was the subject. Store the
DAEMON messages to separate folder.
o *pm-jasubject.rc* -- Standardize Subject "Re[32]: FW: Sv: message"
or any other derivate to de facto "Re: message"
o *pm-janetmind.rc* -- Reformat http://minder.netmind.com/ messages,
The default 4k message is shortened to few important lines.

Miscellaneus modules

o *pm-jaempty.rc* -- check if message body is empty (nothing
relevant) Define variable BODY_EMPTY to "yes" or "no" if
message is empty.
o *pm-janslookup.rc* -- Run nslookup on given address. If you
compose return address with "formail -rt -x To:" you can
verify if domain is registered before sending reply. Uses cache
for already looked up domains.
o *guess-mua.rc* -- Guess the Mail User Agent and set MUA:
MH,PINE,MAIL

Mailing list modules

o *Microlist* a small mailing list by david hunt <[email protected]>
...This version contains vars set for my environment and needs,
and requires resetting of those vars before use. It's exact
function and use will remain a mystery until I get a readme
file written for it. If anyone wants to use it, I suggest you
write to me first. If anyone has any suggestions or criticisms
(no matter how harsh) please write
http://www.west.net/~dh/homedir/microlist/microlist4.3
o *pm-jalist.rc* -- Subroutine to extract mailing list name from
message. Do you need to add new recipe to your .procmailrc
every time you subscribe to new mailing list? If you do,
take a look at thsi module, which examines the message and
defines variable `LIST' to hold the maling list name. You
can use it directly to save the messages adaptively to
correct folders. No more hand work and manual storing
of mailing list messages.

4.8 Where to get Procmail code and modules

"Alan's procmail modules"
Send subject "send procmail library" to Alan Stebbens
<[email protected]> http://reality.sgi.com/aks/

"pm-code, Jari's Procmail modules"
http://www.netforward.com/poboxes/?jari.aalto See pm-code.zip

"Elijah's"
http://www.qz.to/~eli/src/procmail/rc.master.html

"Concordia scripts"
http://alcor.concordia.ca/topics/email/auto/procmail/
...We provide sample sets of recipes to get you started. The great
thing about the concordia scripts is the fact that they are
designed to run from a central location and be called from a
.procmailrc installed in the user's ~/home directory.
<[email protected]>

"Meng on procmail"
http://icg.resnet.upenn.edu/procmail/
http://res2.resnet.upenn.edu/procmail/
...goes into exhaustive detail about how I manage my mailing lists

"David's" David Hunt <[email protected]>
http://www.west.net/~dh/homedir/pmdir/
...My .procmailrc and .forward files can be viewed at

4.9 Procmail code to filter UBE

_Sysadms_ _remember_ : Spam filtering is much more efficiently done
in the MTA, especially if you just looking at From and To lines.
For example, I you can setup in Exim a rule that blocks \d.*@aol\.com
(that is any aol.com local part that begins with a digit). AOL
guarantees that _none_ of their addresses being with a digit. Exim
rejects such bogus addresses at the SMTP level before the message
is received.

"Daniel's smap filter"
1997-09-13 Daniel Smith <[email protected]> sent excellent spam filter
called `spamc.rc'. It used some nice heuristics and filters from
various people, icluding [david] and [philip].
Later Dan made substantial changes to it and the new version is
available from ftp://ftp.bristol.nl/pub/users/DanS/spamcheck

"pm-jaube.rc Jari's ube filter (compiled from others)"
After Daniel Smith posted his spam recipes to procmail mailing
list, Jari investigated them and compiled other recipes to a
general purpose UBE module that needs no special setup and can be
installed via simple INCLUDERC. No additional ube-list files are
used, all UBE all detected happens using procmail rules. The module
is included in kit `pm-code.zip'.

"Catherine A. Hampton's Spambouncer"
http://www.best.com/~ariel/nospam/
...The attached set of procmail recipes/filters, which I call
The Spam Bouncer, are for users who are sick of spam (unsolicited
junk mail email) and want to filter it out of their mail as easily
as possible. These recipes can be used as shared recipes for a
whole system, or by an individual for their own mailbox only.

"Protect yourself from spam: A practical guide to procmail"
http://www.sun.com/sunworldonline/swol-12-1997/swol-12-spam.html
...take you, step by step, through everything you need to know in
order to enlist the aid of a Unix host in filtering unwanted e-mail
traffic. This page is excellent to get you started with procmail
and filtering with simple recipes and how to store messages to
folders. Recommended for newcomers to Procmail.

"Junkfilter" by Gregory Sutter <[email protected]>
http://www.pobox.com/~gsutter/junkfilter/
...Junkfilter is a user-configurable procmail-based filter system
for electronic mail. Recipes include checks for forged headers,
key words, common spam domains, relay servers and many others.

"Download procmail spam filters"
http://www.telebyte.com/stopspamr
This is excellent site and contains many other spam stop pointers.

"SpamDunk"
http://www.interlog.com/~waltdnes
http://www.interlog.com/~waltdnes/beta/techie.htm
...This webpage shows a commented example of a working .procmailrc
file that works for me. I have tried to make things as generic as
possible, but there are no guarantees that it will work for anyone
else.