Path: senator-bedfellow.mit.edu!dreaderd!not-for-mail

Path: senator-bedfellow.mit.edu!dreaderd!not-for-mail
Message-ID: <mail/procmail/[email protected]>
Supersedes: <mail/procmail/[email protected]>
Expires: 2 Jun 2004 12:58:04 GMT
X-Last-Updated: 2002/07/11
Organization: none
From: <[email protected]> (Jari Aalto+mail.procmail)
Subject: Procmail tips page pointer
Summary: This mail contains pointer to Procmail Tips page
at http://www.procmail.org/ which answers most of your procmail
questions about recipes, handling spam, autoreply, calling shell programs
Newsgroups: comp.mail.misc,comp.answers,news.answers
Approved: [email protected]
Followup-To: comp.mail.misc
Precedence: bulk
Originator: [email protected]
Date: 04 May 2004 13:01:16 GMT
Lines: 460
NNTP-Posting-Host: penguin-lust.mit.edu
X-Trace: 1083675676 senator-bedfellow.mit.edu 566 18.181.0.29
Xref: senator-bedfellow.mit.edu comp.mail.misc:79465 comp.answers:57061 news.answers:270894

Archive-name: mail/procmail/tips-pointer
Posting-Frequency: 2 times a month
Maintainer: Jari Aalto <[email protected]>
Last-Modified: $Docid: 2002-07-11 Jari Aalto $

Announcement: "Procmail tips page: all you wanted to know about procmail"

Faq archive is at:

http://www.faqs.org/faqs/

The Procmail documentation project is at

http://pm-doc.sourceforge.net/

Other procmail documents

Era's exellent procmail pages (including procmail faq) are at:

http://www.iki.fi/~era/procmail

Another exellent email filtering tips and recipes (procmail)
Page by Prof. Timo Salmi at

http://www.uwasa.fi/~ts/info/proctips.html

What is Procmail?

Procmail is a mail processing utility, which can help you
filter your mail; sort incoming mail according to sender, Subject
line, length of message, keywords in the message, etc; implement an
ftp-by-mail server, and much more. Procmail is also a complete
drop-in replacement for your MDA. (If this doesn't mean anything to
you, you don't want to know.)

Procmail runs under Unix. See Infinite Ink's Mail Filtering and
Robots page for information about related utilities for various other
platforms, and competing Unix programs, too (there aren't that many
of either).

Description

The procmail tips page contains collection of procmail recipes,
instructions, howtos. The document also contains URL pointers to
the procmail mailing list and sites that fight against Internet
UBE. You will also find many other interesting subjects that
discuss about internet email: haeders and RFCs.

The tips are compiled from the procmail discussion list,
from comp.mail.misc and from the author's own experiences with
procmail.

This document does not teach you basics of procmail; instead you
have to be familiar with the procmail man pages already before this
document's tips are useful to you.

Table of contents as of 2002-07-11

1.0 Document id
1.1 General
1.2 What is Procmail?
1.3 Abbreviations and thanks
1.4 Version information
1.5 Document layout and maintenance
1.5.1 Sending improvements
1.6 About presented recipes
1.7 Variables used in recipes
1.8 About "useless use of cat award"

2.0 Procmail pointers
2.1 Where is procmail developed
2.2 About procmail's Y2K compliance
2.3 Procmail resources
2.4 Procmail mode for Emacs
2.5 Procmail module library project
2.5.1 Where to get the modules
2.5.2 Terminology
2.5.3 Foreword to using modules
2.5.4 Header file modules
2.5.5 General modules
2.5.6 Low-level Date and time handling
2.5.7 Higher-level Date and time handling
2.5.8 Forwarding and account modules
2.5.9 Vacation modules
2.5.10 Message-id based modules
2.5.11 Cron modules
2.5.12 Backup modules
2.5.13 Confirmation modules
2.5.14 File Servers
2.5.15 Mime modules
2.5.16 Filtering message body or headers
2.5.17 Miscellaneous modules
2.5.18 Mailing list modules
2.6 Procmail code to filter UBE

3.0 Dry run testing
3.1 What is dry run testing
3.2 Why the From field is not okay after dry run
3.3 Getting default value of a procmail variable

4.0 Things to remember
4.1 Get the newest procmail
4.2 Csh's tilde is not supported
4.3 Be sure to write the recipe starting right
4.4 Always set SHELL
4.4.1 If system has no /bin/sh and you're forced to use csh/tcsh
4.4.2 Procmail won't work well with SHELL set to csh derivate
4.5 Check and set PATH
4.6 Keep the log on all the time
4.7 Never add a trailing slash for directories
4.8 Remember what term DELIVERED means
4.9 Beware putting comment in wrong place
4.10 Brace placement
4.11 Local lockfile usage
4.12 Global lockfile
4.13 Gee, where do I put all those ! * $ ??
4.14 Sending automatic reply, use X-loop header
4.15 Avoid extra shell layer (check command for SHELLMETAS)
4.16 Think what shell commands you use
4.17 Using absolute paths when calling a shell program
4.18 Disabling a recipe temporarily
4.19 Keep message backup, no matter what
4.20 Order of the procmail recipes

5.0 Procmail flags
5.1 The order of the flags
5.2 Flag w and recipe with |
5.3 Flag w, lock file and recipe with |
5.4 Flag f and w together
5.5 Flags h and b
5.6 Flag h and sinking to /dev/null
5.7 Flag i and pipe flag f
5.8 Flag r
5.9 Flag c's background
5.10 Flag c before nested block forks a child
5.11 Flag c and understanding possible forking penalty
5.12 Flags before nested block
5.13 Flags aAeE tutorial

6.0 Matching and regexps (regular expressions)
6.1 Philosophy of abstraction in regexps
6.2 Matches are not case-sensitive
6.3 Procmail uses multi line matches
6.4 Headers are unfolded before matching
6.5 Improving Space-Tab syndrome
6.6 Handling exclamation character
6.7 Rules for generating a character class
6.8 Matching space at the end of condition
6.9 Beware leading backslash
6.10 Correct use of TO Macro
6.11 Procmail's regexp engine
6.12 Procmail and egrep differences
6.13 Understanding procmail's minimal matching (stingy vs. greedy)
6.14 Explaining \/ and ()\/
6.15 Explaining ^^ and ^
6.16 ANDing traditionally
6.17 ORing traditionally
6.18 ORing and score recipe
6.19 ORing by using De Morgan rules

7.0 Variables
7.1 Setting and unsetting variables
7.2 Variable initialization and sh syntax
7.3 Testing variables
7.4 What does $\VAR mean?
7.5 Common pitfalls when using variables
7.6 Quoting: Using single or double quotes
7.7 Quoting: Passing values to an external program
7.8 Passing values from an external program
7.9 Incrementing a variable by a value N
7.10 Comparing values
7.11 Strings: How many characters are there in a given string?
7.12 Strings: How to strip trailing newline.
7.13 Strings: deriving the last N characters of a string.
7.14 Strings: Getting partial matches from a string.
7.15 Strings: Procmail string manipulation example
7.16 How to raise a flag if the message was filed
7.17 Dollar sign in condition lines.
7.18 Finding mysterious foo variable
7.19 Storing code to variable
7.20 Getting headers into a variable.
7.21 Converting value to lowercase

8.0 Suggestions and miscellaneous
8.1 Speeding up procmail
8.2 See the procmail installation's examples
8.3 Printing statistics of your incoming mail
8.4 Storing UBE mailboxes outside of quota
8.5 Using first 5-30 lines from the message
8.6 Using cat or echo in scripts?
8.7 How to run an extra shell command as a side effect?
8.8 Forcing "ok" return status from shell script
8.9 Make your own .procmailrc available to others
8.10 Using dates efficiently
8.11 Keep simple header log
8.12 Gzipping messages
8.13 Emergency stop for your .procmailrc

9.0 Scoring
9.1 Using scores by an example
9.2 Brief Score tutorial
9.3 Score's scope
9.4 Counting length of a string
9.5 Counting lines in a message (Adding Lines: header)
9.6 Determining if body is longer than header
9.7 Matching last Received header
9.8 How to add Content-Length header
9.9 Testing message size or number of lines
9.10 Counting commas with recursive includerc

10.0 Formail usage
10.1 Fetching fields with formail -x
10.2 Always use formail's -rt switch
10.2.1 For procmail versions prior 3.14
10.3 Using -rt and rewriting the From address
10.4 Formail -rt and Resent-From header
10.5 Quoting the message
10.6 Without quoting the message
10.7 How to include headers and body to the reply message
10.8 Adding text to the beginning of message
10.9 Adding text to the end of message
10.10 Adding text before quoted message
10.11 Adding extra headers from file
10.12 Splitting digest
10.13 Mailbox: Splitting to individual files
10.14 Mailbox: Extracting all From addresses from mailbox
10.15 Mailbox: Applying procmail recipe on whole mailbox
10.16 Mailbox: run series of commands for each mail (split mailbox)
10.17 Option -D and cache
10.18 Option -D and message-id in the body
10.19 Reducing formail calls (conditionally adding fields)
10.20 Formail -A -a options
10.21 Formail -e -s options

11.0 Saving mailing list messages
11.1 Using subroutine pm-jalist.rc to detect mailing lists
11.2 Using plus addressing [email protected]
11.3 Using RFC comment trick for additional information
11.4 Simple mailing list handling
11.5 Archiving according to TO
11.6 Using Return-Path to detect mailing lists

12.0 Procmail, MIME and HTML
12.1 Mime Bibliography
12.2 Mime notes
12.3 Software to deal with mime or HTML
12.4 Mime content type application/ms-tnef
12.5 Trapping HTML mime messages
12.6 Complaining about HTML messages
12.7 Converting HTML body to plain text
12.8 Getting rid of unwanted mime attachments (HTML, vcard)
12.9 Sending contents of a HTML page in plain text to someone

13.0 Simple recipe examples
13.1 Saving: MH folders -- numbered messages
13.2 Saving: to monthly folders
13.3 Modifying: Filtering basics
13.4 Modifying: Squeezing empty lines around message body
13.5 Modifying: shuffling headers always to same order
13.6 Service: Auto answerer to empty messages
13.7 Service: File server -- send file as attachments upon request
13.8 Service: Ping responder
13.9 Service: simple vacation with procmail
13.10 Service: vacation code example
13.11 Service: Auto-forwarding
13.12 Service: forward only specific messages
13.13 Service: Making digests
13.14 Kill: killing advertisement headers and footers
13.15 Kill: simple kill file recipe with procmail
13.16 Kill: duplicate messages
13.17 Kill: spam filter with simple recipes
13.18 Kill: (un)subscribe messages
13.19 Time: Once a day cron-like job
13.20 Time: Running a recipe at a given time
13.21 Time: Triggering mail and using cron
13.22 Decoding: Uudecode
13.23 Decoding: MIME
13.24 How to send commands in the message's body
13.25 Matching two words on a line, but not one
13.26 How to define personal XX macros?
13.27 How to change subject by body match
13.28 How to change Subject according to some other header
13.29 How to call program with parameters

14.0 Miscellaneous recipes
14.1 Matching valid Message-Id header
14.2 Sending two files in a message
14.3 Excessive quoting of message
14.4 Sending message to pager in chunks
14.5 Playing particular sound when message arrives
14.6 Combining multiple Original-Cc and Original-To headers
14.7 Forwarding sensitive messages in encrypted format

15.0 Procmail and PGP
15.1 Decrypt pgp messages automatically
15.2 Getkeys from key server
15.3 Auto grab incoming pgp keys

16.0 Includerc usage
16.1 Using: multiple rc files
16.2 Using: You can call rc file conditionally
16.3 Autoloading an rc file
16.4 Making: naming of the rc file
16.5 Making: Using name space when saving procmail variables
16.6 Making: Public and private variables in rc file
16.7 The rules of thumb for constructing general purpose rc file
16.8 An includerc skeleton

17.0 Mailing list server

18.0 Common troubles
18.1 Procmail modes: normal, delivery, and mail filter.
18.2 Procmail as sendmail Mlocal mail filtering device
18.3 Procmail doesn't pass 8bit characters
18.4 My ISP isn't very interested in installing procmail
18.5 My ISP has systemwide procmailrc; is this a good idea?
18.6 Procmail changes mailbox and directory permissions
18.7 Changing mbox permission during compilation to 660
18.8 The .forward file must be real file
18.9 Using .forward if procmail already is LDA
18.10 Mail should be put in the mailqueue if write fails
18.11 Qmail: how to make it work with procmail
18.12 Qmail: Procmail looks file from /var/spool/mail only
18.13 Qmail: patch to procmail 3.11pre7 to work with Maildirs
18.14 AFS: How to use Procmail when HOME is in AFS cell
18.15 Help, some idiot sent my address to 30 mailing lists
18.16 Help, Procmail beeps and prints to my console
18.17 Help, procmail dumps mail to console
18.18 Help, corrupted From_ line in mailbox
18.19 Directing user's mail to HOME instead of /var/spool/
18.20 NFS mounting /var/mail is a good way to get bad performance
18.21 I can't see the sendmail's response in LOGFILE
18.22 Compiling procmail and choosing locking scheme
18.23 Forwarding lot of mail causes heavy load
18.24 What happens to mail if MDA Procmail fails
18.25 Procmail reads entire 90Mb message into memory
18.26 Help, procmail uses occasionally huge chunk of memory
18.27 Procmail signaled out of memory in my verbose log
18.28 Variables DEFAULT and ORGMAIL
18.29 When DEFAULT cannot be mailed to
18.30 Variable DROPPRIVS
18.31 Variable HOME
18.32 Variable HOST
18.33 Variable LINEBUF
18.34 Variable LOG and LOGFILE
18.35 Variable TRAP
18.36 Variable UMASK
18.37 UMASK and permissions
18.38 Performance difference between back tick and "|" recipe
18.39 Procmail's temporary file names while writing file out
18.40 Parameter $@
18.41 Procmail variables are null terminated (detecting null string)
18.42 FROM_DAEMON TO and TO_ and case-sensitiveness
18.43 TO_ macro deciphered
18.44 TO_ macro and RFC 822
18.45 FROM_DAEMON deciphered

19.0 Technical matters
19.1 List of exit codes
19.2 List of precedence codes
19.3 Sendmail and -t
19.4 RFC822 Reply-To and formail problem with multiple recipients
19.5 Procmail and IMAP server
19.6 Machine which processes mail
19.7 Compiling procmail and MAILSPOOLHOME

20.0 Smartlist
20.1 MLM RFC
20.2 Other mailing list software
20.3 SmartList code (mailing list implementation with procmail)
20.4 Installation trouble: getparams
20.5 Accepting mail only from users in whitelist(s)

21.0 Additional procmail or MUA software
21.1 Comstat to handle multiple mailboxes
21.2 Elm and pgp support (Mutt is the successor to elm.)
21.3 MH sites

22.0 Additional procmail software for Emacs
22.1 What is Emacs
22.2 Emacs and procmail mode and Lint
22.3 Emacs and lining up backslashes
22.4 Emacs and browsing mailbox files
22.5 Emacs and live-mode.el
22.6 Emacs and font-lock.el

23.0 Procmail, Emacs and Gnus
23.1 Gnus pointers
23.2 Why use procmail with Gnus
23.3 Setting up gnus for procmail - Basics
23.4 Gnus for procmail - More gnus
23.5 Emacs and Gnus -- Fiddling with spool files
23.6 Gnus and article snippets
23.7 Emacs GNUS - POP - Procmail

24.0 RFC, Request for comments
24.1 RFCs and their jurisdiction (munged Addresses)
24.2 Comments about addresses munging
24.3 RFC and valid mail address characters
24.4 RFC and login-name@fdqn
24.5 RFCs and messages signature
24.6 RFC and using MIME in Usenet newsgroups
24.7 Some RFC Pointers

25.0 Introduction to E-mail Headers
25.1 To find out more about mail (Resources)
25.2 Lecture by Alan Stebbens
25.3 Applied to received messages
25.4 Bcc lecture by Alan Stebbens
25.5 Bcc lecture by Philip Guenther

26.0 Message headers
26.1 What is correct From address syntax
26.2 What's that X-UIDL header?
26.3 What is that first From_ header?
26.4 Message-Id header
26.5 Received header
26.6 Return-Path
26.7 Errors-To
26.8 X-Subscription-Info
26.9 Reply-To header
26.10 Mail-Copies-To header
26.11 Mail-Followup-To and Reply-To-Personal headers
26.12 Content-Length header and From_ specification
26.13 Moral about CC copies in Usenet

27.0 Other interesting code
27.1 Misc mail related pointers
27.2 Expire mail pointers
27.3 Usenet News related pointers
27.4 Code: Perl Extract procmail man pages from 3.11pre7.tar.gz
27.5 Code: Sh remove matching lines from file

28.0 UBE in Internet
28.1 Terms used and foreword
28.2 UBE strategies
28.2.1 4g. I asked to be "removed" - guess what? I got another U*E
28.2.2 4h. I asked to be "removed" - guess what? The message bounced
28.3 UBE and bouncing message back
28.4 UBE and "I don't mind" attitude
28.5 We need a law against UBE

29.0 Anti-UBE pointers
29.1 NoCEM, CAUCE and others
29.2 General Filtering pages (more than procmail)
29.3 Junk mail and spam
29.4 Comprehensive list of spammers
29.5 Misc pointers
29.6 Questionable UBE stop services
29.7 UBE related newsgroups or mailing lists
29.8 Software: the net abuse page
29.9 Software: adcomplain -- Perl junk mail report
29.10 Software: Ricochet -- Perl junk mail report
29.11 Software: yell -- perl
29.12 Software: RBL lookup tool -- C
29.13 Software: mapSoN
29.14 Software: spamgard
29.15 Software: Spam Be Gone
29.16 Software: TinyGnus - Emacs Gnus plug-in