From: (Jari Aalto+mail.procmail) Subject: Procmail tips page pointer Summary: This mail contains pointer to Procmail Tips page at http://www.procmail.org/ which answers most of your procmail questions about recipes, handling spam, autoreply, calling shell programs Newsgroups: comp.mail.misc, comp.answers, news.answers Followup-To: comp.mail.misc Date: 31 Dec 2003 02:02:23 GMT Organization: none Archive-name: mail/procmail/tips-pointer Posting-Frequency: 2 times a month Maintainer: Jari Aalto Last-Modified: $Docid: 2002-07-11 Jari Aalto $ Announcement: "Procmail tips page: all you wanted to know about procmail" Faq archive is at: http://www.faqs.org/faqs/ The Procmail documentation project is at http://pm-doc.sourceforge.net/ Other procmail documents Era's exellent procmail pages (including procmail faq) are at: http://www.iki.fi/~era/procmail Another exellent email filtering tips and recipes (procmail) Page by Prof. Timo Salmi at http://www.uwasa.fi/~ts/info/proctips.html What is Procmail? Procmail is a mail processing utility, which can help you filter your mail; sort incoming mail according to sender, Subject line, length of message, keywords in the message, etc; implement an ftp-by-mail server, and much more. Procmail is also a complete drop-in replacement for your MDA. (If this doesn't mean anything to you, you don't want to know.) Procmail runs under Unix. See Infinite Ink's Mail Filtering and Robots page for information about related utilities for various other platforms, and competing Unix programs, too (there aren't that many of either). Description The procmail tips page contains collection of procmail recipes, instructions, howtos. The document also contains URL pointers to the procmail mailing list and sites that fight against Internet UBE. You will also find many other interesting subjects that discuss about internet email: haeders and RFCs. The tips are compiled from the procmail discussion list, from comp.mail.misc and from the author's own experiences with procmail. This document does not teach you basics of procmail; instead you have to be familiar with the procmail man pages already before this document's tips are useful to you. Table of contents as of 2002-07-11 1.0 Document id 1.1 General 1.2 What is Procmail? 1.3 Abbreviations and thanks 1.4 Version information 1.5 Document layout and maintenance 1.5.1 Sending improvements 1.6 About presented recipes 1.7 Variables used in recipes 1.8 About "useless use of cat award" 2.0 Procmail pointers 2.1 Where is procmail developed 2.2 About procmail's Y2K compliance 2.3 Procmail resources 2.4 Procmail mode for Emacs 2.5 Procmail module library project 2.5.1 Where to get the modules 2.5.2 Terminology 2.5.3 Foreword to using modules 2.5.4 Header file modules 2.5.5 General modules 2.5.6 Low-level Date and time handling 2.5.7 Higher-level Date and time handling 2.5.8 Forwarding and account modules 2.5.9 Vacation modules 2.5.10 Message-id based modules 2.5.11 Cron modules 2.5.12 Backup modules 2.5.13 Confirmation modules 2.5.14 File Servers 2.5.15 Mime modules 2.5.16 Filtering message body or headers 2.5.17 Miscellaneous modules 2.5.18 Mailing list modules 2.6 Procmail code to filter UBE 3.0 Dry run testing 3.1 What is dry run testing 3.2 Why the From field is not okay after dry run 3.3 Getting default value of a procmail variable 4.0 Things to remember 4.1 Get the newest procmail 4.2 Csh's tilde is not supported 4.3 Be sure to write the recipe starting right 4.4 Always set SHELL 4.4.1 If system has no /bin/sh and you're forced to use csh/tcsh 4.4.2 Procmail won't work well with SHELL set to csh derivate 4.5 Check and set PATH 4.6 Keep the log on all the time 4.7 Never add a trailing slash for directories 4.8 Remember what term DELIVERED means 4.9 Beware putting comment in wrong place 4.10 Brace placement 4.11 Local lockfile usage 4.12 Global lockfile 4.13 Gee, where do I put all those ! * $ ?? 4.14 Sending automatic reply, use X-loop header 4.15 Avoid extra shell layer (check command for SHELLMETAS) 4.16 Think what shell commands you use 4.17 Using absolute paths when calling a shell program 4.18 Disabling a recipe temporarily 4.19 Keep message backup, no matter what 4.20 Order of the procmail recipes 5.0 Procmail flags 5.1 The order of the flags 5.2 Flag w and recipe with | 5.3 Flag w, lock file and recipe with | 5.4 Flag f and w together 5.5 Flags h and b 5.6 Flag h and sinking to /dev/null 5.7 Flag i and pipe flag f 5.8 Flag r 5.9 Flag c's background 5.10 Flag c before nested block forks a child 5.11 Flag c and understanding possible forking penalty 5.12 Flags before nested block 5.13 Flags aAeE tutorial 6.0 Matching and regexps (regular expressions) 6.1 Philosophy of abstraction in regexps 6.2 Matches are not case-sensitive 6.3 Procmail uses multi line matches 6.4 Headers are unfolded before matching 6.5 Improving Space-Tab syndrome 6.6 Handling exclamation character 6.7 Rules for generating a character class 6.8 Matching space at the end of condition 6.9 Beware leading backslash 6.10 Correct use of TO Macro 6.11 Procmail's regexp engine 6.12 Procmail and egrep differences 6.13 Understanding procmail's minimal matching (stingy vs. greedy) 6.14 Explaining \/ and ()\/ 6.15 Explaining ^^ and ^ 6.16 ANDing traditionally 6.17 ORing traditionally 6.18 ORing and score recipe 6.19 ORing by using De Morgan rules 7.0 Variables 7.1 Setting and unsetting variables 7.2 Variable initialization and sh syntax 7.3 Testing variables 7.4 What does $\VAR mean? 7.5 Common pitfalls when using variables 7.6 Quoting: Using single or double quotes 7.7 Quoting: Passing values to an external program 7.8 Passing values from an external program 7.9 Incrementing a variable by a value N 7.10 Comparing values 7.11 Strings: How many characters are there in a given string? 7.12 Strings: How to strip trailing newline. 7.13 Strings: deriving the last N characters of a string. 7.14 Strings: Getting partial matches from a string. 7.15 Strings: Procmail string manipulation example 7.16 How to raise a flag if the message was filed 7.17 Dollar sign in condition lines. 7.18 Finding mysterious foo variable 7.19 Storing code to variable 7.20 Getting headers into a variable. 7.21 Converting value to lowercase 8.0 Suggestions and miscellaneous 8.1 Speeding up procmail 8.2 See the procmail installation's examples 8.3 Printing statistics of your incoming mail 8.4 Storing UBE mailboxes outside of quota 8.5 Using first 5-30 lines from the message 8.6 Using cat or echo in scripts? 8.7 How to run an extra shell command as a side effect? 8.8 Forcing "ok" return status from shell script 8.9 Make your own .procmailrc available to others 8.10 Using dates efficiently 8.11 Keep simple header log 8.12 Gzipping messages 8.13 Emergency stop for your .procmailrc 9.0 Scoring 9.1 Using scores by an example 9.2 Brief Score tutorial 9.3 Score's scope 9.4 Counting length of a string 9.5 Counting lines in a message (Adding Lines: header) 9.6 Determining if body is longer than header 9.7 Matching last Received header 9.8 How to add Content-Length header 9.9 Testing message size or number of lines 9.10 Counting commas with recursive includerc 10.0 Formail usage 10.1 Fetching fields with formail -x 10.2 Always use formail's -rt switch 10.2.1 For procmail versions prior 3.14 10.3 Using -rt and rewriting the From address 10.4 Formail -rt and Resent-From header 10.5 Quoting the message 10.6 Without quoting the message 10.7 How to include headers and body to the reply message 10.8 Adding text to the beginning of message 10.9 Adding text to the end of message 10.10 Adding text before quoted message 10.11 Adding extra headers from file 10.12 Splitting digest 10.13 Mailbox: Splitting to individual files 10.14 Mailbox: Extracting all From addresses from mailbox 10.15 Mailbox: Applying procmail recipe on whole mailbox 10.16 Mailbox: run series of commands for each mail (split mailbox) 10.17 Option -D and cache 10.18 Option -D and message-id in the body 10.19 Reducing formail calls (conditionally adding fields) 10.20 Formail -A -a options 10.21 Formail -e -s options 11.0 Saving mailing list messages 11.1 Using subroutine pm-jalist.rc to detect mailing lists 11.2 Using plus addressing foo+bar@address.com 11.3 Using RFC comment trick for additional information 11.4 Simple mailing list handling 11.5 Archiving according to TO 11.6 Using Return-Path to detect mailing lists 12.0 Procmail, MIME and HTML 12.1 Mime Bibliography 12.2 Mime notes 12.3 Software to deal with mime or HTML 12.4 Mime content type application/ms-tnef 12.5 Trapping HTML mime messages 12.6 Complaining about HTML messages 12.7 Converting HTML body to plain text 12.8 Getting rid of unwanted mime attachments (HTML, vcard) 12.9 Sending contents of a HTML page in plain text to someone 13.0 Simple recipe examples 13.1 Saving: MH folders -- numbered messages 13.2 Saving: to monthly folders 13.3 Modifying: Filtering basics 13.4 Modifying: Squeezing empty lines around message body 13.5 Modifying: shuffling headers always to same order 13.6 Service: Auto answerer to empty messages 13.7 Service: File server -- send file as attachments upon request 13.8 Service: Ping responder 13.9 Service: simple vacation with procmail 13.10 Service: vacation code example 13.11 Service: Auto-forwarding 13.12 Service: forward only specific messages 13.13 Service: Making digests 13.14 Kill: killing advertisement headers and footers 13.15 Kill: simple kill file recipe with procmail 13.16 Kill: duplicate messages 13.17 Kill: spam filter with simple recipes 13.18 Kill: (un)subscribe messages 13.19 Time: Once a day cron-like job 13.20 Time: Running a recipe at a given time 13.21 Time: Triggering mail and using cron 13.22 Decoding: Uudecode 13.23 Decoding: MIME 13.24 How to send commands in the message's body 13.25 Matching two words on a line, but not one 13.26 How to define personal XX macros? 13.27 How to change subject by body match 13.28 How to change Subject according to some other header 13.29 How to call program with parameters 14.0 Miscellaneous recipes 14.1 Matching valid Message-Id header 14.2 Sending two files in a message 14.3 Excessive quoting of message 14.4 Sending message to pager in chunks 14.5 Playing particular sound when message arrives 14.6 Combining multiple Original-Cc and Original-To headers 14.7 Forwarding sensitive messages in encrypted format 15.0 Procmail and PGP 15.1 Decrypt pgp messages automatically 15.2 Getkeys from key server 15.3 Auto grab incoming pgp keys 16.0 Includerc usage 16.1 Using: multiple rc files 16.2 Using: You can call rc file conditionally 16.3 Autoloading an rc file 16.4 Making: naming of the rc file 16.5 Making: Using name space when saving procmail variables 16.6 Making: Public and private variables in rc file 16.7 The rules of thumb for constructing general purpose rc file 16.8 An includerc skeleton 17.0 Mailing list server 18.0 Common troubles 18.1 Procmail modes: normal, delivery, and mail filter. 18.2 Procmail as sendmail Mlocal mail filtering device 18.3 Procmail doesn't pass 8bit characters 18.4 My ISP isn't very interested in installing procmail 18.5 My ISP has systemwide procmailrc; is this a good idea? 18.6 Procmail changes mailbox and directory permissions 18.7 Changing mbox permission during compilation to 660 18.8 The .forward file must be real file 18.9 Using .forward if procmail already is LDA 18.10 Mail should be put in the mailqueue if write fails 18.11 Qmail: how to make it work with procmail 18.12 Qmail: Procmail looks file from /var/spool/mail only 18.13 Qmail: patch to procmail 3.11pre7 to work with Maildirs 18.14 AFS: How to use Procmail when HOME is in AFS cell 18.15 Help, some idiot sent my address to 30 mailing lists 18.16 Help, Procmail beeps and prints to my console 18.17 Help, procmail dumps mail to console 18.18 Help, corrupted From_ line in mailbox 18.19 Directing user's mail to HOME instead of /var/spool/ 18.20 NFS mounting /var/mail is a good way to get bad performance 18.21 I can't see the sendmail's response in LOGFILE 18.22 Compiling procmail and choosing locking scheme 18.23 Forwarding lot of mail causes heavy load 18.24 What happens to mail if MDA Procmail fails 18.25 Procmail reads entire 90Mb message into memory 18.26 Help, procmail uses occasionally huge chunk of memory 18.27 Procmail signaled out of memory in my verbose log 18.28 Variables DEFAULT and ORGMAIL 18.29 When DEFAULT cannot be mailed to 18.30 Variable DROPPRIVS 18.31 Variable HOME 18.32 Variable HOST 18.33 Variable LINEBUF 18.34 Variable LOG and LOGFILE 18.35 Variable TRAP 18.36 Variable UMASK 18.37 UMASK and permissions 18.38 Performance difference between back tick and "|" recipe 18.39 Procmail's temporary file names while writing file out 18.40 Parameter $@ 18.41 Procmail variables are null terminated (detecting null string) 18.42 FROM_DAEMON TO and TO_ and case-sensitiveness 18.43 TO_ macro deciphered 18.44 TO_ macro and RFC 822 18.45 FROM_DAEMON deciphered 19.0 Technical matters 19.1 List of exit codes 19.2 List of precedence codes 19.3 Sendmail and -t 19.4 RFC822 Reply-To and formail problem with multiple recipients 19.5 Procmail and IMAP server 19.6 Machine which processes mail 19.7 Compiling procmail and MAILSPOOLHOME 20.0 Smartlist 20.1 MLM RFC 20.2 Other mailing list software 20.3 SmartList code (mailing list implementation with procmail) 20.4 Installation trouble: getparams 20.5 Accepting mail only from users in whitelist(s) 21.0 Additional procmail or MUA software 21.1 Comstat to handle multiple mailboxes 21.2 Elm and pgp support (Mutt is the successor to elm.) 21.3 MH sites 22.0 Additional procmail software for Emacs 22.1 What is Emacs 22.2 Emacs and procmail mode and Lint 22.3 Emacs and lining up backslashes 22.4 Emacs and browsing mailbox files 22.5 Emacs and live-mode.el 22.6 Emacs and font-lock.el 23.0 Procmail, Emacs and Gnus 23.1 Gnus pointers 23.2 Why use procmail with Gnus 23.3 Setting up gnus for procmail - Basics 23.4 Gnus for procmail - More gnus 23.5 Emacs and Gnus -- Fiddling with spool files 23.6 Gnus and article snippets 23.7 Emacs GNUS - POP - Procmail 24.0 RFC, Request for comments 24.1 RFCs and their jurisdiction (munged Addresses) 24.2 Comments about addresses munging 24.3 RFC and valid mail address characters 24.4 RFC and login-name@fdqn 24.5 RFCs and messages signature 24.6 RFC and using MIME in Usenet newsgroups 24.7 Some RFC Pointers 25.0 Introduction to E-mail Headers 25.1 To find out more about mail (Resources) 25.2 Lecture by Alan Stebbens 25.3 Applied to received messages 25.4 Bcc lecture by Alan Stebbens 25.5 Bcc lecture by Philip Guenther 26.0 Message headers 26.1 What is correct From address syntax 26.2 What's that X-UIDL header? 26.3 What is that first From_ header? 26.4 Message-Id header 26.5 Received header 26.6 Return-Path 26.7 Errors-To 26.8 X-Subscription-Info 26.9 Reply-To header 26.10 Mail-Copies-To header 26.11 Mail-Followup-To and Reply-To-Personal headers 26.12 Content-Length header and From_ specification 26.13 Moral about CC copies in Usenet 27.0 Other interesting code 27.1 Misc mail related pointers 27.2 Expire mail pointers 27.3 Usenet News related pointers 27.4 Code: Perl Extract procmail man pages from 3.11pre7.tar.gz 27.5 Code: Sh remove matching lines from file 28.0 UBE in Internet 28.1 Terms used and foreword 28.2 UBE strategies 28.2.1 4g. I asked to be "removed" - guess what? I got another U*E 28.2.2 4h. I asked to be "removed" - guess what? The message bounced 28.3 UBE and bouncing message back 28.4 UBE and "I don't mind" attitude 28.5 We need a law against UBE 29.0 Anti-UBE pointers 29.1 NoCEM, CAUCE and others 29.2 General Filtering pages (more than procmail) 29.3 Junk mail and spam 29.4 Comprehensive list of spammers 29.5 Misc pointers 29.6 Questionable UBE stop services 29.7 UBE related newsgroups or mailing lists 29.8 Software: the net abuse page 29.9 Software: adcomplain -- Perl junk mail report 29.10 Software: Ricochet -- Perl junk mail report 29.11 Software: yell -- perl 29.12 Software: RBL lookup tool -- C 29.13 Software: mapSoN 29.14 Software: spamgard 29.15 Software: Spam Be Gone 29.16 Software: TinyGnus - Emacs Gnus plug-in