Greylisting with Exim, Python and MySQL – greylisting-py 1.0

I just released a working version of my own greylisting implementation, written in Python and using a MySQL database. I originally wrote this for our mail server at work, as the existing implementations I found online did not satisfy my needs completely.

The script was made for a system running Exim, MySQL 4 and Python 2.3 on Redhat Linux, so it should be easy to install for similar configurations. Of course, it should be possible to run it on any environnement and MTA (if your MTA gives you the possibility of refusing mail based on the result of an external command).

Here is a list of features of my project :

  • Whitelisting of single IP addresses
  • Whitelisting of IP networks (CIDR)
  • Support for SPF (if enabled, messages that pass SPF test will not be greylisted)
  • You can enable greylisting by email adress and/or by domain.
  • Configurable parameters, such as initial delay and expire delay.

You can download greylisting-py (yes, I did not spent much time trying to find a name for it :) from the Google code page : http://code.google.com/p/greylisting-py/. There’s also instructions on how to install it and use it with Exim.
I hope it will be useful for some people!

But what is greylisting?

Greylisting is a technique to help fight spam, very useful as a complement to other filters like SpamAssassin. Basically, it works this way :

  1. MTA 1 receives a message from MTA 2. It dectects a new “triplet” [sender ip, sender address, recipient address], and since it’s the first time it sees that triplet, it refuses the message with a temporary error code (4xx).
  2. Since the error code was temporary, MTA 2 keep the message in queue to try sending it again later (usually after a delay between 1 and 60 minutes)
  3. MTA 1 receives the second delivery attempt. It knows that the same triplet has tried sending a message previously, so it accepts the message.

If MTA 2 was a bot sending spam, chances are that it would never bother trying again, so the spam will never come trough. In my experience, greylisting really helps with spam filtering, but of course with the drawback that some messages are delayed for several minutes.

Greylisting drawbacks

There are some drawbacks to using greylist :

  • Message will be delayed from about 15 minutes to 1 hour, depending on the greylisting implementation and the sending MTA. This delay will only happen the first time a server with a given IP address send a message with a given sender and recipient address (MAIL FROM/RCPT TO).
  • Greylisting may cause problem with some big networks (like gmail, hotmail, …). These networks sometimes send emails from a different IP at each new try, so the greylisting delay will always start back from zero, until the same IP is used again. This may cause messages to be delayed longer than intended, or ultimately fail. This problem can be solved by whitelisting these networks and using SPF to accept message without going trough the greylist (if a server passes SPF test, it will most likely also pass the greylist).
  • Some non-compliant servers could treat the temporary error as an permanent error and drop the message, but I don’t think any important MTA have this problem.

For more details, look at the Wikipedia article.

Tags: , , , , ,

2 Responses to “Greylisting with Exim, Python and MySQL – greylisting-py 1.0”

  1. Justin Brown Says:

    Hi, I was just trying out your greylisting script, but I’m not sure I installed it correctly. When I run your test command at the command line with a domain in GREYLIST_ENABLE I get the response “greylist”. But when I put the configuration in Exim all I get is “Warning: greylisting ACL result: error”. I don’t expect you to give me all kinds of support, but any pointers would be helpful. Thanks.

  2. jphoude Says:

    Sorry for the late response, but I just tried installing my script on an other server and got a similar problem.
    One thing you can do to debug this problem is to change the ACL to display the error message. For example here I added .$value after “error”, so $value will print the error message (probably a python stack trace) :
    warn set acl_m8 = ${run{/usr/local/greylisting-py/greylisting.py from domain $sender_address $sender_host_address $sender_helo_name}\
    {$value}{error.$value}}

    For me, the error was with Python egg cache, that was set to a directory where the Exim user couldn’t write.

    I fixed it by adding the following code at the top of greylisting.py (after the first block of comments) :
    import os
    os.environ['PYTHON_EGG_CACHE'] = ‘/tmp’