Reputation systems from the perspective of Foro ABUSES, the Spanish Abuses Forum

Authors: Foro Abuses: http://www.abuses.es/

José D. Domenech, Sergio Bastian, Jesus Sanz de las Heras, Francisco Monserrat

Index

Introduction to reputation systems

One of the most commonly employed methods for reducing the amount of junk mail reaching inboxes is the use of reputation lists. In general, reputation systems are based on classifying IP addresses according to the activity considered hostile they originate, such as sending spam, phishing or any other type of malicious behaviour. The most frequent hostile activity is sending spam.

Since it is likely that an IP address that has sent spam once will do so again, it is common practice to reject e-mails from addresses with a bad reputation.

Traditional systems are limited to a list of addresses classified as hostile. Access to these lists is generally through the DNS lookup protocol. That is why they used to be called DNSBL (Domain Name Server Black List). They are also known as “blacklists”, since if an address is included in these lists it means its messages will be rejected (or at least penalised) by servers that consult these lists.

The traditional system is not without its problems and sometimes these lists cause more harm than good. These days, the “reputation lists” model is based on more complex criteria such as the proportion of malicious e-mails and “clean” mails originating from the address and, in the case of major anti-spam system providers, on the adding of data from their customers through various sources.

With the advent of "over the counter" solutions or e-mail appliances, a large number of manufacturers include one or several of these reputation lists. This is not always in a way that is visible for the anti-spam system administrator, something which can hugely complicate how a problem is solved. It is therefore necessary to differentiate between the hardware, the e-mail appliance and the reputation list.

Lastly, the major e-mail service providers (ESP) incorporate various solutions in a concerted effort to combat SPAM, mixing hardware solutions with tailor-made developments for combating junk mail and warnings for those who send it.

The purpose of this document is firstly to evaluate reputation systems from the “customer” point of view (the server or e-mail service that uses one of these systems), so as to justify certain reputation lists and advise against others, when they do not comply with a series of minimum requirements.

Secondly, the aim is to gather contact information for different reputation list providers and the main e-mail service providers, in order to be able to rapidly tackle problems caused by the inclusion of an IP address in one of these systems.

Reputation Lists

Classification

There are various types of reputation lists. If we classify them according to who can consult them, there are private lists for exclusive use by an organisation or company, others which security firms offer their clients for a fee and lastly, lists that can generally be freely consulted and which are by far the most widely used. The latter are the ones analysed in this document, because even those who use private or paying lists also tend to consult public reputation lists. For this reason good use of these lists - selecting those which are reliable and avoiding those which are not recommended - is of great importance for the smooth running of e-mail on the internet.

Evaluation criteria

The characteristics a reputation list must have in order to be reliable, based on the data provided by the members of the ABUSES Forum, are as follows:

  1. It should not request money or other donations for removing an IP address from its black list. This would lead to a protection tax and a loss of confidence in one of the weapons for combating SPAM.
  2. It should not list full IP addresses ranges, but rather individual IP addresses that send spam or are infected by trojans etc. (except DUL or PBL lists). This way every effort is made to ensure that IP addresses that send legitimate mail are not affected by the blacklist.
  3. It must have clear exit mechanisms, either automatic (recommended) or manual, for once the problem that caused it to be included in the list has been resolved.
  4. There must be a fast, clear form of contact for communicating with the list's administrators. An IP address should not be listed for longer than necessary.
  5. It must have inclusion and exclusion criteria, which must be in the public domain and clearly defined.
  6. It must provide access to the evidence which occasioned it to be included in the list.
  7. It must possess mechanisms for notifying the person responsible for IP addresses of the inclusion of IP addresses in reputation lists (FBL, ARF etc.).

Common or most widely known/used reputation lists

● Barracuda Networks

○ URL information: http://www.barracudacentral.org/rbl

○ Type: free lookup - download not available

○ URL exclusion: http://www.barracudacentral.org/rbl/removal-request

○ Sends an FBL: No

○ Comments:

■ also included in its commercial e-mail programmes (Appliance)

■ possible to use it (lookups) from outside, by requesting use

■ possibility of downloading from the whole list for inclusion in a reputation list is not known

● Spamhaus

○ URL information: http://www.spamhaus.org/

○ Type: free lookup up to a certain amount, after which you have to pay. Pay to download.

○ URL exclusion: http://www.spamhaus.org/lookup/

○ Sends an FBL: No

○ FAQ: http://www.spamhaus.org/faq/

○ Comments:

■ has different types of lists or areas: SBL, XBL, PBL, DBL, ZEN

● CBL (Composite Blocking List)

○ URL information: http://cbl.abuseat.org/

○ Type: free lookup and download

○ URL exclusion: http://cbl.abuseat.org/lookup.cgi

○ Sends an FBL: no

○ FAQ: http://cbl.abuseat.org/faq.html

○ Comments:

■ already included in the Spamhaus XBL list

● SPAMCOP

○ URL information: http://www.spamcop.net

○ Type: free lookup - download not available

○ URL exclusion: http://www.spamcop.net/bl.shtml

○ Sends an FBL: yes

○ FAQ: http://www.spamcop.net/fom-serve/cache/1.html

○ Comments:

■ allows users to report SPAM (by registering in the system)

● Lashback

○ URL information: http://www.lashback.com/blacklist

○ Type: free lookup and download

○ URL exclusion: http://www.lashback.com/blacklist

○ Sends an FBL: no

○ Comments:

■ requires confirmation (link) by e-mail to complete delisting process

■ only lists IP addresses from e-mail senders whose addressees are registered in opt-out lists (Robinson's list or similar)

● PSBL (Passive Spam Block List)

○ URL information: http://psbl.org/

○ Type: free lookup and download

○ URL exclusion: http://psbl.org/remove

○ Sends an FBL: no

○ FAQ: http://psbl.org/faq/

○ Comments:

■ previously also known as Surriel RBL

■ uses Spamikaze software (http://spamikaze.org/)

● Trendmicro

○ URL information: http://www.trendmicro.es/productos/hosted-email-security/index.html

○ Type: commercial

○ URL exclusion: https://ers.trendmicro.com/reputations

○ Sends an FBL: dk/na

○ Comments:

■ restrictive list; the first exclusion is simple but for requests thereafter it is not so easy to exclude a reoffending IP address.

● SORBS

○ URL information: http://www.sorbs.net/

○ Type: free lookup and download

○ URL exclusion: http://www.sorbs.net/cgi-bin/support

○ Sends an FBL: yes, but only to ISPs that register in its system

○ FAQ: http://www.sorbs.net/information/faq/

○ Comments:

■ management interfaces: http://www.us.sorbs.net/managers/

● Commtouch

○ URL information: http://www.commtouch.com/sp-anti-spam/

○ Type: commercial: pay to look up – download not available.

○ URL exclusion: http://www.commtouch.com/check-ip-reputation/

○ Sends an FBL: no

○ Comments:

■ the system is based on an analysis of message signatures and also uses greylisting mechanisms

● UCEPROTECT

○ URL information: http://www.uceprotect.net/en/index.php

○ Type: free lookup and download

○ URL exclusion: http://www.uceprotect.net/en/rblcheck.php

○ Sends an FBL: no

○ FAQ: http://www.uceprotect.net/en/index.php?m=2&s=0

○ Comments:

■ they request payment to delist IP addresses

■ they register IP address ranges and even complete ASN in their blacklist


Evaluation of reputation lists

Table of results

P1

P2

P3

P4

P5

P6

P7

Average

Barracuda

7.93

8

7.67

6.4

5.07

4.31

4.2

6.23

CBL

8.33

853

7.87

7.6

7.27

6.29

3.09

6.83

Comtouch

10

10

7.67

4

2.67

1.83

1.75

5.42

Lashback

10

10

8

5.6

6.4

2.6

2.5

6.44

PSBL

9.11

7.11

8.3

6.67

6.33

8.33

4.67

7.22

SORBS

6.25

6.38

5.69

5.38

6.06

5.59

4.36

5.69

Spamcop

8.94

8.81

8.25

6.69

8.06

8.4

8.62

8.25

Spamhaus

9.12

7.76

8.71

7.12

8.12

7.06

7.21

7.87

TrendMicro

8.92

7.64

5.67

5.92

6

6

4.8

6.42

UCEPROTECT

1

1

1.9

1

2.9

2.22

1.13

1.59

Colour key

Not very well known, few responses, not evaluated

Use not recommended

May cause some problems, insecure use

Few problems, can be used with caution

No problems, use recommended


Other reputation list and systems

White lists

Indiscriminate use of reputation lists may, on occasions, cause a bigger problem than the harm you are trying to avoid. This is especially important for providers who are "close" to each other to bear in mind, meaning those who are located in the same country.

For example, let us suppose that a Spanish provider has a problem and spam is sent via its servers. Generally (at least this is how it usually occurs), the spam will be sent indiscriminately to countries all over the world. If an Indonesian provider rejects the e-mail, the damage is minimal. However, if another Spanish provider also rejects it, it is highly likely the effect will be much worse than the sending of the spam itself.

To avoid this problem, white lists have been created. They work on the basis of a relationship of trust. Each member of the list is responsible for taking reasonable care to tackle spam incidents. This means that members of the community who use it agree to apply reduced filter measures (or no filters at all), thereby preventing legitimate e-mails from being blocked.

The system offers many more advantages than disadvantages. The origin of spam is very diverse and can come from any number of places. Giving free passage to a few, with the knowledge moreover that incidents will not continue for too long, prevents most of the collateral damage caused by filters. Spam received and accepted will always be a small percentage of the total.

● ESWL/ MTAWL

○ URL information: http://www.abuses.es/eswl

○ Type: free lookup and download

○ URL exclusion: http://www.abuses.es/eswl

○ Sends an FBL: yes, but only to registered IP addresses

○ Comments:

■ list of addresses from Foro Abuses

■ two levels: mail servers from Spanish ISPs (ESWL) and registered mail servers (MTAWL)

■ various download formats

● DNSWL

○ URL information: http://www.dnswl.org/

○ Type: free lookup and download

○ URL exclusion: http://www.dsnwl.org/

○ Sends an FBL: no

○ Comments: One of the first white lists to be created, various download formats.

Reputation for E-mail Service Providers (ESPs)

● Microsoft (Hotmail, Live, MSN, Outlook etc.)

○ General policy: http://postmaster.live.com

○ SMTP error codes: http://mail.live.com/mail/troubleshooting.aspx#errors

○ Solutions for ISPs (SNDS, JMRP, etc.): http://mail.live.com/mail/services.aspx

○ Form for opening a support ticket: https://support.msn.com/eform.aspx?productKey=edfsmsbl2&ct=eformts

○ Comments:

■ the information they provide in SNDS is not updated in real time

■ the response time for problem incidents in the delivery of e-mail messages is 2 to 3 days, which is not fast enough in the majority of cases (customers and users constantly complain because their e-mails do not reach Microsoft mail users)

● Google Mail (free Gmail and paying Gmail)

○ General policy: http://support.google.com/a/bin/answer.py?hl=es&answer=178266&topic=28609&ctx=topic

○ Form for opening a support ticket: http://support.google.com/a/bin/request.py

○ Comments:

■ they only have a forum for users and postmasters to ask questions and solve problems

■ pseudo-FBL: http://blog.returnpath.com/blog/joanna-roberts/3-steps-to-qualify-for-gmails-feedback-loop

● Yahoo

○ General policy: http://postmaster.yahoo.com

○ Solutions for ISPs (FBL): http://feedbackloop.yahoo.net

○ Form for opening a support ticket: http://help.yahoo.com/l/us/yahoo/mail/postmaster/cfl_app.html (Choose Product “Mail” and Category “Postmaster”)

○ Comments:

■ they constantly redirect to comply with their policy for sending newsletters and distribution lists, without dealing with other issues such as the reasons why spam messages are being sent, which could be due to a virus in the computer (and subsequent theft of e-mail credentials) or on the web page (stolen FTP, code injections, etc.)

● AOL

○ General policy: http://postmaster.aol.com

○ SMTP error codes: http://postmaster.aol.com/Postmaster.Errors.php

○ Solutions for ISPs (FBL): http://postmaster.aol.com/Postmaster.FeedbackLoop.php

○ Form for opening a support ticket: http://postmaster.aol.com/SupportRequest.php

○ Comments:

■ they send a lot of information to abuse contact e-mails, interesting information for finding filtered IP addresses of own mail servers, in ARF format

● Telefónica/ Movistar (Telefonica.net, Terra, Mail services for companies)

○ Form for opening a support ticket: http://www.movistar.es/nemesys

○ Comments:

■ Nemesys sends alerts to the abuse@ contact when there is a large number of invalid addressees

Websites for checking reputation on different lists (Appendix)

● http://multirbl.valli.org/lookup

● http://www.mxtoolbox.com/

● http://rosinstrument.com/cgi-bin/blqw.pl

● http://www.dnsbl.info/dnsbl-database-check.php

Other topics

Future considerations

The evolution of Internet to IPv6 throws up several challenges related to reputation lists. On the one hand, the increased assigning of addresses to IPv6 end users (some recommend the use of 2^64 IP addresses different to the link layer), together with automatic assignation and randomisation of addressing mechanisms, may make reputation lists for end addresses unfeasible. Reputation lists for IP ranges and the use of wildcard DNS could be considered as alternatives.

On the other, during this transition period, the effect caused by CGNAT when establishing the reputation of a public IPv4 address, which may be being used simultaneously by different customers, is being noted with concern.

Glossary of terms

CGNAT: Carrier Grade NAT: Mechanism for sharing one public IPv4 address between various customers of the same operator, so that they can all shift to the Internet provider network.

Wildcard DNS: Use of generic registers in DNS so that anything looked up from a particular DNS entry is matched. For example (* search for one that is not JASH.esgay.com *)

DUL: Dynamic User List: reputation list that indicates whether an IP address belongs to an Internet Service Provider that assigns it dynamically to domestic users. Given that end users should not be sending an e-mail as if they were a server, this type of list is used for blocking e-mails sent from end users' computers, usually infected with some kind of malware and sending spam.

PBL: Policy Block List, a list where ISPs' dynamically assigned addresses are published. Being on this list does not mean that these IP addresses are sending spam; it means one should NOT accept mails from them, as they are dynamic address ranges for services unrelated to sending/receiving e-mails.

FBL: The Feedback Loop or Complaint Feedback Loop is a system for providers to receive reports of marketing e-mail messages originating from an IP address in their network.

Revisions

This document was written and reviewed by the members of Foro Abuses, according to their experience with their own e-mail systems and those of their static addressing customers where they host a mail server.

The Abuses Forum will revise this document periodically, in order to re-evaluate the different reputation lists based on P1-P7 criteria. Reputation lists will be included in or deleted from the document based on feedback from ISPs who are members of this forum.

Revisions made, in chronological order:

  1. October - 2008
  2. March 2013