Spam from an ISP perspective
description
Transcript of Spam from an ISP perspective
![Page 1: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/1.jpg)
Spam from an ISP perspective
Simon Lyall, Ihug
Uniforum NZ NetForum ConferenceJuly 2003
![Page 2: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/2.jpg)
Who is Ihug?
• 3rd largest ISP in New Zealand• 90,000 dialup and broadband customers• Email System software: Exim, qmail, openldap
on Debian Linux.• Email system spread across 14 servers.• Duplicate system in Australia.
![Page 3: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/3.jpg)
The Scale of the Problem
Total emails received each day 1,250,000
Estimate Legitimate Emails per day 400,000
Estimated Spam Emails per day 560,000
Sent to invalid addresses 270,000
Proportion of email that is Spam 60%
![Page 4: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/4.jpg)
Spams Blocked per customer
Percentile Spams/day Unblocked
50 15 2
60 20 2
80 35 4
90 60 7
95 85 9
99 150 17
![Page 5: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/5.jpg)
Mid 2004 on Current trends
● Total Email volume has doubled in 12 months● 90% of each customer's email is Spam● Anti-Spam solutions need to block 95-98% of
Spam● 2004's Spam has evolved to beat 2003's filters
![Page 6: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/6.jpg)
If 10% monthly growth for 12 months
Percentile Spams / day Unblocked
2018 2
5046 5
80108 12
90186 21
95264 29
99465 52
![Page 7: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/7.jpg)
History of Spam Filtering at Ihug
Before 2000 Little or no filtering
2000 - Feb 2002 MAPS RBL+
2002 - Jun 2003 MAPS RBL+ & Spamcop
Jun 2003 - present Spamassassin based
![Page 8: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/8.jpg)
Why ISPS care about Spam?
● Customers Unhappy– Close accounts– Blame the provider– Call helpdesk– Install their own anti-spam software
● Resources Wasted● Staff unhappy/offended/wasting time● Money to be made
![Page 9: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/9.jpg)
Customer expectations of Mail and Anti-Spam system
● All email must go through instantly– Bouncing or deleting suspected spam is not an option– False positives must be kept to an absolute minimum
● Except the mail I don't want– 100% hit rate is expected– Each customer's definition of Spam is unique
● Must be free and involve no work for me● Must be optional
![Page 10: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/10.jpg)
Technology for Blocking Spam
● External Solution Providers● DNSBLs● Other methods
![Page 11: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/11.jpg)
External Providers
● Pros●Brandname Product●Support●Pretty Effective
● Cons●Cost●Not 100% effective ●Implementation costs
![Page 12: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/12.jpg)
DNS Block Lists (DNSBLs)
● Used to check IP of sending server● VERY wide variety of DNSBLs and
organisations running them.● Wide range in quality, goals and usefulness.● Spam Blocking vs Behavour Changing vs Group
of Networks
![Page 13: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/13.jpg)
Types of DNSBLs
● Open Relay (ORDB) ● Open Proxy, Open formail (monkeys.com) ● Country or Provider based (blackholes.us,
Cluecentral)● Dialup and Dynamically assigned Ips (MAPS
DUL)● Known Spammers, Spam friendly ISPs, slow
reacting ISPs (Spews, Spamhaus SBL)
![Page 14: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/14.jpg)
Types of DNSBLs (cont)
● Current Spam (bl.spamcop.net , MAPS RSS)● Whitelists (bondedsender.org)● Other
– Random– Everything– Very aggressive– Specific– Private
![Page 15: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/15.jpg)
Using DNSBLs
● Read the listing and delisting criteria● Don't use duplicate lists● They ALL make mistakes● Some make more mistakes than others● Make sure the list's values align with yours● Mirror the lists locally to cut down on delays
![Page 16: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/16.jpg)
Other Technologies
● Distributed Checksums (Razor, DCC)● Suspicious Connection Throttling● Marking non-Spam
– Habeas– Bonded Sender– Challenge response (Earthlink, Actrix)– Force lists to register (AOL)
![Page 17: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/17.jpg)
Other Technologies (cont)
● Baysian – Paul Graham article
● Rules Based– Procmail– Spamassassin– Spambouncer
![Page 18: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/18.jpg)
Integrating Spam filtering
● It must be optional.● Customers must be able to examine and retrieve
deleted email● Remember each customer is different● Plan beforehand what to do about mistakes ● Aim to block at least 90% of Spam● Remember to review and upgrade regularly
![Page 19: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/19.jpg)
Controlling Customers
● Clear AUP ● Ability to suspend and close customers● Monitored abuse address● Open relay scanning● Open proxy scanning● Port 25 blocking● Management support
![Page 20: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/20.jpg)
Questions
● This talk online– http://www.darkmere.gen.nz
![Page 21: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/21.jpg)
Links
● External Providers– Brightmail http://www.brightmail.com– Postini http://www.postini.com– Message Labs http://www.messagelabs.com
● RBLs– Multi RBL lookup http://www.moensted.dk/spam– MAPS http://www.mail-abuse.org– Spamcop http://spamcop.net/bl.shtml– Spamhaus http://www.spamhaus.org
![Page 22: Spam from an ISP perspective](https://reader036.fdocuments.us/reader036/viewer/2022062809/568159f9550346895dc745c2/html5/thumbnails/22.jpg)
More Links
● Software– Spamassassin http://www.spamassassin.org– Procmail http://www.procmail.org
● Baysian Filters– Paul Graham http://www.paulgraham.com/spam.html
– Bogofilter http://bogofilter.sourceforge.net● Reporting Spam
– Abuse.net http://www.abuse.net