Mail-Filters Technical Presentation

22
Mail-Filters Technical Presentation How it works, Why it’s Better

description

Mail-Filters Technical Presentation. How it works, Why it’s Better. Mail-Filter Technology Overview. Why Mail-Filters Bullet Signature Creation Star Engine Process Overview Implementation Options SDK Contents Getting Started The API Commands Testing Options OEM Implementation Examples - PowerPoint PPT Presentation

Transcript of Mail-Filters Technical Presentation

Page 1: Mail-Filters Technical Presentation

Mail-Filters Technical Presentation

How it works, Why it’s Better

Page 2: Mail-Filters Technical Presentation

Mail-Filter Technology Overview

• Why Mail-Filters• Bullet Signature Creation• Star Engine Process Overview• Implementation Options• SDK Contents• Getting Started• The API Commands• Testing Options• OEM Implementation Examples• FAQs

Page 3: Mail-Filters Technical Presentation

Why Mail-Filters

• It’s Fast – 100s of messages per second (or higher)

• It’s Accurate – over 95% of spam caught, less than 1 in 1,000,000 false positive rate

• Many implementation options – the right solution for any environment

• It’s Proprietary – it’s not fooled by spammer tricks - gives time to market and competitive differentiation

• It catches Foreign Language Spam – in over 30 languages – a worldwide solution

• Easy Implementation – usually less than a day• Full Support – Integration, technical support and training,

marketing materials, sales training and lead generation

Page 4: Mail-Filters Technical Presentation

How Mail-Filters Works

2. Human Editors Craft Bullet Signatures

1. Spam Collection occurs from many sources

3. Bullet Signatures Are Updated Every 1-15 Minutes

4. Mail-Filters Technology Integrated into OEM Solutions - Catches Spam, without False Positives

5. Tuning Users and Administrators provide feedback to help identify spam and those that send them.

Page 5: Mail-Filters Technical Presentation

Bullet Signature Creation

Mail-Filters’ Process Overview To Capture Spam & Create Bullet Signatures

wwwwww

Phish Trolling

QualityCheck

Translation Tools

Customer submissionsBullet

Signatures

Aristotle(Signature

Auto-Suggest)

Customer

Mail-FiltersTechnology on

Customer Device

SpamDB

PrioritizationProcess

International SpamHarvester

Partner Collections

Phish Traps

www

Traffic and ConnectionHeuristics

Spam Pre-Qualification

Partner Pre-QualificationExpert

Auto-NominateProcess

Bullet Signature Updater

Scam Sensors

Traffic Analysis

Pre-Qualified &Auto-Nominated

ReputationAnalysis

Human Editors

LanguageAssignment

Spammer Profile Creation

Data QualityManager

CullingEngine

Mail-Filters Data Centers

Message Profile Creation

Traffic Profiles

BulletSignatureUpdates

Page 6: Mail-Filters Technical Presentation

STAR Engine Server

ST

AR

Eng

ine

Man

agem

ent

Mod

ule

MessageNormalizer

SnowFlake Buster

Language Analyzer

Malformed MessageProcessor

Message Analysis

Traffic Analysis

Reputation Analysis

Spammer Profile Check

False PositiveRationalizer

BulletSignatures

Bullet SignatureUpdater

Mail-FiltersData Centers

Known Good Mail

Star Engine Process Overview

Star Engine Interface

OEMSoftware

Is Message Spam?

Yes / No

Page 7: Mail-Filters Technical Presentation

Implementation Options

• Enterprise– Most typical implementation – highest

performance – uses more resources

• Desktop– Small footprint – message is local – scan and

database is remote

• Embedded– Tiny amount of resources required – scanning

is done remotely

Page 8: Mail-Filters Technical Presentation

Star Engine – Enterprise(Very High Performance)

• Can process 100s or even over 1000 messages per second

• Requests Bullet Signature updates every 1-10 minutes (only changes are downloaded)

• The SEI and SES are typically deployed on the same hardware

• The SEI is linked into the OEM application using C or C++

• The SES runs as a Service or Daemon and it manages it’s own Database Updates

• The Database is usually between 3-10MB – will download a fresh DB upon startup if none present

OEM Application

Server or Appliance Hardware

Star Engine Interface (SEI)

Star Engine Server (SES)(Service or Daemon)

Linked Together by OEM at compile

C o

r C

++

AP

I

TC

P /

IP

Mail-FiltersData Centers

TC

P /

IP

Page 9: Mail-Filters Technical Presentation

Star Engine - Enterprise

• The Star Engine Server is fully multi-threaded• The Star Engine Server will run as a Service

under Windows or as a Daemon under Linux, FreeBSD, or Solaris

• TCP/IP outbound on Port 80 is required – IP proxies are supported

• Typical requirements are P4, 100MB RAM, Hard Disk optional

• A unique Mail-Filters Customer ID is required to download the Bullet Signature Database

Page 10: Mail-Filters Technical Presentation

Star Engine – Desktop(Small Footprint)

• Only requires 128kb of RAM• Can process 10s of messages

per second• Secondary server can be

anywhere, including and typically Mail-Filters’ Data Centers

• Database updates are not required on the SEI (just the SES)

• Same exact API as the Enterprise implementation

• Can also be used in a server cluster environment – many SEI’s feeding one SES

OEM Application

PC or Other Device(with limited resources)

Star Engine Interface

Linked Together by OEM at compile

C o

r C

++

AP

I

TC

P /

IP

Mail-FiltersData Centers

TC

P /

IP

Star Engine Server

Separate Server

Page 11: Mail-Filters Technical Presentation

Star Engine – EmbeddedA Completely New Approach

• Anti-Spam detection for edge devices with almost no resource requirements

• OEM code requires less than 10kb of RAM

• No software need be installed on any user PC – the service is turned on or off at the OEM device

• Works with POP3 & IMAP• OEM device intercepts the message

delivery request and sends it to Mail-Filters

• Mail-Filters receives the messages on behalf of the end user, filters for viruses and spam, then sends the clean messages to the end user

• OEM or customer determines what happens to spam (delete, mark with an X-header, decorate the subject line)

• Since spam can be deleted and the downlink speed is probably slower than the link from Mail-Filters’ data centers to the email servers – good mail will get to the end user faster.

WWW

PC

Mail-FiltersData Centers

Email Server

1. Email Client requests mail

2. OEM device intercepts the request based on port the request is made on (Ex. 110 = POP3) – and redirects the request to Mail-Filters’ data centers.

3, Mail-Filters makes the request on behalf of the user, filters the messages, then sends the good mail to the user. No mail is kept at Mail-Filters – it just passes through.

4. Mail-Filters’ authenticates as the user to the ISP or Corporate email servers - the mail is delivered

Page 12: Mail-Filters Technical Presentation

Embedded Architecture

OEM Application

OEM Device

Redirect Code

Outbound Listening Code (Port 110 for POP3 or Port 147 for IMAP Requests)

Customer Premise

The Internet

The Email Client requests email from an email server – it makes the request on port 110 or 147 – the OEM device redirects the request to Mail-Filters. A port is opened by the email server via Mail-Filters to the PC. The email is filtered, a policy is applied, then delivered to the Email Client.

Mail-FiltersData Centers

Email Server

Email Server PCs

Page 13: Mail-Filters Technical Presentation

SDK Contents

• Star Engine Server software executables

• Star Engine Interface libraries in C and C++

• Simple Single-Threaded implementation example application

• Documentation

• Typical integration time is less than a day

Page 14: Mail-Filters Technical Presentation

Getting Started with the SDK

• Install the Star Engine Server• Run the Star Engine Server• Run the Example Application

– This application will scan the files in the directory of choice and all sub-directories to see if they are spam. The results will display on the screen.

• Begin the Integration to the OEM application

Page 15: Mail-Filters Technical Presentation

The Star Engine API(The Star Engine Interface)

• The Commands are Straight-Forward– Initialize – This command establishes a connection to

the Star Engine Server– Shutdown – Used to tear down the thread after a

successful Initialize command– Scan SMTP Buffer – Passes the SES the data to be

scanned – will return TRUE if Spam– SCAN Buffer – Passes the SES data to be scanned –

best used for non-SMTP types of content such as IM, SMS, web pages, etc.

– Version – Returns the versions of all the components currently being used, including the database version date.

Page 16: Mail-Filters Technical Presentation

Testing Options• The Mail-Filters database is culled to eliminate old/unused signatures.

– As a result, the catch rate will suffer on old corpuses of email– Best results are obtained with live (or very close to it) email.

• There are several options to test the Mail-Filters technology– To test for catch rate or false positive rate

• Use the Example scan utility to check individual messages in a directory• Send mail to an account Mail-Filters can set up for you at Cleantree.com.

Good mail will go to the Inbox, spam to the Spam folder. Check results using your browser.

• Integrate into the OEM application and run it to check catch rate.– To test throughput:

• Unfortunately, the Example application is only a single-threaded application and will not show what the SES can achieve throughput-wise (it does fine on catch rate)

• The only fair test is to do an integration and run email through it. Most OEMs fine the solution throughput is the same whether Mail-Filters technology is running or not.

– To test Foreign Language:• Do a beta test with a customer or partner in the region of interest• Mail-Filters have several partners in various regions that may assist in a beta

test, if desired.

Page 17: Mail-Filters Technical Presentation

Implementation Examples

• Enterprise– Most OEMs have implemented the Mail-Filters

technology as the primary anti-spam solution• AV solutions company scans for spam while it has the message

in memory to scan for viruses. Because spam is more prevalent and is a much faster scan, spam is typically scanned for first.

– Some have augmented their own anti-spam technology• Because Mail-Filters technology is both fast and accurate, some

have used it as a pre-processor to their own, more computationally expensive technology, to increase the throughput of the overall solution, and to increase spam catch rates.

Page 18: Mail-Filters Technical Presentation

Implementation Examples

• Desktop– Some devices don’t have the processing power or

resources available for spam detection. For these, the Mail-Filters technology can provide a smaller footprint

• Firewalls, security gateways, messaging gateways, enterprise PCs may prefer a secondary server to handle the scanning to free up resources on their own hardware.

– An MSP has a cluster environment where there are many SEIs feeding one SES per tower. This is very efficient and allows their overall throughput to increase dramatically.

Page 19: Mail-Filters Technical Presentation

Implementation Examples

• Embedded– Ideal for DSL routers, Cable Modems, Wireless

gateways, SMB security gateways etc.– Because it requires no end user software

installation or configuration, it is simple to sign-up and have spam and viruses eliminated.

Page 20: Mail-Filters Technical Presentation

Frequently Asked Questions

• How do I get the SDK?– Sign the Mail-Filters MNDA and we’ll send it to you via email.

• Is the Star Engine Server multi-threaded?– Yes.

• Does it handle messages in double-byte character sets?– Yes, our technology catches spam in over 30 languages,

including multi-byte character sets such as Japanese, Korean, Chinese, Arabic, and Hebrew.

• How is the update interval set – can it be changed?– The update interval is set by the OEM, but can be changed on a

customer by customer basis. The default is an incremental every 10 minutes and a full update written to disk once a week.

• Will this solution work on less than a Pentium IV PC?– Yes, but it works more efficiently on a PIV.

Page 21: Mail-Filters Technical Presentation

Frequently Asked Questions• What happens if the SES can’t get a database, or quits running, or some

other catastrophe?– The SES or SEI will fail safe. It will return a FALSE ( the message isn’t spam)

and continue to process messages while trying to reconnect. The customer will see more missed spam, but won’t miss any messages.

• What if the SES doesn’t have the rights to write the database to disk, or the disk is full?

– The SES will continue to function properly and will acquire updates to the database in memory. The version command will return the database currently being used in RAM.

• Is the API really just 5 functions?– Yes – it doesn’t get much simpler than that.

• Can the SES return a probability of a message being spam?– No - Because the technology uses human editors to craft profiles and message

signatures, we’re very very confident the message is spam if we identify it. Because our false positive rate is so low, our methodology is proven to be correct. A probability is required by technologies that guess or compute whether a message is spam – we know it, so we tell you. For those solutions that require a probability, they set our TRUE response to the highest probability – 10 or 1 or 100.

Page 22: Mail-Filters Technical Presentation

Conclusions

• The Mail-Filters technology is easy to implement and provides options for any situation.

• The underlying technology far surpasses what others are doing, giving the Mail-Filters OEM a significant advantage over competitors in catch rate and accuracy, language coverage, and throughput.

• Human review provides the difference -the technology delivers it.