Scan Sequence and Action in Microsoft Forefront Protection 2010 for Exchange Server

Published: October, 2009

Software version: Forefront Protection 2010 for Exchange Server

Carolyn Liu

Introduction ............................................................................................................................................................................ 1

Exchange Mailbox and Forefront hook .................................................................................................................................... 3

Scan Processes ........................................................................................................................................................................ 4

Scan Process Type ..................................................................................................................................... 5

Actions for Malware Scans and Filters ...................................................................................................... 6

Action Table .............................................................................................................................................. 7

Scan Job and Filter Types .......................................................................................................................... 9

Scan Sequence .......................................................................................................................................................................10

Message Header Scan and Action Sequence .......................................................................................... 10

Message Scan and Action Sequence ....................................................................................................... 10

Summary ................................................................................................................................................................................12

Introduction Microsoft Forefront Protection for Exchange Server (FPE) is a leading solution for securing your

messaging environment. Its multi-engine antimalware solution is a proven security product that

has helped many customers to secure their e-mail system. With the introduction of a Premium

Antispam solution and seamless integration with Exchange Hosted Filtering, FPE will bring pro-

tection for Exchange to the next level.

Users familiar with FPE know that besides malware scanning, there are various filtering options.

This article provides insight into the scanning options, as well as the FPE process sequence for

malware scanning and filtering. Administrators can leverage this knowledge to maintain a se-

cure and sophisticated messaging system.

The concept of server roles was introduced in Exchange Server 2007. Server roles enable Ex-

change to clearly classify different functionalities within Exchange and enable administrators to

categorize one or more roles on different servers and locations in the organization.

Exchange Server 2007 introduced the following five roles: Edge Transport, Hub Transport, Client

Access, Mailbox, and Unified Messaging. There is also a combined Hub Transport/Mailbox role.

For more detail about these server roles, see the following article:

http://www.microsoft.com/exchange/evaluation/features/serverroles.mspx

On Edge and Hub Transport roles, Microsoft Exchange provides a Transport Agent framework.

This is a plug-in architecture that enables Exchange e-mail message security vendors to supply

their own agent to process messages passing through the transport pipeline. An agent processes

messages based on SMTP events and communicates to the Exchange Transport pipeline for

processing results and actions, such as discarding a spam message or adding a legal disclaimer

footer when a message leaves an organization. The SMTP events processing sequence is shown

in the diagram below:

OnH

eloC

omm

and

OnE

hloCom

man

d

OnE

ndOfA

uthen

ticat

ion

OnA

uthCom

man

d

OnC

onnect

OnE

ndOfH

eader

s

OnE

ndOfD

ata

OnR

ejec

t

OnD

isco

nnect

OnR

setC

omm

and

OnN

oopCom

man

d

OnH

elpC

omm

and

OnD

ataC

omm

and

OnR

cptT

oCom

man

d

OnM

ailC

omm

and

Figure 1 SMTP Events Processing in Exchange Transport

The processing sequence moves from left to right.

Based on different mail processing requests and the mail delivery status, each agent may inter-

cept different SMTP events. For example, the OnConnect event is often processed by the anti-

spam agent.

For more information about the Exchange Transport architecture and detailed SMTP events, see

the following article:

http://technet.microsoft.com/en-us/library/aa996349.aspx

In the Categorizer (see Figure 2), the routing agent processes the routing events and categorizes

and routes messages already received by the organization to proper mail store(s) or other or-

ganization(s).

On the Edge and Hub Transport roles, Forefront provides real-time protection via the Exchange

Transport framework. This is processed in several stages. First, Forefront Antispam agents

process e-mails at the Edge role via comprehensive mechanisms (IP block list, Sender ID, SMTP

filtering, Content Filtering), stopping spam e-mails before they enter an organization. Next, the

Forefront Antimalware routing agent passes the e-mail messages to Forefront scanning proces-

es for Malware and filtering processing. The Forefront routing agent in the Categorizer inter-

cepts messages that are passing through in real-time and routes the data to one of the Forefront

scanning processes using an Inter-Process Communication mechanism for malware scanning

and various filtering operations.

Figure 2, below, describes the SMTP events going through an Exchange Edge role and different

process points by Transport agents.

Smtp Receive

Messages

Jet

Transport

SMTP Receive Agents

Connection Filtering Agent

AddressRewritingInbound Agent

Edge Rule Agent

Sender ID Agent

Sender Filter Agent

Recipient Filter Agent

Content Filtering Agent

Protocol Analysis Agent

Attachment Filtering Agent

prio

rity

Mex Event Dispatch

Stranded Mail

Scanner

on restart fork/create

create

TarpittingIP Connection

throttling

Connector

Selection

MEx Event

DispatchInbound TLS Inbound MLS

OnH

eloC

omm

and

OnE

hloCom

man

d

OnE

ndOfA

uthen

ticat

ion

OnA

uthCom

man

d

OnC

onnect

OnE

ndOfH

eader

s

OnE

ndOfD

ata

OnR

ejec

t

OnD

isco

nnect

OnR

setC

omm

and

OnN

oopCom

man

d

OnH

elpC

omm

and

OnD

ataC

omm

and

OnR

cptT

oCom

man

d

OnM

ailC

omm

and

EdgeTransportSvc.exe

Header Firew

all

Figure 2 Exchange Transport

Exchange Mailbox and Forefront hook On the Exchange Mailbox role, Exchange provides a virus scanning API (VSAPI) that enables anti-

virus vendors to scan messages passing through the Exchange Mail Store (mailbox databases).

When a mail client such as Outlook accesses mail, FPE provides real-time protection via the Ex-

change VSAPI plug-in to intercept messages and route the data to one of the FPE scanning

processes for malware scanning and filtering.

This is an additional layer of protection. Because the Mail Store can be very heavily loaded, we

advise customers to deploy their messaging system and protection solution carefully. For exam-

ple, FPE has a virus stamp feature that stamps a message when it is scanned on the Edge or Hub

role so that a redundant scan is not performed when the message is stored in the mailbox.

Internet

FSE-protected

Edge

FSE-protected HubMailbox

Mailbox

FSE-protected Hub

InboundInbound

Inbound

Inbound

Inbound

Inbound

Inbound

Outbound

Outbound

Outbound

Outbound

OutboundOutbound

Figure 3 Exchange and Forefront Topology

Scan Processes For all Exchange roles that have FPE installed, FPE uses a similar common entity to perform

malware scanning and filtering: a scan process that communicates to the hook agent and works

independently to avoid disruption of any Exchange processes.

A scan process analyzes messages and applies appropriate file navigation, filters, and malware

scans for each part of a message.

There are multiple scanning processes per scan job type (default number is 4), configurable by

the administrator, which enable concurrent processing of multiple messages and reduce the

direct impact of the scanning process on the core Exchange process (preventing, for example,

the possibility of crashing due to the deep content inspection of potentially malicious code).

Currently, the FPE scan process encompasses the following scanning technologies:

Malware scan (viruses, spyware, and worms)

Filters, which include:

o Sender-domain: This filter examines an e-mail from particular senders or do-

mains.

o Subject line: This filter examines the subject line of e-mails.

o File: This filter examines file names, file size, file types, or file extensions based

on file content.

o Keyword: This filter compares words and phrases in the message body of an e-

mail.

o Allowed senders: This filter is similar to the sender-domain filter but allows the

administrator to bypass any content protection filters.

Figure 4 Forefront Security for Exchange Server Transport Scan Process

Figure 4 describes the Forefront scan process basic diagram in Exchange Edge and Hub roles.

Scan ProcessScan ProcessScan ProcessScan Process

Quarantine and

ActionsFile Navigators Keyword and

Filtering Engines

AntimalwareEngine

Adapters

Exchange Transport

Forefront

Antimalware AgentAntispam Agents Other Agents

Figure 5 Forefront Security for Exchange Server Scan Process on Mailbox Role

Figure 5 describes the Forefront scan process basic diagram on Exchange Mailbox role

Scan ProcessScan ProcessScan ProcessScan Process

Quarantine and

ActionsFile Navigators Keyword and

Filtering Engines

AntimalwareEngine

Adapters

Exchange VSAPI Framework

Forefront VSAPI

hook agent

.

SCAN PROCESS TYPE

There are four scan process types: Transport, Realtime, Scheduled, and On-demand.

Transport Scan Job

The Transport Scan process (FSCTransportScanner.exe) is installed on the Exchange Edge/Hub

Transport role, and scans messages as they arrive from the Exchange Transport Service (Edge-

Transport.exe) and are intercepted by the FPE transport routing agent (FSEAgent.dll).

Realtime Scan Job

The Realtime Scan process (FSCRealtimeScanner.exe) is installed on the Exchange Mailbox role

and scans messages when a user accesses mail via the mail client (such as Outlook or Outlook

Web Access Client). The messages are intercepted by the FPE VSAPI hook agent.

Scheduled Scan Job

The Scheduled Scan process (FSCScheduledScanner.exe) is architecturally the same as the Rea-

time Scan Job, except the trigger is different. The Scheduled scan job is scheduled via the Win-

dows Task Scheduler and leverages Exchange background scanning – a separate task thread that

traverses through items in the Exchange store database looking for instances of items that have

not been scanned.

On-Demand Scan Job

The On-Demand Scan process has been architecturally redesigned for the this release due to

Exchange Server 2010 architecture changes. For Exchange Server 2010, the on-demand scan

leverages EWS (Exchange Web Services) from the Exchange Client Access Server (CAS) Role. On-

demand scanning in Exchange Server 2007 installations will still use the older design (ADO).

ACTIONS FOR MALWARE SCANS AND FILTERS

When malware is found or a filter is matched, the FPE scan process will take necessary actions

on the relevant message part. It is necessary to have a clear understanding of each action taken

by each FPE scan process. The action definitions are:

Clean

A message part (which could be a message body or an attachment) is cleaned. This option only

applies to virus scans. If cleaning is successful, the original part will be replaced by the cleaned

part and reassembled into the original format of the message. For example, an e-mail contains

the attachment a.zip. This zip file contains two files: b1.doc and b2.exe. If b1.doc is infected but

cleaned by FPE and b2.exe is clean, a modified a.zip that contains the cleaned b1.doc and the

original b2.exe will arrive in the user’s inbox.

Delete

A message part is deleted and replaced with custom defined deletion text. For example, an e-

mail contains the attachment a.zip. This zip file contains two files, b1.doc and b2.exe. If b1.doc is

infected, it will be deleted, and a modified a.zip that contains the deletion text b1.txt and the

original b2.exe will arrive at the user’s inbox.

Deletion Text b1.txt contains the following text by default:

“Forefront Security for Exchange Server detected b1.doc to be infected.”

The FPE administrator can customizethe Deletion Text. For more information on customizing

Deletetion Text, refer the FPE Operations Guide.

Purge

The entire message is deleted and will not be delivered to the recipient(s). This option always

applies to worms (a special virus type). This option is supported in realtime (Exchange Mailbox)

scanning as well. In VSAPI 2.6, the VIRSCAN_DELETE_MESSAGE error code will indicate that the

top level message is deleted, effectively purging the message.

See Table 1 and Table 2 for what this action applies to.

Identify

A user-defined word or phase will be pre-pended to the e-mail subject line. No other action is

taken on the message. This is supported in filtering. It is available for keyword filtering, file filter-

ing, subject line filtering, and sender-domain filtering.

For example, if a keyword is matched within an e-mail message body, text defined by the FPE

administrator will be pre-pended to the e-mail subject line, indicating that a matching keyword

was found. The default pre-pended-text is “SUSPECT:”

FPE administrators can also use this option to add a MIME message header so that it can be

identified later for processing into folders at a user’s inbox or for other purposes identified by

the FPE administrator. By default, X-Junk-Mail is written to the header.

Skip (detect only)

When the Skip (detect only) option is selected, an incident log entry will be created indicating

the infection and filtering information, and the rest of the scanning and filtering process contin-

ues.

ACTION TABLE

The following table shows the action options within FPE filters and default actions among vari-

ous scan job types.

Filter Type

File Filter Keyword Filter

Allowed Sender

Subject Line Sender-Domain Scan Job Type

Hub Transport or Edge Transport

Skip (detect only) Purge Delete Identify Default:

Skip (detect only) Purge Identify Default:

N/A 1 Skip (detect only) Purge Identify Default: Identi-

Skip (detect only) Purge Identify Default: Identi-

Delete Identify fy fy

Mailbox Realtime

Skip (detect only) Purge Delete Default: Delete

N/A N/A 1 Skip (detect only) Purge Default: Skip (detect only)

Skip (detect only) Purge Default: Skip (detect only)

Mailbox Scheduled

Skip (detect only) Purge Delete Default: Delete

N/A N/A 1 Skip (detect only) Purge Default: Skip (detect only)

Skip (detect only) Purge Default: Skip (detect only)

Mailbox On-Demand

Skip (detect only) Purge Delete Default: Delete

N/A N/A 1 Skip (detect only) Default: Skip (detect only)

Skip (detect only) Default: Skip (detect only)

Table 1

Note:

1. The Allowed Sender List is used to identify sender address/domains that are allowed to by-

pass the configured filters (File Filter, Keyword Filter, Subject Line Filter, Sender-Domain Filter).

The following table shows the action choices in FPE among various scan job types for malware

scans.

Malware Type

Virus Spyware

Scan Job Type

Edge Transport

or

Hub Transport

Skip (detect only) Clean Delete Default: Clean

Skip (detect only) Purge Delete Default: Delete

Mailbox Skip (detect only) Clean

Skip (detect only) Purge

Realtime Delete Default: Clean

Delete Default: Delete

Mailbox

Scheduled

Skip (detect only) Clean Delete Default: Clean

Skip (detect only) Purge Delete Default: Delete

Mailbox

On-Demand

Skip (detect only) Clean Delete Default: Skip (detect only)

(2)

Table 2

SCAN JOB AND FILTER TYPES

The following table shows correlation between the scan job and filter types.

Filter Type

File Keyword Allowed

Senders

Subject

Lines

Sender-Domain

Scan Job Type

Hub Transport

or

Edge Transport

Yes Yes Yes Yes Yes

Mailbox

Realtime

Yes No No Yes Yes

Mailbox

Scheduled

Yes No No Yes Yes

Mailbox

On-Demand

Yes No No Yes Yes

Table 3

Scan Sequence When a message is scanned by an FPE scan process, it is processed by antimalware engines and

filtering engines in one pass. This is done by navigating each part of the encoded message or

compressed files in a recursive manner. This maximizes the performance and increases the

complexity of the process. The following diagrams depict the logic flow of the scan and action

sequence for the scan process.

MESSAGE HEADER SCAN AND ACTION SEQUENCE

An

tim

alw

are

/Filte

rin

g A

ge

nt

No

Yes

No

No

Me

ssa

ge

He

ad

er

Sca

nn

ing

No

Process message headers

Yes

[Transport] Is the action identify? Tag(s) added to header(s)Yes

Is the action purge? Message removed from pipelineYes

No

No

Does message match a

sender/domain filter?

Yes

[Transport] Is the action identify? Tag(s) added to header(s)Yes

Is the action purge? Message removed from pipelineYes

No

Does message header

match a subject filter

Does message match an allowed sender list

for subject or sender filtering?

MESSAGE SCAN AND ACTION SEQUENCE

The following diagrams depict the logic flow of the scan and action sequence for the message

body and attachments.

Note:

The scan sequence is a recursive operation based on file navigation flow.

“End of execution” means to go back to the last level of execution of the recursive action. For

example, a message contains a.zip as an attachment, and a.zip contains b.exe and c.doc. If b.exe

is spyware but not a virus, and the spyware scan action is “Delete”, file b.exe will be replaced

with Deletion Text “b.txt”, and the execution will end for b.exe and the flow will go back to the

scan of the next container subpart, c.doc.

A

ntim

alw

are/F

ilte

rin

g A

ge

nt

No

Yes

No

Yes

Yes

No

Yes

Does file contain a worm?

No

Does file contain a virus?Yes

Yes

No

Does message contain spyware?

Yes

No No

No; action is skip

If container, have all subparts been scanned yet?No

Yes

No

Yes

No

Yes

No; action is skip

No; action is skip

Yes

Yes

No

NoYes

Yes

NoYes

Wo

rm

File

Filte

rin

gK

eyw

ord

Filte

rin

g

Process all file parts from message

Message removed from pipeline

Does sender match an allowed sender list

for file filtering?Does file name or type

match a file filter?

Check if is container

[Transport] Is this

file a message

body?

Does sender match

an allowed sender list

for keyword filtering?

Does message body

match a keyword filter?

YesMessage removed from pipelineIs the action purge?

YesTag(s) added to header(s)[Transport] Is the action identify?

Process all file parts from container

Yes Yes

NoYes

Yes

NoYes

No

Is the action purge? Message removed from pipeline

Was part of a container?

Deletion text insertedIs the action delete?

No; action is skip

Yes

NoYes

Yes

NoYes

Is the action purge? Message removed from pipeline

New container replaces old

Treated as corrupted

compressed fileCan file be rebuilt?Was part of a container?

Deletion text insertedIs the action delete?

No

[Transport] Is the action identify? Tag(s) added to header(s)

New container replaces old

Treated as corrupted

compressed file

Can file be rebuilt?

Was part of a container?

Is the action clean? Was clean successful?

Is the action delete? Deletion text inserted

Treated as corrupted

compressed file

New container replaces old

Can file be rebuilt?

No

No

Viru

sS

pyw

are

No

YesWas file a subpart of a container?

No End of execution

Continue to workload pipeline

Summary

We summarized some of the core functionalities in Forefront Protection for Exchange Server

and provided detailed views of malware scanning and filtering. This should give you an in-depth

understanding of the product to leverage the superior protection provided by FPE.

The vision behind this product line is to maximize protection by building a solution that is com-

ponentized and is adaptive to current and future scanning technologies. We are working hard

towards that goal.

Your feedback is critical for improving the existing product and building more successful ones in

the future.