NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident...

33
NATIONAL NETWORK FOR SPAM MONITORING Juan Díez González Security Technician - INTECO-CERT April, 2008 20th Annual FIRST Conference on Computer Security Incident Handling

Transcript of NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident...

Page 1: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

NATIONAL NETWORK FOR SPAM MONITORING

Juan Díez González

Security Technician - INTECO-CERT

April, 2008

20th Annual FIRST Conference on Computer Security Incident Handling

Page 2: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

2

Summary

1. INTECO and INTECO-CERT

2. Spam Monitoring Network

1. Sensors Network in Spain

2. Spam Network description

1. Client side

2. Server side

3. Portal side

3. Spam Web site

4. Problems found

Page 3: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

3

The National Communications Technology Institute

� Convergence of the Spanish and European Information Society � Promotion of regional development creating a high innovation "Cluster-TIC“� Create communications solutions for companies and individuals. � Consolidate as the main Spanish centre of innovative and reference programs and projects.

OBJECTIVES

� Promoted by the Ministry of Industry, Tourism and Trade, � Platform for the development of the Knowledge Society� Foundation : projects in the innovation and technology area.

What is INTECO?

Page 4: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

4

INTECO Security programs

Establish the bases for the coordination of different public initiatives in the information security area

Promoting applied research and specialisedtraining activities in the TIC security area.

Become the national IT Security Reference Centre .

e-trust

Services to SME’s Services to Citizens

INTECO-CERT, Computer Emergency Response Team for SMEs and Citizens

Security Technologies Show-Room for SMEs

Information Security

Observatory

INTECO Security programs

Page 5: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

5

INTECO-CERT. Objectives

Increase the level of awareness in the security area and enforcethe usage of security solutions for SMEs and homes.Provide provision of reactive and preventive services and procedures for security incidents.Present training facilities on technology and information security.Provide best practices, recommendations and advice.Show available security solutions for SMEs and citizens

INTECO-CERT

Page 6: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

6

Information Services:• Subscription to security reports, alerts

• News, events

• Online virus warnings, software vulnerabilities, spam .

Training Services: Tutorials, manuals, online courses.

Protection Services: free tools, software updates.

Response and Support Services:• Security Incidents management.

• Malware infections.• Phishing attacks.

• Legal support.

• Security forums.

INTECO-CERT Services

Page 7: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

7

To obtain real-time information about SPAM to give a general view about how spam affects organizations and citizens

To compare this information with other available sources of information on malware

To share this information with other interested organizations

Objectives

How? INTECO’s Sensor

Network

SPAM monitoring network

Page 8: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

8

More than 150 organizations � more than 100 million real e-mails processed per day.

More than 5 years used to get virus detection information

2,26% infected e-mails detected in almost 30 billion ones analyzed

Sensors Network in Spain

7%

11%

4%

13%

1%

27%

37%

National Administration

Regional Administration

Province Administration

Local Administration

Internationals

Business

University and Research

Sensors network

Page 9: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

9

Functional Diagram

Sensor_script

Report

Organization

Logs

IODEF

bzip2

Delivery

DB Oltp

Validation and DB load

Analysis

Web Portal

Central Server Internet

Antispam

DB Olap

SMIME

Page 10: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

10

Sensor Script

Written in Perl.

Tailored for every organizationSpam detection Report using:

IODEF format

Zip or Bzip2 compressionSMIME delivery

Sensor_script

Report

Organization

Logs

IODEF

bzip2

Delivery

Antispam

SMIME

Report Contains:Report Info. Date, server …Totals Section

Per hourPer method

Email origin IP for every emailDetection method used for every IP

Page 11: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

11

Sensor Script

ReportOrganizacion: Nombre _OrganizaciónASN: Número _ASNSensor: Nombre del SensorFecha: AAAA-MM-DDT HH:MM:SS±UMTTipo Origen: postfixVersion: 3.0Fecha inicio: 2007-02-08T11: 33:48+01:00Fecha final: 2007-02-08T11: 35:31+01:00Numero de relays: 10Mensajes Procesados: 37Spam Detectado: 29 78.38Spam Pasado: 0 0.00Spam Rechazado: 29 100.00Spam Declarado: 0 0.00Spam por Analisis de Contenido: 0 0.00Spam por Politica de Conexion: 0 0.00Spam por otro metodo: 0 0.00Metodo Detectados Rechazados % --------------------------------------------------- -------------Bogofilter 25 25 86.21DSBL 4 4 1 3.79Horas Procesados Detectados Rechazados Declarados Contenido Conexion Otros % --------------------------------------------------- --------------------------------------------------- ---------2007-02-08T11 37 29 29 -1 -1 -1 -1 78.38 Relay Procesados Detectados Rechazados Declarados Contenido Conexion Otros--------------------------------------------------- ------------------------------------------------127.0.0.4 25 25 25 -1 -1 -1 -1127.0.0.2 4 4 4 -1 -1 -1 -183.113.61.243 1 0 0 -1 -1 -1 -181.4.161.50 1 0 0 -1 -1 -1 -162.42.230.12 1 0 0 -1 -1 -1 -161.229.107.225 1 0 0 -1 -1 -1 -1218.81.159.46 1 0 0 -1 -1 -1 -1172.18.0.127 1 0 0 -1 -1 -1 -1202.190.152.140 1 0 0 -1 -1 -1 -182.194.72.78 1 0 0 -1 -1 -1 -1

Relay Metodos--------------------------------------------------- -------------127.0.0.2 DSBL127.0.0.4 Bogofilter

Header

Summary

Ips

Methods

Page 12: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

12

Sensor Script

IODEF format

IODEF (Incident Object Description Exchange Format), defines a data representation to exchange security incidents among different CSIRT.

XML Syntax.Contains security incidents information

Advantages.

Increased automation in incident data processing, since the resources of security analysts to parse free-form textual documents will be reduced;

Decreased effort in standardizing similar data (even when highlystructured) from different sources;

Common format on which interoperable tools for incident handling and subsequent analysis can be built, specifically when data comes from multiple constituencies.

Page 13: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

13

Sensor Script

IODEF schema

Specific ExtensionBasic Model

Page 14: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

14

XML-IODEF spam report

Spam report

IODEF Report

Page 15: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

15

Sensor script

SMIME Delivery

INTECO-CERT CA certificate used to:

Generate one cert per organizationSign every report on SMIME delivery

Verify digital signature on the central server reception

Organization Central Server

Delivery

CertificateCA Certificate

Page 16: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

16

Central Server

SMIME validationDB Loading

Network analysis to get IP info:

DomainASN

Country

Organization…

BD Oltp

Validation and DB loading

Analysis

Central Server Internet

Page 17: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

17

Central Server

Analytical Environment to

Totalize data

Aggregate dataSpeed up web queries

Minimize web response time

BD Oltp

Web Portal

Central Server Internet

BD Olap

Page 18: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

18

Web Components

Spam StatisticsCustom Component

PHP/SWF Charts

• Powerful CMS (specially 1.5)

• Free (as in freedom) software

• Big supporting community

• Fast development (using Joomla API)

• Modular for new interface addition (web service?)

• Easily extensible (thanks to OOP)

PHP/SWF:• Eye-catching flash charts

• Totally customizable.

• PHP API for easyconfiguration.

• Not free, but cheap.

xajax:• Fast AJAX development.

• Easy to integrate if servercode is modular.

� Not fully customizable.

� Not accessible.

Page 19: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

19

Web Site: https://ersi.inteco.es

Page 20: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

20

World map: SPAM by COUNTRY

Page 21: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

21

Interactive bar chart: SPAM by COUNTRY

Page 22: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

22

Interactive bar chart: SPAM by ORGANIZATION

Page 23: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

23

Interactive bar chart: SPAM by DOMAIN

Page 24: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

24

Interactive bar chart: SPAM by ASN

Page 25: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

25

Line chart: TOTAL EVOLUTION of the SPAM

Page 26: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

26

Interactive line chart: MTA EVOLUTION

Page 27: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

27

Interactive area chart: SPAM by HOUR

Page 28: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

28

Interactive bar chart: SPAM by METHOD

Page 29: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

29

Interactive pie chart: SPAM TOTALS

Page 30: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

30

Interactive pie chart: MTA TOTALS

Page 31: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

31

Problems found

Client sideTailored for each organization

Email infrastructure changes.

Anti-SPAM products changesAnti-SPAM versions changes

Anti-SPAM Filter changes

Server side

Huge amount of input data. Sampling.

IP information changes. Ip resolution very often.Whois services response differences (RIPE, ARIN, APNIC..)

Page 32: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

32

Questions?

More info:

https://ersi.inteco.es

Sensors Support. [email protected]

Juan Díez. [email protected]

Luis Fernández. [email protected]

Page 33: NATIONAL NETWORK FOR SPAM MONITORING · 20th Annual FIRST Conference on Computer Security Incident Handling. 2 Summary 1. INTECO and INTECO-CERT 2. Spam Monitoring Network 1. Sensors

www.inteco.es