Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Ulogd2, Advanced firewall logging

Eric Leblond

INL172 rue de Charonne75011 Paris, France

RMLL 2009, July 8, Nantes

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 1/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Some words about me

NuFW main developper

INL co-founderNetfilter hacker

some kernel stuffuserspace libraryulogd2organizer of Netfilter Workshop 2008

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 2/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Netfilter logging history

At the beginning was syslog

Pre Netfilter days

Flat packet loggingOne line per packet

A lot of informationNon searchable

INPUT DROP IN=eth0 OUT= MAC=00:1a:92:05:ee:68:00:b0:8e:83:3b:f0:08:00 SRC=62.212.121.211 \DST=91.121.73.151 LEN=60 TOS=0x00 PREC=0x00 TTL=58 ID=35342 DF PROTO=TCP SPT=59261 \DPT=113 WINDOW=5440 RES=0x00 SYN URGP=0

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 3/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Netfilter logging history

Ulogd days

Netfilter introduces ULOG targetiptables -A INPUT -p tcp -j ULOG --ulog-prefix "bad packet"

Communication via a netlink socketSpecial type of socketused for kernel userspace bidirectionnal communication

Ulogd, a logging daemonSyslog and file outputSQL output: PGSQL, MySQL, SQLite

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 4/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Netfilter logging history

Linux 2.6.14: Netfilter userspace reloaded

Netfilter introduces NFnetlinkRewrote userspace interactionFor logging, queueing and connection trackingMultiple communication on a single netlink socket

Three new librarieslibnetfilter_queue: userspace decisionlibnetfilter_log: logginglibnetfilter_conntrack: connection tracking handling

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 5/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Netfilter logging history

Ulogd2: an ulogd generalisation

Interact with the new libraries

Rewrite of ulogdlibnetfilter_log

Packet loggingIPv6 readyFew structural modification

libnetfilter_conntrackConnection tracking loggingAccounting, loggingCompletely new

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 6/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

1 Introduction

2 Connection tracking

3 Ulogd2 Architecture

4 Using Ulogd2

5 Conclusion

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 7/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Some words about connection tracking

Stateful filtering

Original IP packet filter:Filter only on IP header fieldsHave no idea of the packet history

Stateful filtering is:follow the history of connection

Is packet part of an existing connection ?Is packet correct relatively to the protocol ?

to determine the validity of a packet

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 8/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Some words about connection tracking

Netfilter connection tracking

Netfilter maintains a connection tableValid for "all" protocols

For flow-oriented protocol: TCP, SCTPFor protocol without state: UDP

Support both IPv4 and IPv6

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 9/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Some words about connection tracking

Network Address Translation

Private Network can’t go to internet

Firewall has to modify packet to show its addressTwo way of seeing a connection

From insideFrom outside

Conntrack keep track of the correspondance

tcp 6 431996 ESTABLISHED src=192.168.1.131 dst=91.121.73.151 sport=52964 dport=22\packets=13 bytes=772 src=91.121.73.151 dst=192.168.1.131 sport=22 dport=52964 \packets=11 bytes=7548 [ASSURED] mark=0 secmark=0 use=1 \

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 10/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Some words about connection tracking

libnetfilter_conntrack: Connection tracking handling library

Interrogation:Connections listingRetrieve information about a connection

IP informationAccounting statistics

Modification:Create new entryChange or fix timeoutChange mark

Destruction

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 11/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Some words about connection tracking

Connection tracking events

Send all significative connection related events touserspace :

NEW: connection creationESTABLISHED: Switch from NEW to ESTABLISHEDconnectionDESTROY: connection destruction

Make possible to maintain a connection history inuserspace

Accounting information

NAT decision history

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 12/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Netfilter logging

Ulogd2, a modular daemon

Able to use multiple entriesSupport for packet loggingSupport for flow logging

And multiple outputText basedDB based

Plugin based architectureEntryOutputFilters

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 13/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Netfilter logging

Ulogd2, schema of architecture

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 14/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Netfilter logging

Packet logging

Compatible with old kernelIPv4 support:

ULOGNFLOG

IPv6 support:NFLOG only

Hardware information:Network interfacesHardware header

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 15/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Netfilter logging

Connection tracking event logging

libnetfilter_conntrack based

IPv4 and IPv6

Listen to eventsContains the two IP tuples

Orig IP headerReply IP header

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 16/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

From input to output

The stack concept

Workflow based configuration: stackChoose an inputDescribe treatment and transformation to applyChoose an output

Based on key value propagation trough the stack

stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMUstack=ct1:NFCT,mark1:MARK,ip2str1:IP2STR,pgsql2:PGSQL

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 17/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

From input to output

The stack concept: plugin

Each plugin has :Input keysOutput keys

Plugin structure# /opt/ulogd2/sbin/ulogd --info /opt/ulogd2/lib/ulogd/ulogd_filter_IP2STR.soName: IP2STRInput keys:

Key: oob.family (unsigned int 8)Key: oob.protocol (unsigned int 16)Key: ip.saddr (IP addr)Key: ip.daddr (IP addr)[...]

Output keys:Key: ip.saddr.str (string)Key: ip.daddr.str (string)[...]

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 18/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

From input to output

Ulogd2, the stack concept

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 19/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

From input to output

Ulogd2, the stack concept

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 20/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

From input to output

Various output plugin

File-basedSyslogFilePCAP

DatabasesPGSQLMySQLSqlite (TODO)

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 21/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

From input to output

Treatment and filtering

Treatment plugins:Decoding plugins: BASE, IFINDEXConversion plugins: IP2STR, IP2BIN, MAC2STR

Filtering:Decide if treatment has to be continuedMARK plugin: stop propagation through stack if there is nomatch on mark

Multiplexing:Reusing INPUT dataMultiple logging

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 22/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

SQL output

Really use databases

Let dababase work to the databaseUse database capability

Procedure for insertionExtensible schemas

Optimize schemaAvoid empty fieldsIndex on most frequent request

Autoconfigurationulogd calls a procedureparams are taken from field name in a tableno need to recompile ulogd if we change the DB

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 23/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

SQL output

Easy modification

Procedure can do different things with dataProvided procedure

Insertion of all available data in DBFor connection trackingFor packet logging

Possible extensionArbitrary accountingStatistics

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 24/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

SQL output

Extensible database schemas

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 25/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

SQL output

Extensible database schemas

Easy to extendAdd table with your custom fieldlink ID of the new table with ulog2 ID.

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 26/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

SQL output

Use VIEW for usage ease

VIEW can be built for common task

TCP quad viewCREATE OR REPLACE VIEW view_tcp_quad ASSELECT ulog2._id,ulog2.ip_saddr_str,tcp.tcp_sport,

ulog2.ip_daddr_str,tcp.tcp_dportFROM ulog2 INNER JOIN tcp ON ulog2._id = tcp._tcp_id;

and provide easy select

TCP quad selectulog2=> SELECT ip_saddr_str,tcp_dport FROM view_tcp_quad;ip_saddr_str | tcp_dport---------------+-----------148.60.18.179 | 1194148.60.18.179 | 1194

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 27/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Packet logging

Security interest

Analysed dropped trafficAttack attemptScansWorms or trojan traffic

Analyse authorized trafficKeep a trace of access to critical dataForensic on succesful attackWork with other security subsystem

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 28/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Packet logging

Production interest

A firewall block some packetsPacket necessary for network servicesLogging is critical when setting up a new firewall

and detect misconfigurationPacket necessary for network servicesPackets revealing improper configuration of service

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 29/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Packet logging

Nulog: displaying packet data

Display SQL data (Ulogd1 format)Ulogd2 support in progress

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 30/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Connection event logging

Security interest

Advantages of logging flow over logging packetStart timeEnd timeVolume information

Better view of severity of the eventDuration informationData volumeNAT information

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 31/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Connection event logging

Recover internal IP from external data

Connection logging containsOrig IP tupleReply IP tuple

Someone from outside asks you information about anattack:

Extern world only knows the Reply tupleConnection logging lead you to the IP at the origin of anattack

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 32/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Connection event logging

Accounting

Each connection logging contains:bytes usagepacket usage

Summing usage lead you to global statisticUsing any IP criteria (per port or per IP bandwith)Or using external information (per user bandwith)

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 33/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

NF3D

NF3D

Data visualisation tryout

Represent both packet and connection on a graph

Link packet to their corresponding connection

Connections are displayed in a GANTT fashion

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 34/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

NF3D

NF3D: SSH brute force

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 35/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

NF3D

NF3D: Demonstration

Let’s pray Murphy.

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 36/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

A complete logging system

ulogd2 is a complete logging tool for NetfilterPacket loggingConnection logging

Easy to extendVia pluginVia database modification

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 37/ 38

Introduction Contents Connection tracking Ulogd2 Architecture Using Ulogd2 Conclusion

Questions ?

Contacts:Directly: eric@inl.frMailling List: netfilter-devel@vger.kernel.org

References:Ulogd2:http://netfilter.org/projects/ulogd/index.htmlUlogd2 documentation:http://software.inl.fr/trac/wiki/ulogd2/userNulog:http://software.inl.fr/trac/wiki/EdenWall/NuLogNF3D: http://software.inl.fr/trac/wiki/nf3d

Eric Leblond INL 172 rue de Charonne 75011 Paris, France

Ulogd2, Netfilter logging reloaded 38/ 38