Performance measurements of syslog-ng Premium Edition … ·...

Performance measurements of syslog-ng PremiumEdition 4 F1

October 13, 2011

AbstractPerformance analysis of syslog-ng Premium Edition

Copyright © 1996-2011 BalaBit IT Security Ltd.

Table of Contents1. Preface ............................................................................................................................................. 32. Executive summary ........................................................................................................................... 4

2.1. Optimizing multithreaded performance ..................................................................................... 52.2. Use cases for performance improvement ................................................................................... 5

3. The test environment ......................................................................................................................... 74. Interpreting the measurement results ................................................................................................... 85. Summary ........................................................................................................................................ 10

5.1. About BalaBit ....................................................................................................................... 10Appendix 1. Detailed results .............................................................................................................. 11

2www.balabit.com

1. Preface

This document summarizes the findings of several performance tests and measurements carried out on differentversions of syslog-ng Premium Edition. The main focus was syslog-ng PE version 4 F1, but similar performancetests will be performed and published regularly with every major syslog-ng PE release.

3www.balabit.com

Preface

2. Executive summary

The main findings of the performance tests are as follows:

■ The fastest way the syslog-ng PE application can receive log messages from the network is using plainTCP transport with the syslog() source driver. Using the tcp() driver is not significantly slower.

■ Starting with version , syslog-ng PE can be run in multithreaded mode to scale to multiple CPUs orcores for increased performance. In case of multiple connections, the performance rates have beentested with 10 and 100 connections, and in both cases the results are the same.Depending on the exact syslog-ng PE configuration, environment, and other parameters, syslog-ng PEis capable of processing:

• over 150000 messages per second when receiving messages from a single connection and storingthem in text files;

• over 150000 messages per second when receiving messages from a single connection and storingthem in logstore files;

• over 500000 messages per second when receiving messages from multiple connections and storingthem in text files;

• over 500000 messages per second when receiving messages from multiple connections and storingthem in logstore files;

• over 100000 messages per second when receiving messages from secure (TLS-encrypted) connectionsand storing them in text files.

NoteBy default, syslog-ng PE runs in single-thread mode. Multithreading must be explicitly enabled in the syslog-ngPE configuration file using the threaded(yes) option.

■ The tests were performed using 200-byte log messages and ran for 120 seconds.

NoteIn certain cases, the rate of writing into plain-text files has increased from 180000 messages per second to500000 messages per second since syslog-ng PE version 4.0.

The rate of writing into logstore files has increased in certain cases from 170000 messages per second to 500000messages per second since syslog-ng PE version 4.0.

■ Using TLS-encrypted transport seriously degrades the rate of receiving messages, down to a rate of 30%of normal performance.

■ PatternDB only works on one thread. Therefore, using PattnerDB can significantly decrease performance.If you are receiving lots of messages, but not all of them are processed with PatternDB, there is no sig-nificant performance degradation.

■ If you are receiving messages through one connection only, performance will be better without multi-threading (threaded(no)).

4www.balabit.com

Executive summary

■ Using disk buffer on the client side can significantly degrade performance. This is especially the case ifrelays that are heavily loaded are used together with diskbuffer enabled on relays. Performance can alsobe degraded if lots of messages are received on the syslog-ng server, and are forwarded through thenetwork (for example to a log analyzer or SIEM), and therefore disk buffer is enabled on the server aswell.

■ Regarding flow-control, using soft flow-control is faster by 10% than using hard flow-control.

■ Higher stats_level decreases the performance. For example, stats_level(2) means -10% inperformance.

■ Simple filtering (for example, filtering on facility or tag) decreases the performance only by a few percent.However, regular expressions significantly decrease the message-processing rate, by about 50-75%, de-pending on the complexity of the regular expression.

■ Version 4.1.1 of syslog-ng PE provides true multi-core support. For the best performance, use multi-core hardware for both the source and the destination side. For details on using multi-threading, seeChapter Multithreading and scaling in syslog-ng PE in The syslog-ng Premium Edition Administrator Guide.

2.1. Optimizing multithreaded performance

Destinations that have a queue process that queue in a single thread. Multiple sources can send messages to thesame queue, so the queue can scale to multiple CPUs. However, when the writer thread writes the queue contentsto the destination, it will be single-threaded.

Message parsing, rewrite rules, filters, and other types of message processing is performed by the reader thread ina sequential manner. This means that such operations can scale only if reading messages from the source can bemultithreaded. For example, if a tcp source can process messages from different connections (clients) in separatethreads. If the source cannot use multiple threads to process the messages, the operations will not scale.

To improve the processing power of syslog-ng PE and scale to more processors, use the following methods:

■ To improve scaling on the source side, use more sources, for example, more source files, or receive themessages from more parallel connections. For network sources, you can also configure a part of yourclients to send the messages to a different port of your syslog-ng server, and use separate source definitionsfor each port.

■ On the destination side, when writing the log messages to files, use macros in the filename to split themessages to separate files (for example, using the $HOST macro). Files with macros in their filenamesare processed in separate writer threads.

■ On the destination side, when sending messages to a syslog-ng server, you can use multiple connectionsto the server if you configure the syslog-ng server to receive messages on multiple ports, and configurethe clients to use both ports.

2.2. Use cases for performance improvement

Improving performance with lots of connections.

If there are several thousand active connections simultaneously, it is advised to place relay syslog-ng-s on anothercomputer in front of the syslog-ng. Switching between the active connections is time-consuming, while the amountof incoming messages is usually not significant. This problem is solved by using relays, since they are collecting thelogs. The syslog-ng solution can handle lots of log messages sent from a few connections easily.

5www.balabit.com

Optimizing multithreaded performance

http://www.balabit.com/sites/default/files/documents/syslog-ng-pe-4.1-guides/syslog-ng-pe-v4.1-guide-admin-en.html/chapter-multithreading.html

Storing lots of log messages.

It requires large free disk space if syslog-ng receives lots of messages per second (note that syslog-ng can processmessages at even 100 MB/sec speed). In this case it is advised to use compressed logstore instead of file for storingdata. The size of a compressed logstore (compress(3)) is only a few percent of the file destination. Obviously,the effectiveness of the compression depends on the pattern of the incoming messages, but since most of the timethe log messages consist of simple text messages, they can be compressed quite effectively (around 90% compressionrate).

Writing data into SQL database.

The syslog-ng will only commit after the amount of inserts set in flush-lines(), ifflags(explicit-commints) is enabled and the value of flush-lines() is raised. The speed of writing intoan SQL database can be 2-3 times faster if flush-lines(100) and flags(explicit-commints) is enabled.

Filtering messages.

It is advised to use the simplest filters when filtering incoming messages. If a message can be filtered with severaltypes of filters, check the measured data. For example, if a message is filtered with a complicated regexp, theperformance of syslog-ng can drop down to 5%, whereas if tag or facility, there is no performance decrease.

Identifying bottlenecks.

It is advised to first identify the bottlenecks if the performance of syslog-ng seems to be too low. For example, ifsyslog-ng is writing on slow disks at the destination side, upgrading the source side device will not help increasingthe performance.

6www.balabit.com

Use cases for performance improvement

3. The test environment

The test environment consisted of a single client and a server hardware, connected via a Gigabit switch. Note thatin certain test runs, the client opened several separate connections to the servers to simulate real-life logging envir-onments. The syslog-ng Premium Edition application was installed from the .run package.

The client and the server hardware were identical with the following main parameters:

■ 2x Intel® Xeon® Processor E5620 (12M Cache, 2.40 GHz, 5.86 GT/s Intel® QPI, quad-core)

■ 18 GB RAM

■ Western Digital RE3 WD2502ABYS 250GB 7200 RPM 16MB Cache SATA 3.0Gb/s 3.5" InternalHard Drives

Themain operating system of the hosts was ubuntu-lucid-x64, but the syslog-ng PE client and server applicationswere running in chrooted debian-etch-x64 environments.

7www.balabit.com

The test environment

4. Interpreting the measurement results

Several different measurements were performed in various syslog-ng configurations and settings. These are listedin Appendix 1, Detailed results (p. 11) in the following format.

-log_destination_file_macro-threaded_no-active-idle_100-0-log_source_legacy_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 144494.31 msg/sec, count=17340320, time=120.006, (average) msg size=200, bandwidth=28205.29 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 3.2G

The results can be interpreted as follows:

The configuration parameters of the syslog-ng PE server are listed as a string of name-value pairs. Whitespaces inthe name are replaced with underscores. For example -log_destination_file_macro-threaded_no-active-idle_100-0-log_source_legacy_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_no:

■ hw_config: The identifier of the hardware configuration used in the tests. For details, see Section 3, The testenvironment (p. 7).

■ platform: The operating system and architecture used in the test, for example, debian-etch_amd64.

■ log_source: The type of source driver used in the test. The legacy name refers to syslog-ng drivers usingthe BSD-syslog standard (RFC3164), while syslog to the IETF-syslog standards (RFC5424).

■ log_size: The length of the log messages in bytes.

■ flush_lines: The value of the flush_lines() parameter.

■ modifiers: Modifications (for example, rewriting or filtering) used in the log path (if any).

■ file/logstore: All incoming messages are written into the same file or logstore.

■ file/logstore_macro: All messages from each connection are written into a separate file or logstore.

■ threaded: Whether multi-threading is used or not.

■ active-idle: The number of active and idle connections between the client and the server. Log messageswere transferred only in the active connections.

■ fetch_limit-iw_size: The values of the fetch_limit() and iw_size() parameters, respectively.

■ log_destination: The type of destination driver used in the test.

■ runtime: The length of the test run (in seconds).

■ journal_block_size: The value of the journal_block_size parameter.

8www.balabit.com

Interpreting the measurement results

■ max_connections: The value of the max_connections parameter.

■

Test results measured on the syslog-ng PE client, for example average rate = 182588.16 msg/sec, count=21910693,time=120.000, (last) msg size=200, bandwidth=35641.21 kB/sec:

■ Average rate: The rate of messages the client sent the log messages to the server (messages per second).

■ Count: The total number of log messages sent by the clients during the test run.

■ Time: The length of the test run (in seconds).

■ Msg size: The size of the log messages sent by the client (in bytes).

■ Bandwidth: The total bandwith used by the client connections (in kilobytes per seconds).

9www.balabit.com

Interpreting the measurement results

5. Summary

The test measurements show that the processing capabilities of syslog-ng PremiumEdition have increased significantlyin version 4.1, and that syslog-ng PE is capable of receiving and processing high-volume log traffic.

5.1. About BalaBit

BalaBit IT Security Ltd. is a developer of network security solutions satisfying the highest standards. BalaBit wasfounded and is currently owned by Hungarian individuals. Its main products are the syslog-ng system logging software,which is the most widely used alternative syslog solution of the world; the syslog-ng Store Box logserver appliance;Zorp, a modular proxy gateway capable of inspecting over twenty protocols, including encrypted ones like SSL andSSH, and the Shell Control Box, an appliance that can transparently control, audit, and replay SSH, RDP, VNC, Citrix,and Telnet traffic.

To learn more about commercial and open source BalaBit products, request an evaluation version, or find a reseller,visit the following links:

■ The syslog-ng homepage

■ Shell Control Box homepage

■ syslog-ng Store Box (SSB) homepage

■ Product manuals, guides, and other documentation

■ Register and request an evaluation version

■ Find a reseller

All questions, comments or inquiries should be directed to <[email protected]> or by post to the following address: BalaBit IT Security 1115 Budapest,Bártfai str. 54 Phone: +36 1 3710540 Fax: +36 1 2080875 Web: http://www.balabit.com/

Copyright © 2011 BalaBit IT Security Ltd. Some rights reserved. This document is published under the Creative Commons Attribution NoncommercialNo Derivative Works (byncnd) 3.0 license. All other product names mentioned herein are the trademarks of their respective owners.

The latest version is always available at the BalaBit Documentation Page.

10www.balabit.com

Summary

http://www.balabit.com/network-security/syslog-ng/

http://www.balabit.com/network-security/syslog-ng/log-server-appliance/

http://www.balabit.com/network-security/zorp-gateway/

http://www.balabit.com/network-security/scb/

http://www.balabit.com/network-security/syslog-ng/

http://www.balabit.com/network-security/scb/

http://www.balabit.com/network-security/syslog-ng/log-server-appliance/

http://www.balabit.com/support/documentation/

https://www.balabit.com/mybalabit/

http://www.balabit.com/partnership/commercial/

http://www.balabit.com/

http://www.balabit.com/support/documentation/

Appendix 1. Detailed results

The following measurement results were obtained using syslog-ng PE 4.1.1 in the environment described in Section3, The test environment (p. 7).


-log_destination_file_macro-threaded_no-active-idle_100-0-log_source_leg-acy_tcp_tls-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 41125.41 msg/sec, count=4976358, time=121.004, (average) msg size=200, bandwidth=8027.68 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 926M

-log_destination_file_macro-threaded_no-active-idle_100-0-log_source_legacy_udp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noserver side average rate = 73888 msg/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)

11www.balabit.com

Detailed results

Results:Size of stored outputfiles: 1.7G

-log_destination_file_macro-threaded_no-active-idle_100-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 150482.21 msg/sec, count=18058484, time=120.004, (average) msg size=204, bandwidth=29961.61 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 3.1G

-log_destination_file_macro-threaded_no-active-idle_100-0-log_source_sys-log_tcp_tls-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 42256.31 msg/sec, count=5113231, time=121.005, (average) msg size=204, bandwidth=8413.40 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 888M

-log_destination_file_macro-threaded_no-active-idle_10-0-log_source_legacy_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 151873.38 msg/sec, count=18228666, time=120.025, (average) msg size=200, bandwidth=29645.68 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)

12www.balabit.com

Detailed results



-log_destination_file_macro-threaded_no-active-idle_10-0-log_source_legacy_udp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noserver side average rate = 104504 msg/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 2.3G


13www.balabit.com

Detailed results




14www.balabit.com

Detailed results

-log_destination_file_macro-threaded_no-active-idle_1-0-log_source_legacy_udp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noserver side average rate = 132674 msg/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 1Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 2.9G



-log_destination_file_macro-threaded_yes-active-idle_100-0-log_source_legacy_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_no

15www.balabit.com

Detailed results

average rate = 502478.02 msg/sec, count=60300083, time=120.005, (average) msg size=200, bandwidth=98083.71 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 11G

-log_destination_file_macro-threaded_yes-active-idle_100-0-log_source_leg-acy_tcp_tls-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 121424.46 msg/sec, count=14571587, time=120.005, (average) msg size=200, bandwidth=23702.05 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 2.7G

-log_destination_file_macro-threaded_yes-active-idle_100-0-log_source_legacy_udp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noserver side average rate = 75662 msg/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 1.7G

-log_destination_file_macro-threaded_yes-active-idle_100-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 491634.46 msg/sec, count=58999059, time=120.005, (average) msg size=204, bandwidth=97886.39 kB/sec

16www.balabit.com

Detailed results

Settings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 11G

-log_destination_file_macro-threaded_yes-active-idle_100-0-log_source_sys-log_tcp_tls-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 180296.55 msg/sec, count=21635887, time=120.001, (average) msg size=204, bandwidth=35897.76 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 3.7G

-log_destination_file_macro-threaded_yes-active-idle_10-0-log_source_legacy_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 556482.98 msg/sec, count=66790618, time=120.022, (average) msg size=200, bandwidth=108625.48 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 13G

-log_destination_file_macro-threaded_yes-active-idle_10-0-log_source_leg-acy_tcp_tls-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 184389.45 msg/sec, count=22128464, time=120.009, (average) msg size=200, bandwidth=35992.82 kB/sec

17www.balabit.com

Detailed results

Settings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 4.1G

-log_destination_file_macro-threaded_yes-active-idle_10-0-log_source_legacy_udp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noserver side average rate = 116542 msg/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 2.6G

-log_destination_file_macro-threaded_yes-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 574520.59 msg/sec, count=68945535, time=120.005, (average) msg size=204, bandwidth=114389.35 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 12G

-log_destination_file_macro-threaded_yes-active-idle_10-0-log_source_sys-log_tcp_tls-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 184399.17 msg/sec, count=22130108, time=120.011, (average) msg size=204, bandwidth=36714.61 kB/secSettings:Used log_fifo_size : 10000

18www.balabit.com

Detailed results

Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 3.8G

-log_destination_file_macro-threaded_yes-active-idle_1-0-log_source_legacy_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 113075.15 msg/sec, count=13569130, time=120.000, (average) msg size=200, bandwidth=22072.27 kB/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 1Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 2.5G

-log_destination_file_macro-threaded_yes-active-idle_1-0-log_source_leg-acy_tcp_tls-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 39871.26 msg/sec, count=4784753, time=120.005, (average) msg size=200, bandwidth=7782.87 kB/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 1Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 890M

-log_destination_file_macro-threaded_yes-active-idle_1-0-log_source_legacy_udp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noserver side average rate = 87227 msg/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000

19www.balabit.com

Detailed results

Used log_fetch_limit : 1000Used max_connections : 1Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 2.0G

-log_destination_file_macro-threaded_yes-active-idle_1-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 121715.44 msg/sec, count=14605893, time=120.000, (average) msg size=204, bandwidth=24234.03 kB/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 1Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 2.5G

-log_destination_file_macro-threaded_yes-active-idle_1-0-log_source_sys-log_tcp_tls-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 39731.51 msg/sec, count=4768123, time=120.008, (average) msg size=204, bandwidth=7910.70 kB/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 1Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 828M

-log_destination_file_macro-threaded_no-active-idle_100-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 143315.72 msg/sec, count=17198753, time=120.006, (average) msg size=204, bandwidth=28534.73 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000

20www.balabit.com

Detailed results

Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 3.0G



-log_destination_file_macro-threaded_yes-active-idle_100-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 477960.00 msg/sec, count=57357348, time=120.004, (average) msg size=204, bandwidth=95163.75 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:

21www.balabit.com

Detailed results

Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 9.8G


-log_destination_file_macro-threaded_yes-active-idle_1-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 123077.75 msg/sec, count=14769334, time=120.000, (average) msg size=204, bandwidth=24505.27 kB/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 1Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 2.6G

-log_destination_file-threaded_no-active-idle_100-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 193706.35 msg/sec, count=23246155, time=120.007, (average) msg size=204, bandwidth=38567.71 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:

22www.balabit.com

Detailed results

syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 4.0G

-log_destination_file-threaded_no-active-idle_10-0-log_source_syslog_tcp-plat-form_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 196727.29 msg/sec, count=23618159, time=120.055, (average) msg size=204, bandwidth=39169.19 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 4.1G

-log_destination_file-threaded_no-active-idle_1-0-log_source_syslog_tcp-plat-form_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 210884.41 msg/sec, count=25306158, time=120.000, (average) msg size=204, bandwidth=41987.93 kB/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 1Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 4.3G

-log_destination_file-threaded_yes-active-idle_100-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 403383.35 msg/sec, count=50423789, time=125.002, (average) msg size=204, bandwidth=80315.24 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)

23www.balabit.com

Detailed results


-log_destination_file-threaded_yes-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 468737.67 msg/sec, count=56263034, time=120.030, (average) msg size=204, bandwidth=93327.55 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 9.6G

-log_destination_file-threaded_yes-active-idle_1-0-log_source_syslog_tcp-plat-form_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 152239.01 msg/sec, count=18268687, time=120.000, (average) msg size=204, bandwidth=30311.40 kB/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 1Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 3.1G

-log_destination_logstore_macro-threaded_no-active-idle_100-0-log_source_sys-log_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 136615.68 msg/sec, count=16395047, time=120.008, (average) msg size=204, bandwidth=27200.73 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used compress : 3Used journal_block_size : 1048576Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:

24www.balabit.com

Detailed results

syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 54M

-log_destination_logstore_macro-threaded_no-active-idle_10-0-log_source_sys-log_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 143284.53 msg/sec, count=17197059, time=120.020, (average) msg size=204, bandwidth=28528.52 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used compress : 3Used journal_block_size : 1048576Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 55M

-log_destination_logstore_macro-threaded_no-active-idle_1-0-log_source_sys-log_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 150296.47 msg/sec, count=18035922, time=120.002, (average) msg size=204, bandwidth=29924.63 kB/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000Used compress : 3Used journal_block_size : 1048576Used log_fetch_limit : 1000Used max_connections : 1Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 57M

-log_destination_logstore_macro-threaded_yes-active-idle_100-0-log_source_sys-log_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 411002.81 msg/sec, count=49322230, time=120.004, (average) msg size=204, bandwidth=81832.30 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000

25www.balabit.com

Detailed results

Used compress : 3Used journal_block_size : 1048576Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 157M

-log_destination_logstore_macro-threaded_yes-active-idle_10-0-log_source_sys-log_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 501953.32 msg/sec, count=60236225, time=120.003, (average) msg size=204, bandwidth=99940.91 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used compress : 3Used journal_block_size : 1048576Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 189M

-log_destination_logstore_macro-threaded_yes-active-idle_1-0-log_source_sys-log_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_en-cryption_noaverage rate = 122072.75 msg/sec, count=14648802, time=120.000, (average) msg size=204, bandwidth=24305.17 kB/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000Used compress : 3Used journal_block_size : 1048576Used log_fetch_limit : 1000Used max_connections : 1Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 46M

-log_destination_logstore-threaded_no-active-idle_100-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_no

26www.balabit.com

Detailed results

average rate = 127963.49 msg/sec, count=15356122, time=120.003, (average) msg size=204, bandwidth=25478.04 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used compress : 3Used journal_block_size : 1048576Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 52M

-log_destination_logstore-threaded_no-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 169451.51 msg/sec, count=20339527, time=120.031, (average) msg size=204, bandwidth=33738.47 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used compress : 3Used journal_block_size : 1048576Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 65M

-log_destination_logstore-threaded_no-active-idle_1-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 169138.37 msg/sec, count=20296728, time=120.000, (average) msg size=204, bandwidth=33676.13 kB/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000Used compress : 3Used journal_block_size : 1048576Used log_fetch_limit : 1000Used max_connections : 1Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)

27www.balabit.com

Detailed results

Results:Size of stored outputfiles: 64M

-log_destination_logstore-threaded_yes-active-idle_100-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 159231.46 msg/sec, count=19108020, time=120.001, (average) msg size=204, bandwidth=31703.62 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used compress : 3Used journal_block_size : 1048576Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 70M

-log_destination_logstore-threaded_yes-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 165739.87 msg/sec, count=19893294, time=120.027, (average) msg size=204, bandwidth=32999.47 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used compress : 3Used journal_block_size : 1048576Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 63M

-log_destination_logstore-threaded_yes-active-idle_1-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 153169.18 msg/sec, count=18380582, time=120.001, (average) msg size=204, bandwidth=30496.60 kB/secSettings:Used log_fifo_size : 1000Used log_iw_size : 1000Used flush_lines : 1000Used compress : 3Used journal_block_size : 1048576Used log_fetch_limit : 1000Used max_connections : 1

28www.balabit.com

Detailed results

Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 58M

-log_destination_file_macro-threaded_no-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_all_modifiers-logstore_en-cryption_noaverage rate = 64904.17 msg/sec, count=7793054, time=120.070, (average) msg size=204, bandwidth=12922.68 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : filter(f_filter_easy_regexp)Used modifier : filter(f_filter_tag_regexp)Used modifier : filter(f_filter_facility_filter)Used modifier : parser(pdb)Used modifier : rewrite(rewrite_set)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 1.3G

-log_destination_file_macro-threaded_no-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_complex_regexp-logstore_en-cryption_noaverage rate = 56582.29 msg/sec, count=6794026, time=120.073, (average) msg size=204, bandwidth=11265.76 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : filter(f_filter_complex_regexp)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 1.2G

-log_destination_file_macro-threaded_no-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_easy_regexp-logstore_en-cryption_noaverage rate = 103321.84 msg/sec, count=12402335, time=120.035, (average) msg size=204, bandwidth=20571.79 kB/secSettings:

29www.balabit.com

Detailed results

Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : filter(f_filter_easy_regexp)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 2.2G

-log_destination_file_macro-threaded_no-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_facility_filter-log-store_encryption_noaverage rate = 163012.89 msg/sec, count=19566065, time=120.027, (average) msg size=204, bandwidth=32456.52 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : filter(f_filter_facility_filter)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 3.4G


-log_destination_file_macro-threaded_no-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_patterndb-logstore_en-cryption_noaverage rate = 93069.78 msg/sec, count=11172904, time=120.048, (average) msg size=204, bandwidth=18530.57 kB/secSettings:

30www.balabit.com

Detailed results

Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : parser(pdb)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 1.9G

-log_destination_file_macro-threaded_no-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_rewrite_set-logstore_en-cryption_noaverage rate = 142911.23 msg/sec, count=17153559, time=120.029, (average) msg size=204, bandwidth=28454.20 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : rewrite(rewrite_set)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 2.9G

-log_destination_file_macro-threaded_no-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_tag_filter-logstore_en-cryption_noaverage rate = 151612.02 msg/sec, count=18197879, time=120.029, (average) msg size=204, bandwidth=30186.56 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : filter(f_filter_tag_filter)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 3.1G

-log_destination_file_macro-threaded_yes-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_all_modifiers-logstore_en-cryption_noaverage rate = 42832.42 msg/sec, count=5140790, time=120.020, (average) msg size=204, bandwidth=8528.11 kB/sec

31www.balabit.com

Detailed results

Settings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : filter(f_filter_easy_regexp)Used modifier : filter(f_filter_tag_regexp)Used modifier : filter(f_filter_facility_filter)Used modifier : parser(pdb)Used modifier : rewrite(rewrite_set)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 863M

-log_destination_file_macro-threaded_yes-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_complex_regexp-logstore_en-cryption_noaverage rate = 30426.56 msg/sec, count=3652717, time=120.050, (average) msg size=204, bandwidth=6058.05 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : filter(f_filter_complex_regexp)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 635M

-log_destination_file_macro-threaded_yes-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_easy_regexp-logstore_en-cryption_noaverage rate = 162411.20 msg/sec, count=19491113, time=120.010, (average) msg size=204, bandwidth=32336.72 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : filter(f_filter_easy_regexp)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 3.4G

32www.balabit.com

Detailed results

-log_destination_file_macro-threaded_yes-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_facility_filter-log-store_encryption_noaverage rate = 552573.53 msg/sec, count=66313455, time=120.008, (average) msg size=204, bandwidth=110019.60 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : filter(f_filter_facility_filter)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 12G


-log_destination_file_macro-threaded_yes-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_patterndb-logstore_en-cryption_noaverage rate = 42391.15 msg/sec, count=5088634, time=120.040, (average) msg size=204, bandwidth=8440.25 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : parser(pdb)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 884M

33www.balabit.com

Detailed results

-log_destination_file_macro-threaded_yes-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_rewrite_set-logstore_en-cryption_noaverage rate = 551997.14 msg/sec, count=66256784, time=120.031, (average) msg size=204, bandwidth=109904.84 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : rewrite(rewrite_set)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 11G

-log_destination_file_macro-threaded_yes-active-idle_10-0-log_source_syslog_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_tag_filter-logstore_en-cryption_noaverage rate = 567792.55 msg/sec, count=68189840, time=120.096, (average) msg size=204, bandwidth=113049.77 kB/secSettings:Used log_fifo_size : 10000Used log_iw_size : 10000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 10Modifiers:Used modifier : filter(f_filter_tag_filter)syslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: 12G

-log_destination_file_macro-threaded_yes-active-idle_30-0-log_source_legacy_tcp-platform_debian-etch_amd64-syslogger_syslogd-modifiers_NA-logstore_encryption_noaverage rate = 817309,1 msg/sec, count=490399609, time=600.017, (average) msg size=150, bandwidth=119723,012 kB/secSettings:Used log_fifo_size : 100000Used log_iw_size : 100000Used flush_lines : 1000Used log_fetch_limit : 1000Used max_connections : 100Modifiers:Used modifier : NAsyslog-ng version:syslog-ng 4 F1 (4.1.1)Results:Size of stored outputfiles: NA

34www.balabit.com

Detailed results

Since in certain measurements the maximum capacity of the Gigabit network interface and the hard disk provedto be a bottleneck, a slightly modified environment was used for the last measurement:

■ The server received log messages on 2 gigabit network cards instead of 1

■ The length of the messages was reduced to 150 bytes

■ To overcome the I/O limitation of the hard disk, 30% of the messages was written to /dev/null, becausesyslog-ng reached the maximum performance of the specified hard disk. Note that writing messages to/dev/null is generally slower than writing them to a hard drive.

35www.balabit.com

Detailed results

Performance measurements of syslog-ng Premium Edition … ·...

Documents

Transcript of Performance measurements of syslog-ng Premium Edition … ·...