IP Platforms Best Practices for Performance 010810
-
Upload
amandeepsi -
Category
Documents
-
view
229 -
download
1
Transcript of IP Platforms Best Practices for Performance 010810
-
8/3/2019 IP Platforms Best Practices for Performance 010810
1/45
2010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
IP Platforms BestPractices for Performance
Pierre Lamy
Technical Lead Ottawa TACApril 2010
-
8/3/2019 IP Platforms Best Practices for Performance 010810
2/45
2
22010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Intro and revision history
This document describes methods and
techniques that users can apply on variousCheck Point IP Security Appliances to achieveoptimal performance.
Version 1.0 October 2009 Word format Version 1.1 January 2010 Word + PPT, minor
revisions
Version 1.2 April 2010 Updates
-
8/3/2019 IP Platforms Best Practices for Performance 010810
3/45
3
32010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
General Performance Best Practices
These guidelines are Appliance independent and do not require any specialtuning.
Always use the latest versions of Check Point products. Always upgrade to themost recent HFA (HotFix Accumulator) for a given version.
Create a small block of rules near the top of your rulebase, containing themost heavily used rules. These rules should be fully accelerated withSecureXL.
Keep the rulebase simple and small. Reduce the number of rules bycombining similar rules together. Rules which disable SecureXL accelerationshould be placed very low in the rulebase.
If not using VPN (encryption) on the module, make sure the VPN product is
disabled on that module.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
4/45
4
42010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
General Performance Best Practices
Do not use QoS from IPSO or Floodgate.
Avoid using Domain Objects. DNS lookup takes additional CPU cycles.
Avoid using UFP URI Filtering Protocol as this is resource intensive.
Use Networks instead of address ranges for Network Address Translation.
Keep logging to a minimum. Only business critical rules which will beanalyzed should have logging enabled. Drop rules, Accept rules, Stealth rules,
Cleanup rules and Implied rules should not log unless there is a clear businesscase and the customer intends on analyzing the logs on a regular basis.Otherwise logging should only be used for debug purposes.
IP Cluster members should have exactly the same package lists, havingdissimilar packages can cause state sync issues resulting in a performancereduction.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
5/45
5
52010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
General Performance Best Practices
When you install an ADP interface module in an appliance, the networkprocessor in the card performs all VPN encryption and decryption, even for
VPN packets that ingress or egress through non-ADP interfaces. The built-inNokia encryption accelerator continues to accelerate IKE traffic but does notperform any other processing.
If VPN traffic ingresses or egresses through a non-ADP interface, throughputis negatively affected because the packets must transit the appliancesbackplane to reach the network processor in the ADP module. It isrecommended that one configure VPNs to use only ADP interfaces to avoidthis performance loss.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
6/45
6
62010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
General Performance Best Practices
Uniprocessor systems (IP152, IP292, IP395, IP565, IP1265) shoulduse IPSO 4.2 (this is correct in April 2010) while Multiprocessorsystems (IP695, IP1285, IP2455) should use IPSO 6.2. The latestbuild of any major release should always be used.
Multiprocessor systems should have Check Point R70 installed to take
full advantage of CoreXL technology. sk40465 has some more detailsabout this.
Use R70 + IPSO 6.2 on uniprocessor systems where there is a needfor a specific feature
Do not persist in the use of IPSO 4.2 or R65 once support is no longeroffered for these products.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
7/45772010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
General Performance Best Practices
Using interface flow control can reduce network
throughput on busy interfaces and we do not suggest it beenabled.
Avoid using SmartView Monitor to constantly monitor
system performance or collect historical data, asSmartView Monitor itself has an impact on performance.
Avoid using custom scripts on systems which have
performance issues, as the scripts will incur CPUresources.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
8/45882010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
General Performance Best Practices
A CST from the IPSO system as well as a cpinfo from themanagement station, are CRITICAL to provide to Check Point Supportwhen opening a case for assistance in troubleshooting performanceissues. Without at least those files, Check Point Support will be unableto assist the customer. For systems with extremely high CPU, orwhere running CST may cause problems, it is recommended to run itwith the following syntax: nice +20 cst. The command may take
hours to complete but will not divert critical system resources fromprocessing traffic in a live environment.
Any cpinfo that is provided to Check Point Support MUST begenerated using the latest cpinfo tool downloaded from the Support
site. This requires uninstalling the old cpinfo, and then installing thelatest one. Providing old cpinfo output to Check Point Support willdelay Support response.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
9/45992010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
General Performance Best Practices
Avoid using Standalone installations. Separate the management station fromthe enforcement point by running the management station on another system.
Use the default settings in the capacity optimization tab of the enforcementpoints properties, changing only the total connections number.
General recommendations for these platforms are to use onboard quad portsfor:
Security Gateway State Synchronization traffic Cluster protocol network traffic Policy and Appliance management traffic A path from the enforcement point to the Check Point Log server
SecureXL options should be matched between SXL and the Security Gatewaysettings; for example Sequence Validation and Delayed Notifications.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
10/4510102010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
System Resources
High performance relies on the availability ofkey system resources: CPU, memory, andnetwork interface bandwidth.
Tuning involves better using the currenthardware, not simply upgrading.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
11/4511112010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
System Resources - CPU
Small packet size traffic: The amount of traffic any network device can processis not determined by byte throughput numbers, but rather by the packets per
second. A small packet uses as many resources as a large full size ethernetpacket.
CPU utilization is incurred on a per-packet level, rather than per-byte.Therefore it is critical to note that a system that is processing a large numberof small packets, works as hard as a system processing the same number of
large packets. There may be a very large difference in the apparent bytethroughput between the systems.
State synchronization particularly demands high CPU cycles because thenodes in a cluster perform synchronization for every connection theyencounter. This ensures high availability but causes high CPU usage. This is
especially important to consider, when deciding whether or not to synchronizeshort lived connection types like DNS when using a VRRP pair the failovertime of VRRP exceeds the DNS timeout. It would not be advisable to syncDNS connections in that scenario.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
12/4512122010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
System Resources - CPU
High number of logging rules affects CPU. Logging uses CPUcycles and is discouraged where there is no need.
The Active Log feature in SmartView Tracker will severelycompromise the ability of an enforcement point to processtraffic, and for performance reasons should not be used.
Accounting Logging: By default, accounting logging producestwo kinds of log tracking (one in Log Viewer and one in AccountLog View) for the same connection. Alerts as a tracking option,also have a significant negative impact on performance.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
13/4513132010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
System Resources - CPU
Any configuration which disables SecureXL or forces traffic touse Slowpath affects CPU. This includes rule configurations,SmartDefense protections, and Floodgate.
NAT traffic incurs slightly more CPU impact that non-NATedtraffic. NAT in an ADP environment is strongly discouraged asconnection establishment rate acceleration does not work onNAT traffic.
Traffic which is not connection-rate accelerated, uses moreCPU resources than traffic which is. Connection establishment
and teardown uses (relatively) a lot of CPU resources, evenwhen all SecureXL acceleration is in use.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
14/4514142010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
System Resources Memory
Check Point recommends upgrading any Appliances memory to its full capacity toimprove performance.
The main factors that demand high RAM usage are: Concurrent connections Concurrent VPN tunnels NAT connections Security Servers
Use the Web User Interface to determine how much memory the Appliance hasinstalled. In the Web User Interface navigation tree, select Monitor --> System Utilization--> CPU-Memory Live Utilization. Look for the Total Real Memory value. The topcommand can also be used on recent IPSO versions to view the information on thecommand line.
The amount of memory, allocated to the Check Point Security Gateway to processnetwork traffic, is determined under the capacity optimization tab of the gateway object
properties. The Automatic values should always be used, and the manual setting ofmemory allocation should NEVER be used unless directed to by Check Point Support.
You should always set the capacity optimization to the maximum values (connections)supported by the platform and memory configuration as detailed in the release notes forIPSO. There is no drawback to doing this, but leaving the low defaults in place willresults in insufficient memory on busy systems.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
15/4515152010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
System Resources Memory
Two factors can reduce the number of Security Gatewayconnections that can be supported
Concurrent VPN tunnels are dependent upon the amount ofmemory available in the Appliance. As you add more VPNtunnels, the number of Security Gateway connections anAppliance can support will decrease.
Security Servers will reduce the maximum number ofsupported connections as they write to temporary files anduse 8 entries in the connections table.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
16/4516162010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
System Resources - Network InterfaceBandwidth
When the Security Gateway performance reaches the limitation of theinterface bandwidth and the CPU is still not fully utilized then the bottleneck isthe interface. One option to increase performance in this case is to use moreports via Link Aggregation to achieve the maximum performance.
The limitation of the network interface is determined by the amount of packetsper second it can process. Assuming 1518 byte frames, a 100megabit NICport can sustain ~8234 packets per second (pps ) in each direction (fullduplex). A 1gb NIC port can sustain ~82340 pps in each direction (full duplex).
While the network port may sustain much higher numbers of pps than these and this is often seen in the field and QA there is no guarantee that it WILLsupport more than these standard pps numbers for a given link speed.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
17/4517172010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
System Resources - Network InterfaceBandwidth
If the number of packets per second exceeds those numbers, using Link Aggregation tocombine 2 or more links together is possible, thus increasing the amount of bandwidth inpps that can be processed on that logical network interface.
IPSO Sync should not need more than a 1gb interface, and there can be problems whenIPSO Sync is run over a Link Aggregation group. The speed of the IPSO and CheckPoint sync interface should be as fast as the fastest NIC port on the system. In a 10gbenvironment, typically 1gb for sync is sufficient. State sync should always be on anisolated VLAN or network segment. Note that it is NOT supported to run Check Pointstate sync over any interface VLAN other than 0 / 1 / untagged. Any VLANing done onthe switch access port is fine, but trunking is not supported as Check Point state sync isnot designed to support the extra frame sizes.
Note: Security Gateway state sync link aggregated interfaces should be directlyconnected using cross-over cables, unless you are using 3 or more cluster members.IPSO and Check Point state sync should always be on an isolated VLAN or networksegment.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
18/45
18182010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
IP Clustering
IP Clustering provides both high availability and scalability. IPClustering is useful when the performance of one system aloneis insufficient to provide the desired level of performance. Forexample, when an Appliance CPU reaches ~30%, it would berecommended to add another Appliance to form a two-membercluster that can scale the Security Gateway performance. This
is a capacity planning exercise that Check Point Salesengineers can help with. The 30% number is considered anindustry standard measurement or indicator that suggests morecapacity should be added.
IP Clustering is especially beneficial when using SmartDefensefeatures. With all SmartDefense features enabled, a two-member clusters HTTP transaction rate is about 40% higherthan a Standalone Appliance.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
19/45
19192010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
IP Clustering
Use dedicated interfaces for cluster protocol networks and statesynchronization; do not share interfaces with the production traffic.
It is strongly recommended to use separate interfaces for clusterprotocol network and Security Gateway synchronization traffic so thatthey are separate Broadcast domains.
Use a bandwidth of at least 100 Mbps full duplex for IPSO sync
interface(s). 1gb is recommended.
Use switches, not hubs, and never use crossover cables for IPClustering protocol networks.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
20/45
20202010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
IP Clustering
Do not use IP Clustering Forwarding mode, when performanceis a concern. Unicast and Multicast provide better performanceand less latency. Forwarding mode is a fallback mode, for whenfeature-poor network switches are in use.
If IGMP snooping is in use on the switch, disable it or configurestatic CAMs in order to allow Multicast traffic on specific ports.
Use dynamic cluster work assignment for optimum loadbalancing. This allows the cluster to periodically rebalance theload by moving active connections between nodes.
Use delayed synchronization if your system processes manyshort-lived connections, you are in VRRP or Standalone, andSXL templates are in use. A 30 second delay in synchronizingconnections can boost the performance by about 20%. If youuse Check Point delayed notifications, you must also enableSecureXL delayed notifications.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
21/45
21212010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
ADP
Addition of ADP will increase performance of the appliance with somelimitations explained below. The decision to purchase ADP add oncards should be made in consultation with Check Point Sales. ADPshould be considered if the performance improvement desired falls into one of the following categories:
Packet throughput performance, specifically for small packets. Performance improvements for packet streams with mixed packet
sizes. Encrypted traffic (VPN) forwarding. Long-lived connections performance.
For example data transfer rates for protocols like ftp, http etc. NAT Performance only for long-lived NAT connections. (ADP
accelerates NAT throughput. Connection-rate Acceleration is
not currently supported for NAT connections. XMC cardsshould be used instead for high NAT & CPS.)
Latency for both un-encrypted and encrypted traffic. Multicast throughput performance.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
22/45
22222010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
ADP
Performance issues with mixing ADP with non-ADP interfaces:
The best performance one can get is by not mixing trafficbetween ADP and non-ADP interfaces. Running in mixed mode
will have performance impacts. When run in dual-mode having separate ADP traffic flows and separate non-ADP flowsone can see the Appliance performance scale as opposed tonot using any ADP interfaces at all.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
23/45
23232010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
ADP Benefits
Throughput Acceleration:
The first packet in a connection is sent up the stack to the Security Gateway thatvalidated the packet based on the defined rule base. Once validated the SecurityGateway application tells IPSO via the SecureXL API to handle future packets in thesame connection. IPSO then instructs the ADP sub-system to create a bi-directionalflow for that connection. All future packets for that connection will now be processed bythe ADP sub-system.
The following protocols benefit from SecureXL & ADP Throughput acceleration:
TCP, UDP, & traffic carried over those protocols IPSec VPN acceleration Multicast forwarding PIM (from IPSO 3.9 for IP2250 & IP2255; from IPSO 4.2 for all platforms) GRE & ESP
-
8/3/2019 IP Platforms Best Practices for Performance 010810
24/45
24242010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
ADP Benefits
Connection Rate Acceleration
The first packet in a connection is validated by the Security Gateway application. Oncevalidated, the Security Gateway instructs IPSO to create a template so that IPSO canvalidate future connections where only the Source Port differs. A template consists ofthe following attributes: SrcAddr, SrcPort, Proto, DestAddr, & Dest-Port. IPSO comparesthe first packet in the next connection to its template table. If the packet matches atemplate then IPSO adds the connection to its table, then instructs ADP to create a bi-directional flow for the connection and lastly informs the Security Gateway about thenew connection. All future packets are processed by the ADP module.
The following protocols benefit from SecureXL & ADP Connection-rate acceleration: Unencrypted TCP, UDP, & traffic carried over those protocols Particularly effective on HTTP 1.1 traffic Even more effective on HTTP 1.0 traffic HTTP 1.0: Separate connection for each HTTP component
-
8/3/2019 IP Platforms Best Practices for Performance 010810
25/45
25252010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
ADP Best Practices
Configure traffic to flow in/out of the same ADP subsystem, sincetraversing to another ADP subsystem or worse to a non-ADP interface
will negatively impact throughput performance. There is a 10gb fullduplex bandwidth over the crossbar between 2 ADP subsystems.
Do not use ports connected to the ADP subsystems for clusterprotocol network or for Security Gateway state synchronization. Useonboard ports for Security Gateway state synchronization; this will
guarantee that the synchronization data goes to its own channel andwill avoid sync packets being lost. This also prevents the sync datafrom disrupting the data passing between the ADPs and the mainCPU. Likewise do not log to the management station via the ADPinterfaces.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
26/45
26262010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
ADP Best Practices
Note that the backplane connecting the ADP subsystems to themain CPU has limited bandwidth, and this bottleneck will impactthroughput performance when there is a lot of non-acceleratedtraffic.
Do not combine non-ADP ports and ADP ports in a link
aggregation group or redundancy group.
Do not include interfaces on different ADP I/O cards in the samelink aggregation or redundancy group. IP Security Appliances donot support cross ADP link aggregation.
SecureXL is enabled by default. Note that its critical not to disableSecureXL because SecureXL is required for ADPs to function.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
27/45
27272010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
ADP Best Practices
Avoid performing tcpdump or fw monitoron ADP
interfaces when the interfaces are under heavy load.Performing a tcpdump or fw monitoron an ADP interfaceforces all traffic received or transmitted by the ADPsubsystem, to be copied and piped to IPSO through thebackplane. SecureXL will still be used,and the ADP willstill accelerate, but the backplane will be choked withdata. This causes a significant degradation inperformance due to constricted backplane capacity.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
28/45
28282010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Limitations of ADP
Traffic that is not throughput or connection rateaccelerated will not benefit from ADP acceleration. Alllimitations of SecureXL apply to ADP.
Transparent mode will accelerate traffic normally, howevera special design consideration is that there must be routespointing out of the xmode interfaces, as SecureXLdepends on caching route table lookups.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
29/45
29292010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Limitations of ADP
Enabling Sequence verifier: this solution requires enabling sequence verifier option onIPSO as well as in SmartDashboard.
This solution was suggested after analyzing the CST, where it was observed that most(60 million out of 80 million) of the TCP connections were getting closed with RSTsinstead of the usual 3-way handshake for terminating TCP connections. As part of thestateful inspection, the Security Gateway needs to monitor all TCP RSTs if sequenceverifier is not turned ON, as they are categorized as untrusted RSTs. This behavior ofterminating TCP connections causes additional load on ADP backplane interfaces,where the packet drops were observed.
By turning ON sequence verifier ADP will perform sequence verification on all TCPconnections thereby validating even RSTs that are used to terminate the TCPconnections. Once the TCP RST is validated by ADP and is accepted there is no needto send this packet to the Security Gateway, thereby reducing the backplane traffic andthe overhead of the Security Gateway having to inspect these packets.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
30/45
30302010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Limitations of ADP
Once the sequence verification is turned ON, you should see significantreduction in packets going over the backplane to the Security Gateway. This
can be monitored by executing the following command
ipsctl i net:dev:adp:if:stats with an r option to monitor the rate of tcp_rst
As a result of the reduction in TCP RSTs going over the backplane, we should
observe less drops of data packets on backplane which can be monitored byexecuting the following command.
ipsctl i net:dev:bp:msg:stats with an r option to monitor the rate ofrx_fc_drops
-
8/3/2019 IP Platforms Best Practices for Performance 010810
31/45
31312010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Limitations of ADP
Too many control messages queued up on eth1, will result in data loss on thedata channels eth2-4. The queue depth for the control channel is tunable, thedefault is 64 in IPSO 6.2. Potential values are 128 and 256:
ipsctl -w net:dev:bp:msg:delay_drop_limit 128
You should see a decrease in the rate of dropped packets on data channel.This can be monitored by executing the following command:
ipsctl i net:dev:bp:msg:stats with an r option to monitor the rate ofrx_fc_drops
If this solution does not yield the desired result then the delay_drop_limit canbe easily set back to its default value of 64. Setting of these ipsctl variablestakes effect immediately and is non-intrusive.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
32/45
32322010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Limitations of ADP Turning off the delay_drop variable: this solution requires changing the default value of
delay_drop, an ipsctl tunable in IPSO. IPSO pro-actively drops data packets, when controlchannel is congested. This option can be completely turned off by executing the followingcommand
ipsctl w net:dev:bp:msg:delay_drop 0
The option of dropping data packets when control channel is congested was developed undercertain performance benchmarking conditions, where the box is tested for limits and theaggressive load conditions persist for an extended period of time. This is the reason foraggressive drop_delay_limit to 64 by default.
Unfortunately, this condition also comes into effect when there is transient congestion on thecontrol channel. By turning off this feature, we do not drop the data packets pre-maturely.
Current congestion level and the max congestion level on the control channel can bemonitored by executing the following command:
ipsctl i net:dev:bp:msg:stats with an r option to monitor the rate of bms_scheds andbms_scheds_max
If this solution does not yield the expected result, then you can revert back to default behaviorimmediately by executing the following command.
ipsctl w net:dev:bp:msg:delay_drop 1
-
8/3/2019 IP Platforms Best Practices for Performance 010810
33/45
33332010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Limitations of ADP
The PSL acceleration feature should be enabled on multicoresystems using SecureXL and CoreXL, with or without ADP. Theipsctl tunable can be found using
ipsctl a net:sxl
PSL acceleration allows full acceleration of all but the lastpacket containing the application level Protocol Data Unit
IPS / SmartDefense takes care of a go/no-go to drop deniedconnections.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
34/45
34342010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Security Gateway Performance Tuning
NAT
IP Appliances do not support Network Address Translation(NAT) connection acceleration. The first packets of the firstconnection on the same service are forwarded to the SecurityGateway application. Then a template of that connection is
created so that subsequent TCP establishments on the sameservice, where only the source port is different, will beaccelerated by SecureXL. NAT connections setup and teardowncannot be accelerated because NAT templates are notsupported.
While each connection uses two entries in the flows table,connections involving NAT use four entries. NAT connectionsuse more CPU and memory resources compared to ordinaryconnections.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
35/45
35352010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Security Gateway Performance Tuning
Rulebase Size
Although there is no limit to the number of rules in a Security Gateway database, thereis a performance impact as the number of rules grows. The more rules an Appliancehas, the more it will cost the Appliance in compilation time and runtime efficiency.
Rulebase size affects connections rate performance.
Rulebase order is important and can affect performance. Use the following guidelinesfor organizing the rulebase: The rulebase should be as simple as possible. With fewer rules the rulebase will be
more efficient and less error prone. When creating a rule, be specific. Narrow down the source, destination, and
service. Avoid using Any in the service field. The most active NAT rules should be at the top of the NAT rulebase. Defining Group Objects for networks allows the policy compiler to superset traffic in
the actual rule for a performance gain. Anti spoofing should be configured for all the Security Gateway interfaces.
Avoid using negate in the rulebase (For example, a network exclusion)
-
8/3/2019 IP Platforms Best Practices for Performance 010810
36/45
36362010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Performance Troubleshooting
Follow the guidelines to troubleshoot performance issues and use the bestpractices outlined above to optimize the Appliance overall performance.
Do NOT use fw monitor for performance troubleshooting. Connecting aspan port via a switch, is preferred to tcpdump. Traffic captures sent toCheck Point Support should not exceed 80mb uncompressed. Particularlines within a packet capture should be referenced by the customer as
needed.
Check free disk space using the df k command line tool.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
37/45
37372010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Performance Troubleshooting
Check currently used CPU statistics using the vmstat 1 tool. The last3 columns are significant, customers should never concernthemselves with the other columns. The very last column is CPU Idletime, this is the amount of free CPU cycles, in percentage, since thelast vmstat iteration. The second-last column is System CPU usage in
percentage, this includes IPSO and Check Point kernels as well asinterrupts. The third-from-last column is User CPU utilization, this isusually due to Policy Installation, SmartDefense, Security Servers,and user scripts or commands.
-
8/3/2019 IP Platforms Best Practices for Performance 010810
38/45
38382010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Performance Troubleshooting
The top command line utility is used on recent IPSO versions, thisutility will provide per-process CPU and memory utilization, as well asglobal statistics, and more granular CPU statistics such as % ofinterrupts. The percentage of interrupts includes both softwareinterrupts and hardware interrupts. Hardware interrupts are virtually
never the cause of performance issues, performance issues arevirtually always caused by software interrupts.
If it can be shown that the performance problem is due to a high ratioof interrupts compared to overall CPU utilization, the fix is always first
to properly tune SecureXL.
P f T bl h i
-
8/3/2019 IP Platforms Best Practices for Performance 010810
39/45
39392010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Performance Troubleshooting
SecureXL Acceleration statistics can be verified using the followingcommands:
fwaccel stat
fwaccel stats
fwaccel stats s fwaccel templates s
These commands should provide a good overview of how much SecureXL isin use. The SecureXL and Nokia IPSO Guide
(http://downloads.checkpoint.com/dc/download.htm?ID=10036) should beused to help tune the rulebase, and ensure that as many connections andpackets are accelerated as possible.
P f T bl h i
-
8/3/2019 IP Platforms Best Practices for Performance 010810
40/45
40402010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Performance Troubleshooting
If dropped packets are a concern, they can be checked as a snapshot in IPSOby running the command ipsctl a ifphys | grep qdrop. This will provide anindication of which interfaces are dropping traffic. Please note that thesecounters are incremental since system boot time. For more information aboutqdrops, consult sk39462.
To view realtime statistics for a particular counter, run ipsctl i use use the r command to toggle the rate per second counter refresh.
Any qdrop which is logged will have a corresponding reason codeincremented, in ifphys::errors
ifphys::stats
Common drops are rx_mpc, which represents the Receive, Missed PacketCount, and symerrs, Symbol Errors.
P f T bl h ti
-
8/3/2019 IP Platforms Best Practices for Performance 010810
41/45
41412010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Performance Troubleshooting
rx_mpc is due to the operating system not being able to flush the receivebuffer for the interface fast enough. The receive buffer queues up incomingdata which has been received on the physical media, and is flushed from thequeue every time there is an interrupt. The interrupt is triggered under 2conditions when the rx_ring has reached full, or after a timer is hit. Moreinformation about these processed is detailed in sk39176. There are advanced
tunable variables that can be used under direction of Check Point Support toinfluence this behavior.
Symbol Errors are due to a bad fiber network cable, dirty or dusty NIC port orfiber connectors. It can also be due to a bad NIC card or switch port but this is
very uncommon. Symbol errors only increment for received data for the localside, not sent data. More information about symerrs is detailed in sk39733.
P f T bl h ti
-
8/3/2019 IP Platforms Best Practices for Performance 010810
42/45
42422010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Performance Troubleshooting
A lot of short lived connections transiting the enforcement point cancause slowdowns, as connection establishment and teardown incursCPU utilization. This can be partly mitigated, by ensuring templatescan be created for the most heavily used rules. You may also be ableto use the Fast Expire SecureXL feature.
Fast Expire should be used primarily for short lived connections suchas DNS.
P f T bl h ti
-
8/3/2019 IP Platforms Best Practices for Performance 010810
43/45
43432010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Performance Troubleshooting
Use ifconfig to verify that no interfaces have the PROMISC flag set. Aninterface in promiscuous mode forwards all frames seen on the physicalmedia, to the operating system for Layer 2 filtering. An interface in non-promiscuous mode, uses a MAC chip to filter Layer 2 frames to ensure thatonly frames which are destined for the local machine are passed to theoperating system. This is determined based on Unicast/Broadcast/MulticastMAC Address lists in the Receive Address High and Low registers. PROMISCis set for Transparent Mode and this is normal behavior.
Customers may wish to view the Security Gateway connection tables inhuman readable format to help with rules optimization. Check Point Supporthas internal-only tools to read the output of fw tab u. Customers may beinterested in an unsupported third-party script which can be found athttp://www.fw-1.de/aerasec/download/fw1-tool/fw1-tool.pl
Check Point makes no guarantees about this product, and provides thisinformation as reference only.
Ad d d b i d t i
http://www.fw-1.de/aerasec/download/fw1-tool/fw1-tool.plhttp://www.fw-1.de/aerasec/download/fw1-tool/fw1-tool.pl -
8/3/2019 IP Platforms Best Practices for Performance 010810
44/45
44442010 Check Point Software Technologies Ltd. | [Unrestricted] For everyone
Advanced debugging and tuning
Advanced debugging and tuning should onlybe carried out under the direction of theEscalations group, or Development.
Th E d Q ti ?
-
8/3/2019 IP Platforms Best Practices for Performance 010810
45/45
The End Questions?