Packet Flow Through Checkpoint

CheckPoint Packet Flow

Checkpoint process the packet in the ingress and the egress using two CHAINS.

Basic:

Physical layer - ingress interface
Data Link Layer/Ethernet
Inspect Driver [inspect Engine]

Network Layer/IP Routing

Inspect Driver
Data Link Layer/Ethernet
Physical layer - egress interface

Advance:

1. NIC hardware
-The network card receives electrical signalling from the link partner.

2. NIC driver
-Sanity checks
-The NIC hardware decodes the signal and passes it to the operating system's NIC driver via the PCI bus
-The frame is converted to an mbuf entry and the frame headers are stored for later use.
-NIC driver hands off the data to the operating system's mbuf memory space

3. Operating system IP protocol stack
-The OS performs sanity checks on the packet
-Hand off to SXL if enabled, or to Firewall Kernel if not

4. SecureXL (if enabled)
-SXL lookup is performed, if it matches, bypass the firewall kernel and proceed with (Operating system IP protocol stack, outbound side)

5. Firewall Kernel (inbound processing)
-FW Monitor starts here (so, perhaps you need to disable secureXL [fwaccel off CAUTION] ... )
-Connection state lookups, some protocol inspection, rulebase processing, antispoofing lookups etc
-Processing order can be seen via fw ctl chain
-Bypass complex inspection if not needed

6. Complex protocol inspection (AV is an example)
-Leave the kernel and process under userland.
-Enters back at this same stage if the traffic passed

(inbound processing stops here)

(outbound processing starts here)

7. Firewall Kernel 
-Route lookup
-Check Point sanity checks etc
-FW Monitor ends here
-Pass to operating system

8. Operating system IP protocol stack
-The OS performs sanity checks on the packet
-Pass the mbuf to the NIC driver for the appropriate outbound interface

9. NIC driver
-Tag the packet as an ethernet frame by adding MAC addresses for source and destination

10. NIC hardware
-The NIC hardware encodes the signal and transmits it via wire

                                                       Fig. Ctl Chain

TIP: 
1. DST NAT can happen between i & I (when client side NAT enabled [DEFAULT]) or between o & O (server side NAT) 

2. SRC NAT will happen immediately after o (after routing) regardless the client side NAT enabled or not.

• Static NAT - One to one translation
• Hide/Dynamic NAT - Allows you to NAT multiple IPs behind one IP/Interface

• Automatic NAT - Quick basic address NAT translation.  
• Manual NAT - Allows greater flexibility over automatic NAT; it is preferred over Automatic NAT. If you are using manual rules, you must configure proxy ARPs to associate the translated IP address with the MAC address of the Security Gateway interface that is on the same network as the translated addresses.Configuration depends of  OS.
• ['Global Properties' - go to 'NAT' - go to 'Automatic NAT rules'

• Server Side NAT - destination is NAT`d by the outbound kernel
• Client Side NAT  - destination is NAT`d by the inbound kernel 
• ['Global Properties' - go to 'NAT' - go to 'Manual NAT rules'

Between all the steps there are queues. These queues accumulate packets and on intervals flush them to the next step. All of this happens very very quickly in small CPU time slices.

The INSPECT engine itself is more to do specifically with protocol inspection rather than all of the other steps. INSPECT runs traffic against definitions, if the definitions match it usually means that it hit a protection and the appropriate action is to (drop, log) the traffic.


VM is used for referring firewall Virtual Machine (VM, and it is CP terminology): 
i (PREIN) – inbound direction before firewall VM . 
I (POSTIN) – inbound direction after firewall VM.
o (PREOUT) – outbound direction before firewall VM,
O (POSTOUT) – outbound direction after firewall VM.

                                                     Fig. fw monitor in the Chain. 


TIP: Source NAT occurs after outbound inspection (o), not between (I) and (o), so if we are doing src nat we won't see (O), the same will happen if the traffic flows via a vpn.




(00000001) new processed flows
(00000002) previous processed flows
(00000003) ciphered traffic
(ffffffff) Everything


[Expert@firewall]# fw ctl chain
in chain (19):
0: -7f800000 (f27f9c20) (ffffffff) IP Options Strip (in) (ipopt_strip)
1: -7d000000 (f1ee6000) (00000003) vpn multik forward in
2: - 2000000 (f1ecad30) (00000003) vpn decrypt (vpn)
3: - 1fffff8 (f1ed7550) (00000001) l2tp inbound (l2tp)
4: - 1fffff6 (f27faff0) (00000001) Stateless verifications (in) (asm)
5: - 1fffff5 (f2c6b240) (00000001) fw multik VoIP Forwarding
6: - 1fffff2 (f1ef30b0) (00000003) vpn tagging inbound (tagging)
7: - 1fffff0 (f1ecbb20) (00000003) vpn decrypt verify (vpn_ver)
8: - 1000000 (f2849190) (00000003) SecureXL conn sync (secxl_sync)
9: 0 (f27af6b0) (00000001) fw VM inbound (fw)
10: 1 (f2812680) (00000002) wire VM inbound (wire_vm)
11: 2000000 (f1ecdff0) (00000003) vpn policy inbound (vpn_pol)
12: 10000000 (f284e9e0) (00000003) SecureXL inbound (secxl)
13: 21500000 (f3b2c940) (00000001) RTM packet in (rtm)
14: 7f600000 (f27f0910) (00000001) fw SCV inbound (scv)
15: 7f730000 (f2928620) (00000001) passive streaming (in) (pass_str)
16: 7f750000 (f2a3bbf0) (00000001) TCP streaming (in) (cpas)
17: 7f800000 (f27f9fb0) (ffffffff) IP Options Restore (in) (ipopt_res)
18: 7fb00000 (f2a07540) (00000001) HA Forwarding (ha_for)
out chain (16):
0: -7f800000 (f27f9c20) (ffffffff) IP Options Strip (out) (ipopt_strip)
1: -78000000 (f1ee5fe0) (00000003) vpn multik forward out
2: - 1ffffff (f1eccbd0) (00000003) vpn nat outbound (vpn_nat)
3: - 1fffff0 (f2a3ba70) (00000001) TCP streaming (out) (cpas)
4: - 1ffff50 (f2928620) (00000001) passive streaming (out) (pass_str)
5: - 1ff0000 (f1ef30b0) (00000003) vpn tagging outbound (tagging)
6: - 1f00000 (f27faff0) (00000001) Stateless verifications (out) (asm)
7: 0 (f27af6b0) (00000001) fw VM outbound (fw)
8: 1 (f2812680) (00000002) wire VM outbound (wire_vm)
9: 2000000 (f1ecdaa0) (00000003) vpn policy outbound (vpn_pol)
10: 10000000 (f284e9e0) (00000003) SecureXL outbound (secxl)
11: 1ffffff0 (f1ed71e0) (00000001) l2tp outbound (l2tp)
12: 20000000 (f1ecce40) (00000003) vpn encrypt (vpn)
13: 24000000 (f3b2c940) (00000001) RTM packet out (rtm)
14: 7f700000 (f2a3b810) (00000001) TCP streaming post VM (cpas)
15: 7f800000 (f27f9fb0) (ffffffff) IP Options Restore (out) (ipopt_res)

Acceleration

Performance Pack is a software acceleration product installed on Security Gateways. Performance Pack uses SecureXL technology and other innovative network acceleration techniques to deliver wire-speed performance for Security Gateways.

In a SecureXL-enabled gateway, the firewall first uses the SecureXL API to query the SecureXL device and discover its capabilities. The firewall then implements a policy that determines which parts of what sessions are to be handled by the firewall, and which should be offloaded to the SecureXL device. When new sessions attempt to get established across the gateway, the first packet of each new session is inspected by the firewall to ensure that the connection is allowed by the security policy. As the packet is inspected, the firewall determines the required behavior for the session, and based on its policy it may then offload some or all of the session handling to the SecureXL device. Thereafter, the appropriate packets belonging to that session are inspected directly by the SecureXL device. The SecureXL device implements the security logic required for further analysis and handling of the traffic. If it identifies anomalies it then consults back with the firewall software and IPS engine. In addition, SecureXL provides a mode that allows for connection setup to be done entirely in the SecureXL device, thus providing extremely high session rate.

                                                                     Fig. FW Paths and core processing  

Medium Path is known as PXL (SXL + PSL [PSL passive Streaming Library IPS related])
Slow Path (Firewall Path) is known as F2F (Forwarded 2 Firewall)

SecureXL is implemented either in software (Core), or in hardware (SAM cards on Check Point 21000 appliances; ADP cards on IP Series appliances with CPUs inside). The Dispatcher + Performance Pack combination is known as SND (Secure Network Dispatch).

Fwaccell (Firewall Acceleation feature) is used to check/manage the SecureXL Acceleration Device 

SIM (SecureXL implementation Device) Affinity (Association with a Core) can be  managed automatically by checkpoint (each 60 seconds) or statically configured.
TIP: To achieve the best performance, pairs of interfaces carrying significant data flows (based on network topology) should be assigned to pairs of CPU cores on the same physical processor.

• Active Streaming (CPAS) - Technology that sends streams of data to be inspected in the kernel, since more than a single packet at a time is needed in order to understand the application that is running (such as HTTP data). Active Streaming is Read- and Write-enabled, and works as a transparent proxy. Connections that pass through Active Streaming can not be accelerated by SecureXL.
• Passive Streaming - Technology that sends streams of data to be inspected in the kernel, since more than a single packet at a time is needed in order to understand the application that is running (such as HTTP data). Passive Streaming is Read-only and it cannot hold packets, but the connections are accelerated by SecureXL.
• Passive Streaming Library (PSL) - IPS infrastructure, which transparently listens to TCP traffic as network packets, and rebuilds the TCP stream out of these packets. Passive Streaming can listen to all TCP traffic, but process only the data packets, which belong to a previously registered connection. For more details, refer to sk95193 (ATRG: IPS).
• PXL - Technology name for combination of SecureXL and PSL.
• QXL - Technology name for combination of SecureXL and QoS (R77.10 and above).
• F2F / F2Fed - Packets that can not be accelerated by SecureXL (refer to sk32578 (SecureXL Mechanism)) are Forwarded to Firewall.
• F2P - Forward to PSL/Applications. Feature that allows to perform the PSL processing on the CPU cores, which are dedicated to the Firewall.

HyperThreading 

SMT: New in R77. Minimices Context change inside a physical core and it may improve performance. (or, may not f.e memory used is increased and connection table size can de reduced)

It is not supported in Open Servers.

It is strongly recommended to disable Hyper-Threading in BIOS when CoreXL is enabled (on Check Point appliances this is disabled, by default). Applies to Intel processors prior to "Intel Nehalem (Core i7)", where this technology was improved (called Simultaneous Multi-Threading, or Intel® Hyper-Threading)



Multicore 

Multi Core processing in Checkpoint is known as CoreXL. Thre are some limitations when CoreXL is enabled.


The following features/settings are not supported in CoreXL:

1. Check Point QoS (Quality of Service)⁽¹⁾
2. 'Traffic View' in SmartView Monitor⁽²⁾ (all other views are available)
3. Route-based VPN
4. IP Pool NAT⁽³⁾ (refer to sk76800)
5. IPv6⁽⁴⁾
6. Firewall-1 GX
7. Overlapping NAT
8. SMTP Resource⁽³⁾
9. VPN Traditional Mode (refer to VPN Administration Guide - Appendix B for converting a Traditional policy to a Community-Based policy)

If any of the above features/settings is enabled/configured in SmartDashboard, then CoreXL acceleration will be automatically disabled on the Gateway (while CoreXL is still enabled). In order to preserve consistent configuration, before enabling one of the unsupported features, deactivate CoreXL via 'cpconfig' menu and reboot the Gateway (in cluster setup, CoreXL should be deactivated on all members).

Notes:

• (1) - supported on R77.10 and above (refer to sk98229)
• (2) - supported on R75.30 and above
• (3) - supported on R75.40 and above
• (4) - supported on R75.40 and above on SecurePlatform/Gaia/Linux only

There are two main roles for CPUs applicable to SecureXL and CoreXL:

• SecureXL and CoreXL dispatcher CPU (the SND - Secure Network Distributor)
  You can manually configure this using the sim affinity -s command. (Exception: cpmq if multiqueue)
• CoreXL firewall instance CPU
  You can manually configure this using the fw ctl affinity command 

Traffic is processed by the CoreXL FW instances only when the traffic is not accelerated by SecureXL (if SecureXL is installed and enabled). So, if all the traffic is accelerated, we can have several fw instances idle.

• CoreXL will improve performance with almost linear scalability in the following scenarios:
  
  • Much of the traffic can not be accelerated by SecureXL
  • Many IPS features enabled, which disable SecureXL functionality
  • Large rulebase
  • NAT rules
  
  
• CoreXL will not improve performance in the following scenarios:
• SecureXL accelerates much of the traffic
• Traffic mostly consists of VPN (VPN traffic inspection occurs only in CoreXL FW instance #0)
• Traffic mostly consists of VoIP (VoIP control connections are processed in only in CoreXLFW instance #0)

In some cases it may be advisable to change the distribution of kernel instances, the SND, and other processes, among the processing cores. This configuration change is known as Performance Tuning. This is done by changing the affinities of different NICs (interfaces) and/or processes. However, to ensure CoreXL's efficiency, all interface traffic must be directed to cores not running kernel instances. Therefore, if you change affinities of interfaces or other processes, you will need to accordingly set the number of kernel instances and ensure that the instances run on other processing cores.

Automatic Mode — (default) Affinity is determined by analysis of the load on each NIC. If a NIC is not activated, Affinity is not set. NIC load is analyzed every 60 seconds.

Manual Mode — Configure Affinity settings for each interface: the processor numbers (separated by space) that handle this interface, or all. In Manual Mode, periodic NIC analysis is disabled

The default affinity setting for all interfaces is Automatic. Automatic affinity means that if Performance Pack is running, the affinity for each interface is automatically reset every 60 seconds, and balanced between available cores. If Performance Pack is not running, the default affinities of all interfaces are with one available core. In both cases, any processing core running a kernel instance, or defined as the affinity for another process, is considered unavailable and will not be set as the affinity for any interface. Poor decisions maybe done with automatic affinity.

                                                    Figure. Possible affinity setting. 

                                                  Figure. MultiCore Processing and packet flow paths.

SecureXL and CoreXL connection info exchage:

• Connection offload - Firewall kernel passes the relevant information about the connection from Firewall Connections Table to SecureXL Connections Table.
  Note: In ClusterXL High Availability, the connections are not offloaded to SecureXL on Standby member.

• Connection notification - SecureXL passes the relevant information about the accelerated connection from SecureXL Connections Table to Firewall Connections Table.

• Partial connection - Connection that exists in the Firewall Connections Table, but not in the SecureXL Connections Table (versions R70 and above).
  
  • In Cluster HA - partial connections are offloaded when member becomes Active
  • In Cluster LS - partial connections are offloaded upon post-sync (only for NAT / VPN connections)
  Such connections must be offloaded to SecureXL, since packets in these connections must not be dropped.
  If a packet matched a partial connection in the outbound, then it should be dropped.

• Delayed connection - Connection created from SecureXL Connection Templates without notifying the Firewall for a predefined period of time. The notified connections are deleted by the Firewall.