NSX DFW Enable TCP Strict

Today, I finally had some time to dive a bit deeper into the 6.4.0 release of NSX for vSphere. When I was configuring the Distributed Firewall, I noticed a setting called ‘Enable TCP Strict” which I didn’t immediately recognize. After some investigation, it became clear that this setting strengthens the security of your Distributed Firewall by dropping packets that have not been preceded by a complete 3-way TCP handshake. As you might recall, a 3-way TCP handshake looks like this:

1. The client sends a SYN (synchronize) message to the server
2. The server replies with a SYN-ACK (synchronize acknowledgement) message
3. The client replies with an acknowledgement (ACK) message

Stateful firewalling

The NSX DFW is a stateful firewall. That means that it performs stateful packet inspection and tracks the state of network connections. Packets matching a known active connection are allowed by the firewall and packets which do not, are inspected against the firewall rules. Now, there are specific use cases – such as asymmetric routing – where packets hit the firewall without the firewall being aware of the connection state. Or in other words, where the firewall isn’t aware of the 3-way TCP handshake that preceded the packet.

TCP Split Handshake Attack

In 2011, NSS Labs reported most major firewall vendors were vulnerable to the so called “TCP Split Handshake Attack”. Firewalls were updated to include an option to always enforce a completed  3-way TCP handshake before allowing packets to pass.

TCP Strict configuration changes in NSX 6.4.0

The NSX DFW also includes this option but I noticed the configuration of this option changed in the 6.4.0 release. Prior to 6.4.0, TCP Strict mode was configured at the global DFW level using the API:

PUT /api/4.0/firewall/config/globalconfiguration

<globalConfiguration>
<layer3RuleOptimize>false</layer3RuleOptimize>
<layer2RuleOptimize>true</layer2RuleOptimize>
<tcpStrictOption>false</tcpStrictOption>
</globalConfiguration>

In NSX 6.4.0, this global option is deprecated according to the NSX 6.4.0 Release Notes:

“Setting tcpStrict in the global configuration section of Distributed Firewall is deprecated. Starting in NSX 6.4.0, tcpStrict is defined at the section level. Note: If you upgrade to NSX 6.4.0 or later, the global configuration setting for tcpStrict is used to configure tcpStrict in each existing layer 3 section. tcpStrict is set to false in layer 2 sections and layer 3 redirect sections. See “Working with Distributed Firewall Configuration” in the NSX API Guide for more information.”

Documentation confusion

According to the NSX 6.3 and 6.4 Administration Guides, Strict TCP mode is enabled by default:

“By default, the NSX Distributed Firewall operates in strict TCP mode and when using a default block rule, it drops packets that do not satisfy connection requirements. A connection begins with a three way handshake (SYN, SYN-ACK, ACK) and typically end with a two way exchange (FIN, ACK)”

But when I read the VMware KB Article “Stateful behavior of the NSX Distributed Firewall in an asymmetric routing environment (2145340)”  I must say I am a bit confused. This article states the following fact:

“By default, the NSX Distributed Firewall operates with strict TCP mode disabled, this allows the firewall to be enabled without disrupting existing connections.”

So, what is it? Is Strict TCP mode enabled by default or disabled by default? In the GUI of NSX 6.4.0 the option to configure TCP Strict mode is available on a granular level and it is disabled by default. I am moving forward on the premise that it is disabled by default.

I will try to setup a lab environment to test and validate the behaviour of the NSX DFW. If you have anything to add in the meanwhile, feel free to reach out!