Creating and Managing Custom TCP/IP Stack in VMware

Creating and Managing Custom TCP/IP Stack in VMware

In vSphere 5.1 and earlier version of VMware vSphere there was only one single TCP/IP stack for all the traffic types such as management, vMotion and virtual machine traffic. Because of this common TCP/IP stack method all the configured vmkernels had to use some common parameters like, same default gateway, memory heap, ARP and routing tables. This was a limitation of this edition and it ended up some common issues.

As a result of finding solutions for those issues VMware allowed to create multiple TCP/IP stacks with vSphere 5.5. After releasing the vSphere 6.5 VMware created a separate TCP/IP stack for vMotion and provisioning traffic other than the default stack. But still there is no option in the GUI to create a custom TCP/IP stack and we need to use the esxcli commands in the command line interface to create a custom stack. But still there are some limitations.

Let’s see how we can create a custom TCP/IP stack in VMware

Use esxcli network ip netstack list to view the available TCP/IP stacks and use esxcli network ip netstack add -N “<tcp_ip_stack_name>” to add a new TCP/IP stack

What are the Limitations?

Unfortunately, custom TCP/IP stack aren’t supported for management traffic, fault tolerance traffic, vSphere Replication and VSAN traffic. When you select a created custom TCP/IP stack traffic, those options automatically disable themselves (see below screen capture when adding a vmkernel).

Note: I had to update my post after reading the VMware Storage hub and please note that VMware vSphere Custom TCP/IP Stack doesn’t support the iSCSI protocol. I have read it in a different book and I’d like to correct it in here.

“Using Static Routes or set a Gateway

As mentioned before, for vSphere hosts, the management network is on a VMkernel port and therefore uses the default VMkernel gateway. Only one VMkernel default gateway can be configured on a vSphere host per TCP/IP Stack. You can, however, add static routes from the command line or configure a gateway for each individual VMkernel port. Setting a gateway on a per VMkernel port granular level has been introduced in vSphere 6.5 and allows for a bit more flexibility. The gateway for a VMkernel port can simply be defined using the vSphere Web Client during the creation of the VMkernel interface. It is also possible to configure it using esxcli.

Note: At the time of writing the use of a custom TCP/IP Stack is not supported for iSCSI!”

Read More:

That’s it for now. Hope you enjoyed my post! If you find this as useful please rate the post and share it!

Click to rate this post!
[Total: 2 Average: 5]

4 responses

  1. Richard R. Avatar
    Richard R.

    How did you use custom netstacks for NAS and iSCSI? It never worked for me.

    1. Aruna Lakmal Avatar
      Aruna Lakmal

      Richard : I have updated the post and recently VMware Storage hub was updated and I have updated the post. I’m pretty sure this had been written in a book.

      “At the time of writing the use of a custom TCP/IP Stack is not supported for iSCSI!”

  2. Joe Avatar

    Ever hear of using a custom netstack for guest VMs? Damned if I can find an example on the internets. Why? Because the esxi host is on a particular network x behind a particular switch, and it seems I ought to be able to connect it also to switch y so some guests can be on that subnet with its resources & default gateway. Yes, I know NORMALLY it’d be one switch with vlans, blah, blah.

    1. Aruna Lakmal Avatar

      Joe, honestly, I didn’t hear something like that in VMware with my experience. But, it is something to find out. I will check with few VMware fellows and, give you an answer.

Leave a Reply

Your email address will not be published. Required fields are marked *