How I configured VLANs, VXLANs and OSPF in my nested VMware homelab

How I configured VLANs, VXLANs and OSPF in my nested VMware homelab

Nested homelab introduction

I was chatting with someone the other week about homelab setups. The old debate of running on physical hosts vs. nested came up. I am running a completely nested homelab with stretched vSAN, NSX-v, et cetera and as a follow-up to that conversation I was having, I decided to do a short writeup on how I configured my homelab.

Basic homelab setup

I have a single SuperMicro SuperServer mini tower with 128GB RAM, an 8-core Xeon D-1541 and some fast SSD storage. I run a 4-node stretched vSAN management cluster for all my SDDC components/workloads and a 2-node workload cluster with nested (FreeNAS) NFS storage. I created two standard virtual switches on the physical ESXi host. One vSwitch holds the actual uplink to my home network with the VMkernel management interface and the other is an internal vSwitch without any physical uplinks and is just used for nested virtualization:

My home network is nothing fancy by the way. It’s just an ISP provided internet router with Wi-Fi and switch onboard. From a homelab perspective, I’m just using the vSwitch connected to my home network to do basic administration of the physical ESXi host and remotely access my nested lab environment (I use a Windows domain controller as a simple RDP jumphost).

The Nested-ESXi internal vSwitch holds all the nested virtualization magic (accept forged transmits, promiscuous mode and MAC address changes). I also configured this internal vSwitch for jumbo frames (MTU 9000) so I can add VXLAN overlay networks using NSX-v. Because all my nested virtualization traffic stays on the internal vSwitch, I never have to worry about jumbo frames in my physical home network. VXLAN traffic will never ever hit the actual physical wire. Because everything is running nested, I can even run all-flash vSAN with all data services without having to invest in 10GE physical network hardware. Thanks to the fast NVME SSD, performance is more than fine…

Infrastructure services

I’m running all required infrastructure services for my homelab such as NTP, DNS, AD, et cetera, on the Windows Domain Controller / Jumphost which is running as a VM on the physical ESXi host. Everything that’s running directly on the physical ESXi host can be considered a “physical” workload. So, in the real world analogy I would be running a physical domain controller outside of my VMware environment.

How to use VLANs in nested ESXi

All my nested ESXi hosts have two uplinks to simulate a typical enterprise vSphere deployment. The nested ESXi hosts are simply equipped with two virtual NICs:

The nested view, looking at the nested Distributed Switch physical adapters:

Both uplinks are connected to the VLAN 4095 Trunk port group on the internal Nested-ESXi vSwitch. By using VLAN ID 4095, I basically created a trunk port that allows Guest OS VLAN tagging:

This allows me to do the VLAN tagging on the nested ESXi hosts’ port groups (with ESXi being my Guest OS):

Routing between VLANs

I’m simulating a physical network routing layer by running a pfSense appliance directly on the physical ESXi host. The uplink is connected to the home network portgroup on the actually connected vSwitch. The internal LAN interface is connected to the internal Nested-ESXi vSwitch trunk port. This allows me to create VLAN interfaces on the internal facing interface in pfSense and route between my physical home network and the VLANs living in my nested virtualization world:

I’m allowing pfSense admin traffic on my WAN interface so I can easily manage it directly from my home network. In the screenshot below you can see some of the VLAN interfaces I have created in pfSense. (I have actually logically created a dual site environment so I can use stretched vSAN, that’s why you are seeing references to Site A and Site B):

Enabling VXLAN by configuring jumbo frames end-to-end

As I mentioned before, I configured the internal Nested-ESXi vSwitch with jumbo frames support. To allow the use of jumbo frames, you have to configure a consistent MTU size end to end. In this case the physical ESXi host’s internal vSwitch, the nested distributed vSwitch, the VMkernel interface and the VLAN interface on the pfSense appliance must be configured with jumbo frames support:

NSX logical routing and switching

I deployed a typical NSX logical routing and switching topology in my homelab. I have two NSX edges deployed and they are configured for Equal Cost Multipath Routing to my “physical core network / pfSense appliace” using OSPF:

I am running Quagga OSPF on the pfSense appliance and my NSX edges are peering nicely with it:

Downstream, the NSX Edges are peering with my NSX Distributed Logical Router (Control VM):

The DLR is obviously the first hop gayeway for the logical networks (VXLANs) attached to it:

VPN Remote Access

I was thinking about configuring VPN remote access to my homelab through NSX but in the end I decided I would be better of investing a bit of money in a dedicated out of band VPN appliance. I bought a Raspberry Pi :-). With PiVPN and some basic port forwarding on my internet router, I was quickly VPN-ing into my homelab from all over the world. With the SuperMicro IPMI interface, I can even turn my homelab on or off.

Closing thoughts and future homelab plans

For me, nested virtualization is a perfect fit. I’m running a complete SDDC, following the Multi Availability Zone VMware Validated Design reference architecture, in a single small, relatively low powered box. It is a great environment that will come in really handy when preparing for my VCDX-NV. To create a Dual Region environment with Cross-VC NSX, I am going to add a second region to my homelab by using the Ravello cloud service. As a vExpert, I have access to plenty of free CPU hours, and with Ravello now supporting bare metal virtualization, it seems like a great opportunity to expand my lab even further and do some cross region disaster recovery testing.

If you have any questions about my homelab setup, or if you have useful tips on how to improve my setup, just let me know!

16 thoughts on “How I configured VLANs, VXLANs and OSPF in my nested VMware homelab”

  • 1
    Marc on May 23, 2018 Reply

    Why not use pivpn on a small linux on the host?

  • 2
    jkusters on May 25, 2018 Reply

    I really wanted something out of band. The small linux VM would rely on my physical ESXi host being up and running. Now, I can VPN into my home network and turn on my homelab remotely or troubleshoot it need be. The Raspberry Pi is a really cheap solution. I also wanted to get my hands on a Raspberry Pi in all honesty 🙂

  • 3
    Harold on July 5, 2018 Reply

    So is this infrastructure Windows Server box running on the Home-network vSwitch, or on the Nested-ESXi vSwitch? Trying to recreate your setup, but I can’t figure this out.

  • 4
    jkusters on July 5, 2018 Reply

    It’s on the Home Network. It uses the pfSense appliance as its default gateway so it can act as a jumphost to get from the home network into the nested homelab networks.

  • 5
    jkusters on July 5, 2018 Reply

    The pfSense box is kind of the spider in the center of the web. All the VLANs are routed there. This makes it really easy to configure end-to-end routing in the entire environment using OSPF (in my NSX-v lab) and BGP (in my NSX-T lab).

  • 6
    Tim Maier (@virTimMaier) on August 23, 2018 Reply

    Are you running vCenter, NSX Manager and NSX Controllers in the nested environment or on the physical ESXi host?

  • 7
    Bjørn-Tore on August 29, 2019 Reply

    Hi – Looking to create something similar with a single SuperMicro server. Just to clarify, the physical network is only a single uplink from the SuperMicro server to your homenetwork switch? My SuperMicro server has multiple physical network interfaces, so I just wonder how to best use them with this nested setup. Thanks.

    • 8
      jkusters on August 29, 2019 Reply

      Hi Bjorn. Yes, I simply used a single uplink. It’s just a homelab for me so I didn’t bother with uplink redundancy (and save a port on my physical switch). You could leverage additional uplinks and apply a teaming policy on your vSwitch.

  • 9
    Bjørn-Tore on August 29, 2019 Reply

    Ok I see. Thank you.

  • 10
    Bjørn-Tore on August 29, 2019 Reply

    BTW, great post and blog. I will try this as soon as I get my SuperMicro server 🙂

  • 11
    Michael Grann on October 20, 2019 Reply

    Did you deploy NSX Manager onto a nested ESXi VM? I tried this twice with ESXi 6.7update2 and VMware-NSX-Manager-6.4.5-13282012 and after deployment the NSX Manager VM cannot be powered on, it says no host in the cluster can run it even though its HW resources are much more than the requirement for Tiny deployment. I tried adding the Physical ESXi host to Vcenter (nested) and NSX Manager deploys & runs but there is NO “Networking & Security” icon in Vcenter. I’m unable to add the physical ESXi host into the Mgmt cluster. Any advice would be greatly appreciated.

  • 12
    Lucien on February 25, 2020 Reply

    Hello Jeffrey, I’m in the process of building my nested ESXi homelab and I came across this page.
    I found the idea of having a router in the middle to separate home network and nested network very nice so reproduced the same with a VyOS router instead. Everything works as expected except for one thing:
    My vCenter is a nested VM.
    My DNS server is also a nested VM.
    My pc lays in my home network.
    When I point the dns server of my pc to the nested DNS server to resolve vcenter.lab.local, it doesn’t resolve, although the traffic is open. From my pc I can ping the nested DNS server and telnet nested_dns_server 23 works too.
    The only solution I found so far was to add an entry in my pc’s hosts file.
    I was wondering how you did to resolve your vcenter’s dns name from your home network?

  • 13
    Tone on March 29, 2021 Reply

    Hello Jeffrey, Thank you for the wonderful content I am still struggling to understand how to setup the network part of this. My specific needs are a full air gapped lab no connection to internet. I still need a nested setup to simulate corp structure. I mainly connect to the host sever via a static Ip on and extra computer I have. Should I just static everything and eliminate the need for a router? I don’t have the need to go out online. This setup is mainly to simulate attacks . Any input would greatly help. I am new to all this NESTED stuff for sure. Thank you.

    • 14
      jkusters on March 30, 2021 Reply

      Thank you for the kind words. I created a YouTube channel and recorded a couple of videos you might like in which I explain and demonstrate my nested setup. Check it out at and let me know if you have additional questions, OK?

  • 15
    tryllz on March 20, 2022 Reply


    I’m attempting the same setup but something I’m not understanding.

    Why have VLAN sub-interfaces in the pfSense if the ESXi is to be attached to the Trunk Port ?!

    I thought it would be each sub-interface of pfSense will have a relevant Portgroup with its associated VLAN on the Physical ESXi, then that VLAN Portgroup will be added to the ESXi.

  • 16
    tryllz on March 20, 2022 Reply

    What I meant was :

    The pfSense has a sub-interface with VLAN1611, so I created a portgroup and added 1611 tag to it on the physical ESXi, then added that portgroup as a vNIC to the Nested ESXi..

Leave a Reply