VLAN Configuration Guide for VMware Homelab

Overview

This guide details the VLAN configuration strategy for a VMware homelab environment supporting VCF (VMware Cloud Foundation), NSX-T, and Tanzu Kubernetes Grid (TKG).

VLAN Design Principles

Traffic Isolation

  • Management Traffic: Separate from production workloads
  • Storage Traffic: Isolated for performance and security
  • vMotion Traffic: Dedicated network for VM migrations
  • Overlay Traffic: Separate network for NSX TEPs
  • Workload Traffic: Segmented by purpose (TKG management vs workload)

Network Segmentation Benefits

  • Security: Isolate different traffic types
  • Performance: Prevent bandwidth contention
  • Troubleshooting: Easier to diagnose issues
  • Compliance: Meet security requirements

Complete VLAN Assignment

VLAN ID Name Purpose IP Range MTU Notes
10 Management ESXi mgmt, vCenter, infrastructure 192.168.10.0/24 1500 Untagged/Native VLAN
20 vMotion Live VM migration 192.168.20.0/24 9000 Jumbo frames recommended
30 vSAN Storage cluster traffic 192.168.30.0/24 9000 High bandwidth required
40 NSX-TEP NSX overlay tunnels 192.168.40.0/24 1600+ Increased for encapsulation
50 NSX-Edge-Uplink Edge north-south traffic 192.168.50.0/24 1500 Connects to physical router
100 TKG-Mgmt K8s management clusters 192.168.100.0/24 1500 Control plane components
110 TKG-Workload K8s workload clusters 192.168.110.0/24 1500 Application workloads
200 NUC-Mgmt Testing/troubleshooting only 192.168.200.0/24 1500 Optional - not for production

Physical Switch Configuration

Ubiquiti UniFi Configuration

For UniFi switches, configure VLANs in the controller:

  1. Create Networks:

    Settings → Networks → Create New Network
- Purpose: Corporate
- Network Group: LAN
- VLAN: <ID>
- Gateway IP/Subnet: <IP Range>

  2. Assign to Ports:

    Devices → Switch → Ports → Port Profile
- Select "All" for trunk ports to ESXi
- Select specific VLAN for access ports

Trunk Port Configuration

ESXi host ports should be configured as trunk ports:

Port Configuration:
- Mode: Trunk
- Native VLAN: 10 (Management - untagged)
- Tagged VLANs: 20,30,40,50,100,110,200
- Speed: Auto (or fixed 10Gbps)

This configuration means:
- Management traffic (192.168.10.x) flows untagged
- All other VLANs are tagged when transmitted
- ESXi management interfaces need no VLAN configuration

ESXi Host Network Configuration

Physical NIC Assignment

Intel NUCs (Dual NIC) 

vmnic0 (Built-in 1GbE):
├── Management (Primary)
├── VM Traffic
└── Failover for all traffic

vusb0 (USB 1GbE):
├── vMotion (Primary)
├── Storage (if using iSCSI/NFS)
└── Secondary path for redundancy

MS-A2 Hosts (Quad NIC) 

vmnic0/1 (2.5GbE):
├── Management
└── VM Traffic

vmnic2/3 (10GbE SFP+):
├── vMotion
├── vSAN
└── NSX TEP

VMkernel Adapter Assignment

VMkernel VLAN Service IP Example TCP/IP Stack
vmk0 10 Management 192.168.10.8 Default
vmk1 20 vMotion 192.168.20.8 vMotion
vmk2 30 vSAN 192.168.30.8 Default
vmk3 40 NSX TEP 192.168.40.8 Default

NSX-T Network Architecture

Traffic Flow Overview 

Physical Network (VLAN 50)
        ↕ (North-South)
   NSX Edge Nodes
        ↕
   Tier-0 Gateway
        ↕
   Tier-1 Gateways
        ↕
Overlay Networks (Segments)

The NSX Edge Uplink (VLAN 50) is critical for:

  • North-South Routing: Traffic between overlay networks and physical network
  • External Connectivity: Internet access for workloads
  • BGP Peering: Dynamic routing with physical routers
  • Load Balancer VIPs: External access to services

Edge Node Network Interfaces 

NSX Edge VM:
├── eth0: Management (VLAN 10)
├── fp-eth0: Uplink to physical network (VLAN 50)
├── fp-eth1: TEP for overlay (VLAN 40)
└── fp-eth2: (Optional) Additional uplink for HA

Distributed Switch Configuration

Port Group Settings

Management Port Group 

Name: Management-PG
VLAN Type: VLAN
VLAN ID: 10
Port Binding: Static
Security:
  - Promiscuous Mode: Reject
  - MAC Address Changes: Reject
  - Forged Transmits: Reject

vMotion Port Group 

Name: vMotion-PG
VLAN Type: VLAN
VLAN ID: 20
Port Binding: Static
Teaming and Failover:
  - Load Balancing: Route based on physical NIC load
  - Network Failure Detection: Beacon probing
Advanced:
  - MTU: 9000

vSAN Port Group 

Name: vSAN-PG
VLAN Type: VLAN
VLAN ID: 30
Port Binding: Static
Advanced:
  - MTU: 9000
Traffic Shaping:
  - Enabled for guaranteed bandwidth

Name: NSX-Edge-Uplink-PG
VLAN Type: VLAN
VLAN ID: 50
Port Binding: Ephemeral
Security:
  - Promiscuous Mode: Accept (for nested ESXi/vSAN witness)
  - MAC Address Changes: Accept
  - Forged Transmits: Accept
Advanced:
  - MTU: 1500 (or 9000 if physical network supports)
Notes:
  - Used by NSX Edge VMs for north-south traffic
  - Connects to physical router/firewall
  - May require specific security policies

Uplinks per Host: 2
Uplink Names:
  - Uplink 1: Primary NIC
  - Uplink 2: Secondary NIC
LAG Configuration: Not required for basic setup

Security Considerations

VLAN Security Best Practices

  1. Management VLAN Protection:
    • Restrict access via firewall rules
    • Use separate management workstation VLAN
    • Enable port security on switch
  2. Storage Network Isolation:
    • No routing between storage and other VLANs
    • Dedicated physical NICs when possible
    • Enable jumbo frames for performance
  3. NSX Overlay Security:
    • TEP VLANs should not be routable externally
    • Use dedicated NICs for TEP traffic
    • Monitor for unusual traffic patterns

Troubleshooting VLAN Issues

Common Problems and Solutions

No Connectivity on VLAN 

# Check VLAN configuration on vSwitch/DVS
esxcfg-vswitch -l

# Verify VLAN tagging
tcpdump-uw -i vmk0 -e

# Check physical switch trunk configuration

vMotion Failures

  • Verify MTU settings (9000 on all components)
  • Check VLAN configuration on both hosts
  • Ensure vmkernel has vMotion enabled

Storage Performance Issues

  • Confirm jumbo frames end-to-end
  • Check for VLAN mismatch
  • Verify storage multipathing

Implementation Checklist

  • Configure VLANs on physical switches
  • Set trunk ports for ESXi hosts
  • Create distributed switch in vCenter
  • Create all port groups with correct VLAN IDs
  • Configure uplinks per host
  • Create VMkernel adapters
  • Test connectivity per VLAN
  • Verify MTU settings
  • Document IP assignments
  • Test vMotion between hosts
  • Verify storage connectivity
  • Configure firewall rules

IP Address Planning

Management Network (VLAN 10) - 192.168.10.0/24 

192.168.10.1    - Gateway (Router/Firewall)
192.168.10.2-6  - Network Infrastructure
192.168.10.7    - Mac Pro ESXi
192.168.10.8    - esxi-nuc-01
192.168.10.9    - esxi-nuc-02
192.168.10.10   - esxi-nuc-03
192.168.10.11   - vCenter Server (VCSA)
192.168.10.15   - NSX Manager
192.168.10.20+  - Infrastructure VMs

vMotion Network (VLAN 20) - 192.168.20.0/24 

192.168.20.1    - Gateway (if needed)
192.168.20.8    - esxi-nuc-01 vmk1
192.168.20.9    - esxi-nuc-02 vmk1
192.168.20.10   - esxi-nuc-03 vmk1

vSAN Network (VLAN 30) - 192.168.30.0/24 

192.168.30.1    - No gateway needed
192.168.30.8    - esxi-nuc-01 vmk2
192.168.30.9    - esxi-nuc-02 vmk2
192.168.30.10   - esxi-nuc-03 vmk2

NSX Edge Uplink Network (VLAN 50) - 192.168.50.0/24 

192.168.50.1    - Physical Router/Firewall (Gateway)
192.168.50.10   - NSX Edge Node 1 Uplink
192.168.50.11   - NSX Edge Node 2 Uplink
192.168.50.20+  - Reserved for additional Edge nodes
192.168.50.100+ - Floating IPs for Load Balancer VIPs

NUC Management Network (VLAN 200) - 192.168.200.0/24 (OPTIONAL) 

NOTE: This network is NOT required for normal operation!
Keep Intel NUCs on the main management network (VLAN 10).

Use cases for VLAN 200:
- Temporary access during network troubleshooting
- Testing network changes without affecting production
- Emergency access if main management network has issues
- Lab experiments requiring isolated management

DO NOT move production NUCs to this network permanently.

Next Steps

  1. Physical Switch Setup: Configure VLANs on Ubiquiti switches
  2. ESXi Network Config: Set up distributed switch with port groups
  3. VMkernel Creation: Add VMkernel adapters for each service
  4. Testing: Verify connectivity and performance
  5. Documentation: Record all IP assignments and configurations

This project is for educational and home lab purposes.