MS-A2 NVMe Memory Tiering Configuration Guide

This guide covers the complete setup of NVMe memory tiering on MINISFORUM MS-A2 hosts with AMD Ryzen 9 9955HX processors.

Overview

Hardware Configuration:

  • MS-A2-01: 192.168.10.12 (esxi-ms-a2-01.lab.markalston.net)
  • MS-A2-02: 192.168.10.13 (esxi-ms-a2-02.lab.markalston.net)

Storage Layout per Host:

  • Samsung 980 500GB: Boot drive (ESXi installation)
  • Samsung 990 PRO 4TB: NVMe memory tiering device
  • WD_BLACK SN850X 4TB: VM storage datastore
  • 128GB DDR5: Base system memory

Memory Tiering Benefits

With NVMe memory tiering enabled:

  • Base Memory: 128GB DDR5
  • NVMe Memory Tier: ~512GB (400% of base memory)
  • Total Effective Memory: ~640GB per host
  • Performance: Near-DRAM speeds for frequently accessed memory pages

Storage Configuration

Step 1: Verify Hardware Detection

# SSH to each host and verify all NVMe devices are detected
ssh root@192.168.10.12  # MS-A2-01
ssh root@192.168.10.13  # MS-A2-02

# List all NVMe devices
esxcli storage core device list | grep -E 'Display Name|Size'

Expected Output:

  • Samsung_SSD_980_500GB (476,940 MB)
  • Samsung_SSD_990_PRO_4TB (3,815,447 MB)
  • WD_BLACK_SN850X_4000GB (3,815,447 MB)

Step 2: Create VM Storage Datastores

Configure WD_BLACK SN850X drives for VM storage:

# Check current partition table
partedUtil getptbl /vmfs/devices/disks/t10.NVMe____WD_BLACK_SN850X_4000GB__________________[SERIAL]

# Create GPT partition table with VMFS partition
partedUtil setptbl /vmfs/devices/disks/t10.NVMe____WD_BLACK_SN850X_4000GB__________________[SERIAL] gpt "1 2048 7814037068 AA31E02A400F11DB9590000C2911D1B8 0"

# Create VMFS6 filesystem
vmkfstools -C vmfs6 -S '[DATASTORE_NAME]' /vmfs/devices/disks/t10.NVMe____WD_BLACK_SN850X_4000GB__________________[SERIAL]:1

# Verify datastore creation
esxcli storage filesystem list | grep wd-sn850x

Device Names:

  • MS-A2-01: t10.NVMe____WD_BLACK_SN850X_4000GB__________________E92917428B441B00
  • MS-A2-02: t10.NVMe____WD_BLACK_SN850X_4000GB__________________3C2917428B441B00

Datastore Names:

  • MS-A2-01: esxi-ms-a2-01-wd-sn850x
  • MS-A2-02: esxi-ms-a2-02-wd-sn850x

NVMe Memory Tiering Configuration

Step 3: Configure Memory Tiering

Run on each MS-A2 host:

# Enable memory tiering kernel setting
esxcli system settings kernel set -s MemoryTiering -v TRUE

# Create tier device using Samsung 990 PRO
esxcli system tierdevice create -d /vmfs/devices/disks/t10.NVMe____Samsung_SSD_990_PRO_4TB_________________[SERIAL]

# Set NVMe percentage to 400% (4x base memory = ~512GB tier)
esxcli system settings advanced set -o /Mem/TierNvmePct -i 400

# Reboot to activate memory tiering
reboot

Samsung 990 PRO Device Names:

  • MS-A2-01: t10.NVMe____Samsung_SSD_990_PRO_4TB_________________72A9415145382500
  • MS-A2-02: t10.NVMe____Samsung_SSD_990_PRO_4TB_________________84A9415145382500

Step 4: Verify Memory Tiering Configuration

After reboot, verify the configuration:

# Check tier device status
esxcli system tierdevice list

# Verify memory tiering settings
esxcli system settings kernel list -o MemoryTiering
esxcli system settings advanced list -o /Mem/TierNvmePct

Expected Results:

# Tier device should show full Samsung 990 PRO allocation
DeviceName                                                            PartitionId  StartSector  EndSector
--------------------------------------------------------------------  -----------  -----------  ---------
t10.NVMe____Samsung_SSD_990_PRO_4TB_________________[SERIAL]          1            2048         7814037134

# Memory tiering should be active
MemoryTiering  Bool  TRUE        TRUE     FALSE    Enables Memory Tiering

# NVMe percentage should be 400%
Int Value: 400

Datastore Management

Step 5: Rename Boot Datastores

Use vCenter Web UI for consistent naming:

  1. Access vCenter: https://vcsa.markalston.net
  2. Navigate: Storage → Datastores
  3. Rename Boot Datastores:
    • datastore1esxi-ms-a2-01-boot (MS-A2-01)
    • datastore1 (1)esxi-ms-a2-02-boot (MS-A2-02)

Final Datastore Configuration

MS-A2-01 (192.168.10.12):

  • esxi-ms-a2-01-boot - Samsung 980 500GB (Boot disk)
  • esxi-ms-a2-01-wd-sn850x - WD_BLACK 4TB (VM storage)
  • OSDATA-[UUID] - ESXi system partition

MS-A2-02 (192.168.10.13):

  • esxi-ms-a2-02-boot - Samsung 980 500GB (Boot disk)
  • esxi-ms-a2-02-wd-sn850x - WD_BLACK 4TB (VM storage)
  • OSDATA-[UUID] - ESXi system partition

Verification Commands

Memory and Storage Status

# Check total memory (should show ~128GB base)
esxcfg-info -F | grep "Physical Mem"

# Verify tier device allocation
esxcli system tierdevice list

# Check all datastores
esxcli storage filesystem list | grep VMFS

# Monitor memory tiering performance
esxtop -b -n 1 | grep -E "MEM|TIER"

Performance Monitoring

# Check NVMe device performance
esxcli storage core device stats get -d [DEVICE_NAME]

# Monitor memory tiering activity
vsish -e get /memory/comprehensive | grep -i tier

Troubleshooting

Common Issues

  1. Memory Tiering Not Active
    • Verify kernel setting: esxcli system settings kernel list -o MemoryTiering
    • Check reboot completed: Runtime should show TRUE
  2. Tier Device Creation Failed
    • Ensure Samsung 990 PRO is not already partitioned
    • Verify device path is correct
    • Check for existing tier devices: esxcli system tierdevice list
  3. Datastore Rename Failed
    • Use vCenter Web UI instead of CLI
    • Ensure no VMs are running on the datastore
    • Check vCenter connection status

Recovery Commands

# Remove tier device if needed
esxcli system tierdevice remove -d [DEVICE_NAME]

# Reset memory tiering
esxcli system settings kernel set -s MemoryTiering -v FALSE
esxcli system settings advanced set -o /Mem/TierNvmePct -i 25

Performance Impact

Benefits:

  • 5x increase in effective memory capacity (128GB → 640GB)
  • Reduced memory pressure for large VMs
  • Improved performance for memory-intensive workloads
  • Optimal for VCF management domain deployment

Considerations:

  • Samsung 990 PRO dedicated to memory tiering (not available for VM storage)
  • Slight latency increase for cold memory pages
  • Requires ESXi 8.0+ for optimal performance

References


Configuration completed: 2025-01-15 Tested on: ESXi 8.0.3, MINISFORUM MS-A2 with AMD Ryzen 9 9955HX


This project is for educational and home lab purposes.