Microsoft Azure – NVadsA10 v5-series

Microsoft have released a new instance powered by NVIDIA A10 GPUs and AMD EPYC 74F3V(Milan) CPU with a base frequency of 3.2 Ghz, all cores peak frequency of 4.0Ghz. With NVadsA10v5-series Azure is introducing virtual machines with partial NVIDIA GPUs. Pick the right sized virtual machine for GPU accelerated graphics applications and virtual desktops starting at 1/6th of a GPU with 4-GiB frame buffer to a full A10 GPU with 24-GiB frame buffer.

The preview is currenty availabe in Azure – US South Central and Azure – West Europe regions.

Prices are now GA (21th March 2022) for North Europe, US East 2, US West3

NVIDIA Ampere GPU (A10) is supporting GPU-P (SR-IOV) which is why its a big step for partitioning the GPU and making the price dramatically cheaper in Azure. *note there have not been released any prices yet of the instances (its unknown)

Blogpost last updated: 21th March 2022

NVadsA10v5-series – The CPU

Inside the NVadsA10v5-series there is a AMD EPYC “MILAN” Cpu platform.
More specific its the AMD EPYC 74F3V MILAN processor.

Benefits

Up to 24 Epyc “Milan” Cores, 48 of Threads @3.2 GHz clock base, Turbo up to 4.0 Ghz

Huge dedicated L3 per CCX provides isolation from noisy compute neighbours

NVadsA10v5-series – The GPU

NVIDIA A10 technical specifications and features

FP32	31.2 teraFLOPS
TF32 Tensor Core	62.5 teraFLOPS \| 125 teraFLOPS*
BFLOAT16 Tensor Core	125 teraFLOPS \| 250 teraFLOPS*
FP16 Tensor Core	125 teraFLOPS \| 250 teraFLOPS*
INT8 Tensor Core	250 TOPS \| 500 TOPS*
INT4 Tensor Core	500 TOPS \| 1,000 TOPS*
RT Core	72 RT Cores
Encode/decode	1 encoder 2 decoder (+AV1 decode)
GPU memory	24GB GDDR6
GPU memory bandwidth	600GB/s
Interconnect	PCIe Gen4 64GB/s
SR-IOV support	Supported: 32 VF (virtual functions)
NVIDIA® CUDA® Support	CUDA 11.2 (or later)
vGPU software support	Supports vGPU 12.x (or later): • NVIDIA RTX Virtual Workstation (vWS) • NVIDIA Virtual PC (vPC)/Virtual Applications (vApps) • NVIDIA AI Enterprise • NVIDIA Virtual Compute Server (vCS)

The NVIDIA A10 Tensor Core GPU combines with NVIDIA RTX Virtual Workstation (vWS) software to bring mainstream graphics and video with AI services to mainstream enterprise servers, delivering the solutions that designers, engineers, artists, and scientists need to meet today’s challenges. Built on the latest NVIDIA Ampere architecture, the A10 combines second- generation RT Cores, third-generation Tensor Cores, and new streaming microprocessors with 24 gigabytes (GB) of GDDR6 memory—all in a 150W power envelope—for versatile graphics, rendering, AI, and compute performance. From virtual workstations, accessible anywhere in the world, to render nodes to the data centers running a variety of workloads, A10 is built to deliver optimal performance in a single-wide, full-height, full-length PCIe form factor.

NVIDIA A10 is a powerful GPU look at some of below benchmark that NVIDIA have done

Benefits of SR-IOV

SRIOV is a method for providing each VM with it’s own fully-virtualized GPU (or other PCI) device
Offers exceptional protection and security for each partitioned VM at the host level via HyperV/GPU-P
Delivers Predictable/Fixed Performance
Offers exceptional protection against noisy neighbors

Predictable performance and security with hardware partitioning of the GPU

In Azure, the security of the customer’s workload is always a top priority. SR-IOV based GPU partitioning provides a strong, hardware-backed security boundary with predictable performance for each virtual machine. Microsoft partition a single NVIDIA A10 GPU and allocate it up to six virtual machines. Each virtual machine can only access the GPU resources dedicated to them and the secure hardware partitioning prevents unauthorized access by other VMs.

The diagram describes the interactions between the host and the guest SR-IOV drivers for GPU partitioning

NVadsA10v5-series – Instance types

Size	vCPU	Memory: GiB	Temp storage (SSD) GiB	GPU partition	GPU memory: GiB	Max data disks	Max NICs / Expected network bandwidth (MBps)
Standard_NV6ads_A10_v5	6	55	180	1/6	4	4	2 / 5000
Standard_NV12ads_A10_v5	12	110	360	1/3	6	4	2 / 10000
Standard_NV18ads_A10_v5	18	220	720	1/2	12	8	4 / 20000
Standard_NV36ads_A10_v5	36	440	720	1	24	16	4 / 40000
Standard_NV36adms_A10_v5	36	880	720	1	24	32	8 / 80000
Standard_NV72ads_A10_v5	72	880	1400	2	48	32	8 / 80000

¹ NVadsA10v5-series VMs feature AMD Simultaneous multithreading Technology

NVadsA10v5-series – Availability in which Azure datacenter.

The preview is currenty availabe in following Azure datacenter:

US South Central
West Europe regions.

NVadsA10v5-series – Prices

Prices are now available with Microsoft Azure “NVads A10 v5 series”

https://azure.microsoft.com/en-us/pricing/details/virtual-machines/windows/

There is no prices on NV18ads (yet)

Azure US East2 have the cheapest prices for the NVadsA10v5-series
The cheapest instance is NV6ads_A10-V5 if calculated vGPU with 1 physical GPU A10.

Size	Azure North Europe	Azure US East 2	Azure US West 3
Standard_NV6ads_A10_v5	$0,545/hour	$0,454/hour	$0,59/hour
Standard_NV12ads_A10_v5	$1,09/hour	$0,908/hour	$1,18/hour
Standard_NV18ads_A10_v5	$	$	$
Standard_NV36ads_A10_v5	$3,84/hour	$3,20/hour	$4,16/hour
Standard_NV36adms_A10_v5	$5,424/hour	$4,52/hour	$5,876/hour
Standard_NV72ads_A10_v5	$7,824/hour	$6,52/hour	$8,476/hour

^{Prices are from 21th March 2022 from https://azure.microsoft.com/en-us/pricing/details/virtual-machines/windows/}

NVadsA10v5-series – GPU Driver

To take advantage of the GPU capabilities of Azure NVadsA10v5-series VMs, NVIDIA GPU drivers must be installed.

During preview you need to manually install the NVIDIA GPU-P driver for Linux and Windows. Microsoft will release updated drivers before GA and include it in extensions and all the standard documentation pages.

Verify driver installation

Please note that the Nvidia Control panel is only accessible with the GRID driver installation. If you have installed CUDA drivers then the Nvidia control panel will not be visible.

You can verify driver installation in Device Manager. The following example shows successful configuration of the NVIDIA A10 GPU card on an Azure VM (Standard_NV36ads_A10_v5).

To query the GPU device state, run the nvidia-smi command-line utility installed with the driver.

Open a command prompt and change to the C:\Program Files\NVIDIA Corporation\NVSMI directory.
Run nvidia-smi. If the driver is installed, you will see output similar to the following. The GPU-Util shows 0% unless you are currently running a GPU workload on the VM. Your driver version and GPU details may be different from the ones shown.

NVadsA10v5-series – Supported OS

During preview Microsoft will support the following guest operating systems.

Windows Server 2019(RS5)
Windows 10 20H2
Windows 11
Ubuntu 18.04
Ubuntu 20.04
CentOS 7.9
RHEL 7.9

NVadsA10v5-series – Supported API

NVIDIA vGPU includes support for the following APIs:

Open Computing Language (OpenCL™ software) 3.0
OpenGL^® 4.6
Vulkan^® 1.3
DirectX 11
DirectX 12 (Windows 10)
Direct2D
DirectX Video Acceleration (DXVA)
NVIDIA^® CUDA^® 11.6
NVIDIA vGPU software SDK (remote graphics acceleration)
NVIDIA RTX (on GPUs based on the NVIDIA Volta graphic architecture and later architectures)

Note: These APIs are backwards compatible. Older versions of the API are also supported.

NVadsA10v5-series – Recommended GPO to configure

Configure GPU-accelerated app rendering – Windows VM

By default, apps and desktops running on Windows Server are rendered with the CPU and do not leverage available GPUs for rendering. Configure Group Policy for the session host to enable GPU-accelerated rendering:

Connect to the desktop of the VM using an account with local administrator privileges.
Open the Start menu and type “gpedit.msc” to open the Group Policy Editor.
Navigate the tree to Computer Configuration > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Remote Session Environment.
Select policy Use hardware graphics adapters for all Remote Desktop Services sessions and set this policy to Enabled to enable GPU rendering in the remote session.

Configure GPU-accelerated frame encoding

Remote Desktop encodes all graphics rendered by apps and desktops (whether rendered with GPU or with CPU) for transmission to Remote Desktop clients. When part of the screen is frequently updated, this part of the screen is encoded with a video codec (H.264/AVC). By default, Remote Desktop does not leverage available GPUs for this encoding. Configure Group Policy for the session host to enable GPU-accelerated frame encoding. Continuing the steps above:

Select policy Configure H.264/AVC hardware encoding for Remote Desktop connections and set this policy to Enabled to enable hardware encoding for AVC/H.264 in the remote session. NoteIn Windows Server 2016, set option Prefer AVC Hardware Encoding to Always attempt.
Now that the group policies have been edited, force a group policy update. Open the Command Prompt and type:cmdCopygpupdate.exe /force
Sign out from the Remote Desktop session.

Configure fullscreen video encoding

Note

Fullscreen video encoding can be enabled even without a GPU present.

If you often use applications that produce a high-frame rate content, such as 3D modeling, CAD/CAM and video applications, you may choose to enable a fullscreen video encoding for a remote session. Fullscreen video profile provides a higher frame rate and better user experience for such applications at expense of network bandwidth and both session host and client resources. It is recommended to use GPU-accelerated frame encoding for a full-screen video encoding. Configure Group Policy for the session host to enable fullscreen video encoding. Continuing the steps above:

Select policy Prioritize H.264/AVC 444 Graphics mode for Remote Desktop connections and set this policy to Enabled to force H.264/AVC 444 codec in the remote session.
Now that the group policies have been edited, force a group policy update. Open the Command Prompt and type:cmdCopygpupdate.exe /force
Sign out from the Remote Desktop session.

Verify GPU-accelerated app rendering

To verify that apps are using the GPU for rendering, try any of the following:

For Azure VMs with a NVIDIA GPU, use the nvidia-smi utility as described in Verify driver installation to check for GPU utilization when running your apps.
On supported operating system versions, you can use the Task Manager to check for GPU utilization. Select the GPU in the “Performance” tab to see whether apps are utilizing the GPU.

Verify GPU-accelerated frame encoding

To verify that Remote Desktop is using GPU-accelerated encoding:

Connect to the desktop of the VM using Azure Virtual Desktop client.
Launch the Event Viewer and navigate to the following node: Applications and Services Logs > Microsoft > Windows > RemoteDesktopServices-RdpCoreCDV > Operational
To determine if GPU-accelerated encoding is used, look for event ID 170. If you see “AVC hardware encoder enabled: 1” then GPU encoding is used.

Verify fullscreen video encoding

To verify that Remote Desktop is using fullscreen video encoding:

Connect to the desktop of the VM using Azure Virtual Desktop client.
Launch the Event Viewer and navigate to the following node: Applications and Services Logs > Microsoft > Windows > RemoteDesktopServices-RdpCoreCDV > Operational
To determine if fullscreen video encoding is used, look for event ID 162. If you see “AVC Available: 1 Initial Profile: 2048” then AVC 444 is used.