Citrix 3D Graphics Pack (vGPU)

Citrix 3D Graphics Pack is the new name for the “Citrix Virtual GPU solution” that was introduced in October 2013 in Tech Preview which is NVIDIA vGPU and XenServer/XenDesktop components. The cool thing about this release is that NVIDIA vGPU is now released and no more “technical preview/beta” and the product have been fully built into XenServer and Citrix have created some amazing GUI in XenServer 6.2 XenCenter and XenDesktop 7.1. There is a cool new SDK commands to fully automate the GPU commands if you want to use the CLI method instead of the GUI mode. December 16th, Citrix released support for GPU virtualization using XenDesktop 7.1 HDX 3D Pro with XenServer 6.2 SP1 in the Citrix 3D Graphics Pack (see http://www.citrix.com/go/vgpu). This means that multiple users can share a single GPU, overcoming the 1:1 ratio to achieve higher user densities and create a more cost-effective remote 2D/3D virtualization solution.

• Kudos to NVIDIA for building the vGPU
• Kudos to Citrix XenServer team for integrating the vGPU into the Xen
• Kudos to Citrix XenDesktop team for integrating the vGPU into the XenDesktop
• Kudos to all the Citrix HDX 3D crew

What is Citrix 3D Graphics Pack

The Citrix 3D Graphics Pack enables true hardware GPU sharing of NVIDIA GRID Graphics cards providing the industry’s highest performance virtualized professional graphics app acceleration. This technology was first unveiled at Citrix Synergy 2013 and allows GPU sharing for Virtual Desktop Infrastructure (VDI) for XenServer, XenDesktop and NVIDIA GRID GPUs.

XenServerGPU pass-through improvements including XenCenter configuration.

Citrix/NVIDIA XenServer w. vGPU Architecture

Sizing NVIDIA vGPU profile

NVIDIA vGPU profiles are designed different with amount of memory, CUDA cores and frame buffer, amount of display and display resolution pr GRID GPU type.

I have added more informations than Citrix and NVIDIA does in their branding.
vGPU OS support is one of the important things where you clearly see which OS is supported on Pass-through profile vs vGPU profiles.

Which GRID to choose

Whats new in XenServer 6.2 SP1

• The 3D Graphics Pack supporting NVIDIA GRID GPUs
• Support for Windows 8.1 and Windows Server 2012 R2
• Improvements to the Site Recovery wizard for large deployments
• GPU pass-through improvements including XenCenter configuration.
• New SR wizard allows up to 50 new fibre-channel HBA SRs to be created in a single step.
• Security Hotfix and functional Hotfix roll-up.
• New SDK for XenServer 6.2.0 Service Pack 1 is ideal for developers wishing to access programmatically XenServer’s new management features for GPU virtualization (including the new vGPU and GPU pass-through). The five available XenServer SDKs, one for each of C, C#, Java, PowerShell and Python, expose the new XenAPI commands for working with physical GPUs (pGPUs), GPU groups, virtual GPUs (vGPUs) and virtual GPU types. The GPU technologies for XenDesktop and XenServer do of course also come with rich GUI configuration operations and provisioning via XenCenter, XenDesktop and MCS.Citrix last few development cycles they have invested in re-writing XenServer’s PowerShell API to provide developers and administrators with a PoSH alternative to using the XenServer command line (CLI) interface. In particular this interface is proving popular with those looking to automate bespoke vGPU and GPU pass-through configuration and benchmarking or auto-test frameworks. The PowerShell API is also a popular choice for XenDesktop and Windows administrators working with XenServer. Read more about the SDK here

Whats new in NVIDIA GRID vGPU Pack

GA of NVIDIA GRID vGPU Manager + Windows Display Driver

• Latest NVIDIA GRID vGPU Manager is version  (331.30)
• Latest NVIDIA GRID vGPU Windows Display Driver (332.07) for Windows 7, Windows 8, Server 2008R2, Server 2012.

Important if you implemented XenServer vGPU tech preview

• Customers who have previously installed the vGPU Tech Preview (XS62ETP001) on a host, cannot subsequently install Service Pack 1. Customers wishing to install Service Pack 1 will need to do a fresh installation of XenServer 6.2.0, before installing Service Pack 1.

How to implement Citrix 3D Graphics Pack

Download Citrix XenServer 6.2 + SP1
Download NVIDIA GRID vGPU Pack for GRID K1 or GRID K2
Download Citrix XenDesktop 7.1 99 user trial or licensed software here (require MyCitrix ID)

1. Start with a fresh XenServer 6.2 installation on a GRID supported hardware
2. Install Service Pack 1 on the XenServer 6.2
3. Download the NVIDIA GRID vGPU Pack and unzip the contents; install NVIDIA GRID manager in XenServer from the CLI
4. Create a Windows 7 VM (this will be the base image)
5. From the XenCenter GUI, assign a vGPU type to the base image
6. In the Windows 7 VM:
a. Install NVIDIA GPU guest OS driver (available in the NVIDIA GRID vGPU Pack)
b. Install the XenServer Tools
c. Install the latest version of Citrix HDX 3D Pro VDA 7.1
7. Create a Machine Catalog using MCS to provision new VMs based on the base image or you can also use Citrix Provisioning Services (PVS)
8. Create Delivery Group, assign users, and publish the desktops
9. Access virtual desktops using Citrix Receiver. No GPU is required on the end-point devices
10. Validate GPU sharing by multiple desktops, using monitoring tools like Process Explorer from Microsoft

GUI enhancements in XenCenter for XenServer 6.2 SP1

In the Citrix XenCenter, there is a new tab called “GPU” at the host level. The appropriate vGPU types attached to the host are defined in this GUI, and made available to the virtual machines (VM). Depending on the requirements, one can also define the GPU placement policy here. This tab also makes it very convenient to visualize how many vGPU’s are already attached, and the physical GPU’s where they get placed.
This makes later troubleshooting simpler.

At the VM-level in XenCenter, the vGPU can be selected as part of VM properties or during New VMcreation on GPU enabled hosts. In the tech preview, this was a laborious step in the CLI. Now, simply determine the suitable vGPU Profile for your use-case and select it from the drop-down list. Once the VM is created, it boots into the Windows standard 800X600 VGA resolution. The vGPU features are available once the guest driver is installed in the Windows VM.

GPU performance graphs are available under the Performance tab of XenServer host. On first-run, these graphs have to be added to the view. Subsequently, they can be moved up or down and can show one or more of the installed GPUs.

GUI enhancements in XenDesktop 7.1 with XenServer 6.2 SP1

There are few GPU related enhancements on the XenDesktop consoles, and automated-provisioning of vGPU-enabled VMs using Machine Creation Services (MCS) is the one we’ve been waiting for. Simply attach a vGPU to the base VM, install the virtual delivery agent (VDA for HDX 3D Pro), and install the required graphics apps. Then head over to XenDesktop Studio to create the machine catalog. The only part to exercise caution is not to perform a Sysprep after creating a vGPU-enabled base image, else it wipes out the vGPU information.

In Studio, the vGPU Type must be defined while creating the host settings to be used as a platform for the MCS machines.

Subsequently, proceed to creation of a machine catalog as usual. The exact steps are outlined in the Reviewer’s Guide. At the step where MCS base image is chosen, hovering over the image name shows information to confirm if you have a valid vGPU-enabled master image.

The remaining process to create machine catalog, create delivery group, and assign users is no different than the usual way of delivering desktops and apps. Use the latest Citrix Receiver to access 3D apps.

Tweak XenServer 6.2.x for GPU intense applications/performance

Below articles are critical to follow, no matter if you use GPU pass-through or vGPU profiles with your virtual machines.
I have seen many GPU intense applications that uses the Turbo mode or the max CPU clock frequency and if you think Turbo mode work out of the box, think again. Many virtualize their 3D applications and will have an impact on this if this is not configured.

Follow this article How to use host-cpu-tune to fine tune XenServer 6.2.0 performance

Follow this article How to investigate and use Turbo mode, C-States and P-States in XenServer

Tweak Citrix XenDesktop 7.1 HDX 3D Pro.

I have seen multiple issues with performance, now Citrix have officially shared the informations, so shall I help you position how to tweak XenDesktop 7.1 HDX 3D. Following tweaks are for XenDesktop 7.0 & 7.1 VDA’s.

• With high screen resolutions (such as 2560×1600), a lower than expected Frames per Second (FPS) may be apparent, impacting user experience.
  Change Encodespeed from
  [HKEY_LOCAL_MACHINE\SOFTWARE\Citrix\Graphics]”EncodeSpeed”=dword:00000002
  to [HKEY_LOCAL_MACHINE\SOFTWARE\Citrix\Graphics]”EncodeSpeed”=dword:00000001
  registry key to work around this issue.
• While connecting to high resolution displays (for example: 2560×1600) artifacts of previously opened windows can remain. To ensure that the screen is refreshed, users can add the following registry key:
  [HKEY_LOCAL_MACHINE\Software\Citrix\HDX3D\BitmapRemotingConfig]”HKLM_EnableDirtyRect”=dword:00000000

Source

Software – Download vGPU (GRID Manager + GRID drivers) + XenServer 6.2 SP1 here

Ctx article – How to Resolve GPU Memory Mapping Issues in XenServer *important to check if you want to use vGPU

Citrite Mayunk Jain blogpost Super Easy GPU Sharing with XenDesktop 7.1: Introducing 3D Graphics Pack

Citrite Mayunk Jain Reviewer’s Guide for Delivering 3D Graphics Apps: Part 3 (vGPU)

Citrite Konstantina Chremmou blogpost – Configuring vGPU and GPU pass-through using the PowerShell SDK for XenServer 6.2.0 Service Pack 1

Citrite Rachel Berry blogpost – Configuring XenServer to use Turbo mode – including for 3-D graphically intense applications

Xen Team Advice for developers and partners working with GPUs

Alexander Ervik (CTP)  Shows how to enable NVIDIA vGPU support in XenServer 6.2 SP1 with Dell R720

Citrix blogpost – True hardware GPU sharing with XenDesktop and NVIDIA GRID arrives!

Citrix FlexCast Services: Virtualize 3D professional graphics

Citrix Technical and Training Materials about vGPU & HDX 3D Pro

NVIDIA GRID certified OEM servers

NVIDIA GRID certified applications