Come on down to the UUID Rodeo! – A deep dive into virtual disk architecture for XenDesktop on XenServer

I recently needed to determine if there were any orphaned virtual disks that had been created by XenDesktop and not properly cleaned up. This XenDesktop environment is connected to a Citrix XenServer that has NFS shared storage. I wanted to write this article because navigating to find specific virtual disks in XenServer is not as straightforward as it is in vSphere. This article may also help you better understand XenDesktop’s Linked Clones virtual disk architecture.

For those of you not familiar, Citrix XenDesktop has a method to clone a master image into subsequent virtual desktops. This method is called Machine Creation Services or “MCS.” A group of virtual machines created from the same master image is called a Machine Catalog. These virtual desktops are called “linked clones” because of their virtual disk structure. “Linked Clone” means that each virtual desktop has a delta disk that accepts changes as the VM runs. However, all the delta disks in the Machine Catalog point back to the same, read-only base disk. The base disk is essentially a copy of the master virtual machine’s disk at the time the Machine Catalog was created.

If you are like me, you started your virtualization career working with VMware vSphere. After getting used to how things are done in that ecosystem, well, that’s just how you expect it to be. When I needed to investigate the virtual disks created by MCS on XenServer, it was different enough that I wanted to share the steps.

To begin the investigation of how XenDesktop on XenServer organizes its virtual disks, let’s first look at how it is done with XenDesktop on vSphere. In general, vSphere stores a virtual machine’s files in a folder that is named after the virtual machine. In the screenshot below, we are looking at vSphere’s Datastore Browser. The virtual machine named “XD71_2” is stored in a folder named “XD71_2” and the virtual machine’s files begin with the same name.

In the case of Citrix’s Machine Creation Services, it creates a folder named after the Machine Catalog to hold the base disk for the linked clones. The folder name starts with the name of the Machine catalog, and includes “baseDisk” in the name. The virtual disk (.vmdk) file is also named after the Machine Catalog.

Pretty good, right? It is very straightforward to find what you are looking for. Now, let’s look at how XenServer organizes its virtual disks. XenServer, places all VHD’s (virtual disks) into one directory and all VHD’s are named by UUID. This makes it much harder for the human eye to see the relationships between virtual desktops, machine catalog master-disks, linked-clone base disks, delta disks, and Identity Disks.

**Please note that the following exploration below does not cover Machine Catalogs of the “Pooled” type or with Personal vDisks. Only “Dedicated” Machine Catalogs are covered. “Dedicated” Machine Catalogs are created when the two options of “static” and “dedicated” are chosen in the Machine Catalog Creation wizard. Screenshot below.

DedicatedChoices

Dedicated Machine Catalogs can also be recognized in Citrix Studio by checking that the “User Data” field is set to “On local disk.” See the comparison below of a Dedicated Machine Catalog to a Catalog that uses a Personal vDisk.

StudioViewOfDedicated
In my first step of investigating how Citrix Machine Creation Services organizes virtual disks on XenServer, I ran the following command which shows all VHD’s in the desired Storage Repository (SR). The Storage Repository is referenced by this UUID, “5fe934bb-f1a9-6e3d-38ed-0f49f7f113d5.”

vhd-util scan -f -p -m ‘/var/run/sr-mount/5fe934bb-f1a9-6e3d-38ed-0f49f7f113d5/*.vhd’

The output shown below is structured so that child VHD’s are indented below their parent VHD. Child VHD’s rely on, or point back to, their parent VHD. The “delta” disks of a linked clone are child disks and point back to the base disk.

At the end of each line below, it either says “parent=<UUID>” or “parent=none.” If the VHD has a parent, it notes the parent’s UUID. Alternatively, it says “none” if it is a disk that has no dependency on a parent disk. See the two examples bordered in red below.

CLI_ParentEquals

The output above is from just a handful of machine catalogs and virtual desktops so you can see how investigating your virtual disk architecture in a larger environment could be a little challenging.

The first thing I wanted to do is determine which virtual machine the first parent disk belonged to. This is the first VHD listed in the output above. Its filename ends in “2c016.vhd.” This VHD has no parent as the end of the line notes “parent=none.”

The below command normally returns the virtual machine name related to the virtual disk. I ran this command to determine which virtual machine owned this virtual disk.

xe vbd-list vdi-uuid=<UUID_of_disk>

xe vbd-list vdi-uuid=461ab151-4e27-496f-acf1-abe7ecb2c016

However, running this command to find the name of the VM associated with this VHD fails; it returns nothing. In the screenshot above, there were a number of UUID’s that did not return a related virtual machine. Given this lack of information, I needed to narrow down the number of VHDs to be able to clarify the relationships in figuring this out for the first time. Otherwise, there were too many UUID’s to do a process of elimination and narrow down which UUID belonged to which virtual machine. To narrow this down, I created a new XenServer Storage Repository on the SAN. I created a new Host Connection in XenDesktop to define this new storage location. Then, I created a new machine catalog with only one virtual desktop. With this done, I opened two CLI sessions to the same XenServer host and put the output side by side, as shown below.

In the left pane, the “vhd-util” scan command shows all VHD’s in this new Storage Repository and is structured so that child VHD’s are indented below their parent VHD. This is the same command shown at the beginning of this article. In the right pane, the “vdi-list” command shows all VDI’s or “Virtual Disk Images” on the Storage Repository.  (You’ll need to zoom in on this image a bit . . . )

CLIx2

The text colors map the UUID in the left pane to the same UUID in the right pane. I will elaborate on each UUID, organized by color below:

GREEN: The UUID’s of the Storage Repository (SR).

BLUE:
The blue VHD listed in the left pane corresponds to the blue VDI in the right pane. This blue Virtual Disk Image (VDI) in the right-pane has a name-label of “base copy.” All the virtual disks ending in “-diff” (look at yellow VDI in right-pane) point back to this base virtual disk. This is the base disk that linked-clones in a “Dedicated” Machine Catalog use. Its name-label is always generically named “base copy,” making it harder to match to a virtual desktop or Machine Catalog.

YELLOW:
 The name-label for these Virtual Disk Images ends in “-diff.” Any changes to the virtual desktop after provisioning are captured in this differential/delta disk.

ORANGE:
With a “Dedicated” machine catalog type, this is the virtual disk that newly spawned virtual desktops will be created from. This disk is not attached to a virtual machine so therefore it cannot be started. Changes/updates to it can only be made via XenDesktop PowerShell commands. I coined the term “master-disk” to describe this virtual disk.

It is interesting to note that the “master-disk” of the machine catalog is actually a snapshot on top of the (blue) “base copy” for the Linked Clones base disk. This can be seen below because the (orange) “master-disk” in the left-pane lists its parent disk. Its parent disk is the (blue) base disk. When a Machine Catalog creation is initiated, a relatively long copy process can be seen on the hypervisor. This process is copying the bootable, master virtual machine’s disk to the (blue) Machine Catalog’s base disk.

RED: The Identity Disk is always a 16MB file that contains the computer name and Active Directory account password (among other things). This ensures each VM is unique. This disk is not a Linked Clone itself, even if it is associated with a virtual desktop that is a Linked Clone. That’s why it shows that it has no parent disk.

As you can see, the cross-referencing of UUID’s in XenServer is certainly more time-consuming than the friendly names of folders and files in vSphere. Is this like being at the UUID rodeo? You be the judge. I personally would like XenServer to move towards an organization of virtual disks that is easier on the human eye!

I hope this helps anyone that needs to dive into their XenDesktop virtual disks on XenServer!

-Richard

Advertisements

3 thoughts on “Come on down to the UUID Rodeo! – A deep dive into virtual disk architecture for XenDesktop on XenServer

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s