A few weeks ago, Tim Mueting, AMD Product Manager, Virtualization Solutions, was kind enough to respond to a Q&A concerning AMD-V Rapid Virtualization Indexing (RVI) technology that is leveraged in Windows Server 2008 R2 Hyper-V to enable Second Level Address Translation (SLAT). Tim works closely with AMD's technology partners and customers in the area of virtualization and focuses extensively on the aspects of hardware-assisted virtualization and the industry-leading virtualization software solutions. With more than 20 years of experience in delivering and marketing large scale enterprise IT solutions, Tim is a knowledgeable source on hardware-assisted virtualization.
First, Some Background
In Windows Server 2008 Hyper-V, the Windows Hypervisor manages two levels of address space translation for guest operating systems running in virtual machines. The first level of translation maps the guest operating system virtual address space into its physical address space. The second level of translation maps the guest operating system physical address space into the system physical address space. In order to do this, the Windows Hypervisor maintains a Shadow Page Table that combines the two levels of address space translation into a single page table.
In contrast, Windows Server 2008 R2 Hyper-V eliminates the need to maintain a Shadow Page Table by implementing Second Level Address Translation. In the following section, Tim describes how AMD-V RVI technology maintains and performs the two levels of address space translations required for a virtual machine in hardware, allowing a reduction in the complexity of the Windows Hypervisor in Windows Server 2008 R2 Hyper-V, and context switches needed to manage virtual machine page faults. As you will see, AMD-V RVI and Hyper-V R2 SLAT provide specific advantages for Hyper-V servers that run a large number of virtual machines or virtual machine workloads that very frequently modify memory.
Q&A with Tim Mueting on AMD-V Rapid Virtualization Indexing Technology
1. Can you first define the virtualization features that AMD currently has in its AMD Opteron™ processor line?
Tim Mueting: In 2006 AMD introduced our first-generation hardware support for x86 virtualization called AMD-Virtualization™ (AMD-V™) technology. AMD-V™ technology is a set of hardware extensions to the x86 system architecture designed to help virtualization software solutions more efficiently run applications in separate, isolated environments and hide the complexity of hardware infrastructures to help simplify manageability. AMD-V™ technology leverages the AMD Opteron™ processor and the Direct Connect Architecture to provide efficient memory handling between the multiple virtual machines running on the server platform and enable fast and efficient switching between virtual machines. There are many extension included in AMD-V™ but I would like to address two of the key features that help us to accomplish this: Tagged TLB and Rapid Virtualization Indexing (RVI).
The Tagged TLB is essentially an extension to the translation look-aside buffer (TLB). A TLB is a CPU cache used to improve virtual address translation speed. The TLB contains page table entries which map virtual addresses to physical addresses. The TLB was designed to contain these mappings for a single operating system. AMD-V has extended this design and added a unique Address Space Identifier, or ASID, to each entry which signifies, or tags, the virtual machine the associated memory location belongs to. This identifier alleviates the need to flush the TLB each time processing switches from one virtual machine to another, which improves the efficiency of switching back and forth between virtual machines.
A second key feature of AMD-V™ is RVI. RVI, introduced with Quad-Core AMD Opteron processors in 2007, addresses a critical need for memory management for virtualization or MMU (memory management unit) virtualization.
2. Can you provide more details on the Rapid Virtualization Indexing (RVI) architecture and how it works?
Tim Mueting: Prior to the introduction of RVI, software solutions used something called shadow paging to translate a virtual machine “guest” physical address to the system’s physical address. Because the original page table architecture wasn’t designed with virtualization in mind, a mirror of the page tables had to be created in software, called shadow page tables, to keep information about the physical location of “guest” memory. With shadow paging, the hypervisor must keep the shadow page tables “in sync” with the page tables in hardware. Every time the guest OS modifies its page mapping, the hypervisor must adjust the shadow page tables to reflect the modification. The constant updating of the shadow pages tables takes a lot of CPU cycles. As you might expect, for memory intensive applications, this process can make up the largest part of the performance overhead for virtualization.
With Rapid Virtualization indexing the virtual memory (Guest OS) to physical memory (Guest OS) and the physical memory (Guest OS) to real physical memory translations are cached in the TLB. As described earlier, we also added a new identifier to the TLB called an Address Space Identifier (ASID) which assigns each entry to a specific VM. With this tag, the TLB entries do not need to be flushed each time execution switches from one VM to another. This simplifies the work that the hypervisor needs to do and removes the need for the hypervisor to update shadow page tables. We can now rely on the hardware to determine the physical location of the guest memory.
3. Are processor and memory overhead reduction the primary targets for RVI with respect to Hyper-V?
Tim Mueting: AMD’s design goal for AMD-V technology has always been to help deliver near native performance for applications running in a virtualized environment. RVI is an important component to achieving this goal by offloading a fairly intensive process from software and moving it down into hardware where it can perform efficiently while reducing hypervisor complexity. RVI is not required for Hyper-V to operate effectively but, as noted, it can improve performance for many workloads.
4. Is RVI available on all AMD Opteron processor models?
Tim Mueting: RVI was first introduced with our Quad-Core AMD Opteron processor code named “Barcelona” in 2007 and is in all Quad-Core and Six-Core AMD Opteron processor models.
5. Windows Server 2008 R2 supports Second Level Address Translation (SLAT) which allows Hyper-V to take advantage of RVI – are the performance benefits derived from RVI workload-dependent, and if so, which workloads will see the most benefits?
Tim Mueting: Users of Hyper-V R2 can gain the advantages that RVI offers to many virtualized applications. RVI can help achieve more efficiency and performance for virtualized workloads that stress the MMU. According to Microsoft, CPU time associated with the hypervisor drops from 10% to 2% when RVI is enabled. We have observed that demanding workloads such as terminal services (renamed Remote Desktop Services in Windows Server 2008 R2), database applications, and web services applications specifically can benefit from RVI. Other workloads have been shown to benefit as well, but as always when discussing performance, your mileage may vary. I encourage everyone to do the necessary testing yourself to determine the appropriate workloads. Certain workloads that do not stress MMU as much (do minimal memory translations) may see less benefit than others.
In addition, it is important to point out that a very small percentage of workloads may see degradation in performance with RVI. In certain situations when the TLB doesn’t have the correct entry, called a TLB miss, additional searches must be done which will incur more overhead. To compensate for this we have made the TLBs much larger which reduces the likelihood of a TLB miss. Thankfully, very few applications exhibit this behavior.
If you want to learn more about how AMD virtualization technology supports Windows Server 2008 R2 Hyper-V, take a look at:
Tim Mueting is a Product Marketing Manager at AMD. His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites are provided for convenience and unless explicitly stated, AMD is not responsible for the contents of such linked sites and no endorsement is implied.