Virt < CSLab

CSLab Web>WebGuestLeftBar>Virt (2010-09-06, AnastasiosNanos) (raw view)
---+Virtualization

---++++Intro

Virtualization is the art of subdividing resources provided by modern computers
in order to achieve maximum performance, isolated execution, maximum
utilization, resource managed hardware sharing and so on. Virtualization is not
a novel concept. Mainframes designed in the late '70s incorporated the idea of
isolating hardware design and hardware-agnostic software execution.
Virtualization exists in all aspects of modern high performance computing. We
try to experiment on Virtualized I/O, especially in Network Device
Virtualization.

---++++Storage and Network I/O


We believe that modern High Performance Interconnection Networks provide
abstractions that can be exploited in Virtual Machine execution environments
but lack support in sharing architectures. Previous work has shown that
integrating the semantics of Virtualization in specialized software that runs
on Network Processors can isolate and finally minimize the overhead on the VM
Hypervisor concerning access to the device by Guest VMs. Direct I/O has been
proposed as the solution to the CPU overhead imposed by guest VM transparent
services that can lead to low throughput for high bandwidth links. However, 
minimizing CPU overhead comes at the cost of giving away the benefits of the device driver model.
Integrating protocol offload support (present in most modern NICs) in virtual network device drivers 
can lead to performance improvement. Bypassing the Hypervisor in data movement 
can also minimize the overhead imposed by heavy I/O but at the cost of safety and memory 
protection.

We envision a Virtualization-enabled High performance Network Interface that can
achieve line-rate throughput and optimized sharing of Network I/O in Virtual
Machines by utilizing commodity hardware and innovative resource-sharing
virtualization architectures.

---++++High-performance I/O in Virtualized Environments

Device access in Virtualization environments is often realized by specific
layers within the hypervisor which allow VMs to interface with the hardware. A
common practice for such an interface is a _split driver model_. These
specific layers host a _backend_ driver while guest VM kernels host a
_frontend_ driver exposing a generic device API to guest kernels or
user-space. The backend exports a communication mechanism to the frontend along
with interrupt routing, page flipping and shared memory techniques. 

On the other hand, intelligent interconnects provide NICs that offload protocol
processing and achieve fast message exchange, suitable for HPC applications.
These NICs feature specific hardware such as DMA engines, volatile memory, I/O
or network processors and an interface to the physical medium. To avoid the
overhead associated with user-space -- kernel-space communication, HPC
interconnects often utilize a user-level networking approach. Thus, the NIC can
export a virtual instance of a network interface directly to an
application. Our work is focused on integrating these semantics into the VMM
split driver model. 

To evaluate our framework we experiment on optimizing the data exchange path
between an application running in a Xen VM and a 10Gbps
interface. To provide intelligence to the network controller we choose to apply
our approach on a Myrinet Myri-10G NIC and a custom 10GbE
interface which consists of an I/O processor, a number of DMA engines and a
commodity 10GbE NIC.

The split driver model poses difficulties for user-level direct NIC
access in VMs. To enable VMM-bypass techniques, we need to let
VMs have direct access to certain NIC resources. The building block of our
framework is the _backend_ which allows the _frontend_ to communicate
with the NIC's core driver. The frontend driver communicates with the backend via
an event channel mechanism. Contrary to Xen's netfront / netback architecture,
our framework utilizes the backend in conjunction with the NIC's core driver to grant
pages to the VM user space and install mappings that can simulate the normal
case while the netfront driver uses these channels as a data path (to send or
receive packets).

---++++MyriXen (ongoing)

Data access in HPC infrastructures is realized via user-level
networking and OS-bypass techniques through which nodes can communicate
with high bandwidth and low-latency. Virtualizing physical
components requires hardware-aided software hypervisors to control I/O
device access. As a result, line-rate bandwidth or lower latency message
exchange over 10GbE interconnects hosted in Cloud Computing infrastructures
can only be achieved by alleviating software overheads imposed
by the Virtualization abstraction layers, namely the VMM and the driver
domains which hold direct access to I/O devices.
We have designed MyriXen, a framework in which Virtual Machines
efficiently share network I/O devices bypassing overheads imposed
by the VMM or the driver domains. MyriXen permits VMs to optimally
exchange messages with the network via a high performance NIC,
leaving security and isolation issues to the Virtualization layers. Smart
Myri-10G NICs provide hardware abstractions that facilitate the integration
of the MX semantics in the Xen split driver model. With MyriXen,
multiple VMs exchange messages using the MX message passing protocol
over Myri-10G interfaces as if the NIC was assigned solely to them. We
believe that MyriXen can integrate message passing based application
in clusters of VMs provided by Cloud Computing infrastructures with
near-native performance.

---++++Summary

Our framework allows VMs to share an HPC NIC efficiently and exchange messages
with the network. It is a thin split driver layer running on top of
the NIC's core driver and consists of the backend driver in the driver domain
and the frontend drivers in the VMs. Our current agenda consists of evaluating
our prototype in order to estimate our framework's efficiency. In the future, we
plan to experiment on fine-tuning the NIC's intelligence and propose a
high-performance interconnection architecture for Virtualized environments based
on commodity components.

See the [[http://www.cslab.ece.ntua.gr/bibportal/index.php/topics/single/2][Virtualization Section]] in our [[http://www.cslab.ece.ntua.gr/bibportal][Bibliography Portal]] for selected publications concerning Virtualization techniques.

---+++Publications
   *  A. Nanos, G. Goumas and N. Koziris, &ldquo;[[http://www.cslab.ece.ntua.gr/~ananos/papers/mpi_vm-camera.pdf][Exploring I/O Virtualization Data paths for MPI Applications in a Cluster of VMs: A Networking Perspective]]&rdquo;, Proceedings of the 5th Workshop on Virtualization in High-Performance Cloud Computing, held in conjunction with Euro-par 2010, Ischia - Naples, Italy, 30 Aug. - 3 Sep., 2010 [ [[http://www.cslab.ece.ntua.gr/~ananos/papers/mpi_vm-camera.pdf][pdf]] ] [ [[http://www.cslab.ece.ntua.gr/~ananos/papers/mpi_vm-slides.pdf][slides]] ]
   *  A. Nanos and N. Koziris, &ldquo;[[http://www.cslab.ece.ntua.gr/~ananos/papers/myrixen-camera.pdf][MyriXen: Message Passing in Xen Virtual Machines over Myrinet and Ethernet]]&rdquo;, Proceedings of the 4th Workshop on Virtualization in High-Performance Cloud computing, held in conjunction with Euro-par 2009, Delft, The Netherlands, 24-28 August, 2009 [ [[http://www.cslab.ece.ntua.gr/~ananos/papers/myrixen-camera.pdf][pdf]] ] [ [[http://www.cslab.ece.ntua.gr/~ananos/papers/myrixen-slides.pdf][slides]] ]
Topic revision: r7 - 2010-09-06 - AnastasiosNanos
CSLab
Research

Courses

CSLab

Home
People
No permission to view TWiki.WebLeftBarWebsList
No permission to view TWiki.WebTopBar
No permission to view TWiki.WebBottomBar