Virtlet: Run VMs As Kubernetes Pods - Mirantis

CancelOkCancelOk

Mirantis Selected as Software Infrastructure Partner for NVIDIA AI Factory for Government   |  Learn More

Contact UsDocsStorePartnersTrainingPublic SectorPlatform

k0rdent AI: Provision multi-tenanted AI-ready infrastructure and core services

WHY k0RDENT

VIDEO: Run Anywhere. Automate Everything. k0rdent in 30 seconds.

Solutions

FROM METAL-TO-MODEL™

IaaS

Unifed infrastructure automation

• Multi-Cluster

• Hybrid Multicloud

• Workload Migration

• Virtualization

• Private Clouds

GPU PaaS

GPU provisioning & partitioning

• AI Cloud

• Sovereign Cloud

• Enterprise AI Factory

AI PaaS

Training, inference & self-service AI

• Turnkey Training

• Turnkey Inference

• Self-Service Portals

AI Services

Expert support & services

• MCP AdaptiveOps

ServicesModern Application DeliveryApp ModernizationCloud Platform OperationsAI Inferencing PlatformWorkload MigrationMCP AdaptiveOpsEnterprise SupportResources

KNOWLEDGE BASE

Resource LibraryBlogCase StudiesCloud Native CookbookGuided DemosTech TalksVideosWebinars

COMPANY

Why MirantisAboutPress ReleasesCareersLeadershipEventsGET STARTEDPlatformWhy k0rdent

VIDEO: Run Anywhere. Automate Everything. k0rdent in 30 seconds.

Nebul: Delivering Sovereign AI Clouds for European Enterprises

Discover how a neocloud uses Mirantis k0rdent AI to achieve "shared nothing" security without the pain of Kubernetes sprawl.

SolutionsIaaS— Multi-Cluster— Hybrid Multicloud— Workload Migration— Virtualization— Private CloudsGPU PaaS— AI cloud— Sovereign Cloud— Enterprise AI FactoryAI PaaS— Turnkey Training— Turnkey Inference— Self-Service PortalsAI Services— MCP AdaptiveOpsServicesModern Application DeliveryApp ModernizationCloud Platform OperationsAI Inferencing PlatformWorkload MigrationMCP AdaptiveOpsEnterprise SupportResourcesKNOWLEDGE BASEResource LibraryBlogCase StudiesCloud Native CookbookGuided DemosTech TalksVideosWebinarsCOMPANYWhy MirantisAboutPress ReleasesCareersLeadershipEventsStorePartnersTrainingPublic SectorGet StartedContact Us< BLOG HOMEVirtlet: run VMs as Kubernetes podsIvan Shvedunov - January 01, 2011- k8s, NFV, Tutorial, virtletNot long ago the Virtlet project reached an important milestone: its 1.0 stable release. In this post, we'll describe why Virtlet is important for the Kubernetes ecosystem and how to use it.

Why Virtlet

Virtlet makes it possible to run virtual machines on Kubernetes clusters as if they were plain pods, enabling you to use standard kubectl commands to manage them, bringing them onto the cluster network as first class citizens, and making it possible to build higher-level Kubernetes objects such as Deployments, StatefulSets or DaemonSets composed of them. Virtlet achieves this by implementing the Container Runtime Interface (CRI).

Use cases

An obvious use case for VMs are legacy applications that can't run in containers for one reason or another, or that require extra privileges in order to be containerized. These legacy applications are an important use case that will continue to be important for years to come, but other important uses also exist, such as:

1. NFV: Many VNFs can't be easily containerized because it's often undesirable and unsafe for them to share the kernel with the host system.
2. Non-Linux systems: You may need, for example, a Microsoft Windows environment for some of your CI tasks that are used for Windows desktop apps. True, Windows Containers do exist, but maybe you want to have a uniform Linux- and Kubernetes-based infrastructure or you don't have people with enough Windows knowledge. You might also need to run a test environment for an app that runs on some special purpose OS.
3. Unikernel applications such as such as OSv, Mirage or Rump kernels: One exciting example of this work is the MIKELANGELO project, for which the ability of Virtlet to perform actions such as supporting Deployments of VMs is quite important.
4. Isolation: Sure, there are now solutions that enable you to run container images using VMs, thus providing all the isolation you need in most cases, but sometimes you may want to have a "real OS" and not just a container. Imagine, for example, that you have one big bare metal Kubernetes cluster, but you need to test your Kubernetes-based CI/CD and deployment system. Unfortunately, some of the tests can be rather disruptive, with the ability to affect your cluster's control plane in an undesired way. (See Kubernetes in Kubernetes example for more info.)

An important point to make is that while Virtlet was initially targeted at bare metal Kubernetes installations, GCP and Azure now support nested virtualization, and AWS is providing bare metal instances such as i3.metal, making it possible to use Virtlet on the public cloud, as well.

Virtlet features

Virtlet's approach to running VMs on Kubernetes clusters strives to make it possible to use VMs as if they were plain pods. This includes support for common kubectl commands, such as create, apply, get, delete, logs, attach and port-forward, with exec likely to be implemented in future. The VM pods join the cluster network, getting an IP address from the pod subnet. Moreover, it's possible to create Kubernetes services that point at VM pods. VM pods can also make use of TCP and HTTP readiness and liveness probes. In addition, Virtlet honors CPU and Memory resource limits specified for VM pods.Handling mounts for VM pods differs to some extent from how it's handled for actual containers. Virtlet may gain support for all kinds of Kubernetes volumes over time when 9p support is implemented, but it already supports specifying ConfigMap and Secret mounts, which are actually copied into the VMs using the Cloud-Init mechanism. It's also possible to use Virtlet's flexvolume driver to specify mounting of local block devices, "ephemeral volumes" with their lifetime bound to the one of the pod, and Ceph volumes that are specified as block devices.Virtlet makes extensive use of the Cloud-Init mechanism. For example, it's used to inject ssh keys, create users, run specific commands on VM startup, and pass the network configuration in situations where it's too complex to handle using the standard Virtlet networking based on an internal DHCP server.In order to avoid having a separate complex deployment procedure for the nodes that run VMs, Virtlet makes use of the CRI proxy, so you can run both VM pods and plain Kubernetes pods on the same node. You can also deploy Virtlet itself as a DaemonSet.

Let's try Virtlet: setting up

The easiest way to try Virtlet is to use the demo script, which makes use of a kubeadm-dind-cluster, a tool that makes it possible to run Kubernetes test clusters using just Docker.To make things easier, we'll be needing virtletctl binary that facilitates some of the VM-related tasks. You can get it from Virtlet release page, as follows:

Linux:

sudo wget -O /usr/local/bin/virtletctl https://github.com/Mirantis/virtlet/releases/download/v1.0.0/virtletctl echo '4a0efdfe339f6fb00525bc53428415177bdd5f2391774d60ec1c449a99990461 /usr/local/bin/virtletctl' |  sha256sum -c && chmod +x /usr/local/bin/virtletctl

Mac OS X:

sudo wget -O /usr/local/bin/virtletctl https://github.com/Mirantis/virtlet/releases/download/v1.0.0/virtletctl.darwin echo '8265312a5d9ffe0e8ce1ff66fde187ad025d1ebd780fb500f54512b7f0738bd3 /usr/local/bin/virtletctl' |  sha256sum -c && sudo chmod +x /usr/local/bin/virtletctlNow we need to download the demo script and run it:NOTE: if you're already using kubeadm-dind-cluster, this command will erase and replace your existing test cluster.wget https://raw.githubusercontent.com/Mirantis/virtlet/v1.0.0/deploy/demo.sh chmod +x demo.sh ./demo.shAnswer y to the script's questions and wait until the script completes. The script will create a CirrOS VM for you and display its shell prompt:Successfully established ssh connection. Press Ctrl-D to disconnect. $ Now let's test it out.

Testing the Virtlet installation

Let's make sure the VM has network connectivity and can access Kubernetes cluster services. For example, the demo script also creates an nginx sevice:$ ping -c1 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: seq=0 ttl=58 time=4.134 ms --- 8.8.8.8 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max = 4.134/4.134/4.134 ms $ curl -s http://nginx | head -4 Welcome to nginx!Cool! Let's press Ctrl-D to disconnect from the VM and see what it looks like from the Kubernetes perspective.$ kubectl get pods NAME                     READY STATUS RESTARTS   AGE cirros-vm                1/1 Running 0      11m nginx-7587c6fdb6-crb7z   1/1 Running 0      11m The pod labled cirros-vm pod is the VM. We can look at its boot logs via kubectl logs:$ kubectl logs cirros-vm... /dev/root resized successfully [took 0.01s] login as 'cirros' user. default password: 'gocubsgo'. use 'sudo' for root. cirros-vm login: [  734.232878] random: crng init doneOr we can attach to its serial console (press Ctrl-] to detach):$ kubectl attach -it cirros-vm If you don't see a command prompt, try pressing enter. login as 'cirros' user. default password: 'gocubsgo'. use 'sudo' for root. cirros-vm login: cirros Password: $You can delete the VM pod when you no longer need it:$ kubectl delete pod cirros-vm Very good. Now let's try something more interesting.

Running Windows on a Linux Kubernetes cluster using Virtlet

Let's try running a Windows VM. You can obtain the necessary images, for example, from cloudbase. Unfortunately you can't point Virtlet directly at them because of licensing restrictions, so we'll need a few extra tricks. Let's assume the image is available as http://192.168.0.2:8000/windows.qcow2 (You can achieve this using python3 -m http.server, for example.)First, Virtlet uses the pod spec's image field to specify the image. This field must follow the conventions for container image names, so we can't just put any image URL in it. To solve this problem, we'll need to create a Virtlet image name translation object for it:$ cat >winimage.yaml