6.3. Hotplugging disks
In this section we will hotplug a disk to a running virtual machine.
Task 6.3.1: Starting a virtual machine
First, create a file vm_lab06-cirros.yaml in the folder labs/lab06 and start with the following virtual machine definition:
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: lab06-cirros
spec:
runStrategy: Halted
template:
metadata:
labels:
kubevirt.io/domain: lab06-volume
spec:
domain:
devices:
disks:
- name: containerdisk
disk:
bus: virtio
- name: cloudinitdisk
disk:
bus: virtio
interfaces:
- name: default
masquerade: {}
resources:
requests:
memory: 64M
networks:
- name: default
pod: {}
volumes:
- name: containerdisk
containerDisk:
image: quay.io/kubevirt/cirros-container-disk-demo
- name: cloudinitdisk
cloudInitNoCloud:
userData: |
#cloud-config
Create and start the virtual machine with:
kubectl apply -f labs/lab06/vm_lab06-cirros.yaml --namespace lab-<username>
Start the VM with:
virtctl start lab06-cirros --namespace lab-<username>
Open a console with:
virtctl console lab06-cirros --namespace lab-<username>
Inspect block devices on the virtual machine before hotplugging a disk
Depending on the system you use you have different ways of listing devices. Here is a set of possibilities:
Listing block devices with lsblk:
lsblk -a
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vda 253:0 0 44M 0 disk
|-vda1 253:1 0 35M 0 part /
`-vda15 253:15 0 8M 0 part
vdb 253:16 0 1M 0 disk
loop0 7:0 0 0 loop
[...]
Listing devices in the /dev folder:
ls -d -- /dev/[sv]d[a-z]
/dev/vda /dev/vdb
Some operating systems also list disk devices in /dev/disk with subfolders representing different views to your disks.
On a, e.g., Fedora virtual machine:
ls -lR /dev/disk/*
/dev/disk/by-label:
cidata -> ../../vdb
/dev/disk/by-partuuid:
41222438-01 -> ../../vda1
/dev/disk/by-path:
pci-0000:07:00.0 -> ../../vda
pci-0000:07:00.0-part1 -> ../../vda1
pci-0000:08:00.0 -> ../../vdb
virtio-pci-0000:07:00.0 -> ../../vda
virtio-pci-0000:07:00.0-part1 -> ../../vda1
virtio-pci-0000:08:00.0 -> ../../vdb
/dev/disk/by-uuid:
2024-09-03-15-29-13-00 -> ../../vdb
d1b37ed4-3bbb-40b2-a6ba-f377f0c90217 -> ../../vda1
Devices starting with /dev/vd{a,b,c} are our devices using the virtio bus type. Devices with /dev/s{a,b,c,...} are
SCSI devices. What we see inside the virtual machine reflects our configuration.
Actually, we can see that we have two disks available:
/dev/vda- The first disk and therefore our container disk./dev/vdb- The second disk is the cloud-init disk. If your operating system provides the/dev/diskfolder you may see that the cloud-init disk is labeledcidata(see sample output). This is required for cloud-init to detect the disk as a provider for cloud-init configuration1.
Task 6.3.2: Create a DataVolume to be hotplugged
Create a file dv_lab06-hotplug-disk.yaml in the folder labs/lab06 with the following content:
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
name: lab06-hotplug-disk
spec:
source:
blank: {}
storage:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 128Mi
Create the data volume with:
kubectl apply -f labs/lab06/dv_lab06-hotplug-disk.yaml --namespace lab-<username>
Task 6.3.3: Hotplug a volume to a virtual machine
Hotplug the disk by using virtctl:
virtctl addvolume lab06-cirros --volume-name=lab06-hotplug-disk --namespace lab-<username>
Successfully submitted add volume request to VM lab06-cirros for volume lab06-hotplug-disk
After some time, the device will be hotplugged to your virtual machine. You may get a first hint where your new disk is plugged in by executing dmesg in the VM’s console:
dmesg
[ 78.043285] scsi 0:0:0:0: Direct-Access QEMU QEMU HARDDISK 2.5+ PQ: 0 ANSI: 5
[ 78.056318] sd 0:0:0:0: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatical
[ 78.060497] sd 0:0:0:0: [sda] 262144 512-byte logical blocks: (134 MB/128 MiB)
[ 78.070252] sd 0:0:0:0: [sda] Write Protect is off
[ 78.070767] sd 0:0:0:0: [sda] Mode Sense: 63 00 00 08
[ 78.073753] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 78.075719] sd 0:0:0:0: Attached scsi generic sg0 type 0
[ 78.113862] sd 0:0:0:0: [sda] Attached SCSI disk
You can verify this using the commands above. Check that your /dev/sda device is available.
Note
Disks are always hotplugged using the SCSI bus. This is due to the fact that virtio devices use a PCIe slot which are limited to 32 slots on a system. Further PCIe slots need to be reserved ahead of time.Check with list block devices:
lsblk -a
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 128M 0 disk
vda 253:0 0 44M 0 disk
|-vda1 253:1 0 35M 0 part /
`-vda15 253:15 0 8M 0 part
vdb 253:16 0 1M 0 disk
loop0 7:0 0 0 loop
[...]
Check listing the devices:
ls -d -- /dev/[sv]d[a-z]
/dev/sda /dev/vda /dev/vdb
In our case we can clearly see that our new hotplugged disk is /dev/sda. Let’s mount it.
Task 6.3.4: Format and mount the disk
From the previous steps we know our new disk is /dev/sda. To use it we first have to format the new disk.
Important
When hotplugging volumes and formatting devices always make sure you know where they are plugged in.sudo mkfs.ext4 /dev/sda
mke2fs 1.42.12 (29-Aug-2014)
Discarding device blocks: done
Creating filesystem with 131072 1k blocks and 32768 inodes
Filesystem UUID: 8cceeb09-cf4c-4605-896a-e2ecbe878894
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729
Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
Next we have to create mount points for our new disk:
sudo mkdir /mnt/disk
And finally mount the disk:
sudo mount /dev/sda /mnt/disk/
We can start to use the disk:
sudo touch /mnt/disk/myfile
Task 6.3.5: Removing a disk
You can remove a hotplugged disk with:
virtctl removevolume lab06-cirros --volume-name=lab06-hotplug-disk --namespace lab-<username>
Successfully submitted remove volume request to VM lab06-cirros for volume lab06-hotplug-disk
What happens if you remount the disk again? Is the created file /mnt/disk/myfile still present?
Task hint
After hotplugging and mounting the volume again the file is still present. There is no need to format the device again. However, the mounting needs to be done again. You may mount disks with startup scripts like cloud-init.
Beware that the device might have a different name such as sdb.
Mount the disk again with:
sudo mkdir /mnt/disk
sudo mount /dev/sdb /mnt/disk/
Check whether the file /mnt/disk/myfile still exists.
And don’t forget to remove the disk again:
virtctl removevolume lab06-cirros --volume-name=lab06-hotplug-disk --namespace lab-<username>
Persistent mounting
With the above steps we have hotplugged a disk into the VM. This mount is not persistent. Whenever the VM is restarted
or shut down, the disk won’t be attached again. When you want to mount the disk persistently you can use the --persist flag.
virtctl addvolume lab06-cirros --volume-name=lab06-hotplug-disk --persist --namespace lab-<username>
Successfully submitted add volume request to VM lab06-cirros for volume lab06-hotplug-disk
This will add the relevant sections to your VirtualMachine manifest. You can show the modified configuration with:
kubectl get vm lab06-cirros -o yaml --namespace lab-<username>
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: lab06-cirros
spec:
[...]
template:
[...]
spec:
domain:
devices:
disks:
- name: containerdisk
disk:
bus: virtio
- name: cloudinitdisk
disk:
bus: virtio
- name: lab06-hotplug-disk
disk:
bus: scsi
serial: lab06-hotplug-disk
volumes:
- name: containerdisk
containerDisk:
image: quay.io/kubevirt/cirros-container-disk-demo
- name: cloudinitdisk
cloudInitNoCloud:
userData: |
#cloud-config
- name: lab06-hotplug-disk
dataVolume:
hotpluggable: true
name: lab06-hotplug-disk
[...]