Detail the kvm virtual machine online migration based on local storage

There are four types of kvm virtual machine migration

(1) hot migration is based on Shared storage

(2) hot migration is based on local storage

(3) cold migration is based on Shared storage

(4) cold migration is based on local storage

This is about hot migration based on local storage

Live block migration version requirements

qemu version requires greater than or equal to 0.12.1 (no problem with centos6.7 or above)

rpm -qa|grep qemu
qemu-kvm-0.12.1.2-2.491.el6_8.1.x86_64
qemu-kvm-tools-0.12.1.2-2.491.el6_8.1.x86_64

Target host: 192.168.1.24 machine name: CQ-KVM01

Source host: 192.168.1.26 machine name: CQ-KVM02

Migration USES tcp authentication

kvm virtual machine name, is a virtual machine running nginx: CQ-NGINX01

1. Configure the libvirtd service source host and target host

vi /etc/libvirt/libvirtd.conf
listen_tls = 0  
listen_tcp = 1
tcp_port = "16509"
listen_addr = "0.0.0.0"
auth_tcp = "none"
mdns_adv = 0

explain

listen_tls: turn off tls, which is turned on by default listen_tcp: enable tcp authentication tcp_port: tcp port configuration, default port is 16509 auth_tcp: configure whether password authentication is enabled. It is not enabled by default mdns_adv: turn mdns multicast on or off by default

sed -i 's/^#listen_tls = 0/listen_tls = 0/' /etc/libvirt/libvirtd.conf
sed -i 's/^#listen_tcp = 1/listen_tcp = 1/' /etc/libvirt/libvirtd.conf
sed -i 's/^#tcp_port = \"16509\"/tcp_port = \"16509\"/' /etc/libvirt/libvirtd.conf
sed -i 's/^#listen_addr = \"192.168.0.1\"/listen_addr = \"0.0.0.0\"/' /etc/libvirt/libvirtd.conf
sed -i 's/^#auth_tcp = \"sasl\"/auth_tcp = \"none\"/' /etc/libvirt/libvirtd.conf
sed -i 's/^#mdns_adv = 1/mdns_adv = 0/' /etc/libvirt/libvirtd.conf

cat libvirtd.conf |grep -iE "listen_tls"
cat libvirtd.conf |grep -iE "listen_tcp"
cat libvirtd.conf |grep -iE "tcp_port"
cat libvirtd.conf |grep -iE "listen_addr"
cat libvirtd.conf |grep -iE "auth_tcp"
cat libvirtd.conf |grep -iE "mdns_adv"

2. Configuration of libvirtd listening mode, whether it is tcp, ssh or tls authentication mode, shall be configured, and both the source host and the target host shall be executed

vi /etc/sysconfig/libvirtd
LIBVIRTD_ARGS="--listen"

3, edit/etc/libvirt/qemu conf file source host machine and the target host machine to perform

vi /etc/libvirt/qemu.conf
vnc_listen = "0.0.0.0"

explain

vnc_listen: makes VNC listen out of the virtual machine by default.0.0.0.0

4. Restart the libvirtd service so that the configuration takes effect, both the source host and the target host are executed

/etc/init.d/libvirtd restart

5. Check the port and process, whether libvirtd is listening to port 16509, and both the source host and the target host should be executed

netstat -lntp | grep libvirtd
tcp 0 0 0.0.0.0:16509 0.0.0.0:* LISTEN 48825/libvirtd

Modify the iptables script to release ports 16509 and 49152

#ACCESS kvm
iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 5900:5909 -j ACCEPT
iptables -A INPUT -p tcp --dport 16509 -j ACCEPT
iptables -A INPUT -p tcp --dport 49152 -j ACCEPT

6. Add IP and host resolution to the source and target hosts

#24 On the machine 
echo "192.168.1.26 CQ-KVM02" >> /etc/hosts
#26 On the machine 
echo "192.168.1.24 CQ-KVM01" >> /etc/hosts

7. Connect libvirtd of the target host in the source host and view the virtual machine information

virsh -c qemu+tcp://root@192.168.1.26:16509/system
virsh -c qemu+tcp://root@192.168.1.24:16509/system

8. Create a mirror file of the same size on the target host, with the same path and name

Add preallocation=metadata, otherwise the virtual machine disk image will change to raw after installation using the virt-install command

vi /etc/libvirt/libvirtd.conf
listen_tls = 0  
listen_tcp = 1
tcp_port = "16509"
listen_addr = "0.0.0.0"
auth_tcp = "none"
mdns_adv = 0

0

9, migration, 192.168.1.26 is the target host's Intranet IP, tcp authentication, migration time is a little long need to open an screen window, do not need to cd to the mirror file directory

screen -S migratekvm
virsh migrate --live --copy-storage-all --unsafe --persistent CQ-NGINX01 qemu+tcp://root@192.168.1.26:16509/system

10. When executed on the target host, the paused state will be displayed, that is, the suspended state

watch virsh list --all

11. Execute on the target host and display the migration progress

vi /etc/libvirt/libvirtd.conf
listen_tls = 0  
listen_tcp = 1
tcp_port = "16509"
listen_addr = "0.0.0.0"
auth_tcp = "none"
mdns_adv = 0

3

12. When executed in the target host, the directory of image file is /data/kvmimg/, which shows the size of the virtual machine image. You will see that the virtual machine image will keep growing, indicating that the data has been migrated

Note that preallocation=metadata is used when creating the image file

vi /etc/libvirt/libvirtd.conf
listen_tls = 0  
listen_tcp = 1
tcp_port = "16509"
listen_addr = "0.0.0.0"
auth_tcp = "none"
mdns_adv = 0

4

13. After the virtual machine migration is completed, the source host is in shutdown state, and the target host is in running state

vi /etc/libvirt/libvirtd.conf
listen_tls = 0  
listen_tcp = 1
tcp_port = "16509"
listen_addr = "0.0.0.0"
auth_tcp = "none"
mdns_adv = 0

5

14. Delete the xml configuration file by the source host

vi /etc/libvirt/libvirtd.conf
listen_tls = 0  
listen_tcp = 1
tcp_port = "16509"
listen_addr = "0.0.0.0"
auth_tcp = "none"
mdns_adv = 0

6

Note:

During the migration, ping can get stuck and sometimes lose several packages. If there is a deployment database in the virtual machine, it is better for the application to have a reconnection mechanism

A property method

It is recommended not to delete the image files on the source host, but to keep the image files

Execute on the target host and move back to the source host

virsh dumpxml CQ-NGINX01 >~/CQ-NGINX01.xml
virsh migrate --live --copy-storage-all --unsafe --persistent CQ-NGINX01 qemu+tcp://root@192.168.1.24:16509/system

Note:

The source host and the target host had better keep the hardware 1, so that the probability of migration and migration problems can be minimized!

Migration speed and success rate have a lot to do with the network, the host's network is as far as possible ten thousand or one gigabit network card binding, can improve the migration speed and success rate

# Limiting migration speed   for 600M/s
virsh migrate-setspeed CQ-NGINX01 600

--------------------------------------------------------------------------------

Online migration problem summary

(if the port is not the default ssh 22, can write qemu + ssh: / / target: port/system)

Question 1:

error: internal error hostname on destination resolved to localhost, but migration requires an FQDN

The solution is very simple, because hostname is localhost by default, we need to modify the target hostname by 1, or temporarily.

vi /etc/libvirt/libvirtd.conf
listen_tls = 0  
listen_tcp = 1
tcp_port = "16509"
listen_addr = "0.0.0.0"
auth_tcp = "none"
mdns_adv = 0

9

Question 2:
error: internal error Attempt to migrate guest to the same host 00020003-0004-0005-0006-000700080009

This is a server provider problem in mail list. You can check this ID using # dmidecode-s system-uuid, which is theoretically #virsh sysinfo | grep uuid and # virsh capabilities | grep uuid

The three ID are of one type. (the solution I saw in the forum before was to download a file of the dmidecode command to replace the system, so as to achieve the purpose of modification. However, I just modified the configuration file of libvirt, which may be the version problem.)

Let's modify the configuration file of libvirtd

#sed -i "/#host_uuid/ahost_uuid = \"`uuidgen`\"" /etc/libvirt/libvirtd.conf

Restart the libvirtd service 1 time

# /etc/init.d/libvirtd restart

And then look at 1 again

# virsh capabilities | grep uuid

Question 3:

error: unable to set user and group to '107:107' on '/image/vmtest/disk.img': No such file or directory

If you get into this step, it's easy to create one without directory, and one without disk.img

Note: the disk created should be the same size as the disk of the source virtual machine (larger should be ok) and format (raw or qcow2).

qemu-img create /image/vmtest/disk.img +sizeG

Question 4:

error: Unable to resolve address 'target' service '49155': Name or service not known

"Name or service not known" in /etc/hosts add the resolved ip and target domain names

Question 5:

virsh migrate --live --copy-storage-all --unsafe --persistent CQ-NGINX01 qemu+tcp://root@192.168.1.24:16509/system

error: Unable to read from monitor: Connection reset by peer

The target host and source host may not have enough memory

Recommendation: close some unnecessary virtual machines on both the target host and the source host to free up memory

You can also clear cache to free up memory

Clear cache first execute the sync command

sync
echo 3 > /proc/sys/vm/drop_caches