Macvtap bridge for cloud hypvervisor

Current state:

The right way to run VM with macvtap:

sudo su
./target/release/cloud-hypervisor --kernel ../linux-cloud-hypervisor/arch/arm64/boot/Image --disk path=../focal-server-cloudimg-arm64.raw,iommu=on --disk path=/tmp/ubuntu-cloudinit.img --cmdline 'console=hvc0 root=/dev/vda1 rw' --cpus boot=4 --memory size=0,shared=on --memory-zone id=mem0,size=1G,shared=on,host_numa_node=0 --api-socket /tmp/cloud-hypervisor.sock --net fd=3,mac=c2:67:4f:53:29:cb 3<>/dev/tap9

or

sudo ./target/release/cloud-hypervisor \
        --kernel ../linux-cloud-hypervisor/arch/arm64/boot/Image  \
        --disk path=$ROOTFS,iommu=on --disk path=/tmp/ubuntu-cloudinit.img \
        --cmdline "console=hvc0 root=/dev/vda1 rw" \
        --cpus boot=4   \
        --memory size=0,shared=on  \
        --memory-zone id=mem0,size=1G,shared=on,host_numa_node=0 \
        --api-socket /tmp/cloud-hypervisor.sock \
        3<>$"$tapdevice"

On VM:

ip addr
sudo dhclient ens4p0

Previous state:

We can't use --net fd=3,mac=$mac 3<>$"$tapdevice" in the shell script,it return:

Error booting VM: VmBoot(DeviceManager(CreateVirtioNet(TapError(IoctlError(2147767506, Os { code: 25, kind: Uncategorized, message: "Inappropriate ioctl for device" })))))

Use network bridge instead of macvtap to obtain effective network:

Reference https://github.com/cloud-hypervisor/cloud-hypervisor/issues/4917

On host:
sudo ip tuntap add mytap0 mode tap              #Create tap
sudo brctl addbr br0                            #Create bridge
sudo brctl addif br0 mytap0 enP9p3s0            #Connect tap and netcard with bridge
sudo ifconfig mytap0 0 up                       #Clear IP of tap
sudo ifconfig enP9p3s0 0 up                     #Clear IP of netcard
sudo dhclient br0                               #Get IP for bridge
sudo dhclient enP9p3s0                          #Get IP for netcard

Run cloud-hypervisor with --net "tap=mytap0"

On VM:
sudo dhclient enp0s5                            #You can find enp0s5 after run "ip addr"

We have tried something for macvtap:

Following the macvtap setup doc to setup macvtap, CLH report error.

Details in: https://github.com/cloud-hypervisor/cloud-hypervisor/discussions/5084

Following the advice from michael2012z ,we check a test case:

./scripts/dev_cli.sh tests --integration -- --test-filter test_macvtap_hotplug

dev_cli.sh calls docker:

docker run --workdir /cloud-hypervisor --rm --privileged --security-opt seccomp=unconfined --ipc=host --net=bridge --mount type=tmpfs,destination=/tmp --volume /dev:/dev --volume /home/dom/cloud-hypervisor/cloud-hypervisor:/cloud-hypervisor --volume /root/workloads:/root/workloads --env USER=root --env CH_LIBC=gnu cloudhypervisor/dev:20230116-0 ./scripts/run_integration_tests_aarch64.sh --hypervisor kvm --test-filter test_macvtap_hotplug

Then I run the following command on Arm64 server:

USER=root CH_LIBC=gnu ./scripts/run_integration_tests_aarch64.sh --hypervisor kvm --test-filter test_macvtap_hotplug 2>&1 | tee tlog

The two results were passed.

I add some log in tests/integration.rs .But they were not found in the log file.

So I run cargo test directly with cargo test common_parallel::test_macvtap_hotplug --target aarch64-unknown-linux-gnu -- --nocapture 2>&1 | tee tlog_cargo

command = sudo ip link delete guestmacvtap1
command = sudo ip link delete hostmacvtap1
command = sudo ip link add link enP9p3s0 name guestmacvtap1 type macvtap mod bridge
command = sudo ip link set guestmacvtap1 address 12:34:56:78:90:01 up
command = sudo ip link show guestmacvtap1
command = sudo chown $UID.$UID /dev/tap107
command = sudo ip link add link enP9p3s0 name hostmacvtap1 type macvtap mod bridge
command = sudo ip address add 192.168.1.1/24 dev hostmacvtap1
command = sudo ip link set dev hostmacvtap1 up
==== Start cloud-hypervisor command-line ====

"target/aarch64-unknown-linux-gnu/debug/cloud-hypervisor" "--cpus" "boot=2" "--memory" "size=512M" "--kernel" "/root/workloads/CLOUDHV_EFI.fd" "--cmdline" "root=/dev/vda1 console=hvc0 rw systemd.journald.forward_to_console=1" "--disk" "path=/tmp/chtLUHf9/osdisk.img" "--disk" "path=/tmp/chtLUHf9/cloudinit" "--api-socket" "/tmp/chtLUHf9/cloud-hypervisor.sock" "-v" "-v"

ch-remote /tmp/chtLUHf9/cloud-hypervisor.sock add-net fd=[3,4],mac=12:34:56:78:90:01,num_queues=4
guest:remote_command_w_output,{"id":"_net0","bdf":"0000:00:05.0"}
guest:wait_vm_boot
guest:ssh_command ip -o link
child kill
command = sudo ip link del guestmacvtap1
command = sudo ip link del hostmacvtap1

The test case use ch-remote to add network for the VM,but I want to know how to use --net fd=3,mac=$mac 3<>$"$tapdevice" to start VM.