gdb_command.txt is the flip commands that will execute in gdb, it is generated by gdb.py
gdb.py use iomem.txt to generates gdb_command.txt and simulate the bits flip by running gdb.sh in a while loop.
get_iomem.sh run in qemu guest machine, it will generates iomem.txt
iomem.txt is the /proc/iomem in qemu guest machine. It is generated by the execution of get_iomem.sh
gdb.txt will be created after the execution of gdb.py. It is the gdb logging file.
gdb.sh will be execute by gdb.py using python3 subprocess.run()
gdb/fliputils.py provides some user-defined commands for gdb.
Simulate the random bits flip in qemu guest machine with gdb.
Run git clone https://github.com/yexuanyang/flip_simulation.git or git clone [email protected]:yexuanyang/flip_simulation.git
Store the repo somewhere.
Run these commands in bash to install QEMU from the source. QEMU source tar is obtained from its official website.
wget https://download.qemu.org/qemu-7.1.0.tar.xz –no-check-certificate
xz -d qemu-7.1.0.tar.xz
tar -xvf qemu-7.1.0.tar
cd qemu-7.1.0/
apt-get install -y gcc make ninja-build libglib2.0-dev libmount-dev meson git libfdt-dev libpixman-1-dev zlib1g-dev libcap-ng-dev libattr1-dev
mkdir build
cd build
../configure --enable-kvm --enable-virtfs
make -j$(nproc)
make install
qemu-system-aarch64 --versionEnter make LLVM=1 menuconfig in the root directory of the kernel source. enter / to search these configs and open them.
CONFIG_9P_FS=y
CONFIG_9P_FS_POSIX_ACL=y
CONFIG_9P_FS_SECURITY=y
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NET_9P=y
CONFIG_NET_9P_DEBUG=y （Optional）
CONFIG_NET_9P_VIRTIO=y
For aarch64, also add:
CONFIG_PCI=y
CONFIG_PCI_HOST_COMMON=y
CONFIG_PCI_HOST_GENERIC=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO_NET=y
Add -virtfs local,path=/path/to/share,mount_tag=host0,security_model=passthrough,id=host0 or -fsdev local,security_model=passthrough,id=fsdev0,path=<path-to-shared-dir-in-host> -device virtio-9p-pci,id=fs0,fsdev=fsdev0,mount_tag=hostshare
after your qemu boot command to use 9p fs. I recommend the first way.
Then interacting with the qemu guest machine tty console, run mount -t 9p -o trans=virtio,version=9p2000.L host0 /mnt/shared to mount 9p fs in directory /mnt/shared.
You can also add a new line host0 /mnt/shared 9p trans=virtio,version=9p2000.L 0 0 at the end of /etc/fstab to make the qemu mount 9p automatically.
Add -s -S at the end of your boot command will start a gdb server at localhost:1234 and wait for gdb to continue. Run gdb -ex 'target remote:1234' to attach the server. Enter continue will make the kernel boot.
Append -monitor unix:/tmp/qemu_socket,server,nowait to your boot command, it will start a qemu monitor which can be connected by socket file /tmp/qemu_socket using socat -,echo=0,icanon=0 /tmp/qemu_socket or nc -U /tmp/qemu_socket. And you can send a single command to qemu monitor using echo "info status" | socat - /tmp/qemu_socket.
Note: The socket file /tmp/qemu_socket will be created automatically, you should make sure the filename is unique. Do not change this filename, the script will use this filename to connect to qemu monitor.
Qemu support many format disk image, like raw, qcow2, etc. Because we use loadvm and savevm in qemu monitor, so we need a qcow2 format disk image.
Basically, you can convert a raw ext4 image to qcow2 image by qemu-img convert -f raw -O qcow2 <source filename>.img <dest filename>.qcow2. Note that the boot command maybe change when you use a different format disk image.
Reference:
- qemu.org QEMU disk image utility: qemu-img convert
- openstack qemu-img convert: raw, qcow2, qed, vdi, vmdk, vhd
Just cp or mv the code you cloned before to the shared directory.
Firstly run get_iomem.sh in qemu guest machine to get iomem.txt
Secondly run python3 gdb.py in the host machine to simulate the bits flip.
Thirdly run python3 snap.py in the host machine to tests the failure rate of a machine after a single particle flip occurs and the machine runs for a period of time.
Note: Detach gdb server before python3 gdb.py, otherwise it will blocked because 1234 port is used.
Note: You can modify gdb.py to change the flip area, the flip bits, the flip times, etc.
Firstly run get_iomem.sh in qemu guest machine to get iomem.txt
Secondly run gdb in the host machine and run source gdb/fliputils.py in gdb to load the user-defined commands.
Thirdly use autoinject or snapinject in gdb to simulate the bits flip.
Note: More details about the user-defined commands can be found in gdb/fliputils.py.
In QEMU, run echo 8 > /proc/sys/kernel/printk to get all message in stdout.
Before start QEMU, add | tee <some-file> at the end of command to store all content that showed in stdout and that you put in stdin in QEMU at file <some-file>.
Currently, the forms of simulated flipping include external flipping using scripts and built-in flipping mode using user-defined commands in gdb. The granularity of the interval between two flippings of external flipping can only reach the second level, while the granularity of the interval between two flippings of built-in flipping can reach the millisecond level. Because built-in flipping is implemented using gdb python api and QEMU's built-in timer, the only error is that after the QEMU built-in timer times out, the callback function needs to be executed, and then QEMU returns to the paused state, which takes about 50us. Because external flipping uses scripts to execute gdb to connect to QEMU's gdbserver or uses gdbcontroller in the pygdbmi library to connect to gdbserver to execute the corresponding commands, the delay of the two operations is tens or hundreds of milliseconds. However, the external method takes less time to complete the flip than the built-in method. For example, it takes more than 20s to execute autoinject 10 20ns 100ns ram in GDB, but it only takes more than 10s to use the external method. If you need to specify a fine-grained flipping interval, it is recommended to use the GDB command method. If this is not necessary, it is recommended to use a faster external reversal method.