QEMU Build ARM LINUX Development Environment

Introduction to QEMU:

QEMU is a software-only virtualization simulator that can simulate almost all hardware, including the ARM A9 platform that we will use this time.It works by converting guest schema code to TCG intermediate code and then to host schema code.

Environmental Science:

Virtual Machine: ubuntu18.04 LTS


  1. Install gcc-arm-linux-gnueabi
sudo apt install gcc-arm-linux-gnueabi

Once the installation is complete, we can look at the target architecture supported by the tool chain:

arm-linux-gnueabi-gcc --target

Looking over the results, you can see that:

  Known ARM CPUs (for use with the -mcpu= and -mtune= options):
    arm1020e arm1020t arm1022e arm1026ej-s arm10e arm10tdmi arm1136j-s
    arm1136jf-s arm1156t2-s arm1156t2f-s arm1176jz-s arm1176jzf-s arm2 arm250
    arm3 arm6 arm60 arm600 arm610 arm620 arm7 arm70 arm700 arm700i arm710
    arm7100 arm710c arm710t arm720 arm720t arm740t arm7500 arm7500fe arm7d
    arm7di arm7dm arm7dmi arm7m arm7tdmi arm7tdmi-s arm8 arm810 arm9 arm920
    arm920t arm922t arm926ej-s arm940t arm946e-s arm966e-s arm968e-s arm9e
    arm9tdmi cortex-a12 cortex-a15 cortex-a15.cortex-a7 cortex-a17
    cortex-a17.cortex-a7 cortex-a32 cortex-a35 cortex-a5 cortex-a53 cortex-a57
    cortex-a57.cortex-a53 cortex-a7 cortex-a72 cortex-a72.cortex-a53 cortex-a73
    cortex-a73.cortex-a35 cortex-a73.cortex-a53 cortex-a8 cortex-a9 cortex-m0
    cortex-m0.small-multiply cortex-m0plus cortex-m0plus.small-multiply
    cortex-m1 cortex-m1.small-multiply cortex-m23 cortex-m3 cortex-m33 cortex-m4
    cortex-m7 cortex-r4 cortex-r4f cortex-r5 cortex-r7 cortex-r8 ep9312
    exynos-m1 fa526 fa606te fa626 fa626te fa726te fmp626 generic-armv7-a iwmmxt
    iwmmxt2 marvell-pj4 mpcore mpcorenovfp native strongarm strongarm110
    strongarm1100 strongarm1110 xgene1 xscale

The arch and CPU support is complete and will support cortex-a9, which we are going to emulate this time; even cortex-m support will allow you to compile the cores and applications of MMU-free single-chip machines.

  1. Install qemu Install quickly as follows, or use source code to install.
sudo apt install qemu
  1. Download Kernel and busybox Source
    Download main line source: http://cdn.kernel.org/pub/linux/kernel/v4.x
    Download busybox: https://busybox.net/downloads/busybox-1.31.1.tar.bz2
  2. Compile busybox
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- menuconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- install

At this point, the'_intall'directory has been generated in the busybox directory, which will serve as the directory where we build the root file system and add something to the root file system directory.
Add the following directories:

  • etc: mainly store some configuration files such as inittab(init process parses this file for further action); fstab (mainly including some mounted file systems, such as sys proc) init.rd/rcS (can store some executable scripts for use with inittab)
  • Proc: proc file system mount point
  • Sys:sys file system mount point
  • Tmp:tmp file system mount point
  • dev: device file
  • lib: library file directory (if busybox uses dynamic link libraries, you need to copy the cross-compile chain library files here)

A. Statically create the following nodes in the dev directory:

sudo mknod -m 666 tty1 c 4 1
sudo mknod -m 666 tty2 c 4 2
sudo mknod -m 666 tty3 c 4 3
sudo mknod -m 666 tty4 c 4 4
sudo mknod -m 666 console c 5 1
sudo mknod -m 666 null c 1 3

console and null are required, and if not, errors will be reported.
b. etc/inittab file is as follows, written with reference to busyboxdir/examples/inittab:

::shutdown:/sbin/swapoff -a
::shutdown:/bin/umount -a -r

The c. etc/fstab file contains the following contents, the main purpose of which is to specify some file system mount points:

#device mount-point type option dump fsck order
proc  /proc proc  defaults 0 0
temps /tmp  rpoc  defaults 0 0
none  /tmp  ramfs defaults 0 0
sysfs /sys  sysfs defaults 0 0
mdev  /dev  ramfs defaults 0 0

The d. etc/init.d/rcS file contains the following, and the first item of inittab specifies that scripts are executed from rcS

mount -a
echo "/sbin/mdev" > /proc/sys/kernel/hotplug
/sbin/mdev -s       # Generate device nodes based on configuration in/etc/mdev.conf
mount -a

Incidentally modify permissions for rcS:

chmod 777 etc/init.d/rcS

e. lib file copy because busybox uses the default dynamic link (recommended), which saves the root file system size because the application can also link the appropriate libraries.First view the library files that busybox depends on by either of the three commands below.

arm-linux-readelf -d busybox | grep NEEDED
arm--linux-objdump -x busybox | grep NEEDED
strings busybox | grep ^lib

Note: ld-linux.so.3 does not sometimes appear, and we must copy it as well. If we compile the application later, we will also look at the dependent libraries to make up for the missing library files in the root file system.

cp /usr/arm-linux-gnueabi/lib/ld-linux.so.3 _install/lib/
cp /usr/arm-linux-gnueabi/lib/libc.so.6 _install/lib/
cp /usr/arm-linux-gnueabi/lib/libm.so.6 _install/lib/
cp /usr/arm-linux-gnueabi/lib/libresolv.so.2 _install/lib/
  1. Compile Kernel
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- vexpress_defconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- menuconfig(option)

Copy the zImage dtb file for easy use. If we have developed some drivers, we need to test them. In addition to copying the ko file to the root file system, we should also execute the module installation command. Otherwise, the modules dependencies, parameters, symbols, and so on cannot be used.

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- modules_install INSTALL_MOD_PATH=busybox-dir/_install/

When execution is complete, the modules directory is created under the lib of the root file system.

  1. Create Root File System Mirror File
    Use the dd command to create a blank 32M (as appropriate) file:
dd if=/dev/zero of=rootfs.ext3 bs=1M count=32

Format the blank file into ext3 format (the kernel supports the file system by default, and configures the kernel if you need to use something else):

mkfs.ext3 rootfs.ext3

Mount the blank file in a directory:

mkdir fs
sudo mount -o loop rootfs.ext3 ./fs

Copy the root file system built by busybox to the mount point and uninstall it

sudo cp -rf busybox-dir/_install/* ./fs
sudo umount ./fs
  1. Run with qemu, verify.
    To make it easier not to type a sequence of commands every time, we can create a run.sh script that reads as follows:
qemu-system-arm \
	-M vexpress-a9 \
	-kernel ./zImage \
	-nographic \
	-m 512M \
	-smp 4 \
	-sd ./rootfs.ext3 \
	-dtb vexpress-v2p-ca9.dtb \
	-append "init=/linuxrc root=/dev/mmcblk0 rw rootwait earlyprintk console=ttyAMA0"

In the above script, -M specifies the target board, -kernels specifies the linux kernel image, -nographic s specifies no graphical display, -m 512M specifies the run memory size, -smp specifies 4 cores, -sd specifies an external SD card, the root tfs.ext3 mirror file inside the card, -dtb specifies the device tree file, -append specifies bootargs; init=/linuxrc in bootargs specifies that the init process isLinuxrc (busybox generated) under the root file system, root=/dev/mmcblk0 specifies that the root file system is an SD card, and console specifies ttyAMA0, the console.

Run result:

Booting Linux on physical CPU 0x0
Linux version 4.19.86 (yhangzzz@yhangzzz-virtual-machine) (gcc version 7.4.0 (Ubuntu/Linaro 7.4.0-1ubuntu1~18.04.1)) #2 SMP Sun Dec 1 13:35:37 CST 2019
CPU: ARMv7 Processor [410fc090] revision 0 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
OF: fdt: Machine model: V2P-CA9
Memory policy: Data cache writealloc
random: get_random_bytes called from start_kernel+0x98/0x474 with crng_init=0
percpu: Embedded 16 pages/cpu s32780 r8192 d24564 u65536
Built 1 zonelists, mobility grouping on.  Total pages: 130048
Kernel command line: init=/linuxrc root=/dev/mmcblk0 rw rootwait earlyprintk console=ttyAMA0
log_buf_len individual max cpu contribution: 4096 bytes
EXT4-fs (mmcblk0): mounting ext3 file system using the ext4 subsystem
random: fast init done
EXT4-fs (mmcblk0): recovery complete
EXT4-fs (mmcblk0): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext3 filesystem) on device 179:0.
Freeing unused kernel memory: 1024K
Run /linuxrc as init process
random: crng init done
mount: mounting temps on /tmp failed: No such device

Please press Enter to activate this console. 
/bin/sh: can't access tty; job control turned off
/ # ls
bin         etc         lib         lost+found  sbin        tmp
dev         home        linuxrc     proc        sys         usr
/ # 

In this paper, we use qemu to build arm-a9 virtualization platform for linux kernel development, driver development, root file system construction, application development, or uboot development.Although virtual, it is essentially consistent with the steps in actual development.

Tags: Linux sudo simulator Ubuntu

Posted on Sun, 01 Dec 2019 11:48:32 -0500 by kyin