ARM Memory Types

Even ARM itself thinks Device type and Strongly Ordered are confusing.

In Programmer's Guide for ARMv8-A, it says:

13.1 Memory types

The ARMv8 architecture defines two mutually-exclusive memory types. All regions of memory are configured as one or the other of these two types, which are Normal and Device. A third memory type, Strongly Ordered, is part of the ARMv7 architecture. The differences between this type and Device memory are few and it is therefore now omitted in ARMv8.

ARM Linker Veneer

3.3.7. Veneer generation

Veneers are small sections of code generated by the linker and inserted into your program. armlink must generate veneers when a branch involves a destination beyond the branching range of the current state.

http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0285b/Cchhgfaj.html

Device Tree Blob in Qemu Virtual Machine

When simulate ARM64 on QEMU virtual machine, you will not need to provide dtb any more on Qemu command line.

The virt type machine simulation will create dtb, put it on ROM, and passed it to Linux kernel.

I dumped the dtb created by Qemu and use device-tree-compiler to reverse the binary blob to text format device tree.
It is better way to check what peripherals are supported in virt machine, than to read Qemu source code.

/dts-v1/;

/ {
    interrupt-parent = <0x8001>;
    #size-cells = <0x2>;
    #address-cells = <0x2>;
    compatible = "linux,dummy-virt";

    fw-cfg@9020000 {
        reg = <0x0 0x9020000 0x0 0xa>;
        compatible = "qemu,fw-cfg-mmio";
    };

    virtio_mmio@a000000 {
        interrupts = <0x0 0x10 0x1>;
        reg = <0x0 0xa000000 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a000200 {
        interrupts = <0x0 0x11 0x1>;
        reg = <0x0 0xa000200 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a000400 {
        interrupts = <0x0 0x12 0x1>;
        reg = <0x0 0xa000400 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a000600 {
        interrupts = <0x0 0x13 0x1>;
        reg = <0x0 0xa000600 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a000800 {
        interrupts = <0x0 0x14 0x1>;
        reg = <0x0 0xa000800 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a000a00 {
        interrupts = <0x0 0x15 0x1>;
        reg = <0x0 0xa000a00 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a000c00 {
        interrupts = <0x0 0x16 0x1>;
        reg = <0x0 0xa000c00 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a000e00 {
        interrupts = <0x0 0x17 0x1>;
        reg = <0x0 0xa000e00 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a001000 {
        interrupts = <0x0 0x18 0x1>;
        reg = <0x0 0xa001000 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a001200 {
        interrupts = <0x0 0x19 0x1>;
        reg = <0x0 0xa001200 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a001400 {
        interrupts = <0x0 0x1a 0x1>;
        reg = <0x0 0xa001400 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a001600 {
        interrupts = <0x0 0x1b 0x1>;
        reg = <0x0 0xa001600 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a001800 {
        interrupts = <0x0 0x1c 0x1>;
        reg = <0x0 0xa001800 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a001a00 {
        interrupts = <0x0 0x1d 0x1>;
        reg = <0x0 0xa001a00 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a001c00 {
        interrupts = <0x0 0x1e 0x1>;
        reg = <0x0 0xa001c00 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a001e00 {
        interrupts = <0x0 0x1f 0x1>;
        reg = <0x0 0xa001e00 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a002000 {
        interrupts = <0x0 0x20 0x1>;
        reg = <0x0 0xa002000 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a002200 {
        interrupts = <0x0 0x21 0x1>;
        reg = <0x0 0xa002200 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a002400 {
        interrupts = <0x0 0x22 0x1>;
        reg = <0x0 0xa002400 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a002600 {
        interrupts = <0x0 0x23 0x1>;
        reg = <0x0 0xa002600 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a002800 {
        interrupts = <0x0 0x24 0x1>;
        reg = <0x0 0xa002800 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a002a00 {
        interrupts = <0x0 0x25 0x1>;
        reg = <0x0 0xa002a00 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a002c00 {
        interrupts = <0x0 0x26 0x1>;
        reg = <0x0 0xa002c00 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a002e00 {
        interrupts = <0x0 0x27 0x1>;
        reg = <0x0 0xa002e00 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a003000 {
        interrupts = <0x0 0x28 0x1>;
        reg = <0x0 0xa003000 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a003200 {
        interrupts = <0x0 0x29 0x1>;
        reg = <0x0 0xa003200 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a003400 {
        interrupts = <0x0 0x2a 0x1>;
        reg = <0x0 0xa003400 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a003600 {
        interrupts = <0x0 0x2b 0x1>;
        reg = <0x0 0xa003600 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a003800 {
        interrupts = <0x0 0x2c 0x1>;
        reg = <0x0 0xa003800 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a003a00 {
        interrupts = <0x0 0x2d 0x1>;
        reg = <0x0 0xa003a00 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a003c00 {
        interrupts = <0x0 0x2e 0x1>;
        reg = <0x0 0xa003c00 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    virtio_mmio@a003e00 {
        interrupts = <0x0 0x2f 0x1>;
        reg = <0x0 0xa003e00 0x0 0x200>;
        compatible = "virtio,mmio";
    };

    pcie@10000000 {
        interrupt-map-mask = <0x1800 0x0 0x0 0x7>;
        interrupt-map = <0x0 0x0 0x0 0x1 0x8001 0x0 0x3 0x4 0x0 0x0 0x0 0x2 0x8001 0x0 0x4 0x4 0x0 0x0 0x0 0x3 0x8001 0x0 0x5 0x4 0x0 0x0 0x0 0x4 0x8001 0x0 0x6 0x4 0x800 0x0 0x0 0x1 0x8001 0x0 0x4 0x4 0x800 0x0 0x0 0x2 0x8001 0x0 0x5 0x4 0x800 0x0 0x0 0x3 0x8001 0x0 0x6 0x4 0x800 0x0 0x0 0x4 0x8001 0x0 0x3 0x4 0x1000 0x0 0x0 0x1 0x8001 0x0 0x5 0x4 0x1000 0x0 0x0 0x2 0x8001 0x0 0x6 0x4 0x1000 0x0 0x0 0x3 0x8001 0x0 0x3 0x4 0x1000 0x0 0x0 0x4 0x8001 0x0 0x4 0x4 0x1800 0x0 0x0 0x1 0x8001 0x0 0x6 0x4 0x1800 0x0 0x0 0x2 0x8001 0x0 0x3 0x4 0x1800 0x0 0x0 0x3 0x8001 0x0 0x4 0x4 0x1800 0x0 0x0 0x4 0x8001 0x0 0x5 0x4>;
        #interrupt-cells = <0x1>;
        ranges = <0x1000000 0x0 0x0 0x0 0x3eff0000 0x0 0x10000 0x2000000 0x0 0x10000000 0x0 0x10000000 0x0 0x2eff0000>;
        reg = <0x0 0x3f000000 0x0 0x1000000>;
        bus-range = <0x0 0xf>;
        #size-cells = <0x2>;
        #address-cells = <0x3>;
        device_type = "pci";
        compatible = "pci-host-ecam-generic";
    };

    pl031@9010000 {
        clock-names = "apb_pclk";
        clocks = <0x8000>;
        interrupts = <0x0 0x2 0x4>;
        reg = <0x0 0x9010000 0x0 0x1000>;
        compatible = "arm,pl031", "arm,primecell";
    };

    pl011@9000000 {
        clock-names = "uartclk", "apb_pclk";
        clocks = <0x8000 0x8000>;
        interrupts = <0x0 0x1 0x4>;
        reg = <0x0 0x9000000 0x0 0x1000>;
        compatible = "arm,pl011", "arm,primecell";
    };

    intc {
        phandle = <0x8001>;
        reg = <0x0 0x8000000 0x0 0x10000 0x0 0x8010000 0x0 0x10000>;
        interrupt-controller;
        #interrupt-cells = <0x3>;
        compatible = "arm,cortex-a15-gic";
    };

    flash@0 {
        bank-width = <0x4>;
        reg = <0x0 0x0 0x0 0x4000000 0x0 0x4000000 0x0 0x4000000>;
        compatible = "cfi-flash";
    };

    psci {
        migrate = <0xc4000005>;
        cpu_on = <0xc4000003>;
        cpu_off = <0x84000002>;
        cpu_suspend = <0xc4000001>;
        method = "hvc";
        compatible = "arm,psci-0.2", "arm,psci";
    };

    cpus {
        #size-cells = <0x0>;
        #address-cells = <0x1>;

        cpu@0 {
            reg = <0x0>;
            compatible = "arm,cortex-a57";
            device_type = "cpu";
        };
    };

    timer {
        interrupts = <0x1 0xd 0x101 0x1 0xe 0x101 0x1 0xb 0x101 0x1 0xa 0x101>;
        compatible = "arm,armv8-timer", "arm,armv7-timer";
    };

    apb-pclk {
        phandle = <0x8000>;
        clock-output-names = "clk24mhz";
        clock-frequency = <0x16e3600>;
        #clock-cells = <0x0>;
        compatible = "fixed-clock";
    };

    memory {
        reg = <0x0 0x40000000 0x0 0x80000000>;
        device_type = "memory";
    };

    chosen {
        bootargs = "console=ttyAMA0";
        stdout-path = "/pl011@9000000";
    };
};

Run ARM64 on Qemu

Install Required Tools

  
  sudo apt-get install git libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev ncurses-dev g++

Toolchain Installation
Linaro toolchain is believed to be more stable and more complete, easier to install compared to install from main repository.
https://www.linaro.org/downloads/
Choose the Linux version of "linaro-toolchain-binaries (little-endian)"
Latest link when this document is composed:
https://releases.linaro.org/15.02/components/toolchain/binaries/arm-linux-gnueabihf/gcc-linaro-4.9-2015.02-3-x86_64_arm-linux-gnueabihf.tar.xz
Download toolchain to ~/src

  
  tar xvf gcc-linaro-4.9-2015.02-3-x86_64_aarch64-linux-gnu.tar.xz 
  export PATH=~/src/gcc-linaro-4.9-2015.02-3-x86_64_aarch64-linux-gnu/bin/:$PATH

Build Qemu
Get a copy of Qemu from http://wiki.qemu.org/Download, saved to ~/src

  
  cd ~/src
  tar xvf qemu-2.3.0.tar.bz2
  cd qemu-2.3.0
  mkdir -p bin
  cd bin
  ../configure --enable-debug --target-list=aarch64-softmmu
  make -j9

The executable could be find at ~/src/qemu-2.3.0/bin/aarch64-softmmu/qemu-system-aarch64


Build RAM File System
Get a build root copy from http://buildroot.uclibc.org/download.html, save to ~/src

  
  cd ~/src
  tar xvf buildroot-2015.05.tar.bz2
  cd buildroot-2015.05
  make menuconfig
    have below configuration configured
    * Target Options -> Target Architecture(AArch64)
    * Toolchain -> Toolchain type (External toolchain)
    * Toolchain -> Toolchain (Custom toolchain)
    * Toolchain -> Toolchain origin (Pre-installed toolchain)
    * Toolchain -> Toolchain path (/home/xgu/src/gcc-linaro-4.9-2015.02-3-x86_64_aarch64-linux-gnu)    
    * Toolchain -> Toolchain prefix ($(ARCH)-linux-gnu)
    * Toolchain -> External Toolchain kernel headers series (3.17.x)
    * Toolchain -> External Toolchain C library (glibc/eglibc)
    * Toolchain -> Toolchain has RPC support? (enable)
    * Toolchain -> Toolchain has C++ support? (enable)
    * System configuration -> Run a getty (login prompt) after boot (BR2_TARGET_GENERIC_GETTY)
    * System configuration -> getty options -> TTY Port (ttyAMA0) (BR2_TARGET_GENERIC_GETTY_PORT)
    * Target Packages -> Show packages that are also provided by busybox (BR2_PACKAGE_BUSYBOX_SHOW_OTHERS)
    * Filesystem images -> cpio the root filesystem (for use as an initial RAM filesystem) (BR2_TARGET_ROOTFS_CPIO)
  
    or a simpler configure is to use build-in toolchain (lower version)
    * Target Options -> Target Architecture(AArch64)
    * Toolchain -> Toolchain type (External toolchain)
    * Toolchain -> Toolchain (Linaro AArch64 14.09)
    * System configuration -> Run a getty (login prompt) after boot (BR2_TARGET_GENERIC_GETTY)
    * System configuration -> getty options -> TTY Port (ttyAMA0) (BR2_TARGET_GENERIC_GETTY_PORT)
    * Target Packages -> Show packages that are also provided by busybox (BR2_PACKAGE_BUSYBOX_SHOW_OTHERS)
    * Filesystem images -> cpio the root filesystem (for use as an initial RAM filesystem) (BR2_TARGET_ROOTFS_CPIO)
    
  make

You will find image generated at output/images/rootfs.cpio
 
 
Build Linux Kernel
Get a copy of kernel at https://www.kernel.org/, save it to ~/src

  
  cd ~/src
  tar xvf linux-4.1.1.tar.xz
  cd linux-4.1.1
  export ARCH=arm64
  make defconfig    
  ./scripts/config --file .config --set-str CONFIG_CROSS_COMPILE aarch64-linux-gnu-
  ./scripts/config --file .config --set-str CONFIG_INITRAMFS_SOURCE ~/src/buildroot-2015.05/output/images/rootfs.cpio
  make -j9

You will find image generated at linux_dir/arch/arm64/boot/Image


Run Everything Together

  cd ~/src/qemu-2.3.0/bin
  ./aarch64-softmmu/qemu-system-aarch64 -machine virt -cpu cortex-a57 -nographic -smp 1 -m 512 -kernel ~/src/linux-4.1.1/arch/arm64/boot/Image --append "console=ttyAMA0"

Reference
http://www.bennee.com/~alex/blog/2014/05/09/running-linux-in-qemus-aarch64-system-emulation-mode/
http://wiki.qemu.org/Hosts/Linux

ARM Toolchain Differences

Format
Tool chains have  a loose name convention like
  arch [-vendor] [-os] - eabi

arch
  refers to target architecture (which in our case is ARM)
vendor
  refers to toolchain supplier
os
  refers to the target operating system
eabi
  refers to Embedded Application Binary Interface



arm-none-eabi
This tool chain targets for ARM architecture, has no vendor, does not target an operating system and complies with the ARM EABI.  


arm-none-linux-gnueabi
This toolchain targets the ARM architecture, has no vendor, creates binaries that run on the Linux operating system, and uses the GNU EABI. It is used to target ARM-based Linux systems.



Reference
https://community.freescale.com/thread/313490

How jiffies Updated

On gdb
  watch jiffies
  c

Linux will stops when jiffies is written, then
  bt

We can get the call stack like:

#0  calc_global_load (ticks=1) at kernel/sched/proc.c:344
#1  0xc00552e8 in do_timer (ticks=) at kernel/time/timekeeping.c:1766
#2  0xc0058ec0 in tick_periodic (cpu=) at kernel/time/tick-common.c:86
#3  0xc0058efc in tick_handle_periodic (dev=0xc03e1440 ) at kernel/time/tick-common.c:103
#4  0xc001e194 in sp804_timer_interrupt (irq=, dev_id=) at arch/arm/common/timer-sp.c:125
#5  0xc0048010 in handle_irq_event_percpu (desc=0xc03e6a58 , action=0xc03e14c0 ) at kernel/irq/handle.c:143
#6  0xc0048150 in handle_irq_event (desc=0xc03e6a58 ) at kernel/irq/handle.c:192
#7  0xc004a668 in handle_level_irq (irq=, desc=0xc03ee730 ) at kernel/irq/chip.c:457
#8  0xc0047a6c in generic_handle_irq_desc (desc=, irq=) at include/linux/irqdesc.h:129
#9  generic_handle_irq (irq=) at kernel/irq/irqdesc.c:351
#10 __handle_domain_irq (domain=0xc7806020, hwirq=5, lookup=true, regs=) at kernel/irq/irqdesc.c:388
#11 0xc00087c8 in handle_domain_irq (hwirq=, domain=, regs=) at include/linux/irqdesc.h:147
#12 handle_one_vic (regs=, vic=) at drivers/irqchip/irq-vic.c:222
#13 vic_handle_irq (regs=0x1 <__vectors_start>) at drivers/irqchip/irq-vic.c:255
#14 0xc0017640 in __irq_svc () at arch/arm/kernel/entry-armv.S:206

Debug Linux Versatile Build on Qemu

How to debug Linux4.0 ARM Versatile build on Qemu
Below is tested on Ubuntu 14.04
 
Install toolchain
sudo apt-get install gcc-arm-linux-gnueabi


Install Qemu for ARM
sudo apt-get install qemu-system-arm


Install GDB for ARM
sudo apt-get -o Dpkg::Options::="--force-overwrite" install gdb-arm-none-eabi
Some error happens without force overwrite option


Build Kernel with Device Tree Support
cd Linux4.0
make versatile_defconfig
make xconfig
  On the GUI xconfig window, find below configs (by Ctrl+F) and
    check on CONFIG_USE_OF
    check on CONFIG_MACH_VERSATILE_DT
    check on CONFIG_MACH_DEBUG_INFO
  Save the config
export ARCH=arm
export CROSS_COMPILE=arm-linux-gnueabi-
make -j9


Run On Qemu
qemu-system-arm -M versatileab -nographic -dtb ./arch/arm/boot/dts/versatile-ab.dtb -kernel arch/arm/boot/zImage -append “console=ttyAMA0″
or
qemu-system-arm -M versatileab -nographic -dtb ./arch/arm/boot/dts/versatile-ab.dtb -kernel arch/arm/boot/zImage -append “console=ttyAMA0″ -s -S

The second command will cause the Qemu to pause at start before any gdb client is connected


Debug with GDB
cd Linux4.0
ddd --debugger arm-none-eabi-gdb
On ddd command console
  target remote :1234
  file vmlinx
  b  setup_arch
  c
  ....