GNU's LMA and VMA

What is LMA and VMA
Every loadable or allocatable output section has two addresses. The first is the VMA, or virtual memory address. This is the address the section will have when the output file is run. The second is the LMA, or load memory address. This is the address at which the section will be loaded. In most cases the two addresses will be the same. An example of when they might be different is when a data section is loaded into ROM, and then copied into RAM when the program starts up (this technique is often used to initialize global variables in a ROM based system). In this case the ROM address would be the LMA, and the RAM address would be the VMA.

You can see the sections in an object file by using the objdump program with the -h option.

Configure LMA/VMA in Linker Script
The full description of an output section looks like this:
   

section [address] [(type)] : [AT(lma)]
  {
    output-section-command
    output-section-command
    ...
  } [>region] [AT>lma_region] [:phdr :phdr ...] [=fillexp]

Generating Binary Image (to burn to Flash)
objcopy can be used to generate a raw binary file by using an output target of `binary' (e.g., use `-O binary'). When objcopy generates a raw binary file, it will essentially produce a memory dump of the contents of the input object file. All symbols and relocation information will be discarded. The memory dump will start at the load address of the lowest section copied into the output file.


Move LMA to VMA
GNU toolchain does not have scattering mechanism like that in ARM toolchain. so BSS (zero init) part needs to be init by startup code.  no library to do that.

How to find out the .bss part?  via __bss_start__ and __bss_end__

_cstartup:

    /* Relocate .fastcode section (copy from ROM to RAM) */

    LDR     r0,=__fastcode_load

    LDR     r1,=__fastcode_start

    LDR     r2,=__fastcode_end


    .fastcode : {

        __fastcode_load = LOADADDR (.fastcode);

        __fastcode_start = .;


        *(.glue_7t) *(.glue_7)

        *isr.o (.text.*)

        *(.text.fastcode)

        *(.text.Blinky_dispatch)

        /* add other modules here ... */


        . = ALIGN (4);

        __fastcode_end = .;

    } >RAM AT>ROM

Reference
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Using_ld_the_GNU_Linker/scripts.html#BASIC-SCRIPT-CONCEPTS

https://www.embedded.com/design/mcus-processors-and-socs/4007119/Building-Bare-Metal-ARM-Systems-with-GNU-Part-1--Getting-Started
http://www.delorie.com/gnu/docs/binutils/ld_19.html
http://www.delorie.com/gnu/docs/binutils/ld_33.html
https://ftp.gnu.org/old-gnu/Manuals/binutils-2.12/html_chapter/binutils_3.html

Debug Linux Versatile Build on Qemu

How to debug Linux4.0 ARM Versatile build on Qemu
Below is tested on Ubuntu 14.04
 
Install toolchain
sudo apt-get install gcc-arm-linux-gnueabi


Install Qemu for ARM
sudo apt-get install qemu-system-arm


Install GDB for ARM
sudo apt-get -o Dpkg::Options::="--force-overwrite" install gdb-arm-none-eabi
Some error happens without force overwrite option


Build Kernel with Device Tree Support
cd Linux4.0
make versatile_defconfig
make xconfig
  On the GUI xconfig window, find below configs (by Ctrl+F) and
    check on CONFIG_USE_OF
    check on CONFIG_MACH_VERSATILE_DT
    check on CONFIG_MACH_DEBUG_INFO
  Save the config
export ARCH=arm
export CROSS_COMPILE=arm-linux-gnueabi-
make -j9


Run On Qemu
qemu-system-arm -M versatileab -nographic -dtb ./arch/arm/boot/dts/versatile-ab.dtb -kernel arch/arm/boot/zImage -append “console=ttyAMA0″
or
qemu-system-arm -M versatileab -nographic -dtb ./arch/arm/boot/dts/versatile-ab.dtb -kernel arch/arm/boot/zImage -append “console=ttyAMA0″ -s -S

The second command will cause the Qemu to pause at start before any gdb client is connected


Debug with GDB
cd Linux4.0
ddd --debugger arm-none-eabi-gdb
On ddd command console
  target remote :1234
  file vmlinx
  b  setup_arch
  c
  ....

Kernel Early Print

Before the first early prink, __create_page_tables in arch/arm/kernel/head.S will create mapping entry of IO space for UART.

 /*
  * Map in IO space for serial debugging.
  * This allows debug messages to be output
  * via a serial console before paging_init.
  */
 addruart r7, r3, r0

 mov r3, r3, lsr #SECTION_SHIFT
 mov r3, r3, lsl #PMD_ORDER

 add r0, r4, r3
 mov r3, r7, lsr #SECTION_SHIFT
 ldr r7, [r10, #PROCINFO_IO_MMUFLAGS] @ io_mmuflags
 orr r3, r7, r3, lsl #SECTION_SHIFT

Linux iSCSI Target Implementation

iSCSI command execution
-------------------
target_submit_cmd
target_submit_cmd_map_sgls
target_setup_cmd_from_cdb
  transport->parse_cdb(sbc_parse_cdb)
    sbc_execute_rw
      cmd->execute_rw // se_cmd.execute_rw points to sbc_ops->execute_rw

// destination to file      
fd_sbc_ops.execute_rw(fd_execute_rw)
  fd_do_rw
    vfs_readv
    vfs_writev
     
// destination to block device  
iblock_sbc_ops.execute_rw(iblock_execute_rw)
  // block device read write
  iblock_submit_bios
    block_lba = (cmd->t_task_lba << 3);
    submit_bio

     
iSCSI communication      
-------------------
iscsit_do_tx_data
  kernel_sendmsg
iscsit_do_rx_data
  kernel_recvmsg

iscsit_accept_np
  kernel_accept
iscsit_setup_np
  kernel_bind



iSCSI Authentication      
-------------------
__iscsi_target_login_thread
iscsi_target_start_negotiation
iscsi_target_do_login
iscsi_target_handle_csg_zero
iscsi_target_do_authentication
iscsi_handle_authentication

Linux Timer Interface

/* Initialize a simple mmio based clocksource */
clocksource_mmio_init

/* set up the handler of timer interrupt ,
In Irq handler, clock_event_device.event_handler will be called.
event_handler: Assigned by the framework to be called by the low level handler of the event source
*/
setup_irq

/* Configure and register a clock event device */
clockevents_config_and_register

Linux Interrupt on ARM

Question:
How interrupt information get initialized?

Answers:
Usually, in place like arch/arm/mach-xxx/some_file.c, resource of tyep IORESOURCE_IRQ is defined and get registered by calling platform_device_register(struct platform_device).
platform_device.name is used to map device and  device driver.

Device driver registration by usually happens when device module is loaded, by calling platform_driver_register(struct platform_driver).
platform_driver.driver.name is used map the device and device driver.



Question:
How driver register interrupt handler?

Answer:
In implementation of function platform_driver.probe(),
irq = platform_get_irq(plat_dev, num) // get an IRQ for a device
request_irq(irq, my_driver_irq_handler, IRQF_TRIGGER_RISING, ... )

Then my_driver_irq_handler() will be called when irq with index "num" is raised.



Question:
ARM CPU has only one IRQ input pin, how could it find out which peripheral raised the interrupt?
How driver interrupt handler get called?

Answer:
The Interrupt could be handled in two ways, call stack below:

1. CONFIG_MULTI_IRQ_HANDLER on
/* Macro in entry-armV.S */
irq_handler
/* This is a function pointer set by set_handle_irq, usually it got set during platform initialization, ex, like in function s3c2410_init_irq(), the pointer points to s3c24xx_handle_irq().
The pointer could also be set in setup_arch(), /arch/arm/kernel/setup.c, as mdesc->handle_irq.
mdesc->handle_irq is initialized in macro MACHINE_START, MACHINE_END,
mdesc->handle_irq could be set as gic_handle_irq(). */
handle_arch_irq
/* Here finds the actually irq number!!! */
s3c24xx_handle_irq
handle_IRQ
/* the kernel irq API, which will invoke actually irq handler. */
generic_handle_irq

2. CONFIG_MULTI_IRQ_HANDLER off
/* Macro in entry-armV.S */
irq_handler
/* get_irqnr_and_base() is called, which is a platform specific macro to query irq controller to find out which irq line is raised. */
arch_irq_handler_default
asm_do_IRQ
handle_IRQ
/* the kernel irq API, which will invoke actually irq handler. */
generic_handle_irq



Question:
When ARM's GIC ( General Interrupt Controller) hardware is used, how IRQ number is retrieved?

Answer:
In file arch/arm/common/gic.c, function gic_handle_irq() reads the interrupt controller's CPU interface register ICCIAR ( interrupt acknowledge register ) to get the IRQ number.  The register has 0x08 offset relative to "CPU interface base address".
Code:
irqstat = readl_relaxed(cpu_base + GIC_CPU_INTACK);
Reference:
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ihi0048b/index.html


Reference:
http://lxr.free-electrons.com/source/Documentation/blockdev/mflash.txt
http://pankaj-techstuff.blogspot.jp/2007/11/story-interrupt-handling-in-linux-2611.html
http://hi.baidu.com/sun_yfs/blog/item/e2531ecbf2cde989c9176806.html
http://lxr.linux.no/#linux+v2.6.32.59/drivers/video/s3c2410fb.c
http://lxr.linux.no/#linux+v2.6.32.59/arch/arm/plat-s3c24xx/irq.c
http://code.metager.de/source/xref/denx/u-boot/doc/README.SPL
http://code.metager.de/source/xref/denx/u-boot/doc/README.arm-relocation

Mutual Exclusion in Linux Kernel

Three kinds of Mutual Exclusion:

Mutual exclusion among different process contexts
Semaphore: down, up
Mutex: mutex_lock, mutex_unlock
Preempt disable: preempt_disable, preempt_enable

Mutual exclusion among different interrupt contexts
Hardware interrupt disable: local_irq_disable, local_irq_enable
Software interrupt disable: local_bh_disable, local_bh_enable

Mutual exclusion among different CPUs
Spin lock: spin_lock, spin_unlock
Read write spin lock: write_lock_irq, write_unlock_irq
Atomic operation: atomic_read, atomic_write
Sequence Lock: write_seqlock, write_sequnlock
RCU( Read Copy Update ): rcu_read_lock, rcu_read_unlock
Memory barrier: mb, wmb

Flash Memory Controller

Example code:
http://lxr.linux.no/linux+v2.6.32/drivers/mtd/nand/s3c2410.c

mtd nand driver architecture:
http://kernel.org/doc/htmldocs/mtdnand.html

about ECC of nand:
http://en.wikipedia.org/wiki/Flash_memory
http://en.wikipedia.org/wiki/Error_correcting_code

bad block management
NAND devices also require bad block management by the device driver software, or by a separate controller chip. SD cards, for example, include controller circuitry to perform bad block management and wear leveling. When a logical block is accessed by high-level software, it is mapped to a physical block by the device driver or controller. A number of blocks on the flash chip may be set aside for storing mapping tables to deal with bad blocks, or the system may simply check each block at power-up to create a bad block map in RAM. The overall memory capacity gradually shrinks as more blocks are marked as bad.

nand flash controller diagram

specification of nand interface, including command definition
http://www.micron.com/~/media/Documents/Products/ONFI/onfi_31_spec.pdf

Wear Leveling
JFFS2, YAFFS, and UBIFS include bad block management, wear leveling, error correction and provide reliable filesystems for industrial use on top of NAND Flash.

Some code about Linux file system on Flash

struct nand_chip {
 int (*write_page)(struct mtd_info *mtd, struct nand_chip *chip,
   uint32_t offset, int data_len, const uint8_t *buf,
   int oob_required, int page, int cached, int raw);
            
               
}

nand_write
  nand_do_write_ops
    chip->write_page ( nand_write_page )
      ecc->write_page ( nand_write_page_hwecc )
        chip->write_buf ( s3c2410_nand_write_buf ) 
      chip->cmdfunc( nand_command ) 
        chip->cmd_ctrl( s3c2410_nand_hwcontrol )
    


    
nand_read
  nand_do_read_ops
    chip->cmdfunc ( nand_command )
      chip->cmd_ctrl( s3c2410_nand_hwcontrol )
    ecc->read_page_raw ( nand_read_page_raw )
      chip->read_buf ( s3c2410_nand_read_buf )
      

mtd_read
  mtd->_read ( nand_read )
mtd_write       
  mtd->_write ( nand_write ) 
    
    
ubi_io_write    
  mtd_write
ubi_io_read
  mtd_read  
  
Wear-Leveling is in the drivers\mtd\ubi\wl.c 


UBIFS to UBI
  ubifs_leb_read
    ubi_read
  ubifs_leb_write
    ubi_leb_write
    

UBIFS and UBI are different
  UBI source code 
    drivers\mtd\ubi
  UBIFS source code
    fs\ubifs
  UBIFS's VFS mount point
    fs\ubifs\file.c
    fs\ubifs\dir.c
    fs\ubifs\super.c

ATA in Linux

SG_IO
The scsi-core (also known as the "mid level") contains the core of scsi support.
scsi generics driver (sg.o) represent the upper level drivers.

A significant addition in sg v3 is an ioctl() called SG_IO which is functionally equivalent to a write() followed by a blocking read(). In certain contexts the write()/read() combination have advantages over SG_IO (e.g. command queuing) and continue to be supported.

SG_IO call path, the IO request was put to block layer's queue, it is queue handlers responsibility to actually handle the request.

sd_ioctl
scsi_cmd_blk_ioctl
scsi_cmd_ioctl
sg_io
blk_execute_rq
blk_execute_rq_nowait
blk_mq_insert_request

sd_ioctl is registered as ioctl of block_device_operations, which will be registered to system via add_disk(), in function sd_probe_async().


And Here is the queue handling part:

scsi_queue_rq (queue_rq, registered as a blk_mq_ops)
scsi_dispatch_cmd
ata_scsi_queuecmd ( queuecommand )
__ata_scsi_queuecmd
ata_scsi_translate
ata_qc_issue


Zone ATA Command
sd_ioctl
_report_zones_ioctl
blk_zoned_report
blk_cmd_with_sense
blk_cmd_execute
blk_execute_rq



ATA Command Definition
include/linux/ata.h



LibATA
http://linuxmafia.com/faq/Hardware/sata.html

This is the newer ATA driver set for selected SATA chipsets only, maintained by Jeff Garzik, leveraging the kernel's well-tested SCSI layer. Garzik developed it in the 2.6 kernel series. 2.4 support was available only with a backported patch until libata's inclusion in 2.4.27 and later.

libata causes each SATA port appear as a new SCSI bus. There are individual low-level drivers for the individual SATA chipsets, e.g., ahci, pdc_adma, ata_piix, sata_nv, sata_mv, sata_promise, sata_qstor, sata_sil, sata_sil24, sata_sis, sata_sx4, sata_uli, sata_svw, sata_via, sata_vsc.


http://ftp.dei.uc.pt/pub/linux/kernel/people/jgarzik/libata/libata.pdf
struct ata_port_operations is defined for every low-level libata hardware driver, and it controls how the low-level driver interfaces with the ATA and SCSI layers.

Linux Pipeline Sample

Sample code shows how to use pipelin on Linux:

    pid_t pid = 0; 
    int pipefd[2];
  
    pipe(pipefd);
    pid = fork();
    if( -1 == pid ){
      std::cerr << "Fork failed;" << std::endl;
    }
    else if( 0 == pid ){
      int ret = 0;
      // Child process, will start ffmpeg to get meta data info
      
      close(pipefd[0]);  // not care about pipe read end.
      dup2(pipefd[1], 1); // redirect the stdout to pipe write end
      dup2(pipefd[1], 2); // redirect the stderr to pipe write end
      
      close(pipefd[1]);
      
      ret = execlp("ffmpeg","ffmpeg", "-i", filePath.c_str() , NULL);
      
      exit(126);
    }else {
      // Parent process    
      char buffer[4096];
      // not care about the pipe write end; 
      // if we do not close it, the child pipe will not close fully.
      close(pipefd[1]);  
      FILE * pipeFile = NULL;
      int shouldPrint = 0;
      
      pipeFile = fdopen( pipefd[0], "r");  // read from pipe read end.
      // Wait a while, another option is to use wait pid
      sleep(1);
    
      while(1)
      {
        char * p1;
        char * p2;
        
        p1 = fgets( buffer, 4096, pipeFile );
        if(p1 == NULL){
          break;
        }
        
        if( shouldPrint == 0){
          p2 = strstr(p1, "Metadata");
          if(p2){
            shouldPrint = 1;
          }
        }
        else{
          p2 = strstr(p1, "Duration");
          if(p2){
            break;
          }
          std::cout << p1;
        }
      }
      
      fclose(pipeFile);
    }
  }

Linux IO mapped memory access

APIs

For a device driver, the hardware registers access usually involves following kernel API.

• request_mem_region: Tell kernel that the specific range of physical memory are to be used.
• ioremap: Maps the physical memory to kernel virtual memory that can be accessed by kernel.
• ioreadX, iowriteX: X could be 8, 16, 32, parameter is the kernel virtual memory.
• release_mem_region: tell kernel the range of physical memory is not to be used anymore

ioread & iowirte

There are some drivers for ARM device are using "writel" and  "iowrite32" function to access IO mapped memory.
writeX read X are deprecated functions,  should use ioreadX iowriteX functions.

There is some interesting story about memory barriers of io mapped memory access.
Seems in 2006, writel and iowrite32 is no-barrier.  It is nowadays.

http://lwn.net/Articles/198988/

request_mem_region

About why some code, there is no calling of request_mem_region.
http://stackoverflow.com/questions/7682422/what-does-request-mem-region-actually-do-and-when-it-is-needed

Examples

Some source examples shows how to use io read/write related functions.

Linux Device Model

 This good article in Interface magazine 2008 introduced how code is organized in Linux Device Model.

Will find time to translate it into English.

Linux Device Driver Topics

Tested on Ubuntu 12.04

Where does printk go

The result of the printk wouldn't be displayed on terminal, we can find them at 

tail -f /var/log/kern.log

To load module
use insmod
modprobe may not work
http://stackoverflow.com/questions/3140478/fatal-module-not-found-error-using-modprobe
lsmod can list loaded module


Poll and interrupt
unsigned int (*poll) (struct file *filp, poll_table *wait);

The driver method is called whenever the user-space program performs a poll, select,or epoll system call involving a file descriptor associated with the driver. The device method is in charge of these two steps:
1. Call poll_wait on one or more wait queues that could indicate a change in the poll status. If no file descriptors are currently available for I/O, the kernel causes the process to wait on the wait queues for all file descriptors passed to the system call.
2. Return a bit mask describing the operations (if any) that could be immediately performed without blocking.

Driver entry points such as read() and poll() operate in tandem with interrupt handler roll_interrupt(). For example, when the handler deciphers wheel movement, it wakes up any waiting poll() threads that may have gone to sleep in response to a select() system call issued by an application

Linux Command Examples

Delete a folder recursively
find . -type d -name .svn -exec rm -rf {} \;

Kernel Debug Message
dmesg -wH

Zip and unzip
tar czvf xxx.tgz folder/
tar xvzf file-1.0.tar.gz
tar xvjf file-1.0.tar.bz2
tar xvf file-1.0.tar

Get the log to a file
./some_exe 2>&1 | tee fileName.log

How to find a file by name
find /etc -name "ho*t" -print
updatedb
locate filenamefin

How to grep a string from a folder
grep -R "string string" *

Check the return value of the command
echo $?c

How to patch diff
diff -rupN original/ new/ > some.diff
patch -p0 --dry-run < some.diff
patch -p0 < some.diff
patch -p0 -R < some.diff

Version Control
create branch
svn copy http://svn.example.com/repos/calc/trunk  http://svn.example.com/repos/calc/branches/my-calc-branch  -m "Creating a private branch of /calc/trunk."

delete .svn
rm -rf `find . -type d -name .svn`

delete cvs
find . -name 'CVS' -type d -exec rm -rf {} \;

Device Access
sudo dd of=/dev/zero if=/dev/mtd0 skip=1 bs=512 count=1

Read Memory Usage from Status of Proc

VmPeak: 8368 kB (peak usage of virtual memory)
VmSize: 7344 kB (usage of virtual memory)
VmLck: 0 kB (locked memory)
VmHWM: 2336 kB (peak usage of hardware memory)
VmRSS: 2336 kB (usage of hardware memory)
VmData: 3052 kB (heap)
VmStk: 88 kB (stack)
VmExe: 16 kB (text section)
VmLib: 4080 kB (shared library)
VmPTE: 56 kB (page table entry size)

Explanation comes from:
http://d.hatena.ne.jp/naoya/20080727/1217119867
http://www.kernel.org/doc/man-pages/online/pages/man5/proc.5.html
http://www.linuxquestions.org/questions/programming-9/vmsize-regarding-proc-pid-status-432227/