Flash Memory Controller

Example code:
http://lxr.linux.no/linux+v2.6.32/drivers/mtd/nand/s3c2410.c

mtd nand driver architecture:
http://kernel.org/doc/htmldocs/mtdnand.html

about ECC of nand:
http://en.wikipedia.org/wiki/Flash_memory
http://en.wikipedia.org/wiki/Error_correcting_code

bad block management
NAND devices also require bad block management by the device driver software, or by a separate controller chip. SD cards, for example, include controller circuitry to perform bad block management and wear leveling. When a logical block is accessed by high-level software, it is mapped to a physical block by the device driver or controller. A number of blocks on the flash chip may be set aside for storing mapping tables to deal with bad blocks, or the system may simply check each block at power-up to create a bad block map in RAM. The overall memory capacity gradually shrinks as more blocks are marked as bad.

nand flash controller diagram

specification of nand interface, including command definition
http://www.micron.com/~/media/Documents/Products/ONFI/onfi_31_spec.pdf

Wear Leveling
JFFS2, YAFFS, and UBIFS include bad block management, wear leveling, error correction and provide reliable filesystems for industrial use on top of NAND Flash.

Some code about Linux file system on Flash

struct nand_chip {
 int (*write_page)(struct mtd_info *mtd, struct nand_chip *chip,
   uint32_t offset, int data_len, const uint8_t *buf,
   int oob_required, int page, int cached, int raw);
            
               
}

nand_write
  nand_do_write_ops
    chip->write_page ( nand_write_page )
      ecc->write_page ( nand_write_page_hwecc )
        chip->write_buf ( s3c2410_nand_write_buf ) 
      chip->cmdfunc( nand_command ) 
        chip->cmd_ctrl( s3c2410_nand_hwcontrol )
    


    
nand_read
  nand_do_read_ops
    chip->cmdfunc ( nand_command )
      chip->cmd_ctrl( s3c2410_nand_hwcontrol )
    ecc->read_page_raw ( nand_read_page_raw )
      chip->read_buf ( s3c2410_nand_read_buf )
      

mtd_read
  mtd->_read ( nand_read )
mtd_write       
  mtd->_write ( nand_write ) 
    
    
ubi_io_write    
  mtd_write
ubi_io_read
  mtd_read  
  
Wear-Leveling is in the drivers\mtd\ubi\wl.c 


UBIFS to UBI
  ubifs_leb_read
    ubi_read
  ubifs_leb_write
    ubi_leb_write
    

UBIFS and UBI are different
  UBI source code 
    drivers\mtd\ubi
  UBIFS source code
    fs\ubifs
  UBIFS's VFS mount point
    fs\ubifs\file.c
    fs\ubifs\dir.c
    fs\ubifs\super.c

Data Into Storage

I uploaded a picture to Facebook, here is a my guess how a packet of the picture flies and lands.

• Packet is sent to Linux kernel
• Linux kernel handles it to network adapter driver
• Network adaptor sends packet to network, Packet goes through few routers or any other devices and  reaches server in Facebook data center.
• Server network adapter collects data and sends it HTTP server application.
• HTTP server application wants save this packet permanently, so the packet is sent back to server kernel to be written on a storage device.
• Packet is sent to Fibre Channel driver, Fibre Channel adapter use Fibre Channel protocol to wrappers SCSI write command which contains the packet. 
• Packet in SCSI command flies over SAN and arrives at a storage controller system
• Storage controller system ( may be a RAID controller ) finds out which hard disk to write the data and wrap the data with ATA command.
• ATA command is sent via SATA cable and received by hard disk controller
• Hard disk controller decodes SATA information and the encode packet with EDC+LPDC and write to hard disk. ( a lot of complexity is hidden in this step, that is the my current job ...  )

There is a lot of overhead and complexity here and there. 

Reference

AHCI

AHCI Implementation
drivers/ata/ahci_platform.c
drivers/ata/libahci.c
implements most ata_port_operations operations

ahci_init_one,
  memory mapping
  hpriv->mmio = pcim_iomap_table(pdev)[ahci_pci_bar];  AHCI_PCI_BAR_STANDARD

static inline void __iomem *__ahci_port_base(struct ata_host *host,
     unsigned int port_no)
{
struct ahci_host_priv *hpriv = host->private_data;
void __iomem *mmio = hpriv->mmio;

return mmio + 0x100 + (port_no * 0x80);
}

hpriv->mmio points to the HBA memory registers,  pointed by ABAR(AHCI Base Address).
 __ahci_port_base gets the port register.

Command table is initialized in ahci_port_start(), pp->cmd_tbl = mem;

Below maps to AHCI spec v3.3, port registers.
PORT_LST_ADDR = 0x00, /* command list DMA addr */
PORT_LST_ADDR_HI = 0x04, /* command list DMA addr hi */
PORT_FIS_ADDR = 0x08, /* FIS rx buf addr */
PORT_FIS_ADDR_HI = 0x0c, /* FIS rx buf addr hi */
PORT_IRQ_STAT = 0x10, /* interrupt status */
PORT_IRQ_MASK = 0x14, /* interrupt enable/disable mask */
PORT_CMD = 0x18, /* port command */
PORT_TFDATA = 0x20, /* taskfile data */
PORT_SIG = 0x24, /* device TF signature */
PORT_CMD_ISSUE = 0x38, /* command issue */
PORT_SCR_STAT = 0x28, /* SATA phy register: SStatus */
PORT_SCR_CTL = 0x2c, /* SATA phy register: SControl */
PORT_SCR_ERR = 0x30, /* SATA phy register: SError */
PORT_SCR_ACT = 0x34, /* SATA phy register: SActive */
PORT_SCR_NTF = 0x3c, /* SATA phy register: SNotification */
PORT_FBS = 0x40, /* FIS-based Switching */
PORT_DEVSLP = 0x44, /* device sleep */

Get backtrace

This the place holder for all the investigation made about how to get back trace information at run time.

I have a embedded system running a tiny proprietary real time OS which can handle few tasks. When the system is stopped at debugger break point, from debugger it is easy to find out stack trace of the task get stopped ( or OS kernel ), but I have no way to get other task's stack trace.  ARM's debugging toolchain has little help on this since everything is proprietary.

http://www.dwarfstd.org/doc/DWARF4.pdf
https://github.com/eliben/pyelftools/wiki/User's-guide
http://eli.thegreenplace.net/2011/02/07/how-debuggers-work-part-3-debugging-information/

My girl's math home work

Here is the python script to generate it, she must hate to have a father as a programmer.

DMA Transfer to User Space

http://stackoverflow.com/questions/5539375/linux-kernel-device-driver-to-dma-from-a-device-into-user-space-memory
http://stackoverflow.com/questions/3333959/mapping-dma-buffers-to-userspace
https://www.kernel.org/doc/Documentation/DMA-API-HOWTO.txt

ATA in Linux

SG_IO
The scsi-core (also known as the "mid level") contains the core of scsi support.
scsi generics driver (sg.o) represent the upper level drivers.

A significant addition in sg v3 is an ioctl() called SG_IO which is functionally equivalent to a write() followed by a blocking read(). In certain contexts the write()/read() combination have advantages over SG_IO (e.g. command queuing) and continue to be supported.

SG_IO call path, the IO request was put to block layer's queue, it is queue handlers responsibility to actually handle the request.

sd_ioctl
scsi_cmd_blk_ioctl
scsi_cmd_ioctl
sg_io
blk_execute_rq
blk_execute_rq_nowait
blk_mq_insert_request

sd_ioctl is registered as ioctl of block_device_operations, which will be registered to system via add_disk(), in function sd_probe_async().


And Here is the queue handling part:

scsi_queue_rq (queue_rq, registered as a blk_mq_ops)
scsi_dispatch_cmd
ata_scsi_queuecmd ( queuecommand )
__ata_scsi_queuecmd
ata_scsi_translate
ata_qc_issue


Zone ATA Command
sd_ioctl
_report_zones_ioctl
blk_zoned_report
blk_cmd_with_sense
blk_cmd_execute
blk_execute_rq



ATA Command Definition
include/linux/ata.h



LibATA
http://linuxmafia.com/faq/Hardware/sata.html

This is the newer ATA driver set for selected SATA chipsets only, maintained by Jeff Garzik, leveraging the kernel's well-tested SCSI layer. Garzik developed it in the 2.6 kernel series. 2.4 support was available only with a backported patch until libata's inclusion in 2.4.27 and later.

libata causes each SATA port appear as a new SCSI bus. There are individual low-level drivers for the individual SATA chipsets, e.g., ahci, pdc_adma, ata_piix, sata_nv, sata_mv, sata_promise, sata_qstor, sata_sil, sata_sil24, sata_sis, sata_sx4, sata_uli, sata_svw, sata_via, sata_vsc.


http://ftp.dei.uc.pt/pub/linux/kernel/people/jgarzik/libata/libata.pdf
struct ata_port_operations is defined for every low-level libata hardware driver, and it controls how the low-level driver interfaces with the ATA and SCSI layers.