PCI BAR Assignment

Question:
How PCI BAR is assigned in Linux?

Answer:
Find free space from PCI bus resource in __find_resource, in which resource start and end are allocated ( from the bus IO memory region? )
__pci_assign_resource
pci_bus_alloc_resource
pci_bus_alloc_from_region
allocate_resource
reallocate_resource
__find_resource

In below function, parameter resno is the BAR number. 

static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev, int resno, resource_size_t size, resource_size_t align)

PCI BAR address points device IO. The value of the BAR address is just mapped by OS, not in range of the host physical memory. 

To update the BAR to PCI device. 

pci_assign_resource  // from the res->start ... 

  pci_update_resource

    pcibios_resource_to_bus(dev->bus, &region, res);

      region->start = res->start - offset;

    new = region.start | (res->flags & PCI_REGION_FLAG_MASK);

    reg = pci_resource_bar(dev, resno, &type);   

    pci_write_config_dword(dev, reg, new);

Reference:

http://www.linux-mips.org/wiki/PCI_Subsystem

http://wiki.osdev.org/PCI

Mutual Exclusion in Linux Kernel

Three kinds of Mutual Exclusion:

Mutual exclusion among different process contexts
Semaphore: down, up
Mutex: mutex_lock, mutex_unlock
Preempt disable: preempt_disable, preempt_enable

Mutual exclusion among different interrupt contexts
Hardware interrupt disable: local_irq_disable, local_irq_enable
Software interrupt disable: local_bh_disable, local_bh_enable

Mutual exclusion among different CPUs
Spin lock: spin_lock, spin_unlock
Read write spin lock: write_lock_irq, write_unlock_irq
Atomic operation: atomic_read, atomic_write
Sequence Lock: write_seqlock, write_sequnlock
RCU( Read Copy Update ): rcu_read_lock, rcu_read_unlock
Memory barrier: mb, wmb

Linux IO mapped memory access

APIs

For a device driver, the hardware registers access usually involves following kernel API.

• request_mem_region: Tell kernel that the specific range of physical memory are to be used.
• ioremap: Maps the physical memory to kernel virtual memory that can be accessed by kernel.
• ioreadX, iowriteX: X could be 8, 16, 32, parameter is the kernel virtual memory.
• release_mem_region: tell kernel the range of physical memory is not to be used anymore

ioread & iowirte

There are some drivers for ARM device are using "writel" and  "iowrite32" function to access IO mapped memory.
writeX read X are deprecated functions,  should use ioreadX iowriteX functions.

There is some interesting story about memory barriers of io mapped memory access.
Seems in 2006, writel and iowrite32 is no-barrier.  It is nowadays.

http://lwn.net/Articles/198988/

request_mem_region

About why some code, there is no calling of request_mem_region.
http://stackoverflow.com/questions/7682422/what-does-request-mem-region-actually-do-and-when-it-is-needed

Examples

Some source examples shows how to use io read/write related functions.

Linux Atomic Context and Process Context

In below article, concept of atomic context and process context is clearly explained. 

http://lwn.net/Articles/274695/

 Kernel code generally runs in one of two fundamental contexts. Process context reigns when the kernel is running directly on behalf of a (usually) user-space process; the code which implements system calls is one example. When the kernel is running in process context, it is allowed to go to sleep if necessary. But when the kernel is running in atomic context, things like sleeping are not allowed. Code which handles hardware and software interrupts is one obvious example of atomic context.
...

 There is more to it than that, though: any kernel function moves into atomic context the moment it acquires a spinlock. Given the way spinlocks are implemented, going to sleep while holding one would be a fatal error; if some other kernel function tried to acquire the same lock, the system would almost certainly deadlock forever.
...

 "Deadlocking forever" tends not to appear on users' wishlists for the kernel, so the kernel developers go out of their way to avoid that situation. To that end, code which is running in atomic context carefully follows a number of rules, including
(1) no access to user space, and, crucially,
(2) no sleeping.
Problems can result, though, when a particular kernel function does not know which context it might be invoked in. The classic example is kmalloc() and friends, which take an explicit argument (GFP_KERNEL or GFP_ATOMIC) specifying whether sleeping is possible or not.




Another article

http://www.itechtalk.com/thread216.html

The kernel accomplishes useful work using a combination of process contexts and interrupt contexts. Kernel code that services system calls issued by user applications runs on behalf of the corresponding application processes and is said to execute in process context. Interrupt handlers, on the other hand, run asynchronously in interrupt context. Processes contexts are not tied to any interrupt context and vice versa.

Kernel code running in process context is preemptible. An interrupt context, however, always runs to completion and is not preemptible. Because of this, there are restrictions on what can be done from interrupt context. Code executing from interrupt context cannot do the following:

1. Go to sleep or relinquish the processor.
2. Acquire a mutex.
3. Perform time-consuming tasks.
4. Access user space virtual memory.

Linux Device Driver Topics

Tested on Ubuntu 12.04

Where does printk go

The result of the printk wouldn't be displayed on terminal, we can find them at 

tail -f /var/log/kern.log

To load module
use insmod
modprobe may not work
http://stackoverflow.com/questions/3140478/fatal-module-not-found-error-using-modprobe
lsmod can list loaded module


Poll and interrupt
unsigned int (*poll) (struct file *filp, poll_table *wait);

The driver method is called whenever the user-space program performs a poll, select,or epoll system call involving a file descriptor associated with the driver. The device method is in charge of these two steps:
1. Call poll_wait on one or more wait queues that could indicate a change in the poll status. If no file descriptors are currently available for I/O, the kernel causes the process to wait on the wait queues for all file descriptors passed to the system call.
2. Return a bit mask describing the operations (if any) that could be immediately performed without blocking.

Driver entry points such as read() and poll() operate in tandem with interrupt handler roll_interrupt(). For example, when the handler deciphers wheel movement, it wakes up any waiting poll() threads that may have gone to sleep in response to a select() system call issued by an application