Building Linux Kernel Modules Faster: Internal and external

*For Newbie Linux Kernel and Device Driver developers*.

Most repetitive activity for Kernel and Device Driver developers. is building modules they work on and installing them. So lets discuss few tricks to build them faster way.

Traditionally, when we change a source code of a module, we need to 

a) Build the entire kernel again
make; make module;make modules_install;make install
(optionally using "-jN" option to make)
b) reboot and use the new modules.

But we can avoid reboot by 

a) Build the entire kernel again
b) Remove the old module 
c) Insert the new module.

Building the kernel itself is a time consuming process, especially when we just want to update code specific to our module. So for that we make use linux kernel's support and procedures for building external modules  i.e., modules which are not present in the kernel tree for modules in the kernel tree. Sounds Confusing?

Lets say we have written a new module and wanted to compile it, then we write a makefile which invokes the kernel makefile with below option:

make modules M=<Dir_of_new_Module>

But to compile a internal module (module which is present in the kernel tree) we use the same trick, except the makefile is already present. Just follow the below procedure

Procedure: 

1. make silentoldconfig  (Optional)
• whenever Kconfig is changed., For other cases like make and make modules (without M=) this is run by kernel makefile itself.
2. make modules M=<path_to_dir>
3. make modules_install M=<path_to_dir>
4. Remove the old module and Re-Insert the new one.

Still interested of know-how, see the steps in detail:

Details:

1) Whenever we change Kconfig, .config will be updated and we need to tell the kernel makefile to use the new .config instead of using old one.This does that.

2) This uses the makefile in the module sub directory and compiles the files w.r.t #1 .config options.

3) This will install the compiled modules in to the install path /lib/modules//`uname -r`/kernel/ (or) /lib/module/`uname -r`/extra/ depending on whether the module is internal/external. In our case its always extra as we are using "M=" option for building modules. We can add some prefix (extra folder name under the above paths).

Also it used the depmod utility (through a wrapper shell script: scripts/depmod.sh) to generate the below files which will later be used by modprobe.

• /lib/modules/`uname -r`/modules.alias
• /lib/modules/`uname -r`/modules.alias.bin
• /lib/modules/`uname -r`/modules.builtin
• /lib/modules/`uname -r`/modules.builtin.bin
• /lib/modules/`uname -r`/modules.ccwmap
• /lib/modules/`uname -r`/modules.dep
• /lib/modules/`uname -r`/modules.dep.bin
• /lib/modules/`uname -r`/modules.devname
• /lib/modules/`uname -r`/modules.ieee1394map
• /lib/modules/`uname -r`/modules.inputmap
• /lib/modules/`uname -r`/modules.isapnpmap
• /lib/modules/`uname -r`/modules.ofmap
• /lib/modules/`uname -r`/modules.order
• /lib/modules/`uname -r`/modules.pcimap
• /lib/modules/`uname -r`/modules.seriomap
• /lib/modules/`uname -r`/modules.softdep
• /lib/modules/`uname -r`/modules.symbols
• /lib/modules/`uname -r`/modules.symbols.bin
• /lib/modules/`uname -r`/modules.usbmap

Lets discuss the important ones below: 

• /lib/modules/`uname -r`/modules.dep
• Depmod is a utility which will use the "System.map" and try to work out the dependencies between the modules and create a file called "modules.dep", used by modprobe to insert (insmod) the modules in proper order.
• /lib/modules/`uname -r`/modules.order and /lib/modules/`uname -r`/modules.symbols will be used in that process.
• /lib/modules/`uname -r`/modules.alias
• It identifies the HW devices each device driver supports and writes to a file "module.alias" using which the linux kernel device model can insert the driver along with its dependencies (using modprobe and modules.dep) when it  enumerates a new HW device using PCI/USB ..
• /lib/modules/`uname -r`/modules.builtin
• This gives a list of modules which are staically linked to the kernel forming part of vmlinuz.
• /lib/modules/`uname -r`/modules.<BUS>map
• These files will the known drivers for devices and give informtion regarding the device like vendorid, subsystemid, deviceclass etc.
• This is used to create module.alias

4) Now that we have all the necessary files, use below commands "-v" is for verbose and "-r" is for using rmmod. By default it used insmod.

•     modprobe -v -r 
•     modprobe -v 

Installing depmod and modprobe: Install the "https://www.kernel.org/pub/linux/utils/kernel/module-init-tools/" package with your distro specific instructions.

Common Caveats:

1) If you are not seeing the changes in the new module: 

Probably you are re-inserting the old modules add "-v" to the modprobe and check the timestamps in the path from where modprobe is insmod-ing the module and your module in the kernel tree.  

Running "make modules_install" again should do, if all else fails then try copying manually.

2) If you are seeing a warning about module version check, while inserting the module: 

Probably you are using the above procedure for two dependent modules, like changing a function in cfg80211 and using that one in mac80211. 

Its better to use a directory which hold the directory of both of these, in the above case use "make modules  modules_install M=net/". For more details read Module Versioning Section in R#3.

References:

1) Linux kernel Makefile :-) 

2) Building External Modules

3) Kernel Docs: Building Linux kernel

Wireless LAN and Linux Together

With the advent of opensource the development time has come down hugely and the quality has improved quickly. For all those involved with WLAN/Wi-Fi and Opensource lets take a look at the wlan architecture in linux based on opensource mac80211 framework.

Block diagram explaining linux WLAN architecture. Please see below for details.

User-Space:

Configuration: wpa_supplicant and hostapd:

All the applications which interact directly with the user lie here. They can a GUI/CLI based ones for eg. network manager in ubuntu/fedora distributions are UI based ones, but the core part are the CLI based ones eg. wpa_supplicant for controlling the STA part of it and hostapd for controlling the AP part.

Both are configuration file based along with their cli versions (wpa_cli, hostapd_cli) to send commands on the fly.

They support different features like SME, MLME, Security, Wifi-Direct (P2P),  AP and STA configurations.

Tools:

We also have tools to send commands to the driver directly to set some parameters such as channel, bandwidth, some custom commands etc.

The Bridge: User and Kernel

Now how the various applications in userspace communicate to the core entities in the kernel? Well, we have different approaches but all are based on different socket interfaces.

1. WExt ==> Generic Wireless Extensions: IOCTL Interface
2. NL80211==> Netlink Sockets
3. HostAP,==> Raw Packet Sockets
4. Chipset specific:

•               Atheros==> IOCTL Interface 
•               Prism,IPW etc.

Kernel Space

Configuration and UMAC

For opensource world the framework in kernel for WLAN is mac80211, it separates itself in to 2 kernel modules

• cfg80211.ko: Which handles all the configuration, user space interaction
• mac80211.ko: Protocol: Upper MAC , driver interaction.

Most of the features and management is handled by the mac80211 module with the help of lower MAC.

Lower MAC Drivers

Lowe MAC drivers act as a bridge between the UMAC and the chipset (Firmware and HW). They do all the device initialization, registration with OS, Bug Registration, Interrupts registration etc through the services provided by the linux kernel.

A well written driver follows these conventions

• Maintains a OS Independent Layer: Easy portability to different OSes.
• Maintains a UMAC Independent Layer: Easpy portability to different UMAC's: Proprietary, opensource, 3rd party etc.
• Bus Abstraction Layer: Maintains compatibility across different Physical Buses like PCI, PCIe, AHB, SDIO etc.

Chipset: Firmware and HW

The full 802.11 protocol functionality is implemented here.

The firmware which probably runs on a separate processor/micro-controller configures and controls the hardware and also interacts with the host(The driver) through a messaging interface specific to the chipset (control path)

The Data path generally involves a DMA controller in the HW which takes care of generating interrupts to the host processor and transferring packets to and from the Host to the HW queues.

References:

1. wpa_supplicant_hostapd_devel_doc
2. For details on these click userspace_configurations.
3. wpa_supplicant