Developing Dynamically Loadable Kernel Modules

If a module requires another module in order to work properly, you can specify a dependency between them. The module loader attempts to load all the dependent modules when it finds dependencies during loading of a module. This module dependency mechanism is also applicable to Boot Time Loading.

For the system to work properly, the module developer or system administrator needs to exercise care in determining the configured loading phase and the supported loading phase of modules. A module without BOOT1 or BOOT2 specified in its supported loading phase would not be registered during boot phase 1 or boot phase 2. Such modules cannot be loaded in those phases even if they are defined as dependents of other boot-time-loaded modules. A module loading failure in boot time may be critical for your system if the failed module provides essential functionality for your system (i.e., SCSI disk driver).

If there is a possibility that the module will be referred to as a dependent during boot time, then the module must include BOOT1 or BOOT2 in the supported loading phase specification.

All modules that have BOOT1 in their supported loading phase specification are registered in system boot phase 1. All other modules are registered during boot phase 2, regardless of the values specified in their supported loading phase.

	NOTE: Only miscellaneous modules (MISC) can be registered during boot phase 1 on Itanium-based systems. Other DLKM types supported in HP-UX 11i Version 1.5 will not work properly in early boot. When their type-specific registration is performed, the needed functionality will not be available during the early phases of booting, (i.e., streams, wsio, etc.).

A module having a configured loading phase of BOOT1 will be loaded just after DLKM has initialized in boot phase 1. This is later than initialization steps for spinlocks and the primitive memory allocation method used by boot phase 1, but prior to beginning I/O initialization.

A module having BOOT2 as its configured loading phase will be loaded just after the statically linked device drivers have been initialized. Any modules either referenced by an auto-loading stub linked into the static kernel, or depended-upon by other modules loaded during the boot process, are loaded when they are referenced at boot time. They must, however, be configured with the applicable loading phase in their supported loading phase specification.

The C program source code for a DLKM module, which is compiled and possibly linked to become mod.o, is similar to the source code for a static module except that it contains the following additional information:

Module Wrapper

The source code for a DLKM module must contain a wrapper, defined by the modwrapper structure, specifying the _load() and _unload() function entry points and other data structures used by the module. The name of the modwrapper structure for a DLKM module must be module_name_wrapper.

The wrapper data structures are partially initialized by the DLKM infrastructure using values taken from the module's configuration files. These structures provide information needed during loading and unloading, such as the values needed to populate a loadable driver's device switch table entries for the major device number it supports.

The code definition of the modwrapper structure is defined in header file moddefs.h and shown below. The data structures and #define statement preceding the modwrapper structure are also defined in moddefs.h. The modwrapper structure contains a pointer to the modlink structure, and the modlink structure contains pointers to the mod_operations and mod_type data structures. The #define statement indicates the revision (version) number of the modwrapper structure, which is 1.0 for HP-UX 11.0.

	NOTE: The modwrapper revision numbers will be used to maintain backward/forward compatibility of DLKM modules within the DLKM infrastructure.

Example 1-1 An Extract from moddefs.h File Showing modwrapper Structure

#define MODREV 10 struct modlink { struct mod_operations *ml_ops; void *ml_type_data; }; /* * Module type specific linkage structure. */ struct mod_type_data { char *mtd_info; void *mtd_pdata; }; struct modwrapper { int mw_rev; int (*mw_load)(); int (*mw_unload)(); void (*mw_halt)(); void *mw_conf_data; struct modlink *mw_modlink; };

The elements of the modwrapper structure are:

• mw_rev - module revision number; use MODREV in your module wrapper

• mw_load - pointer to module's _load() function

• mw_unload - pointer to module's _unload() function

• mw_halt - currently unused; use (void (*)())NULL in your module wrapper

• mw_conf_data - pointer to configuration data created by configuration tool config(1M)

• mw_modlink - pointer to a modlink array, which is a NULL terminated array of modlink structures that specify the type-specific operations required by the module

The elements of the modlink structure are:

• ml_ops - pointer to a mod_operations structure that depends on the type of DLKM module being defined:

  Table 1-3 Title not available (Module Wrapper)

  Module Operations	DLKM Module Type
  gio_mod_ops	WSIO class driver
  gio_mod_ops	WSIO interface driver
  str_drv_ops	STREAMS driver
  str_mod_ops	STREAMS module
  mod_misc_ops	Miscellaneous module

   

• ml_type_data - pointer to a mod_type_data structure

The elements of the mod_type_data structure are:

• mtd_info - information string returned to modstat() system call

• mtd_pdata - pointer to type-specific data that depend on the type of loadable module being defined: for both STREAMS drivers and STREAMS modules, use a pointer to the streams_info_t structure; for all other types, use (void *)NULL

A sample wrapper for a WSIO driver is as follows:

Example 1-2 WSIO Driver Wrapper—Example

/* * Wrapper Table */ extern struct mod_operations gio_mod_ops; static drv_info_t module_name_drv_info; extern struct mod_conf_data module_name_conf_data; /* module type specific data */ static struct mod_type_data module_name_drv_link = { "module_name - Loadable/Unloadable Test Module", (void *)NULL }; static struct modlink module_name_mod_link[] = { { &gio_mod_ops, (void *)&module_name_drv_link }, /* WSIO */ { NULL, (void *)NULL } }; struct modwrapper module_name_wrapper = { MODREV, module_name_load, module_name_unload, (void (*)())NULL, (void *)&module_name_conf_data, module_name_mod_link };

When a DLKM module is configured as statically linked, its wrapper is not used by the system.

Each DLKM module has its own master file. The format of the master file includes the following section keywords:

For complete details on the use of these keywords, see the "Modular Master File" section of the master(4) manpage.

Every DLKM module requires a system file. The system file includes the following mandatory section keyword and four optional section keywords:

For complete details on the use of these keywords, see the "Modular System File" section of the system(4) manpage.

An optional component, the space.h file contains storage allocations and initializations of data structures associated with a DLKM module when the size or initial value of the data structures depend on configurable values such as tunable parameters. In order to communicate these values to the rest of the DLKM module, the values are stored in global variables and accessed by the module via extern declarations in the module's mod.o file.

	NOTE: All tunable parameters specified in the master file are defined as global variables in the space.h file. See “Sample DLKM WSIO Class Driver” for clarification.

An optional component, the Modstub.o file is statically configured into the kernel as a "place holder" for functions implemented in a loadable module that will be loaded at a later time. Its purpose is to enable the kernel to resolve references to the absent module's functions.

Modstub.o contains “stubs” for entry points defined in the associated loadable module that can be referenced by other statically linked kernel modules currently configured in the system. Access to a stub causes the kernel to auto load the associated loadable module.

Configuring a module that uses stubs requires a full kernel build so that the stubs can be statically linked to the kernel.

To make a WSIO class driver loadable, the driver must provide a _load() and _unload() function and make minor changes to the way that the driver initializes itself. When a driver is statically linked, it typically links its _init() function into the dev_init chain, and the _init() function calls the next driver's _init() function in the chain.

This mechanism is inappropriate for loadable drivers since the dev_init chain is only used during system boot, and there is no mechanism to remove the entry from the chain when the module is unloaded.

Therefore, the module's _load() function should perform both the installation tasks (those normally done during install for static modules) and the driver initialization tasks, and the dev_init chain should be ignored.

Additionally, the _load() function is passed a pointer to an updated version of the drv_info_t structure; the driver must use this version of the drv_info structure when it registers itself with WSIO. A sample _load() function for a WSIO class driver is as follows:

Example 1-3 WSIO Class Driver _load Function

static wsio_drv_info_t module_name_wsio_info = { ... }; /* * LOAD */ static int module_name_load(void *arg) { if (module_name_debug) printf("module_name> Loading\n"); /* Use drv_info passed to us instead of static version */ module_name_wsio_info.drv_info = (drv_info_t *) arg; /* Register with WSIO */ if (!wsio_install_driver(&module_name_wsio_info)) { printf("module_name> wsio_install_driver failed!!\n"); return (ENXIO); } /* Perform driver-specific initialization, but do not call * next function in the dev_init list. */ (void) module_name_init(); return (0); }

	NOTE: The wsio_install_driver() function call returns 1 on success. However (and inconsistently), the wsio_uninstall_driver() function call returns 0 on success.

Initialization for a statically linked WSIO class driver is unchanged from historical practice. That is, the initial driver entry point is module_name_install. This function typically installs an _init() function in a list of functions that will be invoked later in the system boot process, and then the driver registers itself with WSIO. For example:

Example 1-4 WSIO Class Driver _install Function

static void module_name_linked_init (void); static int (*module_name_saved_init)(); /* * INSTALL * This function is called if module is statically linked. */ int module_name_install(void) { extern int (*dev_init)(void); /* Link my init function into chain */ module_name_saved_init = dev_init; dev_init = (int (*)()) &module_name_linked_init; /* Register driver with WSIO */ return ( wsio_install_driver(&module_name_wsio_info) ); }

The _init() function in the static case must call the next driver's _init() function:

Example 1-5 WSIO Class Driver _init Function

/* * Device initialization Link * Called only for statically linked drivers to link init * routine into list. */ static void module_name_linked_init (void) { /* Perform driver-specific initialization */ (void) module_name_init (); /* Call next init function in chain */ (void) (*module_name_saved_init)(); }

The _unload() function frees resources allocated by the driver and unregisters the driver from WSIO. If it is safe to unload the driver when the number of open devices for the driver goes to zero, then the module need not perform any special checks to determine if it is busy. If any step in the _unload() process fails, the driver must undo any action prior to the failure. A sample _unload() function for a WSIO class driver is as follows:

Example 1-6 WSIO Class Driver _unload Function

/* * UNLOAD */ static int module_name_unload(void *drv_infop) { /* This function is only called when the administrator * attempts to unload the module and there are no open * devices using the module. If there is some reason that * the module should not be unloaded, check it now and * return non-zero. */ if (module_name_no_unload) { printf("module_name> I'm BUSY\n"); return (EBUSY); } /* Unregister with WSIO */ if ( wsio_uninstall_driver(&module_name_wsio_info) ) { /* Uninstall failed! Return to a loaded, functional * state. */ printf("module_name> wsio_uninstall_driver failed!!\n"); return (ENXIO); } /* Cancel pending timeouts, free allocated memory and * resources, etc. */ if (module_name_debug) printf("module_name> Unloaded\n"); return (0); }

	NOTE: The wsio_uninstall_driver() function call returns 0 on success. However (and inconsistently), the wsio_install_driver() function call returns 1 on success.

A WSIO interface driver requires an _attach() function for any interface card it intends to claim. It will also require a _probe() function if it intends to install device probes. When installing a WSIO interface driver, the driver must register itself with WSIO and set up the _attach() function and any device probes.

Because the timing of initialization differs between loadable and statically linked modules, some of the steps that are handled automatically during boot for static drivers must be explicitly performed by the _load() function.

The primary difference in making a WSIO interface driver loadable concerns the handling of the attach list. Historically, drivers have added their _attach() functions to a global list, and the _attach() function was responsible for calling the next driver's _attach() function in the list. However, this method does not allow the driver to remove its _attach() function from the list; thus, this approach cannot support unloading.

New support functions have been added to WSIO to allow interface drivers to add and remove their _attach() functions from the attach list. These functions are:

list_type specifies the attach list to use; valid entries are MOD_WSIO_CORE, MOD_WSIO_PCI, and MOD_WSIO_EISA. Both functions return 0 on success and 1 on failure.

These functions should only be called when the module is dynamically loaded. Statically linked modules should continue to use the existing attach chain.

Device probes are normally associated with interface drivers during the initialization of the WSIO Context Dependent Input/Output (CDIO) at boot time. Since this is only done once, loadable interface drivers must explicitly connect the device probe to the driver's drv_info structure:

void wsio_activate_probe(char *probe_name, struct drv_info *drv_infop);

probe_name is the name of the probe as registered by wsio_register_dev_probe() or wsio_register_probe_func(). drv_infop is a pointer to the drv_info structure for this driver. In general, use a probe provided by the system. For example, parallel_scsi_probe or side_probe. See the HP-UX Driver Development Guide for complete information.

Finally, similar to class drivers, the _load() function for interface drivers is passed a pointer to an updated version of the drv_info_t structure; the interface driver must use this version of the drv_info structure when it registers itself with WSIO.

The sample code below puts all these concepts together to demonstrate the loading and initialization steps for an interface driver. This sample can be configured as either a loadable module or a statically linked module.

Example 1-7 WSIO Interface Driver Loading and Initialization Coding

static wsio_drv_info_t module_name_wsio_info = { ... }; static int (*module_name_saved_attach)(); int module_name_load(void *arg) { /* Use the drv_info passed to us instead of the static * version */ wsio_info.drv_info = (drv_info_t *) arg; /* Register the driver with WSIO */ if (!wsio_install_driver(&wsio_info)) return ENXIO; /* install failed! */ /* Add the attach function to the DLKM attach list. */ mod_wsio_attach_list_add(MOD_WSIO_CORE, &module_name_core_attach); /* Register the device probe. */ wsio_register_dev_probe(IF_CLASS, probe_func, "probe_name"); /* The following step is only required for dynamically * loaded modules: attach the probe function to the * drv_info structure */ wsio_activate_probe("probe_name", wsio_info.drv_info); return 0; } int module_name_install(void) { /* Add the attach function to the list. */ extern init (*core_attach)(); module_name_saved_attach= core_attach; core_attach= module_name_attach_linked; /* Register the device probe. */ wsio_register_dev_probe(IF_CLASS, probe_func, "probe_name"); /* Register the driver with WSIO */ return (wsio_install_driver(&wsio_info); } /* Common attach function for both dynamic and * static modules */ int module_name_attach(int id, struct isc_table_type *isc) { /* Normal attach function operations */ ... } /* Attach function called from attach chain for static modules * only */ int module_name_attach_linked (int id, struct isc_table_type *isc) { module_name_attach(id, isc); return ((*module_name_saved_attach)(id, isc)); }

The _unload() function determines if the driver is still busy, and if not, it cleans up all resources that were obtained by the driver. It should free memory that it allocated and remove the attach() function from the attach list. It should also unregister any _probe() function pointers that the driver has registered with other kernel services; for example, the _unload() function should unlink the interrupt service function. If an operation fails while unloading, the _unload() function must be able to restore the driver to a working state and return a nonzero value.

Unlike class drivers, there is no automatic method for the system to determine if an interface driver is still busy. This problem can sometimes be avoided by making class drivers that use the interface driver dependent upon the interface driver. With this dependency relationship in place, the system will not allow the interface driver to be unloaded as long as any class driver that depends upon it is still loaded. If it is not appropriate for the interface driver to rely on dependencies, then it must determine via other means if it is possible to unload the driver. The _unload() function is always free to return a nonzero value, and the module will remain loaded. A sample _unload() function for a WSIO interface driver is as follows:

Example 1-8 WSIO Interface Driver _unload Function

int module_name_unload(void *arg) { int ret; struct isc_table_type *isc; void *token, *priv_ptr; return (EINVAL); /* Remove the attach function from the DLKM attach list. */ if (mod_wsio_attach_list_remove(MOD_WSIO_CORE, &module_name_core_attach)) { return(ENXIO); } /* Unregister the device probe. */ wsio_unregister_dev_probe(IF_CLASS, probe_func, "probe_name"); /* For each ISC belonging to the driver: * Cancel pending timeouts, free allocated memory, * unregister ISR routine, etc. */ if (wsio_uninstall_driver(&module_name_wsio_info)) { /* uninstall failed - go back to loaded state * undo what has been done in _unload routine */ return ENXIO; } return(0); }

The _load() function for a monolithic driver must effectively be the union of the _load() functions for the class and interface drivers described previously. Similarly, the _unload() function must effectively be the union of the _unload() functions for the class and interface drivers.

Initialization of STREAMS drivers is very similar for both the loadable and statically linked module cases. The only difference is that loadable drivers must use the drv_info_t structure that is passed as an argument to the _load() function. The major numbers from this structure must also be used in the streams_info_t structure passed to str_install(). Sample _load() and _install() functions for a STREAMS driver are as follows:

Example 1-9 STREAMS Driver _load and _install Functions

static drv_info_t str_drv_info = { ... }; static drv_info_t *drv_info_p = &str_drv_info; static streams_info_t str_info = { ... }; static drv_ops_t module_name_str_drv_ops = { ... }; int module_name_load(void *arg) { int retval; /* Use the drv_info passed to us instead of the static * version */ drv_info_p = (drv_info_t *) arg; str_info.inst_major = drv_info_p->c_major; if (module_name_install()) return ENXIO; return 0; } int module_name_install(void) { int retval; /* Install in cdevsw */ if ((retval=install_driver(drv_info_p, &module_name_str_drv_ops)) != 0) return (retval); /* install failed */ /* Install in Streams */ if ((retval = str_install(&str_info)) != 0) { uninstall_driver(drv_info_p); return retval; } return 0; }

STREAMS drivers, like WSIO class drivers, automatically track open() and close() system calls for the STREAMS device. The system will prevent a STREAMS driver from unloading whenever the device has one or more open file handles. Of course, the driver can still disallow an unload if this check is insufficient for its needs. A typical _unload() function for a STREAMS driver is as follows:

Example 1-10 STREAMS Driver _unload Function

module_name_unload(void *arg) { int retval; /* Uninstall from Streams */ if ((retval = str_uninstall(&str_info)) != 0) return retval; /* Free module specific resources, etc. */ ... return 0; }

Loadable STREAMS modules have no special requirements during initialization. The argument passed to the _load() function should be ignored. Sample _load() and _install() functions are as follows:

Example 1-11 STREAMS Module _load and _install Functions

static streams_info_t str_info = { ... }; int module_name_load(void *arg) { int retval; if (module_name_install()) return ENXIO; return 0; } int module_name_install(void) { /* Install in Streams */ if ((retval = str_install(&str_info)) != 0) return retval; return 0; }

The system automatically tracks pushes and pops of a STREAMS module on active streams; a module cannot be unloaded while it is pushed onto one or more streams. A typical unload() function for a STREAMS module is as follows:

Example 1-12 STREAMS Module _unload Function

int module_name_unload(void *arg) { int retval; /* Uninstall from Streams */ if ((retval = str_uninstall(&str_info)) != 0) return retval; /* Free module specific resources, etc. */ ... return 0; }

Miscellaneous modules can implement any feature within the kernel. As such, a miscellaneous module's _load() function must address all of the module's specific needs. Similarly, the module's _unload() function must determine for itself if it is safe to unload. The system will not allow a module to be unloaded if other loaded modules are dependent upon the module. Other than this check, the system performs no other checks when the administrator attempts to remove a miscellaneous module from the kernel.

The argument to the _load() function is not meaningful and should be ignored.

Create the mod.o, master, system, space.h (optional), and Modstub.o (optional) files for the DLKM module in a single directory. It is suggested that you use a subdirectory of /usr/conf such as /usr/conf/module_name.

When choosing a name for your module, choose a unique name to avoid conflict with kernel functions and global variables. Consider using your company's name and something that indicates the module's purpose.

To compile your module source code, use the ANSI C compiler /opt/ansic/bin/cc and the compiler options shown below. You can also use the compiler options found in the makefile /stand/build/config.mk or
/stand/build/config.mod.

To compile your module source code, follow these steps:

Once the module's mod.o, master, system, and optional files have been created, call the kminstall command to copy those files to certain subdirectories of /usr/conf and /stand. The kminstall command creates the required subdirectories if they do not exist.

kminstall expects to find the module's component files in the current working directory. If module_name already exists on the system, kminstall prints a message and fails.

To install a DLKM module's components, follow these steps:

kminstall copies the module's component files to the appropriate locations. For example, kminstall copies the master file to /usr/conf/master.d/module_name/0.1.0, and the system file to /stand/system.d/module_name/0.1.0. File /usr/conf/master.d/module_name/0.1.0 is known as the module's master configuration file. File /stand/system.d/module_name/0.1.0 is known as the module's system description file.

	NOTE: The file locations receiving copies of the module's component files may change in future HP-UX releases.

After a module's component files have been installed using the kminstall command, the module is ready to be configured and registered with the running kernel. Follow the procedures in “DLKM Procedures for Dynamically Configured Loadable Modules” for instructions on how to prepare, configure, register, load, and manage your DLKM as dynamically loadable, on the target system.

To configure and test your DLKM as statically linked refer to “DLKM Procedures for Statically Configured Loadable Modules”.

Before devices supported by a driver type DLKM module can be accessed by the system, you need to create special files in the /dev directory using the mknod(1M) command. You do not have to create these files every time you build and boot a new kernel--only when you first add the new module. There must be a special file for each device on your system.

When you set up the special files for a newly added device driver, you must specify the device type (character or block) and the major and minor device numbers. The kernel recognizes any single device by the major-minor number combination encoded in the device special files.

For a dynamically loadable driver, the system assigns a major number to the driver when it first registers with the system. For a statically linked driver, the system assigns a major number to the driver during system boot. The system remembers assigned major numbers from system boot to system boot.

To have the system dynamically assign a major number to your driver, follow these steps:

To create device special files for your driver, follow these steps:

	NOTE: You may be able to use the ioscan and insf commands to install special files in the /dev directory. If no options are specified, insf creates special files for all new devices in the system. New devices are those devices for which no special files have been previously created. See the ioscan(1M) and insf(1M) manual pages for more details.

Testing a DLKM module consists of testing all of its functions under a variety of operating conditions and for both configurations: dynamically loadable and statically linked. Debugging a module is largely a process of analyzing the code to determine what could have caused a given problem.

The most common debugging technique is monitoring the kernel code using the printf() function to print statements included in the module source code so that you know what your module is doing during run-time operation. There are also debuggers such as Q4 to examine the kernel code. See the Q4 description file in the /usr/contrib/doc directory for more detail.

You can use a combination of C preprocessor macros, conditional compilation, and control variables to turn printf() messages on or off. The sample template character driver, “dlclass.c”, includes a control variable named dlclass_debug to determine whether or not to generate the messages. All of the messages can be disabled at once by using kmtune -s to change the value of dlclass_debug to 0 before configuring dlclass.

Correcting errors in the DLKM module is largely a process of analyzing the code to determine what could have caused a given problem. Some common errors are as follows:

Use the -u option of the kminstall command to update a DLKM module's components in the subdirectories of /usr/conf and /stand. Use this command when you modify one or more of the module's component files. If you modify the module source code, you must first recompile the source code in accordance to “Creating the Module's Component Files”.

kminstall expects to find the module's component files in the current directory. If module_name already exists on the system, kminstall updates the module. If module_name does not exist on the system, kminstall prints a warning and adds the module's components to the system.

When updating an existing module, kminstall takes the values of the tunable parameters and the $CONFIGURATION and $LOADABLE flags from the module's current system description file and saves them to the module's new system description file.

To update a DLKM module's components, follow these steps:

kminstall copies the module's component files to the appropriate locations, thereby overwriting the same named files at those locations.

Use the -d option of the kminstall command to remove a DLKM module's components from the system. If the specified module is listed in the /etc/loadmods file, kminstall prints a warning message and removes the module entry from /etc/loadmods. If the specified module is currently loaded, kminstall tries to unload the module. If the unload fails, it prints a message and exits with an error; otherwise, kminstall tries to unregister the module. If the unregistration fails, kminstall prints a message and exits with an error.

To remove a DLKM module's components, execute the following kminstall command:

   /usr/sbin/kminstall -d module_name

kminstall deletes the module's components from the /usr/conf and
/stand subdirectories.

	NOTE: kminstall does not delete the original component files for the module.

If the removed DLKM module is statically linked into the running kernel, you will have to execute config -u to reconfigure the kernel once the module has been removed. Then you will have to shutdown and restart the system for the new configuration to take effect.