TLVC Technical design notes

A collection of notes, musings and random thoughts while developing TLVC. Like the OS itself, these notes are continuously evolving. You're welcome to contribute.

About raw (char) disk/floppy IO

Raw IO (aka 'direct IO' or 'character IO') provides direct access to storage devices, bypassing the buffer system. This is important functionality for maintenance (such as creating or checking filesystems, diagnosing physical media, performance testing), for backup (copying entire volumes), for performance testing and when using disk partitions for non-file system purposes (such as database systems). Generally speaking, the buffer system is for file system access, raw IO is for device access. All device access except mount/umount should use the raw devices even though block access would work. Typical commands that should default to raw devices are fsck, mkfs, fdisk.

In his famous "The Design of the UNIX Operating System" Maurice J Bach writes about raw device access (pp328): "The advantage of using the raw interface is speed, assuming there is no advantage to caching data for later access. Processes accessing block devices transfer blocks of data whose size is constrained by the file system logical block size. For example, if a file system has a logical block size of 1K bytes, at most 1K bytes are transferred per I/O operation. However, processes accessing the disk as a raw device can transfer many disk block during a disk operation, subject to the capabilities of the disk controller. Functionally, the process sees the same result, but the raw interface may be much faster."

In the TLVC case, the file system block size is indeed 1K, and Bach's observations are relevant. However, speed is the least important of the reasons to have a raw interface. On a busy system, avoiding cache pollution when copying disks (for example) would be a big thing, but a TLVC system will rarely be busy in that sense.

The notion of a 'character device' may seem illogical when referring to storage devices which at the lowest level always use blocks for physical storage. The term is historical and 'raw' is more appropriate. On the other hand, reading or writing a single character (or byte) from/to a storage device via the 'character device' is supported and works fine.

In short, the primary goal for the raw drivers is to provide direct access without bloating the kernel. Second to that is speed, to take advantage of whatever options are available in (standard) hardware to maximize IO performance. Since the raw drivers are closely coupled to the block drivers, speed enhancements in one may positively affect the other.

The raw device drivers are optional via menuconfig and may be deselected for memory starved environments. The drivers are included in the default configuration.

For more about the raw drivers, see the development notes Wiki.

About the TLVC buffer system

TBD