OSXFAQ - Technical News and Support for Mac OS X:

I Thought Mac OS X Was Supposed To Be The End Of Extensions Hell ??

by Dr. John Timmer, Contributing Editor
Question: I thought Mac OS X was supposed to be the end of extensions hell, but I still see a folder called Extensions in the System's Library Folder. What's the deal?

Short answer: Yes, they are extensions, but these are fundamentally different. They're extensions to the kernel itself, and are used in a way that should prevent conflicts.


Long Answer (with a few more questions thrown in):

Q: Just what is a Kernel extension?

We all know how troublesome extensions were in Mac OS X. By attempting to load code into the OS themselves, they often lead to conflicts and crashes. Apple promised to leave all that behind in Mac OS X, but if you dig around the System folder of Mac OS X, you'll still find an Extensions folder. What resides inside, however, is nothing like you've been familiar with. Hang on while I describe what they are, how they work, and how they affect the stability of Mac OS X.

Answer:

One of the kernel's jobs is to allow the software running on it to communicate with its underlying hardware. In many Unix based OS'es, the kernel would have to contain all the code necessary for this communication. Considering the variety of hardware a desktop OS will come in contact with, that could mean a lot of memory wasted. It also might not work well with USB and Firewire devices, which may appear and disappear at fairly random moments. In order to use memory more efficiently, Mac OS X (and several other modern Unixes) allows code such as device drivers to be loaded into the kernel dynamically.

Since Apple had to generate this mechanism for altering kernel behavior anyway, they also chose to use it for other forms of kernel communication, such as adding file system and networking capabilities. In short, anything that alters fundamental OS behavior or capabilities but doesn't affect the GUI is likely to require a kernel extension. Most of the kernel extensions you'll see, however, are designed for communications with hardware.

If kernel extensions are so useful and flexible, why aren't they used for a lot more? Unfortunately, just like the extensions of Mac OS X, they come with a price in terms of stability. Mac OS X's stability is largely due to its memory protection, a function that is provided by the kernel. The flipside is that, when code is loaded into the kernel itself, it's able to overstep the memory protection that the kernel provides. Thus, Apple advises developers to stay out of the kernel if at all possible. That means finding a way to provide system wide services without resorting to a kernel extension if at all possible.


Q: How do kernel extensions get loaded during booting? I don't see any rows of dancing icons...

If kernel extensions are required for all hardware communications, how does the kernel even get started without them?

Answer:

Some basic hardware information is acquired early in the boot process by the machine's firmware, but only enough to find a startup disk and locate the operating system on it. The firmware then hands over the booting process to a program called BootX, which (among other things) loads the kernel extensions. Most of its work is done before Mac OS X's Core Graphics code is loaded, though, so drawing dancing icons isn't an option.

BootX's extensions loading has two features that have been added in order to speed the loading process and, correspondingly, the boot time. The first is that the OS caches a lot of the kernel extension information in a single, structured file that can be loaded into the kernel quickly. All kernel extensions contain a property list that includes a flag for whether the hardware it supports is essential for booting or not. During the first boot, all those marked as essential are put into a cache file (it can be found in System:Library, called "Extensions.mkext"). The OS continues to use that file until its modified date is different from that of the Extensions folder itself; at that point, it reverts to loading them from the Extensions folder and then re-creates the Extensions.mkext file.

The second is that the OS loads first and asks questions later. In other words, BootX loads every extension without determining whether the hardware it supports is present or not (more on that below). No time wasted making decisions makes for fast booting.



Question: Isn't loading every kernel extension a bit inefficient? How does the kernel know which are actually needed?

Early in this article, I mentioned that having the kernel have all the code necessary for any possible hardware is a pretty good way to waste memory space. Then i turned around and said that the OS loads them all anyway. Believe it or not, both are true!

Answer: One of the first processes to be launched after the Kernel itself is the Kernel Extension Daemon or kextd, which handles their loading and unloading (if you check your machine now, you'll find it's still running). One of kextd's first tasks is to determine which extensions are actually needed. To do this, it starts doing what's called hardware matching. Each kernel extension contains information as to what sort of hardware it can support. The kextd checks this information against the actual hardware present; if a given kextd doesn't have any hardware that it supports, then it's unloaded, freeing up memory.

Hardware matching is actually a bit more complicated, though. Apple uses both ATI and NVidia video cards, and the extensions folder contains drivers for each - how does the OS know which one to use on a given machine? It turns out that there are several levels of matching that go on. On a generic level, a driver may support a class of hardware, such as a video card or a hard drive. It may also enable features for a more specific type of hardware, such as a firewire hard drive. Finally, the driver can support very specific features, such as those provided by drives using the Oxford Firewire bridges. The kextd tries to find this last, specific match first. If it fails to find that, it looks for something less specific, and finally defaults to a basic driver, such as one for all firewire disk devices. If a driver never winds up matching any of the existing hardware, or can only match with hardware that's already made a better match, it gets unloaded. Not only does this careful matching process save on memory used by the kernel, but it should eliminate the possibility of two extensions trying to perform the same function, thus taking care of extensions conflicts.



Question: If an unused kernel extension gets unloaded, how come something still manages to recognize a firewire drive when it's plugged in?

There's a related question that provides the answer to this one: why is kextd still running three weeks after the boot process is complete?

Answer: The best thing about kernel extensions is that they're dynamic - they can be loaded or unloaded at any time. kext hangs around waiting for new hardware to show up. If and when it does, it goes through the hardware matching process all over again, allowing things such as USB CD burners and Firewire hard drives to be recognized whenever they get plugged in.



To sum up, there are still extensions around in Mac OS X, but they are unlikely to be the cause of instability. That's because their role has been reduced to extending the capacity of the kernel to communicate with hardware and software devices such as graphics cards and network interfaces. In addition, a carefully designed matching system helps prevent two extensions from conflicting.