Linux Lexicon — How Does Linux Kernel Work?

Short Bytes: The Linux kernel is the most used piece of software in the history of computing. It’s everywhere. It’s in computers, servers, phones, TVs, set-top boxes, submarines, airplanes, self-driving cars, and it even runs the International Space Station. But how does it work? That’s more than a mouthful, so I’ll try to break it up into some more manageable chunks, here.

The Linux kernel is made up of over 21 million lines of code, each specifying their own important instructions. Because of the sheer volume of code in the Linux kernel, we would need an equally long article to fully describe how exactly each task is performed. To spare you the gritty detail, we’ll cover the basics in a very high-level fashion.

But, before discussing how any given operating system kernel works, we should first look at what a kernel is responsible for. The kernel provides abstractions for the programmers that develop applications for the kernel’s platform. These abstractions are simplifications for accomplishing complex tasks.

The kernel itself isn’t the only piece of software in an operating system that provides abstractions, but it’s surely one of the more important sets of abstractions–another being the abstractions provided by hardware drivers. Hardware drivers speak the language of the kernel so that the kernel doesn’t need to know how to speak to every single piece of hardware ever created. This is what allows a single kernel to run on many different brands and models of hardware.

So, when talking about abstractions, it’s important to know exactly what is meant by the term. Much like how in art an abstraction or abstract piece is meant to represent something that is not an inherent or innate quality, an abstraction in computer science is a way of hiding the sometimes overwhelming amount of detail behind a process.

Take, for example, reading and writing variables in RAM. Each motherboard can have a wildly different memory controller, your computer might run an ARM, Intel, AMD, SPARC, POWERPC, or MIPS CPU, but that should not matter to your Hello World program, and it doesn’t, because the kernel abstracts the difference in hardware in the form of a uniform interface.

This interface looks to programmers like any other function call but is special because it’s a system call. A system call is just a function that requests something from the kernel, this is where the kernel will carry out the request regardless of the underlying hardware. The Linux kernel implements the POSIX standard of systems calls.

Now, we need to know what the Linux kernel is responsible, that is, what abstractions should we expect any modern operating system kernel to provide for us?

• Data Storage
  • Random Access Memory – Reading and Writing Variables and Data in Memory
  • Permanent Storage – Reading and Writing files on Permanent Storage Devices
  • Virtual Filesystem
• Network Access – Sending and Receiving Data Over a Computer Network
  • Physical Media Agnostic (Ethernet, Wireless, LTE, Dialup)
  • Partially Protocol Agnostic
• Task Scheduling
  • CPU Time Sharing
  • Load Balancing and Prioritizing
• Device Protocols (USB, FireWire, Serial, Parallel)
  • USB Removable Media
  • Webcams
  • Mouse and Keyboard
• Security
  • User and Group Permissions
  • Resource Access Permissions

The Linux kernel makes development easier by providing the above services in the form of a variety of system calls. We’ll take a closer look at a few of them and how they make development easier and more productive.

Data Storage

There are two forms of data storage, namely, temporary storage and permanent storage. Temporary storage, which may not be obvious to some readers, is RAM. Not everything that goes into RAM must be permanently stored. A good example of this is when you browse the internet, you don’t want every single web page you visit to be permanently stored on your computer. Permanent storage, more obviously, is your hard drive or SSD/flash storage.

The Linux kernel provides completely transparent reading and writing of data in RAM, regardless of the hardware platform. It doesn’t matter if you’re running Linux on an older Intel i386 computer or on the newest ARM-based Android phone, the code does not have to be altered before it’s compiled to ensure compatibility, and this is a very powerful idea. Keep in mind, though, that the Linux kernel is what’s modified in order to support different hardware platforms, and it’s the modifications of the kernel that enable it to provide the uniform interface despite the not-so-uniform hardware.

Furthermore, the kernel isolates each process in its own memory space which means the process does not need to know what memory belongs to it because all memory visible to the process does belong to it. This memory partitioning also increases security at no extra cost to the developer.

Very similarly, with permanent storage, the kernel hides the differences of communicating with SATA, PATA, SCSI, USB, M.2, and other storage protocols and allows a single program to read and write files on any medium connected using any supported protocol and any supported filesystem without any changes to the program. And that puts a massive amount of power in the hands of both the developer and the user. This also increases code reusability and developer productivity because there is no need for specialized code.

Network Access

When it comes to networking, it’s a little different because every protocol has its own address format, so there is an inherent need for specialized code to support each networking protocol. Thankfully, only IPv4 and IPv6 are commonplace. As for the many other protocols that have come to pass like DECnet, IPX, and AppleTalk, there is Linux support, but it’s hardly beneficial to support these protocols in any modern application.

Back to IPv4 and IPv6, the two have very different IP address forms, but this is just as much an asset as it is an inconvenience. The type of protocol needed can easily be inferred by the IP address. Additionally, the kernel provides support for TCP, UDP, SCTP, and ICMP, all of which can easily be used by making system calls. It doesn’t matter if your computer is connected using the Ethernet protocol, LTE, or Dialup, the system calls are still the same.

Imagine needing different versions of Chrome or Firefox depending on whether you’re using WiFi or Ethernet, it would be much more cumbersome for developers. This abstraction, again, is very powerful and provides a flexibility that increases developer productivity and end-user flexibility.

Task Scheduling

Task scheduling is a very large topic that happens to be very opinionated. So, we won’t bother talking about scheduling algorithms and we’ll just talk about the responsibilities of the kernel with respect to ensuring each process gets their turn on the CPU, even when there’s hundreds of them.

Before the dawn of multicore CPUs, computers could actually only do a single thing at any given time. Each process was given its fair share of time being computed one by one, but this was done so quickly that it created the illusion of concurrently running processes. Before multicore CPUs, computer manufacturers would actually put more than one CPU on the motherboard to allow for more than one process to be run at a time. This is still done today, but with multicore CPUs, and even with hyperthreading which allows two processes per core on some Intel CPUs (this leads to systems that can support over a hundred threads at a time).

Every process wants time on the CPU, and the kernel is what makes sure that everyone gets their turn in a timely manner. Beyond this, some processes require delays, maybe while waiting for I/O, or a game waiting for a timed interval, and this can also be assisted by the kernel.

Instead of the process occupying the CPU while it waits, another process can be run and the original process can be returned to after the required time, thereby increasing overall performance. In general, the task scheduling means that a developer does not need to worry about the other processes running on the computer, they only need to worry about their program.

The number of abstractions in the Linux kernel is probably too large for any single (or sane) person to attempt to recite. And while the number of abstractions provided by the Linux kernel might be overwhelming, the amount of detail any developer would need to know in order to implement an application without them would be orders more so. Because of that sheer number of abstractions that occur in the Linux kernel, we simply just can’t cover them all. But, if you have any requests of specific abstractions that you’d like covered in more depth, we’d love to oblige, let us know in the comments below.