This content originally appeared on DEV Community 👩💻👨💻 and was authored by Maria 🍦 Marshmallow
In the modern world, we are surrounded by a huge number of electronic devices of varying degrees of complexity. If the device is more or less complex, for example, a TV, router, or smartphone, then with a high degree of probability it is running the Linux operating system, and this thought haunts me.
What haunts me even more is the fact that all the kernels of the Linux operating system that run on various devices and servers are built from the source code located in the repository on the kernel.org website.
Such different devices, and the operating system running on them is assembled from the same source code! This statement is, of course, only partly true, since in fact the kernel is usually extended and modified by the developers of specific Linux distributions as well as the developers of specific devices, but there is a lot of common source code.
I've always wanted to build the Linux operating system myself from source code, but the process has always seemed complicated and confusing, and there's a lot I don't understand. But still, at a certain point in time, I accumulated enough knowledge to fulfill my dream. In this post, I'll show you how to compile and run minimal Linux from source on your computer.
It will not allow you to use all the features of your computer, but it will have the main thing — a command-line interface. Trust me, getting a working Linux command-line interface on your real computer will be an incredible experience.
Surprisingly, the minimum set required to obtain the Linux command line only consists of two files: the Linux kernel file and the root-filesystem initial image file. Obviously, a bootloader is needed that will load these two files and initiate the execution of the kernel, with passing it the image of the initial root filesystem and other parameters, if necessary.
Minimal Linux OS
In order for you to somehow be able to work with the operating system, you need four components:
Bootloader: a special program that allows the processor to start executing machine instructions located in the kernel file of the OS;
Kernel: the program code that contains:
- abstractions for the various physical I/O devices that the processor can work with (device drivers),
- abstractions of data structures for storage (file systems),
- abstractions for the time separation of program instructions (processes, threads),
- other abstractions.
Because of the kernel, the developer of application programs often doesn't care which video card, keyboard, or hard drive is installed on the computer. He or she simply writes code that works with I/O devices, processes, files, sockets, etc.
Initial root filesystem: is needed in order for the kernel to perform the initial loading of files necessary for further loading of the OS. The most important are the Linux kernel modules that are not included in the kernel file but are needed for further loading and the file on the basis of which the very first process will be created when the OS boots (init).
Set of utilities that is an interface to the OS kernel: allows you to work with abstractions found in the kernel of an OS. Depending on the complexity and purpose of the device on which the operating system will run, this set may vary. Utilities define the functionality and user interface. For example, a router will have one; a phone will have another; and a personal computer will have a third.
Loading the Linux operating system
The boot of the Linux operating system may differ on different architectures and computers, but for the x86 architecture, the boot looks like this (in most cases):
- The computer turns on.
- The BIOS or UEFI finds the OS bootloader on the computer and transfers control to it.
- The OS bootloader loads the Linux kernel file and the initial filesystem image file (the
initrdfile) into RAM. - The OS bootloader transfers control to the Linux OS kernel.
- The OS kernel performs initialization.
- The OS kernel accesses the files that are in the initial filesystem image (mounts the image).
- The kernel looks for an
initfile in the initial filesystem and starts the very first user process based on it. - The
initprocess mounts an already persistent filesystem, continues to initialize the OS, transfers the root of the Linux filesystem to the mounted filesystem, and starts other processes that are needed for initialization.
Linux distributions
A distribution is a Linux kernel and a set of libraries, utilities, and programs that are installed on a computer or device.
At the moment, the number of different distributions is huge. You can view a list of them on the DistroWatch.
Modern Linux distributions are usually distributed in the form of ISO images and allow you to install updates and additional programs (packages), but we are making a minimal distribution, so we will not have such an opportunity.
One of the most complete instructions on how to build a Linux distribution from scratch is here. Building a Linux distribution is an interesting process that will allow you to learn a lot of new things, but it is very time-consuming, which means you need a lot of willpower to carry it out from start to finish.
I tried to simplify the creation of the distribution kit to a minimum: we will not mount a permanent filesystem, but as an init file, we will use a script file that will perform minimal initialization and start the sh shell.
OS bootloading
Over the years of its existence, Linux has been ported to many hardware platforms. The Linux boot for each platform is different. For x86, the download may differ as follows:
- Will it be used to boot BIOS or UEFI?
- On which storage device the BIOS or UEFI will look for the bootloader (hard drive, flash drive, optical drive, computer network)?
- How the hard disk or flash drive is marked (MBR or GPT)?
- On what media and in what file system (FAT, NTFS, EXT, CDFS, etc.) will the kernel file and the file with the image of the initial root filesystem, called
initrd, be located?
The structure of the initial root filesystem
The initial root file system contains the minimum number of files and directories necessary for further Linux operation. In our case, these are the bin, dev, proc, and sys directories. The bin directory contains utilities for working with the Linux kernel.
Utility kits
Minimal Linux is a kernel and a set of command-line utilities. The kernel and command-line utilities are developed by different teams of programmers.
The most common sets are:
- GNU Core Utils;
- BusyBox.
Due to its simplicity and low disk space, BusyBox is often used on embedded devices. I will use it for simplicity.
Creating a Linux Build Environment
Paradoxical as it may sound, Linux is usually compiled into Linux. To do this, you need a Linux OS, which contains programs that allow you to build the Linux kernel and a set of utilities for building.
For example, on Ubuntu 22.10, we need to install the following packages: make, build-essential, bc, bison, flex, libssl-dev, libelf-dev, wget, cpio, fdisk, extlinux, dosfstools, and qemu-system-x86.
Note that for other systems, the set of packages may differ.
Building Minimal Linux on Ubuntu 22.10
- Install the assembly-required packages.
$ cd ~
$ mkdir -p simple-linux/build/sources
$ mkdir -p simple-linux/build/downloads
$ mkdir -p simple-linux/build/out
$ mkdir -p simple-linux/linux
$ sudo apt update
$ sudo apt install --yes make build-essential bc bison flex libssl-dev libelf-dev wget cpio fdisk extlinux dosfstools qemu-system-x86
- Download the source code for the Linux kernel and BusyBox from the Internet.
$ cd simple-linux/build
$ wget -P downloads https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.79.tar.xz
$ wget -P downloads https://busybox.net/downloads/busybox-1.35.0.tar.bz2
- Unpack archives with source code.
$ tar -xvf downloads/linux-5.15.79.tar.xz -C sources
$ tar -xjvf downloads/busybox-1.35.0.tar.bz2 -C sources
- Build BusyBox binaries for the Linux kernel as well. This process will take some time, about 10 minutes or even more, so do not be alarmed.
$ cd sources/busybox-1.35.0
$ make defconfig
$ make LDFLAGS=-static
$ cp busybox ../../out/
$ cd ../linux-5.15.79
$ make defconfig
$ make -j8 || exit
$ cp arch/x86_64/boot/bzImage ~/simple-linux/linux/vmlinuz-5.15.79
- Create an
initfile.
$ mkdir -p ~/simple-linux/build/initrd
$ cd ~/simple-linux/build/initrd
$ vi init
Instead of the vim editor (vi command), you can use another text editor such as gedit.
init file:
#! /bin/sh
mount -t sysfs sysfs /sys
mount -t proc proc /proc
mount -t devtmpfs udev /dev
sysctl -w kernel.printk="2 4 1 7"
/bin/sh
poweroff -f
- Create the structure of directories and files.
$ chmod 777 init
$ mkdir -p bin dev proc sys
$ cd bin
$ cp ~/simple-linux/build/out/busybox ./
$ for prog in $(./busybox --list); do ln -s /bin/busybox $prog; done
- Place the structure in the
initrdfile, which is ourcpioarchive.
$ cd ..
$ find . | cpio -o -H newc > ~/simple-linux/linux/initrd-busybox-1.35.0.img
- Launch the built image in the QEMU emulator.
$ cd ~/simple-linux/linux
$ qemu-system-x86_64 -kernel vmlinuz-5.15.79 -initrd initrd-busybox-1.35.0.img -nographic -append 'console=ttyS0'
- Let's try to enter the Linux commands you know. Exit the emulated Linux by typing the exit command.
Create a boot image for a flash drive
If we want to run our Linux on real hardware, then probably the easiest way is to create an image to put on a boot drive and burn it. Creating such an image and booting from the drive is, in my opinion, the most difficult process and requires more advanced knowledge from you.
When creating an image, there are several decisions to be made:
- What will initiate the boot (BIOS or UEFI)?
- Which drive will you be using (CDROM, flash drive, or hard drive)?
- How will you partition the drive (MBR, GPT), and will you partition it at all?
- Which bootloader will you use?
- What file system will be used and where the Linux and bootloader files will be located?
I used a flash drive with the MBR and EXTLINUX bootloader installed and one FAT32 partition where the files reside. The BIOS started the boot process for me (the Legacy boot option if your computer has a UEFI BIOS).
The algorithm for creating a bootable flash drive image is as follows:
- Create an image file.
$ dd if=/dev/zero of=boot-disk.img bs=1024K count=50
- Create a boot partition inside the image file.
$ echo "type=83,bootable" | sfdisk boot-disk.img
- Set up a loopback device on a boot partition inside the
boot-disk.imgfile.
$ losetup -D
$ LOOP_DEVICE=$(losetup -f)
$ losetup -o $(expr 512 \* 2048) ${LOOP_DEVICE} boot-disk.img
- Create a filesystem on the loopback device.
$ mkfs.vfat ${LOOP_DEVICE}
- Mount the loopback device.
$ mkdir -p /mnt/os
$ mount -t auto ${LOOP_DEVICE} /mnt/os
- Copy the Linux kernel file and the
initrdfile to the first partition inside theboot-disk.imgfile.
$ cp vmlinuz-5.15.79 initrd-busybox-1.35.0.img /mnt/os
- In the
boot-disk.imgfile, place the EXTLINUX bootloader.
$ mkdir -p /mnt/os/boot
$ extlinux --install /mnt/os/boot
- Create a configuration file for the bootloader, in which we indicate what exactly to load.
$ echo "DEFAULT linux" >> /mnt/os/boot/syslinux.cfg
$ echo " SAY Booting Simple Linux via SYSLINUX" >> /mnt/os/boot/syslinux.cfg
$ echo " LABEL linux" >> /mnt/os/boot/syslinux.cfg
$ echo " KERNEL /vmlinuz-5.15.79" >> /mnt/os/boot/syslinux.cfg
$ echo " APPEND initrd=/initrd-busybox-1.35.0.img nomodeset" >> /mnt/os/boot/syslinux.cfg
- Unmount the loopback device.
$ umount /mnt/os
$ losetup -D
- Install the MBR bootloader at the beginning of the disk inside the
boot-disk.imgfile.
$ dd if=/usr/lib/syslinux/mbr/mbr.bin of=boot-disk.img bs=440 count=1 conv=notrunc
The boot-disk.img file will contain the boot image of the flash drive.
Using Docker to build Linux
The algorithms described above contain many commands and parameters; it is easy to make a mistake when typing. Commands can be combined into bash scripts, and in order to be able to build Linux on the Windows 10 or 11 operating system, it is rational to use Docker Desktop.
The essence of Docker is as follows:
- In a Dockerfile, you describe the environment structure for your program or script.
- Using the Docker utility, based on the Dockerfile, you create an image of this environment in a specific format.
- Using the same utility, you can launch an image-based instance of your program or script that runs in an isolated environment and is called a Docker container in Docker terminology.
- The images you create can be stored in a repository and reused. Docker containers created from the same image will run identically on all computers that can run them.
- A Dockerfile is easy to read and learn and easy to distribute.
Below are the contents of the Dockerfile:
FROM ubuntu:22.10
RUN apt update && apt install --yes make build-essential bc bison flex libssl-dev libelf-dev wget
RUN apt install --yes cpio fdisk extlinux dosfstools qemu-system-x86
RUN apt install --yes vim
ARG APP=/app
ARG LINUX_DIR=$APP/linux
ARG FILES_DIR=$APP/files
ARG SCRIPTS_DIR=$APP/scripts
ENV BUILD_DIR=$APP/build
ENV LINUX_DIR=$LINUX_DIR
ENV FILES_DIR=$FILES_DIR
ENV LINUX_VER=5.15.79
ENV BUSYBOX_VER=1.35.0
ENV BASH_ENV="$SCRIPTS_DIR/bash-env/env"
COPY ./scripts $APP/scripts
COPY ./files $APP/files
RUN mkdir -p $LINUX_DIR
RUN ln -s $APP/scripts/start-linux.sh /usr/bin/start &&\
ln -s $APP/scripts/build-linux.sh /usr/bin/build &&\
ln -s $APP/scripts/build-image.sh /usr/bin/image
WORKDIR $APP/scripts
CMD build
Let's take a quick look at the main commands our Dockerfile:
The FROM command is the most important one; it specifies which filesystem image our Linux build image will be based on. In this case, it is Ubuntu 22.10.
The RUN command runs commands inside the image we are creating. That is, the commands that follow RUN will be executed as they would be if you were running Ubuntu 22.10 and executed them on the command line. As a result of the command, your file system image will change since these commands change the file system inside it.
The COPY command copies files from the file system of our OS into the created image. Like RUN, it modifies the file system within the image.
The ARG and ENV commands are confusing. I don’t know if I’ll make it clear or not, but ARG is the creation of variables that are used when creating an image, and ENV is the creation of variables that are used when a container has already been created based on this image, and these variables will be visible inside it.
The WORKDIR command specifies which directory will be working when starting a container created based on our image.
The CMD command specifies which command will be executed by default inside the container when it is started.
Running and building minimal Linux with Docker
You can experiment with my project. On Windows, it's best to run Docker in PowerShell.
- Create a Docker image.
$ git clone https://github.com/artyomsoft/simple-linux.git
$ cd simple-linux
$ docker build --build-arg APP -t simple-linux .
- Running minimal Linux.
$ mkdir linux
$ cd linux
$ docker run -v ${pwd}:/app/linux --rm -it simple-linux build
The linux directory you created will contain the built Linux kernel file and the initial rootsystem image file.
- Create a boot image for the flash drive.
Note that you need to use the
--privilegedoption in Docker as the image script uses a loopback device.
$ docker run -v ${pwd}:/app/linux –-privileged --rm -it simple-linux image
If you use Docker Desktop for Linux, Docker will need to be run using sudo, and ${pwd} will need to be used instead of $(pwd).
Burning a boot image for a flash drive to media
The created image file for the flash drive (linux-5.15.79-busybox-1.35.0-disk.img) can be written to the flash drive using the Win32DiskImager utility.
It should be noted that when writing, you will lose all the data stored on the flash drive, so it is better to use a drive that does not contain any files!
After writing the image to the flash drive, restart your computer and choose to boot from USB-HDD, i.e., from the flash drive you created. Most likely, before that, you will need to select "Legacy Boot" and disable "Secure Boot" in the BIOS.
Conclusion
In this post, I have provided detailed instructions on how you can get a working Linux system from source code.
To some, this post may seem very simple and not worthy of attention. But, in order not to scare you away with details, I did not delve into topics such as BIOS, UEFI, file systems, bootloaders, the glibc library, the detailed operating system boot process, various specifications, dynamic and static linking, Linux kernel modules,... I just gave the minimum amount of theory that will allow you to understand what, in fact, you did and understand the topic much faster than me, without collecting all the information bit by bit.
You will hardly use the resulting operating system in the future, but I hope that your abstract knowledge of Linux will turn into understanding and skill for you.
Was this post helpful? Have you ever tried to build a minimal Linux OS from the source code? Share your experience in the comments!
Btw, you can support my work by buying me a coffee! I'll leave here a few links for you:)
You can also support me on Coinbase
This content originally appeared on DEV Community 👩💻👨💻 and was authored by Maria 🍦 Marshmallow
Maria 🍦 Marshmallow | Sciencx (2023-02-15T16:05:18+00:00) Building a minimal Linux OS from source code 🏗. Retrieved from https://www.scien.cx/2023/02/15/building-a-minimal-linux-os-from-source-code-%f0%9f%8f%97/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.