Contents: Strategies and advice Ingredients and preparation Building the Linux kernel Building uClibc Building BusyBox Building Util-linux Building E2fsprogs Building LILO Creating the boot disk Creating the root disk Creating the supplemental disk Installing Linux Conclusion Resources About the author Rate this article Related content: Linux system development on an embedded device Embedded Linux applications: An overview Dual-booting Linux Subscriptions: dW newsletters dW Subscription (CDs and downloads)

Todd E. Sundsted (todd-p2p@etcee.com)
Vice president, Etcee LLC
1 October 2002

Hardware is only as old as the software it runs: a modern operating system and up-to-date applications return an older system to productivity. This article provides best practices and step-by-step guidance on how to build a working Linux system on older hardware or on modern hardware with limited memory and storage.

It's a conspiracy!

Too often, modern operating system vendors treat hardware as if it were disposable -- use it for a year and then throw it away. One might be tempted to believe that secret backroom meetings are going on between vendors of operating systems and computer hardware manufacturers. New operating systems and applications demand the latest, most powerful hardware. The newest hardware works best only with the latest, most feature-rich software. I'm sure the churn helps someone's bottom line, but it does nothing for mine.

I refuse to bow to the conspiracy. When I came across a mint condition IBM ThinkPad 755C in a local garage sale, I realized that I had an opportunity to make a point. Hardware is only as old as the software that it runs.

In this article I will describe how I brought the laptop back to life; and I will describe what you need to do if you find yourself in a similar situation. The usefulness of this material isn't limited to rescuing old laptops from the dump. You might need to build a Linux system to run on hardware with resource constraints, such as in an embedded application. The approach is the same.

Strategies and advice
I planned to use the laptop for writing and for remote access to my more powerful desktop development system. Therefore, I needed a system with network support, a shell, a text editor like vi, CVS for versioning my documents, and SSH for secure remote access. On the resource side, I had 12 MB of memory and a 540-MB hard drive to work with.

Since I was heading into territory I had not explored fully, I chose a path with many small milestones. I bring that approach to this article. Frequent milestones helped me determine whether or not I was making progress solving the problem; frequent milestones will help you debug the process when a step doesn't work as advertised. At several points in this article I encourage you to take your system for a spin, even though the system is not yet complete. These tests will help your judge whether or not your work up to that point is correct.

My first bit of advice is to forget about using any of the established Linux distributions. I had originally hoped to avoid starting from scratch by installing an existing Linux distribution. I quickly learned that they all require more resources than were available. Even Slackware, long known for its support of low memory systems, needs at least 12 MB to install the distribution and even then it's a very tight fit.

One possibility that you might want to consider before embarking on this journey is to start with and then extend one of the existing "single disk" Linux distributions like tomsrtbt (a single disk rescue floppy distribution), Trinux (a single disk Linux security toolkit), or even Tin Foil Hat Linux (a secure Linux distribution designed for encrypting, signing, and wiping files).

I looked at several of these mini-distributions before striking out on my own and I learned a lot in the process. While none of the distributions that I looked at were specifically oriented toward resource-constrained hardware like mine, they did share a common goal -- cramming as much functionality as possible into as small a space as possible. While many of the trade-offs they made to achieve this goal were not acceptable to me, I was introduced to both uClibc and BusyBox, both of which I ended up using.

uClibc
Every Linux system needs a C library. The C library provides common file operations (open, read, write), memory management operations (malloc, free), and many other functions that make a Linux system a Linux system. Most Linux systems use Glibc. Glibc is mature, well tested, and actively being developed. Unfortunately, it also uses an unacceptable amount of memory.

Rather than use Glibc, I recommend uClibc, a modern, stable, highly compatible replacement for Glibc. uClibc was developed for embedded systems and is therefore intended to be complete but lightweight. Trade-offs between speed and size were decided in the direction of size. In spite of this, in almost all cases, applications and tools compiled against uClibc are indistinguishable from those compiled against glibc. The uClibc Web site (see Resources) lists over one hundred applications that are known or have been reported to work with uClibc. The list includes standard utilities like Gzip and Lilo, as well as slightly less common utilities like Lame (an MP3 encoder) and Freeswan (a VPN implementation). Since I measure the usefulness of a system by the maturity of the tools and applications that run on the system, it was important to me that the C library I chose be capable of supporting the tools I needed.

Dietlibc is another alternative to Glibc. My research indicated that while Dietlibc requires less memory that uClibc, it realizes this gain by sacrificing compatibility with Glibc. In the end, I went with uClibc because it appeared to support more of the applications that I needed to use.

BusyBox
Users of Linux systems demand a work environment filled with a rich collection of command-line tools. Since building all of the necessary tools one-by-one looked like a time-consuming chore, I once again pulled a page from the embedded systems playbook and selected BusyBox, a jack-of-all-trades application that provides implementations of most of the tools you expect to see in a Linux environment -- in one binary.

By using BusyBox I conserved precious resources and saved myself a huge investment of time. I found nothing else like it.

Ingredients and preparation
To proceed, you will need access to an existing Linux system -- preferably one running on the same microprocessor family as your target system. An older laptop running Linux is a great environment for writing and editing, but it cannot compete with modern hardware when it come to compiling software packages. We will use the existing Linux system to build the kernel and the supporting software, and to create the disks necessary to boot the target system into Linux. Once we have Linux installed on the target system, we will continue to use the existing system to build applications before transferring the finished product to the target system. I will refer to the existing Linux system as the "build system" and the other Linux system as the "target system".

Begin by downloading the following packages (see Resources for links):

• linux-2.4.19.tar.gz (the kernel)
• uClibc-0.9.15.tar.gz (the C library)
• busybox-0.60.4.tar.gz (useful command-line tools)
• util-linux-2.11u.tar.gz (the fdisk executable)
• e2fsprogs-1.27.tar.gz (for filesystem creation)
• lilo-22.3.2.tar.gz (the boot loader)

In order to build and link BusyBox with uClibc, you will need to install the uClibc development environment in the /usr directory of the target system, so you'll need write access to that directory.

Part of the process involves building a simple root filesystem. I have simplified this step by providing a suitable root filesystem skeleton, complete with the necessary configuration files and device files. You can download the file skeleton.tar.gz from Resources.

You'll need to read and digest the necessary documentation, including README and INSTALL documents. There are simply too many options and special cases to cover all permutations in this article. In the sections below I present my recommended configurations.

Let's build the kernel.

Building the Linux kernel
It's possible to build a working Linux kernel without changing the default configuration. A few well thought-out changes, however, will lead to a system that better suits our needs. In particular, I've kept in mind the requirement for network connectivity. The table below lists the options I changed and provides suggestions as to what you may want to change for your system.

Kernel configuration

Option	Comments
CONFIG_BLK_DEV_RAM=y	Ramdisk Support is necessary during the system boot.
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_BLK_DEV_INITRD=y
CONFIG_FAT_FS=y	These options make it easier to access MSDOS disks. They are optional but can be useful if you intend to dual boot the laptop or exchange floppy disks.
CONFIG_MSDOS_FS=y
CONFIG_VFAT_FS=y
CONFIG_PCMCIA=m	PCMCIA support (see Note below).
CONFIG_I82365=y
CONFIG_I82092=y
CONFIG_TCIC=y
CONFIG_NET_RADIO=y	Wireless support (see Note below).
CONFIG_HERMES=m
CONFIG_PCMCIA_HERMES=m
CONFIG_NET_WIRELESS=y
CONFIG_NFS_FS=y	NFS Support enables you to move files quickly between your target system and another system once basic networking support is in place.
CONFIG_NFS_V3=y
CONFIG_NFSD=y
CONFIG_NFSD_V3=y

Note: Laptop networking may require PCMCIA support. Older laptops won't support the newer Cardbus (32-bit) cards, but that's usually not a problem since there are still many 16-bit networking cards for sale. If you are planning on connecting to the network via wire you will need to select the appropriate drivers under "PCMCIA network device support," which is not included in the table above. I went the wireless route. Many 16-bit PCMCIA cards use the Hermes chipset. You may need to alter the configuration slightly to suit your PCMCIA controller chipset and PCMCIA card.

You should configure the kernel to specifically support your CPU. If you're building a kernel for an older machine on a new machine, you'll need to select the appropriate processor such as an Intel 386 or 486. In the interest of saving space you might what to disable everything you don't think you'll need, in particular SCSI support.

The following steps build the kernel:

1. make xconfig
2. make dep
3. make bzImage
4. make modules

I'll describe how to install the kernel later.

Building uClibc
Building uClibc is more challenging than building the kernel. The uClibc package builds two related components. The first component is the runtime libraries that will support your target system's utilities and applications. The second component is a development environment. The uClibc development environment makes it easy to build utilities and applications that use uClibc, even on a system that doesn't itself use uClibc. uClibc creates and installs wrappers for gcc and related tools. Once the uClibc development environment has been installed you can compile and link most applications against uClibc instead of glibc.

uClibc configuration

Option	Comments
DO_C99_MATH = true	Full math support
DOLFS = true	Large file support
INCLUDE_RPC = true	Remote Procedure Call (RPC) support

The three options in the table above make it possible to build and install software like SSH and NFS if you should later choose to do so. You also need to specify the location of the Linux kernel source.

The following steps summarize how to build the uClibc package. They assume you've extracted the archive into a directory named uClibc-0.9.15:

1. cd uClibc-0.9.15
2. ln -s ./extra/Configs/Config.i386 ./Config
3. Edit the Config file. Enable the options specified in the table above.
4. make

The instructions above assume you're building uClibc for a Intel microprocessor. If you're building uClibc for a different microprocessor, create a link to the appropriate Config file in step two, above.

The make command builds the package. In order to compile and link other utilities and applications with uClibc, you need to install the development environment. The make install command installs the development environment.

Once the development environment is installed, you can use the uClibc development tools in place of the standard Glibc-based development tools by changing the PATH environment variable, as follows:

export PATH=<path to dev environment>/usr/bin:$PATH

Once you have changed the PATH environment variable you will find that most development commands (gcc, ld, ldd, etc.) now point at uClibc wrappers. The commands should, however, still work the same. It's very important that you set the PATH environment variable correctly before building any of the software below.

The latest release of uClibc (version 0.9.15) won't build out of the box on my Redhat 7.3 system. The technique used to locate the gcc headers changed with the release of version 0.9.12. If uClibc won't build on your system, apply the uclibc patch supplied in the Resources section of this article. It ports the previous technique to the latest release of uClibc.

I will describe how to install the runtime environment on the boot and root disks later.

Building BusyBox
If there is one task that will simplify the process of building a Linux system from scratch more than any other it is building and installing the BusyBox package. BusyBox is a single executable that provides the functionality of many other common command line tools, all rolled into one. The documentation for BusyBox claims that all you need to build a working system is BusyBox and "/dev, /etc, and a kernel" -- and they're not kidding.

The table below describes that changes you should make to the Config.h file.

BusyBox configuration

Option	Comments
#define BB_HOSTNAME	Enable networking support.
#define BB_IFCONFIG
#define BB_PING
#define BB_ROUTE
#define BB_FEATURE_NFSMOUNT
#define BB_INSMOD	Enable module support.
#define BB_RMMOD
#define BB_FEATURE_NEW_MODULE_INTERFACE
#define BB_VI	Enable editing support.

Make sure you select ASH as your default shell -- it is the most complete Bourne-compatible shell available in BusyBox. It will run most shell scripts, including those necessary to set up networking.

If you enable NFS mount support, pay attention to the comment that says you must mount with the "-o nolock" option since you most likely won't be running the portmapper daemon.

You must also change the Makefile and enable large file support. Set DOLFS to true.

The make command builds BusyBox.

Building Util-linux
Util-linux is a collection of low-level system utilities. We're interested in the fdisk utility: we need it in order to partition the laptop hard drive.

Util-linux is easy to configure and build. Type configure and then make.

Building E2fsprogs
E2fsprogs is a collection of utilities for creating and manipulating filesystems in ext2 and ext3 format. Once fdisk has been used to partition a hard disk, the e2fsprogs utilities will be used to create a filesystem.

The following steps build the E2fsprogs package.

1. mkdir build
2. cd build
3. ../configure --enable-elf-shlibs
4. make
5. make distribution_tar_file

The undocumented makefile target in the last step creates a TAR file containing the necessary libraries and executables. We will use this to install the package later.

Building LILO
LILO is a Linux bootloader. It is responsible for loading and starting the Linux kernel. There are alternatives to LILO such as GRUB. I selected LILO because I am most familiar with it and its configuration. If you're feeling adventurous, you might consider one of the alternatives.

The make command builds LILO.

Creating the boot disk
The boot disk contains the Linux kernel you built earlier. You will use the boot disk to load the kernel into your target system when you turn the power on or reset the system.

There are several different ways to build a boot disk. The Linux kernel will boot directly from a floppy disk if configured to do so. Most Linux users boot into Linux with the help of a boot loader like LILO or GRUB, however. The Boot Disk HOWTO (see Resources) describes both of these methods and one or two others. I'm going to describe how to build my version of a LILO based boot disk. I recommend the LILO-based method because the LILO-based method allows parameters to be passed to the kernel at boot time. This is very important when dealing with older hardware, which often suffers from troubling idiosyncrasies. In my case, the Linux floppy driver requires the parameter "floppy=thinkpad" be specified in order to get around problems with the floppy drive on my ThinkPad model.

The LILO method needs the LILO executable installed on the system on which you're building the software to make the boot floppy bootable, so you must have LILO installed to proceed. If you can not or will not install LILO, I suggest trying one of the other procedures for creating a book disk described in the Boot Disk HOWTO.

You will need to be root in order to perform the following operations:

1. place a floppy in the floppy drive
2. mke2fs -N 24 -m 0 /dev/fd0
3. mount -o dev /dev/fd0 /mnt
4. mkdir /mnt/boot
5. mkdir /mnt/dev
6. cp -R /dev/null /mnt/dev
7. cp -R /dev/fd0 /mnt/dev
8. cp /boot/boot.b /mnt/boot

Now you must copy the kernel from <linux source>/arch/i386/boot directory to the floppy disk and set the ramdisk word. Replace <linux source> in the path below with the directory in which you built the kernel.

1. cp <linux source>/arch/i386/boot/bzImage /mnt
2. rdev -r /mnt/bzImage 49152

In order to make the floppy boot, you must run LILO. First, create a LILO configuration file named /mnt/bdlilo.conf. I created one with the following information:

    
boot      =/dev/fd0
install   =/boot/boot.b
map       =/boot/map
read-write
backup    =/dev/null
compact
image     =/bzImage
label     =Linux
root      =/dev/fd0

The following command puts the boot loader in place on the boot disk.

1. lilo -v -C bdlilo.conf -r /mnt

If you're feeling brave you can now try to boot your target system from the boot disk. You won't get to a command prompt because we're still missing a root disk, but you'll get a feeling for whether or not you've successfully executed the steps above.

Creating the root disk
If you attempted to boot your target system, you will have seen the familiar lines of kernel output as various parts of the kernel were initialized. The kernel boot process will complete with a line similar to the following:

VFS: Insert root floppy disk to be loaded into RAM disk and press ENTER

The bootdisk we created in the section above holds only the kernel. In order to create a working floppy-based Linux system we need a complete filesystem and the BusyBox software we built earlier.

In the Resources section of this article, I provide a link to a file containing a filesystem skeleton. The filesystem skeleton contains the appropriate directory hierarchy and also defines enough devices in the /dev directory to get your system running. To this filesystem, we will add files, libraries, and executables from the packages we built earlier.

You will need to be root in order to perform the following operations:

1. place a floppy in the floppy drive
2. dd if=/dev/zero of=/tmp/fsfile bs=1k count=4096
3. mke2fs -m 0 -N 2000 /tmp/fsfile
4. mount -t ext2 -o loop /tmp/fsfile /mnt
5. cd /mnt
6. tar xvzf <location of skeleton.tar.gz>

We will now install uClibc and BusyBox onto the new filesystem.

1. cd <uClibc build directory>
2. make PREFIX=/mnt install_target
3. cd <BusyBox build directory>
4. make PREFIX=/mnt install

Copy the filesystem to disk and you're finished.

1. cd ..
2. umount /mnt
3. dd if=/tmp/fsfile bs=1k | gzip -v9 > /tmp/fsfile.gz
4. dd if=/tmp/fsfile.gz of=/dev/fd0 bs=1k

The boot disk and root disk are all you need to boot a system into Linux. The next step is to install Linux on the laptop hard drive.

Creating the supplemental disk
The supplemental disk holds the tools necessary to partition and build a filesystem on the laptop hard drive.

1. place a floppy in the floppy drive
2. dd if=/dev/zero of=/tmp/sufile bs=1k count=1440
3. mke2fs -m 0 -N 2000 /tmp/sufile
4. mount -t ext2 -o loop /tmp/sufile /mnt
5. cd /mnt

We will now install the remaining tools and utilities onto the new filesystem.

1. cd <Util Linux build directory>
2. mkdir /mnt/sbin
3. cp fdisk/fdisk /mnt/sbin
4. cd /mnt
5. tar -xvzf <E2fsprogs build directory>/build/e2fsprogs-1.27-elfbin.tar.gz
6. cd <LILO build directory>
7. make ROOT=/mnt install

Write the files to disk and you're finished. The supplemental disk will be mounted as a normal disk, so it's important not to compress this image.

1. cd ..
2. umount /mnt
3. dd if=/tmp/sufile of=/dev/fd0 bs=1k

You now have all of the files you need to install Linux on the laptop on three floppy disks.

Installing Linux
It's time to enjoy the fruits of your labor. Place the boot disk in your target system's floppy drive and turn on the power. The Linux kernel should boot. After it boots it will ask you to insert the root disk. Insert the root disk. The kernel will load the compressed filesystem from the root disk onto a ramdisk and complete the system boot process. The figures below show the results.

Figure 1. Booting the kernel

Figure 2. Loading the root filesystem

Once Linux has booted, mount the supplemental disk and copy the files from that disk to the root filesystem on the ramdisk. Once you have copied fdisk, mke2fs, lilo, and their supporting files, you are ready to partition and build a filesystem on the target system's hard drive.

Fdisk is a menu-driven utility for creating and manipulating partition tables. It is relatively easy to use: m prints the available commands, n adds a new partition, p prints the partition table, and w writes the partition table and exits. Please refer to the man page for more detailed information.

When partitioning the hard drive, don't forget to add a swap partition. I recommend creating a swap partition of three to four times the size of system memory.

Mke2fs creates an ext2 or ext3 filesystem on a partition. The command mke2fs -j /dev/hda1 will create an ext3 partition on /dev/hda1. I recommended using the newer ext3 filesystem rather than ext2. The extra cost of the journaling activity is worth the added reliability.

Once the hard drive is partitioned and the filesystem is built, mount the hard drive and copy the files from the ramdisk-based root filesystem to the new filesystem. The only files that you will need that are not present on the ramdisk-based root filesystem are the Linux kernel and the LILO configuration file. You can get these from the boot disk that you created earlier. Insert and mount the boot floppy and copy the kernel and LILO configuration files to the hard drive. Place both in the /boot directory of the hard drive filesystem. Rename the LILO configuration file to lilo.conf and edit it so that it looks like the following:

    
boot      =/dev/hda1
install   =/boot/boot.b
map       =/boot/map
read-write
backup    =/dev/null
compact
image     =/boot/bzImage
label     =Linux
root      =/dev/hda1

Finally, install LILO as follows (assuming the hard drive is mounted on /hd):

1. chroot /hd /bin/ash
2. /sbin/lilo

That should do it. You've built the necessary software, used it to boot the target system into Linux from floppies, partitioned and created a filesystem on the target system, and set the system up it boot directly from the hard drive. Not bad for a day's work!

Conclusion
The combination of Linux kernel, uClibc libraries, and BusyBox software makes building a Linux system from scratch nearly painless. The process and rationale described above is excellent for building recovery disks, creating your own custom Linux distribution, building a small footprint distribution for older hardware or for non-traditional embedded applications.

Next time we'll build the packages necessary to install Linux on the hard drive of the laptop, and discuss PCMCIA support and network support.

Resources

• Download my root filesystem skeleton, skeleton.tar.gz.
  
  
• Download the Linux kernel source at the Linux Kernel Archives.
  
  
• Tomsrtbt is a single disk rescue floppy distribution.
  
  
• Trinux is a single disk Linux security toolkit.
  
  
• Tin Foil Hat Linux is a secure Linux distribution designed for encrypting, signing, and wiping files.
  
  
• Download the uClibc source at uClibc.org.
  
  
• Diet libc provides an alternative to both Glibc and uClibc.
  
  
• Download the BusyBox source at BusyBox.net.
  
  
• Download Linux utilities from kernel.org.
  
  
• Download e2fsprogs from the E2fsprogs page on SourceForge.
  
  
• Download LILO from ibiblio's Linux archive.
  
  
• Learn more about GNU GRUB, the GRand Unified Bootloader. Also check out the developerWorks tutorial "Getting to know GRUB" (developerWorks, January 2001).
  
  
• The Linux Kernel HOWTO describes how to configure and build the Linux Kernel. See also the developerWorks tutorial "Compiling the Linux Kernel" (developerWorks, August 2000).
  
  
• The Bootdisk HOWTO explains how to build boot and root disks.
  
  
• Don't forget to read the uClibc README file.
  
  
• Darrick Addison describes how to set up an embedded Linux environment in "Embedded Linux applications: an overview" (developerWorks, 2001).
  
  
• Look at Anand Santhanam's and Vishal Kulkarni's article on embedded Linux, "Linux system development on an embedded device" (developerWorks, March 2002).
  
  
• Check out IBM's embedded Linux products and developer support.
  
  
• Visit IBM's home for embedded Linux.
  
  
• Find the Linux resource you're looking for in the developerWorks Linux zone.
  
  

What do you think of this document? Killer! (5) Good stuff (4) So-so; not bad (3) Needs work (2) Lame! (1) Comments?

developerWorks > Linux | Wireless