Preboot Execution Environment

From ArchWiki
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Related articles

From Wikipedia:Preboot Execution Environment:

The Preboot eXecution Environment (PXE, most often pronounced as pixie) specification describes a standardized client-server environment that boots a software assembly, retrieved from a network, on PXE-enabled clients. On the client side it requires only a PXE-capable network interface controller (NIC), and uses a small set of industry-standard network protocols such as DHCP and TFTP.

In this guide, PXE is used to boot the installation media with an appropriate option-rom that supports PXE on the target. This works well when you already have a server set up.

Preparation

Overview

It is useful to give an overview of the PXE boot process in order to understand the #Server setup, the #Installation on the client side and the Arch Linux files needed.

The client starts by broadcasting packets asking for a DHCP server and containing specific PXE options. The DHCP server responds with networking information (the IP address assigned to the client) and also provides, by using specific bootstrap protocol (BOOTP) parameters of the DHCP, additional information like the TFTP server address, the path of the initial network bootstrap program (NBP) to download or the boot configuration file name.

The NBP is transferred from the PXE server to the client using TFTP, to be loaded into memory and executed. The kernel and the initramfs are also transferred this way.

Then the root filesystem is transferred using one of the following protocols: HTTP, NFS or NBD.

Boot from install media

In order to gather the files that will be transferred from the server to the client for booting, get the latest official install media from the download page.

Next mount the image: 

# mkdir /mnt/archiso
# mount -o loop,ro archlinux-release_date-x86_64.iso /mnt/archiso

where release_date is the release date in the ISO filename like, e.g., 2020.10.01.

Note: The Arch ISO currently only supports BIOS-style PXE booting. See FS#50188 for more information.

Boot from netboot

Arch Linux netboot images can be used to download the latest Arch Linux release on the fly upon system boot. Download the image compatible with your configuration.

Note: The Arch netboot supports both BIOS and UEFI booting.

Pixiecore

A all-in-one solution is provided by pixiecore

  1. Install pixiecore-gitAUR
  2. Run pixiecore quick arch --dhcp-no-bind as root
  3. Boot via PXE

Server setup

You will need to setup a DHCP server, a TFTP server for transferring the NBP and one of the following services for transferring the root filesystem: HTTP server, NFS or NBD.

Network

Bring up your wired network adapter, and assign it an address appropriately. 

# ip link set eth0 up
# ip addr add 192.168.0.1/24 dev eth0

DHCP + TFTP

You will need both a DHCP server and a TFTP server to configure networking on the install target and to facilitate the transfer of files between the server and the client. dnsmasq does both, and is extremely easy to set up.

Install the dnsmasq package.

dnsmasq needs to be configured. See instructions on how to setup a dnsmasq#TFTP server and a dnsmasq#PXE server.

Are provided below some common configuration instructions. tftp_root is the directory where the Arch ISO is mounted (e.g. /mnt/archiso) or where the network boot program is located. 

# /etc/dnsmasq.conf
# Listen only to the specified interface
interface=eth0

# Don't function as dns server
port=0

# tftp server setup
enable-tftp
tftp-root=tftp_root

# Log extra information about dhcp transactions (for debug purposes)
log-dhcp

To enable the DHCP server and give IPv4 addresses within a range, add to the configuration file a line similar to: 

dhcp-range=192.168.0.50,192.168.0.150

Or in case there is already a DHCP-server running on the network and you want to interoperate with it, see dnsmasq#Proxy DHCP.

Two examples covering different boot style and installation media are provided below.

Once configured according to your needs, start dnsmasq.service.

BIOS boot from install media

The path of the initial bootstrap program to be transferred is defined with the dhcp-boot option in the configuration file. 

dhcp-boot=/arch/boot/syslinux/lpxelinux.0

In order to send specific bootstrap protocol (BOOTP) parameters, like the configuration file path, the dhcp-option-force=flag,value line is used. 

dhcp-option-force=209,boot/syslinux/archiso.cfg
dhcp-option-force=210,/arch/

UEFI boot from netboot

To send a file depending on the architecture, here the netboot image for UEFI-style boot, use: 

pxe-service=BC_EFI, "Boot from network BC EFI", ipxe.efi
pxe-service=X86-64_EFI, "Boot from network X86-64 EFI", ipxe.efi

If using netboot, the rest of the server setup section which focuses on the Arch ISO does not apply.

Transferring archiso root filesystem

Thanks to archiso_pxe_http, archiso_pxe_nfs and archiso_pxe_nbd initcpio hooks in archiso, it is possible to boot using HTTP, NFS or NBD. Boot time is approximately the same in all three methods, but HTTP method allows you to watch a state of downloading airootfs.sfs in percents.

HTTP

Among all alternatives, darkhttpd is by far the most trivial to setup (and the lightest-weight).

First, install the darkhttpd package.

Then start darkhttpd using our /mnt/archiso as the document root: 

# darkhttpd /mnt/archiso
darkhttpd/1.8, copyright (c) 2003-2011 Emil Mikulic.
listening on: http://0.0.0.0:80/
Note: It is important that the server is running on port 80. If you start darkhttpd without root access it will default to 8080, the client will still try to access port 80 and the boot will fail.

NFS

You will need to set up an NFS server with an export at the root of your mounted installation media, which would be /mnt/archiso if you followed #Preparation. After setting up the server, add the following line to your /etc/exports file: 

/etc/exports
/mnt/archiso 192.168.0.0/24(ro,no_subtree_check)

If the server was already running, re-export the filesystems with exportfs -r -a -v.

The default settings in the installer expect to find the NFS at /run/archiso/bootmnt, so you will need to edit the boot options. To do this, press Tab on the appropriate boot menu choice and edit the archiso_nfs_srv option accordingly: 

archiso_nfs_srv=${pxeserver}:/mnt/archiso

Alternatively, you can use /run/archiso/bootmnt for the entire process.

After the kernel loads, the Arch bootstrap image will copy the root filesystem via NFS to the booting host. This can take a little while. Once this completes, you should have a running system.

NBD

Install the nbd package and configure it: 

/etc/nbd-server/config
[generic]
[archiso]
    readonly = true
    exportname = /srv/archlinux-release_date-x86_64.iso

where release_date is the release date in the ISO filename like, e.g., 2020.03.01.

Start nbd.service.

Existing PXE server

If you have an existing PXE server with a PXELINUX system setup (e.g. a combination of DHCP and TFTP), you can add the following menu items to your /tftpboot/pxelinux.cfg/default file in order to boot Arch via your preferred method: 

LABEL archlinux
        MENU LABEL Arch Linux x86_64
        LINUX /path/to/extracted/Arch/ISO/arch/boot/x86_64/vmlinuz-linux
        INITRD /path/to/extracted/Arch/ISO/arch/boot/intel-ucode.img,/path/to/extracted/Arch/ISO/arch/boot/amd-ucode.img,/path/to/extracted/Arch/ISO/arch/boot/x86_64/initramfs-linux.img
        APPEND archisobasedir=arch archiso_http_srv=http://httpserver/path/to/extracted/Arch/ISO/
        SYSAPPEND 3
        TEXT HELP
        Arch Linux 2020.10.01 x86_64
        ENDTEXT

You can replace archiso_http_srv with archiso_nfs_srv for NFS or archiso_nbd_srv for NBD (see usage examples in arch/boot/syslinux/archiso_pxe.cfg file resided on ArchLinux iso). Whichever method you choose, you must pass ip= parameter to instruct the kernel to bring up the network interface before it attempts to mount the installation medium over the network. Passing BOOTIF= is required when there are several wired interfaces on the client side and/or you want resolv.conf to be already configured inside booted archiso. You can use sysappend mask 3 (which is 1+2) to pass these parameters automatically. For available boot parameters see README.bootparams.

Installation

For this portion you will need to figure out how to tell the client to attempt a PXE boot; in the corner of the screen along with the normal post messages, usually there will be some hint on which key to press to try PXE booting first. On an IBM x3650 F12 brings up a boot menu, the first option of which is Network; on a Dell PE 1950/2950 pressing F12 initiates PXE booting directly.

Boot

Looking at journald on the PXE server will provide some additional insight to what exactly is going on during the early stages of the PXE boot process: 

# journalctl -u dnsmasq.service -f
dnsmasq-dhcp[2544]: DHCPDISCOVER(eth1) 00:1a:64:6a:a2:4d 
dnsmasq-dhcp[2544]: DHCPOFFER(eth1) 192.168.0.110 00:1a:64:6a:a2:4d 
dnsmasq-dhcp[2544]: DHCPREQUEST(eth1) 192.168.0.110 00:1a:64:6a:a2:4d 
dnsmasq-dhcp[2544]: DHCPACK(eth1) 192.168.0.110 00:1a:64:6a:a2:4d 
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/syslinux/pxelinux.0 to 192.168.0.110
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/syslinux/archiso.cfg to 192.168.0.110
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/syslinux/whichsys.c32 to 192.168.0.110
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/syslinux/archiso_pxe_choose.cfg to 192.168.0.110
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/syslinux/ifcpu64.c32 to 192.168.0.110
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/syslinux/archiso_pxe_both_inc.cfg to 192.168.0.110
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/syslinux/archiso_head.cfg to 192.168.0.110
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/syslinux/archiso_pxe32.cfg to 192.168.0.110
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/syslinux/archiso_pxe64.cfg to 192.168.0.110
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/syslinux/archiso_tail.cfg to 192.168.0.110
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/syslinux/vesamenu.c32 to 192.168.0.110
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/syslinux/splash.png to 192.168.0.110

After you load pxelinux.0 and archiso.cfg via TFTP, you will (hopefully) be presented with a syslinux boot menu with several options, where you can select Boot Arch Linux (x86_64) (HTTP).

Next the kernel and initramfs (appropriate for the architecture you selected) will be transferred, again via TFTP: 

dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/x86_64/vmlinuz to 192.168.0.110
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/x86_64/initramfs-linux.img to 192.168.0.110

If all goes well, you should then see activity on darkhttpd coming from the PXE-target; at this point the kernel would be loaded on the PXE-target, and in init: 

1348347586 192.168.0.110 "GET /arch/aitab" 200 678 "" "curl/7.27.0"
1348347587 192.168.0.110 "GET /arch/x86_64/root-image.fs.sfs" 200 107860206 "" "curl/7.27.0"
1348347588 192.168.0.110 "GET /arch/x86_64/usr-lib-modules.fs.sfs" 200 36819181 "" "curl/7.27.0"
1348347588 192.168.0.110 "GET /arch/any/usr-share.fs.sfs" 200 63693037 "" "curl/7.27.0"

After the root filesystem is downloaded via HTTP, you will eventually end up at the normal live system root zsh prompt.

Post-boot

Unless you want all traffic to be routed through your PXE server (which will not work anyway unless you set it up properly), you will want to stop dnsmasq.service and get a new lease on the install target, as appropriate for your network layout.

You can also kill darkhttpd; the target has already downloaded the root filesystem, so it is no longer needed. While you are at it, you can also unmount the installation image: 

# umount /mnt/archiso

At this point you can follow the Installation guide.

Low memory systems

The copytoram initramfs option can be used to control whether the root filesystem should be copied to ram in its entirety in early-boot.

It highly recommended to leave this option alone, and should only be disabled if entirely necessary (systems with less than ~256MB physical memory). Append copytoram=n to your kernel line if you wish to do so.

Note: As this requires loop-mounting squashfs from a mounted remote filesystem, copytoram=n and archiso_pxe_http are mutually exclusive.

Sharing internet with PXE clients

If your network for PXE clients is private (for example, 192.168.1.0/24), and you want them to be able to access internet (for example, for packages installation), you should configure masquerade/source nat properly. Your PXE server must have a separate NIC connected to the internet. You can use such command to pass through the internet to clients: 

iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -j MASQUERADE

To make this rule persistent after reboot, run the following command: 

iptables-save -f /etc/iptables/iptables.rules

and enable iptables.service.

See Simple stateful firewall#Setting up a NAT gateway and Internet sharing#Enable NAT for more information.

Troubleshooting

DHCP interface rename bug

FS#36749 causes default predictable network interface renaming to fail and then DHCP client to fail because of it. A workaround is to add the kernel boot parameter net.ifnames=0 to disable predictable interface names.

VirtualBox cannot boot while real machines can

When using VirtualBox to test your configuration, the virtual machine may get stuck at: 

Probing EDD (edd=off to disable)... ok

While PXE booting with a real machine works fine. The problem may be because you have set several CPU cores to your client machine, and you set its type as Other and version as Other/Unknown (64 bit). So VirtualBox does not know which paravirtualization interface to use by default.

Adding loglevel=7 to the kernel parameters lets you see where it actually got stuck: 

[    0.063697] smp: Bringing up secondary CPUs...
[    0.103768] x86: Booting SMP configuration:

To resolve this, either use one CPU core, or go to Machine > Settings > System > Acceleration and set one of the following paravirtualization interface: Minimal, Hyper-V or KVM.