Open in app

Sign in

Write

Sign in

PXE boot Cisco UCSX-210C-M6 with Cisco Intersight

Afzal Muhammad
10 min readJun 30, 2021

Overview

PXE (Preboot eXecution Environment) Server enables automated or unattended OS installation over the Network. The main benefit of PXE is that we don’t need any bootable drive to boot OS(Operating System) and we do not need to to burn any ISO file into DVD or usb device.

Once the PXE server is setup, we can install hundreds of System at the same time over the network. As it works on Client-Server architecture, to get the OS installation on clients, boot the clients via PXE option. On the client side, it needs PXE-capable NIC and uses other industry standard network protocols such as DHCP and TFTP. In modern datacenter, PXE is the most preferred choice for booting the OS among enterprises.

In this section, I would be setting up PXE server on RHEL 7.9.

Assumptions

This article assumed the following:

  1. PXE setup requirements such as DHCP, TTFP, HTTP is hosted in a single server, although it is not mandatory.
  2. The PXE server can reach the internet.
  3. The PXE server is setup on Red Hat Enterprise Linux (RHEL) 7.9
  4. Cisco UCSX in Cisco Intersight is claimed and configured. To learn more about Cisco Intersight, visit https://www.cisco.com/c/en/us/products/cloud-systems-management/intersight/index.html

PXE Configuration in Cisco Intersight

Following configuration is required in Cisco Intersight for PXE boot Cisco UCSX 210c servers.

Login to Cisco Intersight and modify the boot policy by adding a boot device for PXE boot as shown below:

eth0 is the interface that will get the IP address via DHCP.

You can capture the interface information in Cisco Intersight in server profiles in inventory tab. Also note down the interface MAC address for configuring IP reservation in DHCP configuration file which will be covered later in this article.

Configure the Boot order in Cisco Intersight as shown below:

Install required packages

Following packages are required to setup PXE server in Linux. Run the following command to install required packages

# yum install dhcp tftp tftp-server syslinux vsftpd xinetd

Configure DHCP Server

Environment:

PXE server IP: 10.10.1.5

DNS: 10.10.1.4

Gateway: 10.10.1.4

PXE Server OS: RHEL 7.9

Firewall: disabled

For configuring DHCP, specify the subnet and the range used for offering IP address via DHCP. You can also specify lease time.

In this configuration, we have specified IP reservation for nodes with the MAC address of the interface configured for PXE boot.

Below are the steps to set up the DHCP to support PXE boot for UEFI servers

Note: This DHCP configuration will not work for BIOS or legacy mode. For this to work, you need to specify filename “pxelinux.0”. To work for both BIOS and UEFI mode, you can also specify conditional checks on vendor-class-identifier for specifying filename. More details on this can be found at https://www.redhat.com/sysadmin/pxe-boot-uefi

Configure DHCP using the following conf file. perform changes according to your environment

[root@e26-linuxjb images]# cat /etc/dhcp/dhcpd.conf
#
# DHCP Server Configuration file.
#   see /usr/share/doc/dhcp*/dhcpd.conf.example
#   see dhcpd.conf(5) man page
#

ddns-update-style interim;
ignore client-updates;
authoritative;
allow booting;
allow bootp;
allow unknown-clients;

# internal subnet for my DHCP Server
subnet 10.10.1.0 netmask 255.255.255.0 {
range 10.10.1.45 10.10.1.100;
option domain-name-servers 10.10.1.4;
option domain-name "sjc02-cdip.cisco.local";
option broadcast-address 10.10.1.255;
option routers 10.10.1.4;
default-lease-time 600;
max-lease-time 7200;

next-server 10.10.1.5;
filename "grubx64.efi";

host boootstrap {
 hardware ethernet 00:25:b5:00:26:06;
 fixed-address 10.10.1.80;
}
host master0 {
 hardware ethernet 00:25:B5:00:26:00;
 fixed-address 10.10.1.50;
}
host master1 {
 hardware ethernet 00:25:B5:00:26:01;
 fixed-address 10.10.1.51;
}
host master2 {
 hardware ethernet 00:25:B5:00:26:02;
 fixed-address 10.10.1.52;
}
host worker0 {
 hardware ethernet 00:25:B5:00:26:03;
 fixed-address 10.10.1.53;
}
host worker1 {
 hardware ethernet 00:25:B5:00:26:04;
 fixed-address 10.10.1.54;
}
}

start and enable the DHCP service. Every time when you make a change in the above configuration file, don’t forget to restart the DHCP service.

# systemctl start dhcpd 
# systemctl status dhcpd 
# systemctl enable dhcpd

Edit and configure TFTP server

TFTP (Trivial File Transfer Protocol) is used to transfer files from server to clients without any kind of authentication. In the case of PXE, tftp perform bootstrap loading.

TFTP server is needed to provide the following:

  1. initrd.img — The “boot loader” which will be loaded to RAM disk.
  2. vmlinuz — A compressed bootable Linux Kernel.

To configure tftp, edit the following configuration file.

[root@e26-linuxjb ~]# cat /etc/xinetd.d/tftp
# default: off
# description: The tftp server serves files using the trivial file transfer \
#	protocol.  The tftp protocol is often used to boot diskless \
#	workstations, download configuration files to network-aware printers, \
#	and to start the installation process for some operating systems.
service tftp
{
	socket_type		= dgram
	protocol		= udp
	wait			= yes
	user			= root
	server			= /usr/sbin/in.tftpd
	server_args		= -s /var/lib/tftpboot
	disable			= no	
	per_source		= 11
	cps			= 100 2
	flags			= IPv4
}

All the network boot related files are to be placed in tftp root directory “/ var/lib/tftpboot “

Run the following commands to copy required network boot files in ‘/var/lib/tftpboot/’

[root@pxe ~]# cp -v /usr/share/syslinux/pxelinux.0 /var/lib/tftpboot [root@pxe ~]# cp -v /usr/share/syslinux/menu.c32 /var/lib/tftpboot [root@pxe ~]# cp -v /usr/share/syslinux/memdisk /var/lib/tftpboot [root@pxe ~]# cp -v /usr/share/syslinux/mboot.c32 /var/lib/tftpboot [root@pxe ~]# cp -v /usr/share/syslinux/chain.c32 /var/lib/tftpboot [root@pxe ~]# mkdir /var/lib/tftpboot/pxelinux.cfg 
[root@pxe ~]# mkdir /var/lib/tftpboot/networkboot

Create subfolder in /var/lib/tftpboot/networkboot for each OS being configured for PXE boot. For example, in this case, RHEL 7.8 and CoreOS 4.6.8

[root@pxe ~]# mkdir /var/lib/tftpboot/networkboot/rhel78 
[root@pxe ~]# mkdir /var/lib/tftpboot/networkboot/coreos46

Mount ISO file

Download the iso file for RHEL 7.8 and CoreOS 4.6.8 and move it to PXE server.

For example, run the following to download the CoreOS 4.6.8 iso.

# curl -L -J -O https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/4.6/latest/rhcos-4.6.8-x86_64-live.x86_64.iso

Create a sub folder in ‘/var/ftp/pub’ for each OS to store the boot image files.

# mkdir /var/ftp/pub/coreos468 
# mkdir /var/ftp/pub/rhel78

Run the following commands to mount iso file both for rhel 7.8 and coreOS 4.6.8 and then copy its contents in ftp server’s directory ‘/var/ftp/pub/coreos468 ‘

Perform the following for each OS iso file. In this example, I setup two OS, RHEL 7.8 and CoreOS 4.6.8

[root@pxe ~]# mount -o loop rhcos-4.6.8-x86_64-live.x86_64.iso /mnt/ mount: /dev/loop0 is write-protected, mounting read-only 
[root@pxe ~]# cd /mnt/ 
[root@pxe mnt]# cp -av * /var/ftp/pub/coreos468

Contents of coreos468 is shown as below:

[root@e26-linuxjb ~]# ls -ll /var/ftp/pub/coreos468
total 887512
dr-xr-xr-x. 3 root root        20 Dec  5  2020 EFI
dr-xr-xr-x. 3 root root        60 Dec  5  2020 images
dr-xr-xr-x. 2 root root       156 Dec  5  2020 isolinux
-r--r--r--. 1 root root       132 Dec  5  2020 zipl.prm
[root@e26-linuxjb ~]#

Contents of RHEL 7.8 folder is shown is below

[root@e26-linuxjb ~]# ls -ll /var/ftp/pub/rhel78/
total 412
dr-xr-xr-x. 4 root root     54 Jun 26 21:50 addons
dr-xr-xr-x. 3 root root     18 Jun 26 21:50 EFI
-r--r--r--. 1 root root   8266 Jun 26 21:50 EULA
-r--r--r--. 1 root root   1455 Jun 26 21:50 extra_files.json
-r--r--r--. 1 root root  18092 Jun 26 21:50 GPL
dr-xr-xr-x. 3 root root     57 Jun 26 21:50 images
dr-xr-xr-x. 2 root root    217 Jun 26 21:50 isolinux
dr-xr-xr-x. 2 root root     43 Jun 26 21:50 LiveOS
-r--r--r--. 1 root root    114 Jun 26 21:50 media.repo
dr-xr-xr-x. 2 root root 282624 Jun 26 21:50 Packages
dr-xr-xr-x. 2 root root   4096 Jun 26 21:50 repodata
-rw-r--r--. 1 root root    963 Jun 26 21:50 rhel78.cfg
-r--r--r--. 1 root root   3375 Jun 26 21:50 RPM-GPG-KEY-redhat-beta
-r--r--r--. 1 root root   3211 Jun 26 21:50 RPM-GPG-KEY-redhat-release
-r--r--r--. 1 root root   1796 Jun 26 21:50 TRANS.TBL
[root@e26-linuxjb ~]#

Copy ISO file contents to FTP server folder

Copy Kernel file (vmlimz) and initrd file from mounted iso file

For CoreOS copy to ‘/var/lib/tftpboot/networkboot/coreos468 ‘

For RHEL copy to ‘/var/lib/tftpboot/networkboot/rhel78 ‘

# cp /var/ftp/pub/coreos468/images/pxeboot/* /var/lib/tftpboot/networkboot/coreos468/.[root@e26-linuxjb ~]# ls -ll /var/lib/tftpboot/networkboot/coreos468
total 887508
-r--r--r--. 1 root root  79512484 Jun 28 11:58 initrd.img
-r--r--r--. 1 root root 820364800 Jun 28 11:59 rootfs.img
-r--r--r--. 1 root root   8924528 Jun 28 11:57 vmlinuz
[root@e26-linuxjb ~]# 
[root@e26-linuxjb ~]# 

[root@e26-linuxjb ~]# ls -ll /var/lib/tftpboot/networkboot/rhel78/
total 63148
-r--r--r--. 1 root root 57894232 Jun 25 20:25 initrd.img
-r-xr-xr-x. 1 root root  6762800 Jun 25 20:25 vmlinuz
[root@e26-linuxjb ~]#

Note: In case of CoreOS, you can also download kernel, initramfs, and rootfs from Red Hat Mirror site (https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/4.6/latest ) and store it in the /var/lib/tfpboot/networkboot/coreos468 folder instead of getting it from iso file.

unmount the iso file using ‘umount’ command

[root@pxe ~]# umount /mnt/ 
[root@pxe ~]#

Now you can verify the content of ftp server in the browser as shown below. Make sure your ftp service is running.

rhel78 folder contents from browser is shown below

You can also open the ftp site in window explorer to perform CRUD operations.

Configure grub.cfg for UEFI or pxelinux.cfg/default for creating PXE menu

[root@e26-linuxjb images]# cat /var/lib/tftpboot/grub.cfg
set timeout=60

# for bootstrap node

menuentry 'Install RHEL CoreOS 4.6.8 Bootstrap Node' --class fedora --class gnu-linux --class gnu --class os {
        linuxefi /networkboot/coreos468/vmlinuz inst.repo=ftp://10.10.1.5/pub/coreos468 coreos.live.rootfs_url=http://10.10.1.5:8080/ignition-install/rhcos-4.6.8-x86_64-live-rootfs.x86_64.img nomodeset rd.neednet=1 coreos.inst=yes coreos.inst.install_dev=sda coreos.inst.image_url=http://10.10.1.5:8080/ignition-install/rhcos-4.6.8-x86_64-metal.x86_64.raw.gz coreos.inst.insecure coreos.inst.ignition_url=http://10.10.1.5:8080/ignition-install/bootstrap.ign
        initrdefi /networkboot/coreos468/initrd.img
}
# for master node

menuentry 'Install RHEL CoreOS 4.6.8 Master Node' --class fedora --class gnu-linux --class gnu --class os {
        linuxefi /networkboot/coreos468/vmlinuz inst.repo=ftp://10.10.1.5/pub/coreos468 coreos.live.rootfs_url=http://10.10.1.5:8080/ignition-install/rhcos-4.6.8-x86_64-live-rootfs.x86_64.img nomodeset rd.neednet=1 coreos.inst=yes coreos.inst.install_dev=sda coreos.inst.image_url=http://10.10.1.5:8080/ignition-install/rhcos-4.6.8-x86_64-metal.x86_64.raw.gz coreos.inst.insecure coreos.inst.ignition_url=http://10.10.1.5:8080/ignition-install/master.ign
        initrdefi /networkboot/coreos468/initrd.img
}

# for worker node

menuentry 'Install RHEL CoreOS 4.6.8 Worker Node' --class fedora --class gnu-linux --class gnu --class os {
        linuxefi /networkboot/coreos468/vmlinuz inst.repo=ftp://10.10.1.5/pub/coreos468 coreos.live.rootfs_url=http://10.10.1.5:8080/ignition-install/rhcos-4.6.8-x86_64-live-rootfs.x86_64.img nomodeset rd.neednet=1 coreos.inst=yes coreos.inst.install_dev=sda coreos.inst.image_url=http://10.10.1.5:8080/ignition-install/rhcos-4.6.8-x86_64-metal.x86_64.raw.gz coreos.inst.insecure coreos.inst.ignition_url=http://10.10.1.5:8080/ignition-install/worker.ign
        initrdefi /networkboot/coreos468/initrd.img
}

menuentry 'Install RHEL 7.8' {
         linuxefi /networkboot/rhel78/vmlinuz inst.repo=ftp://10.10.1.5/pub/rhel78
         initrdefi /networkboot/rhel78/initrd.img
}
[root@e26-linuxjb images]#

For BIOS boot, pxelinux.cfg/default is needed and following file can be used as a reference.

[root@e26-linuxjb ~]# cat /var/lib/tftpboot/pxelinux.cfg/default
default menu.c32
prompt 0
timeout 30
MENU TITLE Cisco SJC LAB PXE Menu
LABEL rhel7_x64
MENU LABEL RHEL 7_X64
KERNEL /networkboot/rhel78/vmlinuz
APPEND initrd=/networkboot/rhel78/initrd.img inst.repo=ftp://10.10.1.5/pub/rhel78 
ks=ftp://10.10.1.5/pub/rhel78/rhel78.cfg
[root@e26-linuxjb ~]#

Create KickStart

Before creating kickstart file, let’s first create the root password in an encrypted string because we will using that encrypted password string in kickstart file.

[root@pxe ~]# openssl passwd -1 Pxe@123# $1$e2wrcGGX$tZPQKPsXVhNmbiGg53MN41 
[root@pxe ~]#

System default kickstart file is placed under /root with name ‘ anaconda-ks.cfg’. We will be creating a new kickstart under the folder /var/ftp/pub/rhel78 with the name ‘ rhel78.cfg ‘

Copy the following content into the new kickstart file. Please modify the kickstart file as per your needs.

[root@pxe ~]# vi /var/ftp/pub/rhel78/rhel78.cfg

#platform=x86, AMD64, or Intel EM64T
#version=DEVEL
# Firewall configuration
firewall --disabled
# Install OS instead of upgrade
install
# Use FTP installation media
url --url="ftp://172.168.1.11/pub/"
# Root password
rootpw --iscrypted $1$e2wrcGGX$tZPQKPsXVhNmbiGg53MN41
# System authorization information
auth useshadow passalgo=sha512
# Use graphical install
graphical
firstboot disable
# System keyboard
keyboard us
# System language
lang en_US
# SELinux configuration
selinux disabled
# Installation logging level
logging level=info
# System timezone
timezone Europe/Amsterdam
# System bootloader configuration
bootloader location=mbr
clearpart --all --initlabel
part swap --asprimary --fstype="swap" --size=1024
part /boot --fstype xfs --size=300
part pv.01 --size=1 --grow
volgroup root_vg01 pv.01
logvol / --fstype xfs --name=lv_01 --vgname=root_vg01 --size=1 --grow
%packages
@^minimal
@core
%end
%addon com_redhat_kdump --disable --reserve-mb='auto'
%end

Start and enable services

[root@pxe ~]# systemctl start xinetd
[root@pxe ~]# systemctl enable xinetd
[root@pxe ~]# systemctl start dhcpd.service
[root@pxe ~]# systemctl enable dhcpd.service
Created symlink from /etc/systemd/system/multi-user.target.wants/dhcpd.service to /usr/lib/systemd/system/dhcpd.service.
[root@pxe ~]# 
[root@pxe ~]# systemctl start vsftpd
[root@pxe ~]# systemctl enable vsftpd
Created symlink from /etc/systemd/system/multi-user.target.wants/vsftpd.service to /usr/lib/systemd/system/vsftpd.service.
[root@pxe ~]#

In case SELinux is enabled, then set the following selinux rule for ftp server.

[root@pxe ~]# setsebool -P allow_ftpd_full_access 1 
[root@pxe ~]#

Following is required for UEFI

# cp /var/ftp/pub/rhel78/EFI/BOOT/grubx64.efi /var/lib/tftpboot/

Launch vKVM

Launch vKVM from Cisco Intersight and login to server vKVM console.

Click Power →Power Cycle System

Click Boot Device → Select LAN

The following screen will display and launch the PXE boot menu as defined in grub.cfg

Originally published at https://community.cisco.com on June 30, 2021.

Pxe
Bootstrapping
Linux
Azure
AWS

--

--

Afzal Muhammad

Written by Afzal Muhammad

69 Followers

Innovative and transformative cross domain cloud solution architect @Microsoft (& xCisco). Helping companies to digitally transform!!!

Help

Status

About

Careers

Blog

Privacy

Terms

Text to speech

Teams