Am I Ubuntu-hater?

Automating Ubuntu 22.04 installations is ...

What can be worse than Ubuntu installation? Two Ubuntu installations. But frankly speaking the Internet in the hotel is even worse. Both Ubuntu and the hotel’s Internet drive me mad. If I read something in documentation, I expect that it works like it is written in the documentation. Maybe my English knowledge is detoriating, or I miss something there like a sign: “ATTENTION, BETA-RELEASE! Some features may not work as expected”. I did it, it works. Am I satisfied? No.

Ubuntu installer

Ubuntu used Debian installation for years. Maybe there are people in the world who were satisfied with it, but I didn’t meet such people. Ubuntu wanted new installation program since years and they did it in the latest releases. But it was optional. I think Ubuntu 24.04 is the first release where you can’t use anything except Subiquity - new Ubuntu installer. But maybe I am wrong.

The idea was good. Make Ubuntu installations easy automatable and compatible with cloud-init. Ubuntu is one of the most popular Linux distributions in cloud environments. Highly automated and repeatable installations must be very easy with Ubuntu.

The implementation is … OK, it is my opinion. You must not trust me. If you did zillions of Ubuntu installations, you know better than me how to do it and maybe you can share your knowledge in the comments, what I did wrong. But as for me it is really deep beta version and not ready for enterprise deployments.

Autoinstall profile

Let’s start automating Ubuntu installation.

In Red Hat you work with plain text files written in their own Kickstart syntax.

In SUSE you work with XML files.

What did we miss? Yes, Ubuntu uses YAML files similar to cloud-init and Ansible. How to write the Ubuntu YAML files and what options are available, you can find in the Ubuntu installation documentation.

As for me YAML is better than XML. But don’t forget about indentation!

Starting autoinstall profile

The installation configuration must start with:

#cloud-config
autoinstall:
  version: 1

If you don’t write it, Ubuntu installation will fail. Even if the file starts with #cloud-config, it is not cloud-init configuration file! Don’t mess them up!

Everything what I write below must be under version: 1, so don’t forget indentation!

Localization

We start configuring our localization. As you probably know I use English as the main language in the system and German keyboard layout:

  locale: en_US.UTF-8
  keyboard:
    layout: de
    toggle: null
    variant: ''

Time

My next step similar to Red Hat and SUSE would be configure timezone and NTP servers. But… we can configure only timezone in the file:

  timezone: "Europe/Berlin"

Want to configure NTP servers? Make your own post-install script and run it! We will talk later about post-installation scripts.

Network configuration

Switching to the network interfaces configuration. What’s the problem? The only problem - there is no documentation how to do it in the installation guide. There is only reference to netplan documentation. I copied some example, changed it a little bit according to my environment and it works.

  network:
    version: 2
    ethernets:
      ibmveth3:
        match:
          macaddress: 02:00:91:6F:64:DA
        addresses:
        - 10.0.3.3/24
        gateway4: 10.0.3.254
        nameservers:
          addresses:
          - 10.0.2.1
          search:
          - power-devops.cloud

There are two problems with this network configuration.

First I thought I can change the interface name from ibmveth3 to something else, but it doesn’t work. Where should I find the name of the future network interface if I didn’t install Ubuntu? In my case it is easy - it is virtual slot 3 and I can be sure that the network interface will be called ibmveth3. But if I use another slot? Should I know the number of the slot before the installation additionally to all other parameters?

According to the Netplan documentation:

You can use any ID that describes the interface and match the actual network card using the match key.

So I wrote something like:

network:
  ethernets:
    eth0:
      match:
        macaddress: MY:MA:CA:DD:RE:SS

It didn’t work for me as expected. At the end the system was installed, I could connect and it had the correct IP address. The interface was still ibmveth3.

The second problem with the network configuration above is the option gateway4. It works but after the new system is booted you can see (or find it in logs) the warning, that the option is deprecated. The correct syntax according to the Netplan documentation is:

  network:
    version: 2
    ethernets:
      eth0:
        match:
          macaddress: 02:00:91:6F:64:DA
        addresses:
        - 10.0.3.3/24
        routes:
        - to: default
          via: 10.0.3.254

Hostname and default user

Next step is to specify the hostname.

  identity:
    hostname: ubuntu.power-devops.cloud
    username: ubuntu
    password: $y$j9T$wagcbpxWVcv5i5xVKGy.J/$zilDtaR/U9htTE.cSXoDY1tGp770MyV9J11/do.TPQ2

In the same block you must specify some username with encrypted password. The user will be created and the password will be set to the user. By the way the password you see is abc123. Feel free to use! :-)

I understand the idea that we always must have some user and never login directly with root. Fully agree! But why it should be specified here? Why I should have only one user? In a normal environment you have a team of administrators with personal accounts.

There is a way to create such users in the file. We will talk about it later.

Storage configuration

Now is the longest part of the configuration - storage configuration. I repeat it one more time - I have only one disk during installation and it is easy to find it.

  storage:
    config:
    - type: disk
      path: /dev/sda
      id: disk-sda
      ptable: gpt
      wipe: superblock-recursive
      preserve: false
      grub_device: false

With the configuration above I remove everything what exists on /dev/sda and create a new partition table. Which partitions must be there? Let’s write the configuration further.

    - type: partition
      device: disk-sda
      id: sda1
      number: 1
      path: /dev/sda1
      size: 8M
      wipe: zero
      flag: prep
      preserve: false
      grub_device: true
      offset: 1048576
    - type: partition
      device: disk-sda
      id: sda2
      number: 2
      path: /dev/sda2
      size: 1G
      wipe: superblock
      preserve: false
      grub_device: false
    - type: partition
      device: disk-sda
      id: sda3
      number: 3
      path: /dev/sda3
      size: -1
      wipe: superblock
      preserve: false
      grub_device: false

These are three partitions - /dev/sda1, /dev/sda2 and /dev/sda3. The first partition is the special PReP boot partition which we need on IBM Power. The second partition is the future /boot file system and the third partition will contain my rootvg.

Let’s describe /boot partition:

    - type: format
      volume: sda2
      id: boot
      label: /boot
      fstype: ext4
      preserve: false
    - type: mount
      device: boot
      id: mboot
      path: /boot
      options: "noatime"

As for me it is too much to write but it works. I don’t understand why Ubuntu developers would like to have both format and mount sections and can’t unite them into one section. At the end I need three different IDs - for partition, for format and for mount, and I must be careful with them. If I mix IDs, nothing would work.

We move on to the next part of storage configuration - LVM. First my rootvg:

    - type: lvm_volgroup
      devices:
      - sda3
      id: rootvg
      name: rootvg
      preserve: false

Now we can define logical volumes which are called LVM partitions in the configuration.

    - type: lvm_partition
      volgroup: rootvg
      name: hd4
      id: hd4lv
      size: 4G
      preserve: false
      wipe: superblock
    - type: lvm_partition
      volgroup: rootvg
      name: hd9var
      id: hd9varlv
      size: 4G
      preserve: false
      wipe: superblock
    - type: lvm_partition
      volgroup: rootvg
      name: hd3
      id: hd3lv
      size: 1G
      preserve: false
      wipe: superblock
    - type: lvm_partition
      volgroup: rootvg
      name: hd1
      id: hd1lv
      size: 1G
      preserve: false
      wipe: superblock
    - type: lvm_partition
      volgroup: rootvg
      name: hd10opt
      id: hd10optlv
      size: 1G
      preserve: false
      wipe: superblock
    - type: lvm_partition
      volgroup: rootvg
      name: hd6
      id: hd6lv
      size: 4G
      preserve: false
      wipe: superblock

Ready with logical volumes? Let’s “format” them:

    - type: format
      volume: hd4lv
      id: hd4
      label: /
      fstype: ext4
      preserve: false
    - type: format
      volume: hd9varlv
      id: hd9var
      label: /var
      fstype: ext4
      preserve: false
    - type: format
      volume: hd3lv
      id: hd3
      label: /tmp
      fstype: ext4
      preserve: false
    - type: format
      volume: hd1lv
      id: hd1
      label: /home
      fstype: ext4
      preserve: false
    - type: format
      volume: hd10optlv
      id: hd10opt
      label: /opt
      fstype: ext4
      preserve: false
    - type: format
      volume: hd6lv
      id: hd6
      label: swap
      fstype: swap
      preserve: false

Formatted? You can start “mounting”:

    - type: mount
      device: hd4
      id: mhd4
      path: /
      options: "noatime"
    - type: mount
      device: hd9var
      id: mhd9var
      path: /var
      options: "noatime"
    - type: mount
      device: hd3
      id: mhd3
      path: /tmp
      options: "noatime"
    - type: mount
      device: hd1
      id: mhd1
      path: /home
      options: "noatime"
    - type: mount
      device: hd10opt
      id: mhd10opt
      path: /opt
      options: "noatime"
    - type: mount
      device: hd6
      id: mhd6
      path: swap

A lot to write I must say! But if you want to get reasonable LVM configuration, there is no other way.

SSH configuration

Because we must be able to work with our future Ubuntu server, we must install and configure SSH. I never thought that it should be a separate section in a modern Linux distribution, but why not?

  ssh:
    install-server: true
    allow-pw: true
    authorized-keys:
    - "ssh-rsa AAAAB info@power-devops.cloud"

If you don’t need password authentication, you can switch it off. The authorized keys are for the default user, we created in the identity section. Here is the place where my head starts exploding. I really don’t understand why user name and its password must be in the identity section and its SSH public key in the ssh section. At the same time all other users must be in another section, including their SSH keys. What?

Installation sources and packages

The standard way of Ubuntu installation is to find some Ubuntu mirror in the Internet and download everything from the mirror. It is very interesting idea in an air-gapped environment which will probably (almost 100%) not work.

If you work with Ubuntu and don’t want that it downloads packages from the Internet, you should create your own repository with the packages. I don’t need such repository. I want to install Ubuntu when I need it - no more, no less. But even if I don’t have repository, I must specify one. That’s why I did it easy for me.

  apt:
    fallback: offline-install
    mirror-selection:
      primary:
      - uri: http://nowhere/ubuntu/
    preserve_sources_list: false

I specified a non existent URL and defined the fallback way to my offline installation medium which is mounted under /cdrom.

We must specify how “much” Ubuntu we want to install and we do it in the source section:

  source:
    id: ubuntu-server
    search_drivers: false

My first idea was to make a minimal Ubuntu installation. Officially I can write ubuntu-server-minimal into the id field and get my minimal installation. Yes, I’ve got my minimal installation.

• This minimal installation ignored all my custom configuration, which we will talk about later, and didn’t create even my default user.

• I couldn’t install any additional packages, because there are no installation repositories (see above).

• The whole system was “minimized”. Very interesting concept of Ubuntu, when you have e.g. the command man but it doesn’t work unless you “unminimize” the system.

That’s why I decided to use “standard” Ubuntu Server installation instead of the “minimal”.

I don’t know if I must specify the kernel section, but I did it:

  kernel:
    package: linux-generic

Now the next my-head-exploding issue. According to the documentation, you can specify additional packages to install like:

  packages:
    - firewalld

It works, yes. But only if you have specified a repository. If you don’t have repository like in my configuration, the installation program will fail trying to install the package. If you think this is because there is no package firewalld on my medium, you are wrong. I tested it even with simpler packages like vim or less which are definitely on the medium and can be installed. They are installed if I do “standard” installation. But the installation program fails if they are specified in the packages section.

Want to have additional packages? Create your own Ubuntu repository. Anyway you need it if you manage Ubuntu installations.

Post-installation configuration

Now we are the point where I write my post-installation scripts. There are two ways how to configure the fresh installed Ubuntu system.

The first way is to use late-commands section. The commands in this section will run after the installation in the installation program environment:

  late-commands:
    - echo 'PermitRootLogin yes' >/target/etc/ssh/sshd_config.d/01-permitrootlogin.conf
    - curtin in-target -- useradd -d /home/test -G sudo -m -s /bin/bash -p '$y$j9T$wagcbpxWVcv5i5xVKGy.J/$zilDtaR/U9htTE.cSXoDY1tGp7
70MyV9J11/do.TPQ2' test

If you need to change some file on your server, you must prepend the path to the file with /target.

If you need to run some command inside of your future server’s environment, prepend the command with curtin in-target.

See the example above.

Another way to make post-installation configuration is to use cloud-init. The last section you can add into your installation profile is user-data and it can everything what cloud-init understands. The only prerequisite - it must be “full” or “standard” Ubuntu installation, not minimal! Even if cloud-init is installed and works in the minimal installation, user-data was never processed in my tests with the minimal installation.

  user-data:
    ntp:
      enabled: true
      servers:
      - 0.rhel.pool.ntp.org
    disable_root: false
    ssh_pwauth: true
    chpasswd:
      expire: false
      users:
      - name: root
        password: abc123
        type: text
    users:
    - default
    - name: admin01
      gecos: Ubuntu Administrator 01
      primary_group: admin01
      groups: users, admin, sudo
      sudo: ALL=(ALL) NOPASSWD: ALL
      shell: /bin/bash
      ssh_authorized_keys:
      - ssh-rsa AAAA admin01@power-devops.cloud

Here you can configure NTP, set passwords, create new users… Almost everything. You can read about it in the cloud-init documentation. The only question which I have after it is, why should I write the whole installation profile which is small and scarce documented subset of cloud-init options?

Providing autoinstall profile to the installer

But anyway we finished creating the installation profile and we must provide it somehow to the installation program. The official way to do it is to use subiquity.autoinstallpath option to the kernel. Of course you can do it either if the file resides on some web server, which I don’t have, or directly on the CD-ROM with Ubuntu, which I don’t have either.

After some difficult minutes breaking my head how to provide the file I found the solution. I don’t have CD-ROM, but my installation is done with NFS and the whole Ubuntu image is mounted under /cdrom during the installation. So I must place my autoinstall profile under /cdrom.

On my NIM server I created a directory for the future autoinstall profiles directly in my Ubuntu NFS share and copied the profile there:

# mkdir -p /nim/linux/ubuntu/24.04/install
# cp ubuntu.yml /nim/linux/ubuntu/24.04/install/

Now when the Ubuntu installer starts and mounts the NFS share, the profile is automatically in /cdrom/install/ubuntu.yml. I must not do anything more here.

GRUB configuration

Add the option subiquity.autoinstallpath to your kernel configuration and you are (almost) ready! Another option to add is autoinstall. If you forget the autoinstall option, you can forget about automated installations. The installer will start, read your configuration and wait for your confirmation before doing any changes.

# cat >/tftpboot/boot/grub2/grub.cfg-0A000303 <<EOF
echo "Welcome to the Ubuntu 24.04.1 installer!"
echo ""

set timeout=60

menuentry 'Install Ubuntu 24.04.1 (automatic NFS)' --class debian --class gnu-linux --class gnu --class os --id ubuntu24041nfs {
  linux (tftp,10.0.2.4)/tftpboot/ubuntu/24.04/vmlinux ro ip=10.0.3.3::10.0.3.254:255.255.255.0:ubuntu.power-devops.cloud::none:10.0.2.1 root=/dev/nfs nfsroot=10.0.2.4:/nim/linux/ubuntu/24.04 nfsopts=vers=3 netboot=nfs nfsrootdebug autoinstall quiet subiquity.autoinstallpath=/cdrom/install/ubuntu.yml
  initrd (tftp,10.0.2.4)/tftpboot/ubuntu/24.04/initrd
}
EOF

Consider supporting Power DevOps Newsletter!

Upgrade to our paid tier to unlock every article in the archive. Become a Founding Member for a little bit extra and book a 1-to-1 coffee chat with Andrey Klyachkin.

Ready to start the installation!

Not very spectacular, but it works. Ubuntu is installed and configured after several minutes of waiting. You can use it!

What I learned today?

If you don’t know something and start working with it, it looks strange and produces many problems. Headache is guaranteed. I had so many problems with the Ubuntu installer and did so many tests installing Ubuntu before I got the working configuration! But at the end after I understood the basic principles of the Ubuntu installation, it works and I see no problems changing its behaviour one more time.

It is always great to learn something new, even if you may not need it in the nearest future.

Have fun with the Ubuntu installer!

Andrey

Hi, I am Andrey Klyachkin, IBM Champion and IBM AIX Community Advocate. It means I don’t work for IBM. Over last 20 years I worked with many different IBM Power customers all over the world both on-premise and in cloud. I specialize in automating IBM Power infrastructures making them even more robust and agile. I co-authored several IBM Redbooks and IBM Power certifications. I am an active Red Hat Certified Engineer and Instructor.

Follow me on LinkedIn, Twitter and Youtube.

Meet me on events like IBM TechXchange, Common Europe Congress and GSE Germany’s IBM Power Working group sessions.