Skip to content

Ubuntu OS Upgrade Guide (20.04 to 22.04 to 24.04)

1. Prerequisites

Python version: 3.12.X

Rdaf CLI version: 1.4.0

RDAF Infra Tags: 1.0.3

RDAF Platform/Worker/App tags: 8.0.0

Important

Internet should be available on all nodes.

2. Preparation Steps

  • Always take VM-level snapshots in vCenter for all the VMs.

  • Always choose default options when prompted during upgrade.

  • Each VM upgrade is approximately 1.5 hours.

3. Upgrade Order

  • APP/Platform VMs

  • Worker VMs

  • Infra VMs

  • Registry VM

  • Gateway VMs

4. Upgrade Process

4.1 Upgrade Ubuntu 20.04 to 22.04

Step 1. Take a backup of sysctl.conf & sudoers file.

cp /etc/sysctl.conf /home/rdauser/
sudo cp /etc/sudoers /home/rdauser

rdauser@aiaperf-worker-10853:~$ cp /etc/sysctl.conf /home/rdauser/
rdauser@aiaperf-worker-10853:~$ sudo cp /etc/sudoers /home/rdauser

Step 2. Verify UBUNTU Version

  • To verify current ubuntu version use the following command.
lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.4 LTS
Release:        20.04
Codename:       focal

Step 3. System Upgrade

To Update the package lists and upgrade all packages to their latest available versions please use the below given command.

sudo apt update -y && sudo apt full-upgrade -y

Step 4. Update Docker Configuration

  • Before making any changes, back up the current Docker configuration file 
cp /etc/docker/daemon.json /home/rdauser

Note

Once the daemon.json file has been modified or updated, there is no need to repeat the process.

  • Edit the Docker configuration file using a text editor using this command.
sudo vi /etc/docker/daemon.json
  • Replace the content with the following configuration given below.
{
 "tls": true,
 "tlscacert": "/etc/tlscerts/ca/ca.pem",
 "fixed-cidr-v6": "fdc6:ef49:83db::/64",
 "ip6tables": true,
 "exec-opts": [
  "native.cgroupdriver=cgroupfs"
 ],
 "tlsverify": true,
 "default-cgroupns-mode": "host",
 "storage-driver": "overlay2",
 "hosts": [
  "unix:///var/run/docker.sock",
  "tcp://0.0.0.0:2376"
 ],
 "tlskey": "/etc/tlscerts/server/server.key",
 "ipv6": true,
 "debug": false,
 "tlscert": "/etc/tlscerts/server/server.pem",
 "userland-proxy": false,
 "experimental": true,
 "live-restore": true
}

Step 5. Edit Docker Service Configuration

  • Edit the Docker service file to apply necessary custom configurations, then follow with the command.
sudo vi /usr/lib/systemd/system/docker.service
  • Comment out the existing ExecStart line.
#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
  • Add the following new ExecStart line.
ExecStart=/usr/bin/dockerd

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd
#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always

Step 6. Reload & Restart Docker Service

Reload the systemd manager configuration and restart the Docker service.

sudo systemctl daemon-reload && sudo systemctl restart docker

Note

When stopping services on the infrastructure VM, ensure you first stop the MariaDB services using docker stop -t 120 <container_ID>, followed by stopping all other services.

Step 7. Stop All Docker Containers

  • Run the following command to stop all running containers on this VM.
docker stop $(docker ps -a -q)

0a33f1394630
1b3f3ec30ecb
afb59cf11563
ea187f89505f

Each line represents a container ID that has been successfully stopped.

  • Restart the system using the below given command
sudo init 6

Step 8. Start Docker Containers After Reboot

  • Once the VM is back online, start all previously stopped containers using the command.
docker start $(docker ps -a -q)

0a33f1394630
1b3f3ec30ecb
afb59cf11563
ea187f89505f
  • Each line represents a container that has been successfully started.

4.2 Ubuntu Upgrade

Important

This applies exclusively to the REGISTRY VM.

  • Before upgrading the OS on the Registrry VM, Ensure the Python symlink is correctly set by running the following commands.
sudo rm -f /usr/bin/python3
sudo ln -s /usr/bin/python3.8 /usr/bin/python3

Note

This step is mandatory to avoid Python-related issues during the OS upgrade.

Step 1. Perform Distribution Upgrade

sudo do-release-upgrade -f DistUpgradeViewNonInteractive

Step 2. Update Docker Config

  • Open and edit the Docker service configuration file using the vi editor.
sudo vi /usr/lib/systemd/system/docker.service
  • Comment out the following line in the file.

#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
* Add the following line above the existing ExecStart line.

ExecStart=/usr/bin/dockerd

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd
#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always

Step 3 Reload and Restart Docker Service

  • To apply configuration changes and restart the Docker service, run the following command.
sudo systemctl daemon-reload && sudo systemctl restart docker

Note

When shutting down services on the infrastructure VM, first stop the MariaDB service using the command given below, and then proceed to stop all other services.

docker stop -t 120 <container_ID>
  • To stop all running and stopped Docker containers on this VM, execute

docker stop $(docker ps -a -q)
The above command retrieves all container IDs and stops them.

0a33f1394630
1b3f3ec30ecb
afb59cf11563
ea187f89505f
  • First, restart the VM using the following command
sudo init 6
  • Once the VM has restarted, start all Docker containers using the following command
docker start $(docker ps -a -q)

0a33f1394630
1b3f3ec30ecb
afb59cf11563
ea187f89505f
  • To verify current ubuntu version use the following command
lsb_release -a

4.3 Upgrade Ubuntu 22.04 to 24.04

Step 1. Use the below command to Update your system packages.

sudo apt update -y && sudo apt full-upgrade -y
Step 2. Perform Distribution Upgrade

  • To upgrade ubuntu from 22.04 to 24.04 please use the command given below.
sudo do-release-upgrade -f DistUpgradeViewNonInteractive

Step 3. Update Docker Config

  • To update Docker configuration, edit the service file.
sudo vi /usr/lib/systemd/system/docker.service
  • Comment out the following line in the file
#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
  • Add the following new ExecStart line.
ExecStart=/usr/bin/dockerd

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd
#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always

Step 4 Reload and Restart Docker Service

  • To reload and restart the Docker service use the following command.
sudo systemctl daemon-reload && sudo systemctl restart docker

Step 5. Stop All Docker Containers

  • To stop all running containers on this VM use the following command.
docker stop $(docker ps -a -q)

0a33f1394630
1b3f3ec30ecb
afb59cf11563
ea187f89505f
  • To restart the VM use the following command
sudo init 6

Step 6. Start Docker Containers After Reboot

  • After the VM boots up, start all containers.
docker start $(docker ps -a -q)

0a33f1394630
1b3f3ec30ecb
afb59cf11563
ea187f89505f
  • To verify current ubuntu version use the following command.
lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 24.04.2 LTS
Release:        24.04
Codename:       noble

5. Post-Upgrade Steps on Registry VM Only

  • Remove the deadsnakes PPA
sudo add-apt-repository --remove ppa:deadsnakes/ppa
  • Clean and update package lists
sudo apt clean && sudo apt update
  • Remove existing Python3 using the command given below.
sudo apt remove python3
  • Install specific Python 3.12 standard library.
sudo apt install libpython3.12-stdlib=3.12.3-1ubuntu0.5
  • Use the following command to install required tools.
sudo apt install vim ufw
  • Configure pip to break system packages
python3 -m pip config set global.break-system-packages true
  • Install the specific rdafcli package
pip install rdafcli-1.4.0.tar.gz

6. Post-Upgrade Validation

  • Run the following commands to verify the system
rdaf infra status
rdac pods
rdac healthcheck

7. UI Validation

Log in to the UI and verify the following.

  • Alerts

  • Incidents

  • Dashboards

  • PStreams

8. Troubleshooting

Traceback (most recent call last):
  File "/usr/lib/cnf-update-db", line 8, in 
    from CommandNotFound.db.creator import DbCreator
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 12, in 
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'
  • Run the following command to create a symbolic link and resolve the Issue.
sudo ln -s /usr/lib/python3/dist-packages/apt_pkg.cpython-38-x86_64-linux-gnu.so /usr/lib/python3/dist-packages/apt_pkg.so