Upgrade from 3.7.2 to 8.1.0.1 Using Offline Bundles.
1. Upgrade From 3.7.2 to 8.1.0.1 and 7.7.2 to 8.1.0.1 Using Offline Bundles
-
RDAF Infra Upgrade: From 1.0.3, 1.0.3.3 (haproxy) to 1.0.4
-
RDAF Platform: From 3.7.2 to 8.1.0.1
-
OIA (AIOps) Application: From 7.7.2 to 8.1.0.1
-
RDAF Deployment
rdafCLI: From 1.3.2 to 1.4.1 -
RDAF Client
rdacCLI: From 3.7.2 to 8.1.0.1 -
Python: From 3.7 to 3.12
2. Prerequisites
Before proceeding with this upgrade, please make sure and verify the below prerequisites are met.
-
RDAF Deployment CLI version: 1.3.2
-
Infra Services tag: 1.0.3 / 1.0.3.3 (haproxy)
-
Platform Services and RDA Worker tag: 3.7.2
-
OIA Application Services tag: 7.7.2
-
CloudFabrix recommends taking VMware VM snapshots where RDA Fabric infra/platform/applications are deployed
Important
-
If the Webhook URL is currently configured with port 7443, it should be updated to port 443. Below are the steps to update Webhook URL:
-
Login to UI → Click on Administration → Organization → click on Configure → click on Alert Endpoints → click on required Endpoint and edit to update the port (Note: Please ignore if the Alert Endpoint is already configured with port 443)
3. Upgrade Steps
3.1 Python Upgrade from 3.7 to 3.12
Note
The following steps involve installing Python 3.12 on a machine with internet connectivity. These commands should be executed only on the on-premises Registry/CLI VM.
- Please refer Python Upgrade Guide From Python 3.7 to Python 3.12
Important
Ensure that the highlighted disk size (Use% column) should be less than 50%. If any disk exceeds this percentage, consider taking appropriate actions like cleaning up unused files or expanding storage.
rdauser@oia-125-216:~/collab-3.7-upgrade$ df -kh
Filesystem Size Used Avail Use% Mounted on
udev 32G 0 32G 0% /dev
tmpfs 6.3G 357M 6.0G 6% /run
/dev/mapper/ubuntu--vg-ubuntu--lv 48G 12G 34G 26% /
tmpfs 32G 0 32G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 32G 0 32G 0% /sys/fs/cgroup
/dev/loop0 64M 64M 0 100% /snap/core20/2318
/dev/loop2 92M 92M 0 100% /snap/lxd/24061
/dev/sda2 1.5G 309M 1.1G 23% /boot
/dev/sdf 50G 3.8G 47G 8% /var/mysql
/dev/loop3 39M 39M 0 100% /snap/snapd/21759
/dev/sdg 50G 541M 50G 2% /minio-data
/dev/loop4 92M 92M 0 100% /snap/lxd/29619
/dev/loop5 39M 39M 0 100% /snap/snapd/21465
/dev/sde 15G 140M 15G 1% /zookeeper
/dev/sdd 30G 884M 30G 3% /kafka-logs
/dev/sdc 50G 3.3G 47G 7% /opt
/dev/sdb 50G 29G 22G 57% /var/lib/docker
/dev/sdi 25G 294M 25G 2% /graphdb
/dev/sdh 50G 34G 17G 68% /opensearch
/dev/loop6 64M 64M 0 100% /snap/core20/2379
3.2 RDAF Deployment CLI Upgrade
Note
Upgrade the RDAF Deployment CLI on both the on-premise Docker registry VM and the RDAF Platform's management VM if they are provisioned separately.
Perform these steps on the VM where the RDAF Deployment CLI was installed, whether it is managing Docker registry or non-Kubernetes deployments.
Note
This process ensures the RDAF CLI is upgraded without requiring internet access on the management VM.
- Download the RDAF Deployment CLI's newer version 1.4.1 bundle and copy it to RDAF management VM on which
rdafdeployment CLI was installed.
wget https://macaw-amer.s3.us-east-1.amazonaws.com/releases/rdaf-platform/1.4.1/offline-ubuntu-1.4.1.tar.gz
- Extract the
rdafCLI software bundle contents
- Change the directory to the extracted directory
- Upgrade the
rdafCLI to version 1.4.1
- Verify the installed
rdafCLI version
3.3 Run Registry Upgrade Script
- Please run the downloaded python upgrade script rdaf_upgrade_132_141.py as shown below
Note
The above command will show the available options for the upgrade script
usage: rdfa_upgrade_132_141.py [-h] {registry_upgrade,upgrade,haproxy_upgrade,cleanup_haproxy,migrate_dataset} ...
options:
-h, --help show this help message and exit
options:
{registry_upgrade,upgrade,haproxy_upgrade,cleanup_haproxy,migrate_dataset}
Available options
registry_upgrade Upgrade on prem registry if any
upgrade upgrade the setup
haproxy_upgrade Upgrade HAProxy with VIP
cleanup_haproxy cleaning up haproxy and keepalived
migrate_dataset Migrate dataset using script
- If there is already an on-premises registry, upgrade it with the command below.
Creating backup rdaf-registry.cfg
Login Succeeded
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /home/rdauser/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Updating docker-compose binary...
- Download the Archive using the below given command.
- Use the command given below and Extract the downloaded
tar.gzfile in/home/rdauser
All-Infra.tar.gz - 4.306160542 gigabyte
rda-platform-opensearch.tar.gz - 1029.852369 megabyte
rda-platform-haproxy.tar.gz - 182.07278 megabyte
rda-platform-busybox.tar.gz - 2.149061 megabyte
rda-platform-kubectl.tar.gz - 109.690924 megabyte
rda-platform-kafka.tar.gz - 229.351941 megabyte
rda-platform-mariadb.tar.gz - 133.30942 megabyte
rda-platform-nats.tar.gz - 12.396015 megabyte
rda-platform-prometheus-nats-exporter.tar.gz - 7.565907 megabyte
rda-platform-nats-box.tar.gz - 41.139901 megabyte
rda-platform-telegraf.tar.gz - 222.317013 megabyte
rda-platform-logstash.tar.gz - 1023.630364 megabyte
rda-platform-kube-arangodb.tar.gz - 66.64376 megabyte
rda-platform-nats-boot-config.tar.gz - 12.971209 megabyte
docker-registry.tar.gz - 295.227198 megabyte
rda-platform-filebeat.tar.gz - 285.381289 megabyte
minio.tar.gz - 61.436229 megabyte
rda-platform-arangodb-starter.tar.gz - 12.01858 megabyte
rda-platform-arangodb.tar.gz - 240.012853 megabyte
mc-RELEASE.2024-11-21T17-21-54Z.tar - 227.08992 megabyte
Note
Make sure all the above tar files are copied to /opt/rdaf-registry/import/ on the VM hosting RDAF registry server role.
- Using command below Load the Docker image from the extracted file.
rdauser@hariofflineregistry108131:~$ docker load -i docker-registry.tar.gz
ec34fcc1d526: Loading layer [==================================================>] 5.811MB/5.811MB
7b35f2def65d: Loading layer [==================================================>] 736.3kB/736.3kB
f93fff1ab6f7: Loading layer [==================================================>] 18.09MB/18.09MB
6a4340199717: Loading layer [==================================================>] 4.096kB/4.096kB
4977497eb9c0: Loading layer [==================================================>] 2.048kB/2.048kB
39dbfc8f73cb: Loading layer [==================================================>] 6.144kB/6.144kB
7e6538a47f65: Loading layer [==================================================>] 673.4MB/673.4MB
2219e255dd2d: Loading layer [==================================================>] 34.67MB/34.67MB
a33e534d7d88: Loading layer [==================================================>] 252.2MB/252.2MB
789bacf5468f: Loading layer [==================================================>] 3.072kB/3.072kB
bcabefa6aac9: Loading layer [==================================================>] 3.072kB/3.072kB
87c19978dcb7: Loading layer [==================================================>] 2.56kB/2.56kB
Loaded image: docker1.cloudfabrix.io:443/external/docker-registry:1.0.4
- Use this command to check the list of Docker images to confirm that the image has been loaded successfully.
3.4 Upgrade On-Prem Registry
- Please update the registry by using the below command
4. Fetching Latest Tags
To update to the latest tags, ensure that all files are copied into the directory /opt/rdaf-registry/import.
Note
Infrastructure files are imported separately, while platform and application files are managed using packaged tar files.
- Use the following command to download the comprehensive package containing all necessary platform components.
wget https://macaw-amer.s3.us-east-1.amazonaws.com/releases/RDA/8.1.0.1/All-Platform-tarfiles.tar.gz
- Execute this command to obtain the full package required for OIA integration.
- Retrieve the full package needed for on-premises setup using the command below.
5. Running the Registry Import to Fetch Tags
- Use the following command to import each package or tag file into the registry.
rdaf registry import --file All-onprem.tar.gz
rdaf registry import --file All-Platform.tar.gz
rdaf registry import --file All-OIA.tar.gz
rdaf registry import --file mc-RELEASE.2024-11-21T17-21-54Z.tar
rdaf registry import --file minio.tar.gz
rdaf registry import --file rda-platform-arangodb-starter.tar.gz
rdaf registry import --file rda-platform-arangodb.tar.gz
rdaf registry import --file rda-platform-busybox.tar.gz
rdaf registry import --file rda-platform-filebeat.tar.gz
rdaf registry import --file rda-platform-haproxy.tar.gz
rdaf registry import --file rda-platform-kafka.tar.gz
rdaf registry import --file rda-platform-kube-arangodb.tar.gz
rdaf registry import --file rda-platform-kubectl.tar.gz
rdaf registry import --file rda-platform-logstash.tar.gz
rdaf registry import --file rda-platform-mariadb.tar.gz
rdaf registry import --file rda-platform-nats-boot-config.tar.gz
rdaf registry import --file rda-platform-nats-box.tar.gz
rdaf registry import --file rda-platform-nats.tar.gz
rdaf registry import --file rda-platform-opensearch.tar.gz
rdaf registry import --file rda-platform-prometheus-nats-exporter.tar.gz
rdaf registry import --file rda-platform-telegraf.tar.gz
- To view the list of tags available in the registry, use the below given command
+---------------------------------------+---------------------------------------+
| Service-Name | Tags |
+---------------------------------------+---------------------------------------+
| cfx-rda-notification-service | 7.7.2, 7.7, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| rda-platform-arangodb-starter | 1.0.3, 1.0.4 |
+---------------------------------------+---------------------------------------+
| ubuntu-cfxdx-nb-nginx-all | 3.7, 3.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| cfx-rda-resource-manager | 3.7, 3.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| ubuntu-rda-worker-all | 3.7, 3.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| rda-platform-haproxy | 1.0.3, 1.0.3.3, 1.0.4 |
+---------------------------------------+---------------------------------------+
| rda-platform-filebeat | 1.0.4 |
+---------------------------------------+---------------------------------------+
| ubuntu-rda-asm | 3.7, 3.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| rda-platform-k8s-haproxy | |
+---------------------------------------+---------------------------------------+
| cfx-rda-event-consumer | 7.7, 7.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| ubuntu-rdac | 3.7, 3.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| rda-platform-mariadb | 1.0.3, 1.0.4 |
+---------------------------------------+---------------------------------------+
| cfx-rda-webhook-server | 7.7, 7.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| cfx-rda-configuration-service | 7.7, 7.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| minio | RELEASE.2023-09-30T07-02-29Z, RELEASE.2024-12-18T13-15-44Z |
+---------------------------------------+---------------------------------------+
| rda-platform-busybox | 1.0.3, 1.0.4 |
+---------------------------------------+---------------------------------------+
| cfx-rda-app-controller | 7.7.2, 7.7, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| ubuntu-rda-registry | 3.7.2, 3.7, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| ubuntu-rda-collector | 3.7, 3.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| ubuntu-rdac-full | 3.7, 3.7.2 |
+---------------------------------------+---------------------------------------+
| cfxcollector | 3.7, 3.7.2 |
+---------------------------------------+---------------------------------------+
| ubuntu-rda-identity | 3.7, 3.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| onprem-portal-nginx | 3.7, 3.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| ubuntu-rda-bulkstats | 3.7, 3.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| rda-platform-kubectl | 1.0.3, 1.0.4 |
+---------------------------------------+---------------------------------------+
| cfx-rda-ingestion-tracker | 7.7, 7.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| ubuntu-rda-client-api-server | 3.7, 3.7.2, 8.1.0.1 |
+---------------------------------------+---------------------------------------+
| cfx-rda-file-browser | 7.7, 7.7 |
+---------------------------------------+---------------------------------------+
- If necessary, kindly remove outdated image tags from the on-premise registry that are no longer in use by executing the following command:
- Please proceed with the full upgrade following the instructions in this Document
Note
There's no need to execute the registry commands.