If you must find a disk for altinst_rootvg on 100 servers, it drives you crazy
The Quest for altinst_rootvg
It would be so easy if everything were standardized! Let’s imagine: all AIX LPARs have hdisk0 for rootvg and hdisk1 for altinst_rootvg. Next time I must upgrade an LPAR, I know that I will use hdisk1 to create altinst_rootvg. The only problem is that after the reboot, the disk for the next altinst_rootvg will most probably be hdisk0. The biggest problem is that even in highly standardized environments, it is rarely the case, and you must search for a suitable disk.
The task is easy - upgrade many AIX servers to AIX 7.3. The process is well known. I already wrote about it and plan to write more. But before the upgrade, we need a disk. Last time, I used a variable to specify the disk for altinst_rootvg. It's okay if you have a dozen servers. You can check them and define variables manually. If you have more than a dozen, it's a waste of time. Ansible can do it for me.
How do we search?
If I search for altinst_rootvg manually, my first choice is to use the disk in the volume group altinst_rootvg. Sometimes, altinst_rootvg was already created for backup purposes or for other reasons. I can take the disk and use it to create a new altinst_rootvg.
The next candidate is a disk from the old_rootvg volume group. If there is such a volume group on my AIX, it means AIX was already once rebooted from altinst_rootvg. After the reboot, altinst_rootvg becomes rootvg, and the old rootvg becomes old_rootvg.
If I have neither altinst_rootvg nor old_rootvg, I must search for a suitable disk. If I have LDEV from storage, I can search for it. Otherwise, I search for a suitable disk with the same size as rootvg.
Do you have any additions to the search process? Right them down into comments!
Side note
The code below searches only for one disk. If you have a mirrored rootvg with two disks, you must search for two disks. It will be your homework to change the playbook to find two suitable disks for altinst_rootvg.
Do we have altinst_rootvg?
To answer the question, we first need to obtain a list of all disks on AIX.
- name: Get hdisk information
ansible.builtin.command:
cmd: lspv -L
register: hdisks
changed_when: falseThe big “-L” in lspv means avoid LVM locking. If you perform an operation on LVM, such as synchronizing a volume group, LVM locks it. The next LVM command you issue will wait till the lock is gone. Usually, it is not a problem because most LVM operations are relatively short. But some of them, like the aforementioned synchronization, can last hours. I am not so patient and don’t want to wait only because I need a list of physical volumes.
- name: Find if we have altinst_rootvg
ansible.builtin.set_fact:
altinst_hdisk: "{{ item | split | first }}"
when: item.split()[2] == 'altinst_rootvg'
loop: "{{ hdisks.stdout_lines }}"I hope IBM will not change the lspv output in the near future. Now every hdisk device has one line in the output. It makes it easy to loop through all lines of the output and compare if we have altinst_rootvg in the volume groups column. If it is found, we take the first column, which is the disk name, and assign it to the variable altinst_hdisk.
Do we have old_rootvg?
We already have the list of disks and corresponding volume groups. We must only repeat the last task, but with another volume group name.
- name: Find if we have old_rootvg
ansible.builtin.set_fact:
altinst_hdisk: "{{ item | split | first }}"
when:
- altinst_hdisk == ''
- item.split()[2] == 'old_rootvg'
loop: "{{ hdisks.stdout_lines }}"We do completely the same, but compare with old_rootvg.
The only difference is one more condition:
- altinst_hdisk == '' We don’t want to reassign the variable if we have already found our disk for altinst_rootvg. That’s why, if we see that the variable has some value, we skip all tasks.
Search for specific LDEV
There are many ways to search for a specific LDEV. You can use lspv, lscfg, lsmpio, and storage-specific commands, if you have them. I will use here the first option - lspv.
If you issue lspv -u, you get additional information about disks:
The first 3 or 4 columns are the same as in the lspv output (without any options).
The last column is the UUID of the disk. We don’t need it in our case.
The column before UUID contains a very big string. The string ends with “vscsi” if it is a VSCSI disk or with “fcp” if it is an NPIV disk.
You can also find information about the disk driver. In my case, it is always IBM. If you look precisely at the string, you will see the same number as you see on the SVC.
Check hdisk6 from the screenshot above and the screenshots below. The first is from the SVC command line interface, the second is from the GUI.
I am sure if you use storage from another vendor, you will find similar information.
It means we must issue lspv -u and compare the output with the number we have. There are two issues with the command.
First, there is no option “-L” (ignore LVM locks) for it, and I don’t know how it works if there is an LVM operation in progress.
Second, the column with the LDEV information can be 4th or 5th, depending on whether the volume group on the disk is activated or not. But this problem is solvable. We simply count the columns from the end, and not from the beginning.
Let’s start collecting the information.
- name: Get additional hdisk information
ansible.builtin.command:
cmd: lspv -u
register: hdisks
changed_when: false
when:
- altinst_hdisk == ''
- ldev is defined and ldev != ''We need this information only if we still don’t have a disk for altinst_rootvg and an LDEV is defined.
- name: Find the suitable disk
ansible.builtin.set_fact:
altinst_hdisk: "{{ item | split | first }}"
when:
- altinst_hdisk == ''
- ldev is defined and ldev != ''
- ldev.upper() in item.split()[-2]
loop: "{{ hdisks.stdout_lines }}"Next, we go through the lspv -u output and check if we find our LDEV information in the second column from the end. If it is found, we set the same variable as before. Of course, we do nothing if the variable is already set.
Search for a volume with a specific size
Before searching for a volume, we must obtain the size of the disk for rootvg. This is a two-step process. We first get the name of the disk in rootvg:
- name: Get disk with rootvg
ansible.builtin.shell:
cmd: getlvodm -w $(getlvodm -v rootvg)
changed_when: false
when:
- altinst_hdisk == ''
register: rootvg_hdiskAs you see, I use the command getlvodm for this purpose. The command is not documented. At least you can’t find any information about it in the official AIX documentation. But it was documented in the ancient LVM Redbook (page 291, chapter B.7).
After we got the disk name, we can get its size.
- name: Get size of rootvg hdisk
ansible.builtin.command:
cmd: "getconf DISK_SIZE /dev/{{ rootvg_hdisk.stdout_lines.0 | split | last }}"
changed_when: false
register: rootvg_hdisk_size
when:
- altinst_hdisk == ''In this case, I use the documented and POSIX-compliant command getconf. If you don’t like it because of its syntax, you can use the undocumented command bootinfo -s to get the same information. In my experience, bootinfo -s didn’t work with some volume types with third-party storage drivers. But with IBM storage, it works without any problems.
We have the rootvg’s disk size. Our next step is to find free volumes that are not in use by any volume group. We use the non-documented getlvodm again.
- name: Get list of free disks
ansible.builtin.command:
cmd: "/usr/sbin/getlvodm -F"
changed_when: false
register: free_disks
when:
- altinst_hdisk == ''We also need disk sizes for our free disks.
- name: Get sizes of free disks
ansible.builtin.command:
cmd: "getconf DISK_SIZE /dev/{{ item }}"
changed_when: false
register: free_disks_sizes
loop: "{{ free_disks.stdout_lines }}"
when:
- altinst_hdisk == ''We can search now!
- name: Search for the disk with the same size as rootvg
ansible.builtin.set_fact:
altinst_hdisk: "{{ item.cmd.2 | regex_replace('\/dev\/', '') }}"
loop: "{{ free_disks_sizes.results }}"
loop_control:
label: "{{ item.cmd.2 }}"
when:
- altinst_hdisk == ''
- item.stdout == rootvg_hdisk_size.stdoutWe loop through the results of the last task. If the output for some disk is the same as the size of the rootvg hdisk (item.stdout == rootvg_hdisk_size.stdout), this is our candidate for altinst_rootvg. We must only get its name, which is in item.cmd.2, and get rid of the /dev/ prefix.
Support the Power DevOps Newsletter!
If you like reading technical articles about IBM Power, AIX, and Linux on IBM Power, consider upgrading to the paid tier to show your support. As a paid subscriber, you not only get regular posts, but you will get additional posts with the full code and further explanations, access to the whole archive of the blog, and take part in our monthly calls where you can ask your questions and propose topics for future newsletters. Be an active member of our community!
Finish? Start!
The playbook is finished. But in reality, you don’t need the playbook. You want to find a suitable disk for altinst_rootvg because you intend to use it for something. Like, I want to use it to perform an AIX upgrade. It means I need a role that I can use in other playbooks. Or a simple file with tasks that I can include in another playbook.
Think about what you need and use the code above in the way you need. Don’t stupidly copy the code and hope that it magically works in your environment!
Have fun creating altinst_rootvg!
Andrey
Hi, I am Andrey Klyachkin, IBM Champion and IBM AIX Community Advocate. This means I don’t work for IBM. Over the last twenty years, I have worked with many different IBM Power customers all over the world, both on-premise and in the cloud. I specialize in automating IBM Power infrastructures, making them even more robust and agile. I co-authored several IBM Redbooks and IBM Power certifications. I am an active Red Hat Certified Engineer and Instructor.
Follow me on LinkedIn, Twitter and YouTube.
You can meet me at events like IBM TechXchange, the Common Europe Congress, and GSE Germany’s IBM Power Working Group sessions.