Argh! I've got a problem with NIM!

Yes, I am testing it for you ;-)

I installed my new NIM server last week, together with the security patch. Meanwhile, the new 7.3 TL4 became available with support for YubiKey! Mmm. I want it! Of course, I updated my freshly installed NIM to AIX 7.3 TL4. Oops, it stopped working!

OK, thanks to the friendly help from IBM Software Lab in Hyderabad I’ve got it working again. But there are still questions…

Why did I update to TL4 the next day?

You know this old sysadmin’s truth - never ever install fresh software! Especially not on Friday!

A seasoned administrator with many decades of experience installs the update, which has been available for only a few hours. What?

The reason is easy. I’ve got the feedback after the last newsletter, which I forgot to mention:

NIM should always have the same or newer AIX version than the systems it manages.

Of course, it makes NIM the best place to test new service packs and TLs.

No, the real reason: I wanted to test YubiKey with AIX, and it was the nearest installed server that I could update to TL4.

But you probably shouldn’t do such tests on your production NIM server.

As for me, it was my freshly installed lab NIM server, and I can do everything on it that I wish.

What was the problem?

My new NIM environment had the security fix installed. On the NIM server itself and on all clients.

To update the NIM server, I had to remove the fix, and it made all my NIM clients stop working. This is not the problem. Because the NIM server was updating anyway.

But my NIM clients couldn’t communicate with the NIM server even after the update.

My first thought was that IBM forgot to incorporate the fix into the new TL4.

Fortunately, it is not the case.

5 MINUTES later, after I asked the question, Carl Burnett confirmed that the fix is in TL4. It was the fastest and most helpful answer! Moreover, he connected me with the development team at the IBM Software Lab to identify the probable cause. I don’t know if you read my newsletter, Carl, but thank you! You helped me one more time.

I can’t say that I liked the solution, but it was very easy:

nimconfig -c

After that, I reconnected all my clients to the NIM server, and everything is good again.

Why didn’t I like this?

The reason is that I had to reconnect all my clients. OK, I have only two clients in my test lab, and this is not a problem. But if I have 100 AIX clients? In another environment, where I thought about installing TL4, I have more than 500 AIX clients:

I will not jump between AIX LPARs to reconnect them to the NIM server.

Yes, I will use Ansible for this task ;-)

Do you want more problems?

No, it is not a real problem. It is something that I could understand if I had read everything carefully. But who reads README files?

After installing the fix or updating the NIM server to TL4, NIM uses a different SSL mechanism that is incompatible with previous AIX versions.

If you have older AIX versions in your environment, and you can’t get the fix for them from IBM, you better stay without the fix on your NIM server and encryption.

Don’t configure SSL, and you still can use your brand-new AIX 7.3 TL4 NIM environment with your old (and unsupported) AIX LPARs. I tested it, and it works.

Remember that running unencrypted traffic in your network means security problems. Everyone can read what you send.

It is more secure to update the NIM server to TL4, install the fix across the entire environment, and use encrypted communications.

But now we have a new problem, and I don’t have a solution for it yet.

Let’s say you have AIX 7.2 TL5 SP10 with the fix, or you have AIX 7.3 TL3 SP1 with the fix, and you want to update the system to the latest version.

To update it, you must uninstall the NIM fix. But if you uninstall it, you lose your connection to the NIM server.

If you have any ideas, write them to me, and I will test them. Or write in the comments under the newsletter.

Support the Power DevOps Newsletter!

If you like reading technical articles about IBM Power, AIX, and Linux on IBM Power, consider upgrading to the paid tier to show your support. As a paid subscriber, you not only get regular posts, but you will get additional posts with the full code and further explanations, access to the whole archive of the blog, and take part in our monthly calls where you can ask your questions and propose topics for future newsletters. Be an active member of our community!

What about YubiKey?

My YubiKey works!

I didn’t do anything special. I copied my SSH public key to the NIM server, and could connect to it using YubiKey.

If I have enough time by Friday, I’ll do a 2-minute YouTube video.

Have fun with AIX 7.3 TL4!

Andrey

Hi, I am Andrey Klyachkin, IBM Champion and IBM AIX Community Advocate. This means I don’t work for IBM. Over the last twenty years, I have worked with many different IBM Power customers all over the world, both on-premise and in the cloud. I specialize in automating IBM Power infrastructures, making them even more robust and agile. I co-authored several IBM Redbooks and IBM Power certifications. I am an active Red Hat Certified Engineer and Instructor.

Follow me on LinkedIn, Twitter and YouTube.

You can meet me at events like IBM TechXchange, the Common Europe Congress, and GSE Germany’s IBM Power Working Group sessions.