Continuing from a previous post

I realized that I could move the storage connections to the new NFS on the backend without migrating the apps just yet. I decided this would be my next move.

Old Apps Storage Migration

I have the media storage connected using custom yaml files with ArgoCD, and thought I could just update/replace those with new ones linked to the new NFS. The immediate issue that I found was that NFS was not working from K8s although I was able to mount the NFS storage to the VM.

Persistent Storage

I decided to start with the PVC, and googled Fud my way into getting PVs to deploy on the NFS share. In doing this I realized that I had never set up NFS how I thought and it was always using the hostpath storage.

Rather than solve this on an outgoing server, I noted the issue and left these apps using “local” storage. I validated that the PVs were linking to the correct folders on the local mount instead of the NFS share.

Media Apps

I did one final sync of the media folder between old and new and then fired up the Apps. Everything seemed to work as before, so I let this run for ~24 hours for any issues.

Media App Migration

Finally it was time to move to media apps running on the new home server, but first I needed to fix NFS storage. Like the old server, NFS was not actually working from K8s and what seemed to be working was using hostpath storage. I started by unmounting the NFS share from the underlying VM, and stopping all the apps that were using it.

Storage Fix

I started with the PVC again and copied my work from the old server to get dynamic PVs to deploy on the NFS share. It took me far longer than it should have to notice where in the code was setting the PV to filesystem but I found it with some help from Gemini. I used a single app that links to the media storage to repeatedly deploy and test where the storage was linked.

Once the storage pieces were actually connecting to the NFS shares from K8s, I updated my copy scripts and recopied the app config files from old PV folders to new. I still was not sure if this was going to work, so I prepared myself to rebuild the apps if needed.

App Migration

Now that the config files were linked correctly on the backend, I experienced none of the issues I was expecting and the media apps booted right up. I did some testing to make sure it could still playback and transcode, but the apps all functioned as if nothing was different.

Hot Spare & Cleanup

I shut down the old PC and removed it from under my desk. I gathered the 12TB HDD from the PC, the external 1TB drive, and cables to install in the cupboard. Once gear was installed and the new server rebooted, I noticed I still had a boot issue with the proxmox CT.

With a little more Google Fu I found that adding a “root delay” on the kernel cmd line could solve the issue. I found where to add the config and rebooted the server to verify - CT now boots and the VM waits.

Lastly I added the 4th 12TB drive as a hot spare to the ZFS Pool and enabled auto replace. This gives me 21TB of usable space and an immediate fix for a single drive failure.

Migration Complete!

Project is continued here

---

← Back to blog