Both nodes must have:
sudo dnf install -y drbd-utils drbd-selinux
sudo lvcreate -L 20G -n navidrome almalinux
sudo lvcreate -L 1G -n navidrome-meta almalinux
Create /etc/drbd.d/navidrome.res:
resource navidrome {
protocol C;
disk {
on-io-error detach;
}
net {
verify-alg sha256;
}
on thing-1.customstack.nyc {
device /dev/drbd0;
disk /dev/almalinux/navidrome;
address 10.0.0.21:7789;
meta-disk /dev/almalinux/navidrome-meta;
}
on thing-2.customstack.nyc {
device /dev/drbd0;
disk /dev/almalinux/navidrome;
address 10.0.0.22:7789;
meta-disk /dev/almalinux/navidrome-meta;
}
}
Critical: No [0] index on meta-disk — that triggers stacked mode detection and breaks the RA.
Critical: Use FQDNs in the on clauses. DRBD matches via uname -n — if the hostname doesn't match exactly, DRBD will say "not defined in your config" and refuse to start. Verify with uname -n on each node before writing this file.
sudo drbdadm create-md navidrome
sudo drbdadm up navidrome
sudo drbdadm status navidrome
sudo drbdadm primary --force navidrome
sudo drbdadm status navidrome
Wait for sync to complete — both nodes should show UpToDate.
sudo mkfs.xfs /dev/drbd0
sudo drbdadm secondary navidrome
The DRBD kernel modules must be loaded before Pacemaker tries to start the resource. Without this, Pacemaker will fail with "DRBD resource not found" after a reboot.
echo -e "drbd\ndrbd_transport_tcp" | sudo tee /etc/modules-load.d/drbd.conf
sudo systemctl enable drbd
Before handing control to Pacemaker, confirm the DRBD device has the right database. DRBD syncs block-level data — whatever was on the Primary node's LV is what gets replicated. Verify you have the right database before proceeding.
sudo mount /dev/drbd0 /mnt
sudo ls -lh /mnt/navidrome.db
sudo sqlite3 /mnt/navidrome.db "SELECT COUNT(*) FROM library;"
sudo sqlite3 /mnt/navidrome.db "SELECT COUNT(*) FROM user;"
sudo umount /mnt
sudo drbdadm secondary navidrome
Expected: 122 libraries, 37 users. If numbers are wrong, restore from backup before continuing — once Pacemaker takes over it's harder to intervene.
sudo pcs resource create drbd-navidrome ocf:linbit:drbd \
drbd_resource=navidrome \
ignore_missing_notifications=true \
op monitor interval=30s
sudo pcs resource promotable drbd-navidrome \
promoted-max=1 promoted-node-max=1 \
clone-max=2 clone-node-max=1 \
notify=true
The filesystem resource mounts /dev/drbd0 on whichever node Pacemaker has promoted to DRBD Primary. It follows the DRBD Primary around the cluster via colocation constraint — you never need to mount it manually.
What it does:
mount /dev/drbd0 /var/lib/navidromeumount /var/lib/navidrome before DRBD demotessudo pcs resource create fs-navidrome ocf:heartbeat:Filesystem \
device=/dev/drbd0 \
directory=/var/lib/navidrome \
fstype=xfs \
op monitor interval=30s
# Filesystem must run on whichever node has DRBD promoted
sudo pcs constraint colocation add fs-navidrome with Promoted drbd-navidrome-clone INFINITY
# Filesystem starts after DRBD is promoted
sudo pcs constraint order promote drbd-navidrome-clone then start fs-navidrome
# Navidrome starts after filesystem and senuti NFS mount
sudo pcs constraint order start fs-navidrome then start ha-navidrome
sudo pcs constraint order start ha-nfs-senuti-mount then start ha-navidrome
sudo pcs constraint colocation add ha-navidrome with fs-navidrome INFINITY
sudo pcs resource enable ha-navidrome
sudo pcs status
sudo drbdadm status navidrome
# Both nodes should show UpToDate
# One node role:Primary, one role:Secondary
sudo pcs status
# drbd-navidrome-clone: Promoted on one node, Unpromoted on other
# fs-navidrome: Started on same node as Promoted
# ha-navidrome: Started on same node as fs-navidrome
/dev/drbd0 manually while Pacemaker is managing it/dev/almalinux/navidrome directly — always go through DRBD/dev/drbd0 or /dev/almalinux/navidrome to /etc/fstab — Pacemaker owns that mount/usr/local/bin/navidrome-backup.sh nightly at 3am, DRBD-aware