🚀 Migrating Prometheus & Grafana from RHEL 7 to RHEL/Rocky/Alma 8

Published on August 3, 2025

A practical guide to moving your monitoring stack safely with zero data loss.

🧩 Objectives

Preserve Prometheus data from local storage
Migrate Grafana dashboards (SQLite, MySQL, or PostgreSQL)
Set up systemd services with proper permissions
Update Prometheus config for new hostnames

📦 Required Packages

sudo dnf install -y wget curl git vim tar
sudo dnf install -y grafana
sudo useradd --no-create-home --shell /bin/false prometheus

📁 Step 1: Backup Existing Data

# Prometheus data
tar -czf prometheus-data.tar.gz /var/lib/prometheus

# Grafana SQLite DB
cp /var/lib/grafana/grafana.db ~/grafana.db.backup

🐬 For MySQL:

mysqldump -u root -p grafana > grafana.sql

🐘 For PostgreSQL:

pg_dump -U grafana grafana > grafana.pgsql

📂 Step 2: Transfer Files to New Server

scp prometheus-data.tar.gz root@newhost:/tmp/
scp grafana.db.backup root@newhost:/tmp/
scp grafana.sql root@newhost:/tmp/        # If using MySQL
scp grafana.pgsql root@newhost:/tmp/      # If using PostgreSQL

⚙️ Step 3: Set Up Prometheus on RHEL 8

Download Prometheus:

cd /opt
wget https://github.com/prometheus/prometheus/releases/latest/download/prometheus-*.linux-amd64.tar.gz
tar -xzf prometheus-*.tar.gz
mv prometheus-*/ prometheus/

Create directories and users:

mkdir -p /etc/prometheus /var/lib/prometheus
cp /opt/prometheus/prometheus.yml /etc/prometheus/
useradd --no-create-home --shell /bin/false prometheus

Restore Prometheus data:

tar -xzf /tmp/prometheus-data.tar.gz -C /
chown -R prometheus:prometheus /etc/prometheus /var/lib/prometheus /opt/prometheus

🔧 Update prometheus.yml for new hostname:

- job_name: 'node_exporter'
  static_configs:
    - targets: ['new-server-name:9100']

Create systemd service:

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
ExecStart=/opt/prometheus/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/var/lib/prometheus

[Install]
WantedBy=multi-user.target

systemctl daemon-reexec
systemctl daemon-reload
systemctl enable --now prometheus

📊 Step 4: Set Up Grafana

Install Grafana:

sudo dnf install -y grafana

Restore based on your DB engine:

SQLite:

cp /tmp/grafana.db.backup /var/lib/grafana/grafana.db
chown grafana:grafana /var/lib/grafana/grafana.db

MySQL:

mysql -u root -p grafana < /tmp/grafana.sql

PostgreSQL:

psql -U grafana -d grafana < /tmp/grafana.pgsql

systemctl enable --now grafana-server

🔐 Permissions Recap

chown -R prometheus:prometheus /etc/prometheus /var/lib/prometheus /opt/prometheus
chown -R grafana:grafana /var/lib/grafana

🧪 Final Checks

✔️ Prometheus at http://<ip>:9090
✔️ Grafana at http://<ip>:3000
✔️ Dashboards and metrics successfully migrated

✅ Summary

This guide helps you migrate your Prometheus and Grafana stack with zero data loss and full operational continuity. From storage to systemd, it aligns with upstream best practices and sets you up for future scalability.