Zero-Downtime SAN Storage Migrations with Oracle ASM: Adding & Dropping Disks Live

Introduction: Why SAN Migrations No Longer Require Downtime

When enterprise storage arrays reach end-of-life or when migrating from traditional Fibre Channel SAN to high-performance NVMe storage, storage engineers often request multi-hour database maintenance windows.

With Oracle Automatic Storage Management (ASM), taking the production database offline for a storage migration is completely unnecessary.

Because ASM abstracts physical disks into extent-based allocation units, DBAs can present new storage LUNs to the OS host and issue a single SQL command to add new disks and drop old disks in a single atomic rebalance operation. The database remains 100% online with zero application disruption.

---

📋 SAN Storage Migration Workflow Overview

[ Old SAN Array (LUNs 1-4) ]                     [ New NVMe Array (LUNs 5-8) ]
           \                                                    /
            \--->  [ Oracle ASM Diskgroup: DATA ]  <-----------/
                     (Atomic Live Rebalance Operation)

1. Storage Provisioning: Zone and present new SAN LUNs to Linux database hosts.
2. OS Device Configuration: Configure multipathd and udev permission rules.
3. Atomic ASM Migration: Execute ALTER DISKGROUP ... ADD DISK ... DROP DISK.
4. Rebalance Monitoring: Monitor I/O speed via V$ASM_OPERATION and ASM_POWER_LIMIT.
5. Storage Reclamation: Unpresent old LUNs after rebalance status reaches COMPLETED.

---

Step-by-Step Migration Execution

Step 1: Discover and Prepare New OS Storage LUNs

After your storage administrator presents the new LUNs, scan the SCSI bus and verify multipath bindings:

# Rescan SCSI host buses on Linux
for host in /sys/class/scsi_host/host*/scan; do echo "- - -" > $host; done

# Verify Multipath devices
multipath -ll | grep -E "mpath|size"

# Configure udev ownership (Example: /dev/oracleasm/disks/NEW_DATA*)
chown oracle:dba /dev/mapper/mpath_new_data*
chmod 0660 /dev/mapper/mpath_new_data*

Step 2: Execute Atomic ADD and DROP Operation

Log into the ASM instance (+ASM1 or +ASM) as SYSASM:

-- Single Atomic Command: Add New Storage LUNs and Drop Old LUNs Simultaneously
ALTER DISKGROUP DATA 
  ADD DISK 
    '/dev/mapper/mpath_new_data1' NAME DATA_NEW_01,
    '/dev/mapper/mpath_new_data2' NAME DATA_NEW_02,
    '/dev/mapper/mpath_new_data3' NAME DATA_NEW_03,
    '/dev/mapper/mpath_new_data4' NAME DATA_NEW_04
  DROP DISK 
    DATA_OLD_01,
    DATA_OLD_02,
    DATA_OLD_03,
    DATA_OLD_04
  REBALANCE POWER 16;

Why Atomic Execution Matters: Combining ADD and DROP in a single SQL statement ensures ASM rebalances data directly from old disks to new disks in one pass, avoiding unnecessary double-rebalancing I/O overhead.

---

⚡ Tuning & Monitoring ASM_POWER_LIMIT

The POWER clause controls how many parallel I/O slave processes (ARBn) perform the background block copy.

• POWER 1 to POWER 4: Minimal I/O impact on active OLTP transactions. Use during peak business hours.
• POWER 16 to POWER 32: High-speed rebalance for off-peak maintenance windows (10GbE / Fibre Channel).
• POWER 64 to POWER 1024: Maximum NVMe line-rate transfer speed.

Dynamic Power Adjustment

You can dynamically adjust the rebalance speed on the fly without stopping the operation:

-- Dynamically Increase Rebalance Speed During Off-Peak Hours
ALTER DISKGROUP DATA REBALANCE POWER 32;

---

🔍 Diagnostic Queries: Monitoring Rebalance Progress

Run these diagnostic SQL queries in the ASM instance to track remaining rebalance time and read/write throughput:

Diagnostic 1: Monitor ASM Rebalance Progress and Estimated Time Remaining

SELECT 
    group_number, 
    operation, 
    state, 
    power, 
    actual, 
    sofar, 
    est_work, 
    est_rate, 
    est_minutes 
FROM v$asm_operation;

Diagnostic 2: Verify Disk Status in Diskgroup

SELECT 
    group_number, 
    disk_number, 
    name, 
    path, 
    mount_status, 
    header_status, 
    mode_status, 
    state 
FROM v$asm_disk 
WHERE group_number = (SELECT group_number FROM v$asm_diskgroup WHERE name = 'DATA');

📚 Official Documentation & Technical References

• MOS Doc ID 438580.1: How to Move / Migrate ASM Disk Group to New Storage / LUNs — My Oracle Support

• Oracle Automatic Storage Management Administrator’s Guide — Administering ASM Diskgroups

• Oracle Automatic Storage Management Administrator’s Guide — ASM Rebalance Operations

• ORACLE-BASE — Automatic Storage Management (ASM) in Oracle Database

---

Need assistance planning a zero-downtime SAN migration or optimizing ASM rebalance throughput? Request a Storage Architecture Audit or Contact our Enterprise DBA Team.