Raid Upgrade Trick

Premise:  Here's my situation.  I have a raid 5, with 3 1 TB drives, which gives me about 1.8 give or take of storage.  I had one drive go bad on it, and I found a great deal on amazon, so I decided to upgrade to a new raid 5 configuration with 3x 2 TB drives.

To recap.

Current setup:

• 3 drives
• 1 TB of space for each
• Raid 5
• total raid capacity:  around 1.8
• space used:  1.3 TB

New Setup:

• 3 drives
• 2 TB of space for each
• Raid 5
• total raid capacity once completed:  around 3.8 TB

Problem:

1.  I don't have enough ports on my machine to plugin all of the drives for my 2 raids.  Ideal would require 6 sata ports for the 2 raids and a 7th one for my OS drive. Even if I futz with things, and allow for one drives to be missing from each.  (ie. run a degraded raid), I would still need 5 ports, which I don't have. 

Solution:

Step 1.  Copy data

Note:  This can be done a bit differently...but I think my approach will save you time in the long run. 

I replaced the dead 1 TB drive with one of the new drives.  The computer starts up, It brings up the old raid /dev/md0 in a degraded state per usual, and I now have 1 new drive to play with that I know is good.

Using the new drive, I'm creating a new degraded raid 1 device.

mdadm --create /dev/md1 -l raid1 -f -n 1 /dev/sdd1

mkfs -t xfs (or whatever you prefer) /dev/md1 

sudo mount /dev/md1 /media/newRaid

rsync -avP /media/oldRaid/ /media/newRaid/    ## be careful the / matter and change rsync's behavior.

So at this point, we have 2 raids up and running, and I'm copying data from the old raid to the new one. 

Step 2.  Replace Drives

At this point, shut down the machine, remove your remaining drives from the old raid, and replace them with the new one.  comment out your /etc/fstab if you're mounting your raid by default.

It will probably create some random /dev/md* depending on your OS and intelligent it is.  Ubuntu re-created the raid array as /dev/md127, which I'll use for now.  we can easily rename it to md0 once everything is done.

Let's add another drive to the array.

mdadm --grow /dev/md0 -n 2  
mdadm --manage /dev/md1 --add /dev/sda1

If the above command works, then if you issue a 

cat /proc/mdstat 

You should see data being synced.

VERY IMPORTANT... wait for the sync to finish before doing anything else at this point.  

This gives me a new functional raid 1.  We're not done.  The next step is to convert a raid 1 to raid 5, and since I still have 2 functional 1 TB drives where I can reconstruct my old raid with, I also have a backup of the data.. so if I lose everything.. I just lose another day or so to copy data back.

Side note:  ETA to copy 1.1 TB of data was about +8 hours.  I started around 6 pm and it was still going at 2 am.

ETA to do the sync above is estimated at 4-5 hours.  Though thankfully I can leave my machine up and running and use it while it's synching.

(I forgot to mention, if you want to access your data while it's synching.. you can mount the drive:

sudo mount /dev/md127 /media/data  

naturally it's recommended not to use your computer while doing that.. but meh... next step will take even longer... (Raid 1 to Raid 5 conversion).  I was actually playing a game while I was copying the data (rsync).  Though I did re-run the command once more once everything was done to make sure it got everything.

Look forward to the next article while will talk about the raid conversions.

References:
 1.  Create A Degraded Raid 1 Array