Creating and Configuring a Large Redundant Storage Array for a Network Share in Ubuntu using MDADM

We had a hardware RAID card that worked well in Windows, but was giving some issues in Linux (Ubuntu 12.04). So, we decided to try to setup a software array using mdadm. This is how we did it.

First, make sure your hardware RAID card has "non-RAID" mode. Basically that it lets each of the attached drives show up as a single drive. Ensure this is enabled. On some cards you will have to flash the BIOS with a non-RAID version.

• Do not create a hardware RAID array using the RAID configuration menu
• Make sure no drives are assigned to a RAID array
• Install the newest version of Ubuntu Server
• When asked for partitioning information
• Select manual partitioning
• Select "Create Software RAID”
• “Create MD device”
• Select RAID5   (we are using RAID5)
• Active devices = #   (where # is the number of drives you want in the array)
• Spare devices = #
• Select all the drives you want in the array
• Select OK

After the array is created our new device has about 21TB. In versions of Ubuntu before 12.04 it was difficult to create a partition using the whole 21TB, but now you should be able to do it from the install menu.

• Create a partition on the newly created device (usually md0)
• Format as ext4
• Save and continue with install per usual
• You might want to select “ssh” from the package selection

Once install is done, and you boot into the OS:

• Make sure array device has been created
• Sudo fdisk –l
• Look for /dev/md0
• Check status of the software array
• Sudo mdadm --detail /dev/md0
• If the status of the array is “building” or “syncing”, let the process finish (may take several hours)
• Create a mount point for the array
• Sudo mkdir /media/RAIDStorage
• Modify fstab to mount partition on boot
• Sudo nano /etc/fstab
• Add a new line:
• /dev/md0 /media/RAIDStorage ext4 defaults 0 0
• Save and exit
• Remount
• Sudo mount –a
• Check device was mounted to /media/RAIDStorage
• Sudo mount | grep RAIDStorage
• Share RAIDStorage with NFS
• Sudo apt-get install nfs-kernel-server
• Edit exports file
• Sudo nano /etc/exports
• Add line
• /media/Storage/Case_Data 10.1.1.0/24(rw,insecure,async,no_subtree_check)
• Save and exit
• /etc/init.d/nfs-kernel-server restart

In this case, permissions are NOT being set up on NFS. If you need a more secure environment, make sure you set it up.

Also, we are using 'async' instead of 'sync'. For some reason, when writing very large files sync have very, very bad performance, while async allowed for maximum write speeds.

If there are write permission errors, check that the permissions on the folder (/media/RAIDStorage) on the server are set correctly for the user