Reading:
RAID
RAID levels
RAID is a method of joining together a number of disks into a virtualized unit so as to increase aggregate performance and/or add data redundancy.
There are various "numbered" RAID schemes, each with their own plusses and drawbacks. The more common of them are as follows. In the diagram depiction for each the letters represent a "stripe" which is like a virtual block of data that is then subdivided into physical disk blocks.
Block-level "striping" across 2 or more disks, No parity or mirroring, so no fault tolerance. Read/write performance improved by number of drives in the system.
Drive0 Drive1 Drive3 ┌──────┐ ┌──────┐ ┌──────┐ │ A1 │ │ A2 │ │ A3 │ ├──────┤ ├──────┤ ├──────┤ Capacity(N drives) = N │ B1 │ │ B2 │ │ B3 │ ├──────┤ ├──────┤ ├──────┤ │ ... │ │ ... │ │ ... │ └──────┘ └──────┘ └──────┘
Block-level "mirroring" across 2 or more disks. No parity, data is duplicated in whole. Mirror drives can be read as per RAID 0, improving read performance.
Drive0 Drive1 Drive3 ┌──────┐ ┌──────┐ ┌──────┐ │ A1 │ │ A1 │ │ A1 │ ├──────┤ ├──────┤ ├──────┤ Capacity(N drives) = 1 │ B1 │ │ B1 │ │ B1 │ ├──────┤ ├──────┤ ├──────┤ │ ... │ │ ... │ │ ... │ └──────┘ └──────┘ └──────┘
Bit-level striping w/ dedicated hamming code parity. Requires 3 or more drives and can tolerate one drive failure. No longer used.
Drive0 Drive1 Drive3 Drive4 Drive5 Drive6 ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │ A1 │ │ A2 │ │ A3 │ │ Ap1 │ │ Ap2 │ │ Ap3 │ ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤ Capacity(N drives) = (N-p) │ B1 │ │ B2 │ │ B3 │ │ Bp1 │ │ Bp2 │ │ Bp3 │ ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤ │ ... │ │ ... │ │ ... │ │ ... │ │ ... │ │ ... │ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘
Byte(3)/Block(4)-level striping w/ dedicated parity. 3+ disks / recovers 1 failure Block-level striping w/ dedicated parity. 3+ disks / recovers 1 failure Neither 3/4 are widely used, replaced by RAID 5.
Drive0 Drive1 Drive3 Drive4 ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │ A1 │ │ A2 │ │ A3 │ │ Ap │ ├──────┤ ├──────┤ ├──────┤ ├──────┤ Capacity(N drives) = N-1 │ B1 │ │ B2 │ │ B3 │ │ Bp │ ├──────┤ ├──────┤ ├──────┤ ├──────┤ │ ... │ │ ... │ │ ... │ │ ... │ └──────┘ └──────┘ └──────┘ └──────┘
Block-level striping w/ distributed parity. 3+ disks required and can recover from one failure. Both reads and writes can be distributed and fast, but updates require reading all blocks in the stripe, and updating both data and parity.
Drive0 Drive1 Drive3 Drive4 ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │ A1 │ │ A2 │ │ A3 │ │ Ap │ ├──────┤ ├──────┤ ├──────┤ ├──────┤ Capacity(N drives) = N-1 │ B1 │ │ B2 │ │ Bp │ │ B3 │ ├──────┤ ├──────┤ ├──────┤ ├──────┤ │ C1 │ │ Cp │ │ C2 │ │ C3 │ ├──────┤ ├──────┤ ├──────┤ ├──────┤ │ Dp │ │ D1 │ │ D2 │ │ D3 │ └──────┘ └──────┘ └──────┘ └──────┘
Spreading the parity across disks eliminates the wear that a dedicated parity drive would experience (every update requires updating both the data block and the parity block associated with the stripe.)
Extends RAID 5 by adding an additional parity block, allowing the array to recover from up to two simultaneous failures. 4+ disks are required.
Drive0 Drive1 Drive3 Drive4 Drive5 ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │ A1 │ │ A2 │ │ A3 │ │ Ap │ │ Aq │ ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤ Capacity(N drives) = N-2 │ B1 │ │ B2 │ │ Bp │ │ Bq │ │ B3 │ ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤ │ C1 │ │ Cp │ │ Cq │ │ C2 │ │ C3 │ ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤ │ Dp │ │ Dq │ │ D1 │ │ D2 │ │ D3 │ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘
Creates a second striped set to mirror the primary striped set. Can operate with so long as one side of the mirror has no failures.
┌─────── Raid 1 ───────┐ Raid 0 Raid 0
Creates a striped set from multiple mirrored sets. Can operate so long as both mirrors remain operational.
┌─────── Raid 0 ───────┐ Raid 1 Raid 1
A striped (RAID 0) set of multiple RAID 5 arrays.
A striped (RAID 0) set of multiple RAID 6 arrays.
Drives that are live in the system, but are unused, but can immediately replace a failed drive. Might be powered-down until needed to improve longevity.
Reading: > man 4 md > man 8 mdadm > man 5 mdadm.conf
> man 4 md
> man 8 mdadm
> man 5 mdadm.conf
The Linux MD (Multiple-Device) driver is a kernel module that can be compiled in or as a loadable module (though it should then be loaded via an initial ram disk image so that it can be used to assemble and ready RAID arrays at boot time.) The LVM or Linux Volume Manager has much of the same functionality, plus more, and is often used in lieu of MD, but for purposes of this class MD is easier to use and test RAID concepts, so we will use it.
The MD driver supports RAID levels 0, 1, 4, 5, 6 and 10. Typical functions are:
Create a RAID array from a set of devices (drives or partitions on drives):
# mdadm --create md-device -l RAID-level -n #-drives -x #-spares devices
# mdadm --create
-l
-n
-x
example:
A RAID 0 device w/ 2 drives: # mdadm --create /dev/md0 -l 0 -n 2 /dev/sdb1 /dev/sdc1
# mdadm --create /dev/md0 -l 0 -n 2 /dev/sdb1 /dev/sdc1
A RAID 6 device w/ 4 drives + 1 hot spare: # mdadm --create /dev/md0 -l 6 -n 4 -x 1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
# mdadm --create /dev/md0 -l 6 -n 4 -x 1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
# mdadm -A
# mdadm -A --scan
# mdadm --monitor -m
--daemonize
Stop the array /dev/md0: # mdadm -S /dev/md0
# mdadm -S /dev/md0
Fail /dev/sdb1 in the /dev/md0 array, and remove it. # mdadm --manage /dev/md0 -f /dev/sdb1 # mdadm --manage /dev/md0 -r /dev/sdb1
# mdadm --manage /dev/md0 -f /dev/sdb1
# mdadm --manage /dev/md0 -r /dev/sdb1
Add back /dev/sdb1 after replacing drive: # mdadm --manage /dev/md0 -a /dev/sdb1
# mdadm --manage /dev/md0 -a /dev/sdb1
Display the state of an array: # mdadm -D /dev/md0 > cat /proc/mdstat
# mdadm -D /dev/md0
> cat /proc/mdstat
Examine a particular drive: # mdadm -E /dev/sda1
# mdadm -E /dev/sda1
/etc/mdadm.conf
Example: ARRAY /dev/md0 UUID=9e5edea7:58c60aba:c286d3f4:ca20cebc Defines the array with given UUID to use the device name /dev/md0
Example: ARRAY /dev/md0 UUID=9e5edea7:58c60aba:c286d3f4:ca20cebc
ARRAY /dev/md0 UUID=9e5edea7:58c60aba:c286d3f4:ca20cebc