Logo  

CS469 - Linux and Unix Administration and Networking

RAID - Redundant Array of Inexpensive/Independent Disks

RAID is a method of joining together a number of disks into a virtualized unit so as to increase aggregate performance and/or add data redundancy.

There are various "numbered" RAID schemes, each with their own plusses and drawbacks. The more common of them are as follows. In the diagram depiction for each the letters represent a "stripe" which is like a virtual block of data that is then subdivided into physical disk blocks.

RAID Level Description
0

Block-level "striping" across 2 or more disks, No parity or mirroring, so no fault tolerance. Read/write performance improved by number of drives in the system.

     Drive0   Drive1   Drive3
    ┌──────┐ ┌──────┐ ┌──────┐
    │  A1  │ │  A2  │ │  A3  │
    ├──────┤ ├──────┤ ├──────┤  Capacity(N drives) = N
    │  B1  │ │  B2  │ │  B3  │
    ├──────┤ ├──────┤ ├──────┤
    │ ...  │ │ ...  │ │ ...  │
    └──────┘ └──────┘ └──────┘
1

Block-level "mirroring" across 2 or more disks. No parity, data is duplicated in whole. Mirror drives can be read as per RAID 0, improving read performance.

     Drive0   Drive1   Drive3
    ┌──────┐ ┌──────┐ ┌──────┐
    │  A1  │ │  A1  │ │  A1  │
    ├──────┤ ├──────┤ ├──────┤  Capacity(N drives) = 1
    │  B1  │ │  B1  │ │  B1  │
    ├──────┤ ├──────┤ ├──────┤
    │ ...  │ │ ...  │ │ ...  │
    └──────┘ └──────┘ └──────┘
2

Bit-level striping w/ dedicated hamming code parity. Requires 3 or more drives and can tolerate one drive failure. No longer used.

     Drive0   Drive1   Drive3   Drive4   Drive5   Drive6
    ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐
    │  A1  │ │  A2  │ │  A3  │ │ Ap1  │ │ Ap2  │ │ Ap3  │
    ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤   Capacity(N drives) = (N-p)
    │  B1  │ │  B2  │ │  B3  │ │ Bp1  │ │ Bp2  │ │ Bp3  │
    ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤
    │ ...  │ │ ...  │ │ ...  │ │ ...  │ │ ...  │ │ ...  │
    └──────┘ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘
3 & 4

Byte(3)/Block(4)-level striping w/ dedicated parity. 3+ disks / recovers 1 failure Block-level striping w/ dedicated parity. 3+ disks / recovers 1 failure Neither 3/4 are widely used, replaced by RAID 5.

     Drive0   Drive1   Drive3   Drive4
    ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐
    │  A1  │ │  A2  │ │  A3  │ │  Ap  │
    ├──────┤ ├──────┤ ├──────┤ ├──────┤ Capacity(N drives) = N-1
    │  B1  │ │  B2  │ │  B3  │ │  Bp  │
    ├──────┤ ├──────┤ ├──────┤ ├──────┤
    │ ...  │ │ ...  │ │ ...  │ │ ...  │
    └──────┘ └──────┘ └──────┘ └──────┘
5

Block-level striping w/ distributed parity. 3+ disks required and can recover from one failure. Both reads and writes can be distributed and fast, but updates require reading all blocks in the stripe, and updating both data and parity.

     Drive0   Drive1   Drive3   Drive4
    ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐
    │  A1  │ │  A2  │ │  A3  │ │  Ap  │
    ├──────┤ ├──────┤ ├──────┤ ├──────┤ Capacity(N drives) = N-1
    │  B1  │ │  B2  │ │  Bp  │ │  B3  │
    ├──────┤ ├──────┤ ├──────┤ ├──────┤
    │  C1  │ │  Cp  │ │  C2  │ │  C3  │
    ├──────┤ ├──────┤ ├──────┤ ├──────┤
    │  Dp  │ │  D1  │ │  D2  │ │  D3  │
    └──────┘ └──────┘ └──────┘ └──────┘

Spreading the parity across disks eliminates the wear that a dedicated parity drive would experience (every update requires updating both the data block and the parity block associated with the stripe.)

6

Extends RAID 5 by adding an additional parity block, allowing the array to recover from up to two simultaneous failures. 4+ disks are required.

     Drive0   Drive1   Drive3   Drive4   Drive5
    ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐
    │  A1  │ │  A2  │ │  A3  │ │  Ap  │ │  Aq  │
    ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤    Capacity(N drives) = N-2
    │  B1  │ │  B2  │ │  Bp  │ │  Bq  │ │  B3  │
    ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤
    │  C1  │ │  Cp  │ │  Cq  │ │  C2  │ │  C3  │
    ├──────┤ ├──────┤ ├──────┤ ├──────┤ ├──────┤
    │  Dp  │ │  Dq  │ │  D1  │ │  D2  │ │  D3  │
    └──────┘ └──────┘ └──────┘ └──────┘ └──────┘
0+1(01)

Creates a second striped set to mirror the primary striped set. Can operate with so long as one side of the mirror has no failures.

      ┌─────── Raid 1 ───────┐
    Raid 0                 Raid 0
1+0(10)

Creates a striped set from multiple mirrored sets. Can operate so long as both mirrors remain operational.

      ┌─────── Raid 0 ───────┐
    Raid 1                 Raid 1
5+0(50)

A striped (RAID 0) set of multiple RAID 5 arrays.

6+0(60)

A striped (RAID 0) set of multiple RAID 6 arrays.

Hot-spare:

Drives that are live in the system, but are unused, but can immediately replace a failed drive. Might be powered-down until needed to improve longevity.

MD - The Linux Multiple-Device driver aka Linux Software RAID

Reading:
> man 4 md
> man 8 mdadm
> man 5 mdadm.conf

The Linux MD (Multiple-Device) driver is a kernel module that can be compiled in or as a loadable module (though it should then be loaded via an initial ram disk image so that it can be used to assemble and ready RAID arrays at boot time.) The LVM or Linux Volume Manager has much of the same functionality, plus more, and is often used in lieu of MD, but for purposes of this class MD is easier to use and test RAID concepts, so we will use it.

The MD driver supports RAID levels 0, 1, 4, 5, 6 and 10. Typical functions are:

Create (--create/-C):

Create a RAID array from a set of devices (drives or partitions on drives):

# mdadm --create md-device -l RAID-level -n #-drives -x #-spares devices

example:

  • A RAID 0 device w/ 2 drives:
    # mdadm --create /dev/md0 -l 0 -n 2 /dev/sdb1 /dev/sdc1

  • A RAID 6 device w/ 4 drives + 1 hot spare:
    # mdadm --create /dev/md0 -l 6 -n 4 -x 1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1

Assemble (--assemble/-A):

  • Start a device:
    # mdadm -A device
    # mdadm -A --scan

Monitor (--monitor/-F/--follow):

  • Email an alert if something happens to a md-device:
    # mdadm --monitor -m email-addr --daemonize [specific md-devices]

Manage:

  • Stop the array /dev/md0:
    # mdadm -S /dev/md0

  • Fail /dev/sdb1 in the /dev/md0 array, and remove it.
    # mdadm --manage /dev/md0 -f /dev/sdb1
    # mdadm --manage /dev/md0 -r /dev/sdb1

  • Add back /dev/sdb1 after replacing drive:
    # mdadm --manage /dev/md0 -a /dev/sdb1

Misc options:

  • Display the state of an array:
    # mdadm -D /dev/md0
    > cat /proc/mdstat

  • Examine a particular drive: # mdadm -E /dev/sda1

Files:

/etc/mdadm.conf

  • Configuration for md drives, defines devices and arrays, used by mdadm to configure arrays in specific ways.

Example:
ARRAY /dev/md0 UUID=9e5edea7:58c60aba:c286d3f4:ca20cebc

  • Defines the array with given UUID to use the device name /dev/md0