Disk Mirroring


This is an example of the setup needed for our baseline V880 servers. They came with six drives, we are typically putting the first four to use with system related stuff in a mirrored partition layout. Typically the "State Databases" needed by Volume Manager are being placed in dedicated 100Mb slices, usually slice 3 of the drive. For this example the commands needed to mirror the system drive are given so only partitions on the first drive will be addressed but the metadb command shown mentions slices on other drives - when setting up a machine you should provide at least three places to store the "State Databases". This walk-through is based on setting up robin.cse.buffalo.edu, a V880. The system drive partition table is:

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm   15332 - 16040     1000.15MB    (709/0/0)    2048301
  1       swap    wu       0 - 15331       21.12GB    (15332/0/0) 44294148
  2     backup    wm       0 - 24619       33.92GB    (24620/0/0) 71127180
  3 unassigned    wm   16041 - 16111      100.16MB    (71/0/0)      205119
  4 unassigned    wm   16112 - 17529        1.95GB    (1418/0/0)   4096602
  5        var    wm   17530 - 21074        4.88GB    (3545/0/0)  10241505
  6        usr    wm   21075 - 24619        4.88GB    (3545/0/0)  10241505
  7 unassigned    wm       0                0         (0/0/0)            0

Note the Solaris-9-ism of slice 0 not starting on the first cylinder. The partition table of c1t1d0 was set up to be identical to this drive. Slice 3 is for the State Database, the others are:

  • Slice 0: root filesystem
  • Slice 1: swap space
  • Slice 2: not used
  • Slice 4: /tmp
  • Slice 5: /var
  • Slice 6: /usr

The partition layout when we start looks like:

robin 3 % df -t ufs
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/dsk/c1t0d0s0     962573   66885  837934     8%    /
/dev/dsk/c1t0d0s6    5040814 1044915 3945491    21%    /usr
/dev/dsk/c1t0d0s5    5040814   28771 4961635     1%    /var
/dev/dsk/c1t2d0s4    10080200  576788 9402610     6%    /opt
/dev/dsk/c1t0d0s4    1984564      26 1925002     1%    /tmp
robin 4 % 

First set initialize the State Databases with:

  metadb -a -f c1t0d0s3 c1t1d0s3 c1t2d0s3 c1t3d0s3

Now create the two metadevices that will make up the mirror. Since these metadevices are typically "invisible" and it's the mirror sets that wind up appearing in df listings and such we have been naming the "invisible" components starting with d10 for the first disk d20 for the second ... and the "visible" components starting with d0.

  metainit -f d10 1 1 c1t0d0s0
  metainit d20 1 1 c1t1d0s0

Next create the mirror device, initially having it contain only the previously existing filesystem so there is no confusion about what will need to be copied to what when the full-blown mirror gets created and the data resync is necessary.

  metainit d0 -m d10

Because this is the root filesystem a special step is now needed:

  metaroot d0

This will handle editing /etc/vfstab for you (and it probably does other stuff...). REBOOT NOW!!!!! Having not done that on one machine it seems like many but not all disk updates get suspended at this point, you will probably have quite a mess if you leave the machine running in this state very long.

To complete the setup attach the second sub-mirror with:

  metattach d0 d20

You can check things with the metastat command. To take care of /var, /usr, and /tmp the sequence of commands is:

  metainit -f d15 1 1 c1t0d0s5
  metainit d25 1 1 c1t1d0s5
  metainit d5 -m d15
  metattach d5 d25
  metainit -f d16 1 1 c1t0d0s6
  metainit d26 1 1 c1t1d0s6
  metainit d6 -m d16
  metattach d6 d26
  metainit -f d14 1 1 c1t0d0s4
  metainit d24 1 1 c1t1d0s4
  metainit d4 -m d14
  metattach d4 d24

Edit /etc/vfstab to change device names to /dev/md/dsk/d5 for /var, d6 for /usr, and d4 for /tmp. Wait for metastat to say all the devices have completed resyncing and then reboot...

This last step is crucial... Record the device the mirror of the root partition is on so we can boot from it if needed. It can be determined with:

  ls -l /dev/rdsk/c1t1d0s0

For robin.cse.buffalo.edu "ls -l" said:


but the "ssd" needed to be replaced by "disk" when configuring the boot PROM.

NOTE: that "w21000004cf785e73" part is the WWN (World Wide Name) of the FCAL disk currently installed there. If the drive gets swapped that WWN will change, and that will not be the correct device name on other V880's with identical disk layouts/mirror setups...

Now to set up for booting when the system drive is lost at the boot PROM do:

  nvalias backup_root /pci@8,600000/SUNW,qlc@2/fp@0,0/disk@w21000004cf785e73,0:a
  printenv boot-device
  setenv boot-device  backup_root net

This should allow the machine to autoboot if the system drive dies. A manual boot command at the PROM monitor would be:

  boot backup_root

It doesn't appear to hurt giving that a try to make sure it works once you think you have the machine set up properly but, reboot off the primary disk ASAP. DiskSuite is a sort of one way mirror so, it gets very unhappy if you make changes to files while running off the backup root unless you use some as yet unknown tool to resync the disks.