Recalculating the --size parameter when recovering a failed array

[Tim Nufire <linux-raid_tim@xxxxxxxxx>]

Hello,

An array that I created using a custom --size parameter has failed and needs to be recovered. I am very comfortable recovering arrays using --assume-clean but due to a typo at creation time I don't know the device size that was originally used. I am hoping this value can be recalculate from data in the superblocks but this calculation is not obvious to me. 

Here's what I know... I'm using metadata version 1.0 with an internal bitmap on all my arrays. I ran some experiments in the lab with 3TB drives and found that when I specified a device size of 2929687500 when creating an array, 'mdadm -D' reported a 'Used Dev Size' of 5859374976. The value specified on the command line is in kilobytes so I was expecting 3,000,000,000,000 bytes to be used on each device. The value reported by mdadm is in sectors (512 bytes) so turning this into bytes I get 2,999,999,987,712 bytes. This is off by 12,288 bytes (12kb) which I assume is used by the v1.0 superblock and/or the internal bitmap. I also tried creating an array with 2TB drives (Requested Size: 1953125000, Used Dev Size: 3906249984) and got a difference of 8kb (2,000,000,000,000 vs 1,999,999,991,808 bytes) so clearly the amount of extra space used depends on the size of the device in some way.

The array that I'm trying to recover reports a 'Used Dev Size' of 5858574976. This is just 800,000 sectors less than I got when requesting an even 3 trillion bytes so I know the size to use on the command line is close to 2929687500. But I don't know how to calculate the exact size... Can someone help me? 

Once I know the size I will recreate the array using the following:

size=???
md11='/dev/sdc /dev/sdf /dev/sdi /dev/sdl /dev/sdo /dev/sdr /dev/sdu /dev/sdx missing missing /dev/sdag /dev/sdaj /dev/sdam /dev/sdap /dev/sdas'
mdadm --create /dev/md11 --metadata=1.0 --size=$size --bitmap=internal --auto=yes -l 6 -n 15 --assume-clean $md11

Just incase it helps, here's the full output from mdadm -D for the array I'm trying to recover and mdadm -E for the first device in that array:

mdadm -E /dev/sdc
/dev/sdc:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 560bd0d9:a8d4758c:9849143c:a2ef5b8e
           Name : sm345:11  (local to host sm345)
  Creation Time : Sat Dec 17 07:22:56 2011
     Raid Level : raid6
   Raid Devices : 15

 Avail Dev Size : 5860532896 (2794.52 GiB 3000.59 GB)
     Array Size : 76161474688 (36316.62 GiB 38994.68 GB)
  Used Dev Size : 5858574976 (2793.59 GiB 2999.59 GB)
   Super Offset : 5860533152 sectors
          State : clean
    Device UUID : f3ad57be:c0835578:4f242111:fb465c0a

Internal Bitmap : -176 sectors from superblock
    Update Time : Sat Jun 16 05:45:17 2012
       Checksum : 4936d9a7 - correct
         Events : 187674

     Chunk Size : 64K

    Array Slot : 0 (empty, 1, 2, 3, 4, 5, 6, 7, failed, failed, 10, 11, 12, 13, 14)
   Array State : _uuuuuuu__uuuuu 2 failed

mdadm -D /dev/md11
/dev/md11:
        Version : 01.00
  Creation Time : Sat Dec 17 07:22:56 2011
     Raid Level : raid6
     Array Size : 38080737344 (36316.62 GiB 38994.68 GB)
  Used Dev Size : 5858574976 (5587.17 GiB 5999.18 GB)
   Raid Devices : 15
  Total Devices : 15
Preferred Minor : 11
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Sat Jun 16 05:45:17 2012
          State : active, degraded
 Active Devices : 12
Working Devices : 15
 Failed Devices : 0
  Spare Devices : 3

     Chunk Size : 64K

           Name : sm345:11  (local to host sm345)
           UUID : 560bd0d9:a8d4758c:9849143c:a2ef5b8e
         Events : 187674

    Number   Major   Minor   RaidDevice State
       0       0        0        0      removed
       1       8       80        1      active sync   /dev/sdf
       2       8      128        2      active sync   /dev/sdi
       3       8      176        3      active sync   /dev/sdl
       4       8      224        4      active sync   /dev/sdo
       5      65       16        5      active sync   /dev/sdr
       6      65       64        6      active sync   /dev/sdu
       7      65      112        7      active sync   /dev/sdx
       8       0        0        8      removed
       9       0        0        9      removed
      10      66        0       10      active sync   /dev/sdag
      11      66       48       11      active sync   /dev/sdaj
      12      66       96       12      active sync   /dev/sdam
      13      66      144       13      active sync   /dev/sdap
      14      66      192       14      active sync   /dev/sdas

       0       8       32        -      spare   /dev/sdc
      15      65      160        -      spare   /dev/sdaa
      16      65      208        -      spare   /dev/sdad


Thanks,
Tim





--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html