RE: How to grow RAID1 mirror on top of LVM?

["David Lethe" <david@xxxxxxxxxxxx>]

-----Original Message-----
From: Lars Täuber [mailto:taeuber@xxxxxxx] 
Sent: Tuesday, April 29, 2008 2:56 AM
To: David Lethe
Cc: Neil Brown; Anton Altaparmakov; linux-raid@xxxxxxxxxxxxxxx
Subject: Re: How to grow RAID1 mirror on top of LVM?

Hallo David!

"David Lethe" <david@xxxxxxxxxxxx> schrieb:
> Lars:
> Even if you *could* do this, then your design would still be an awful idea.  You'd effectively have a random I/O filesystem.  Any write on a RAID1 disk would generate writes on all of the other disks.  Even with a small number of RAID1 targets and default journaling, I wouldn't be surprised if a single application write translated into at least a dozen physical disk writes.  
> 
> What if (when) both the RAID1 & RAID6 became corrupted?  How would you proceed?  How would you propose the md engine deal with this?   What if you lose a disk on the RAID6 while expanding the LVM and have a bad sector on remaining disks, or fsck shows filesytem corruption while you are expanding a md volume. 
> 
> Maybe you are starting to get the point.

No, sorry. I really don't understand. But maybe because I didn't tell detailed enough what our systems looks like.

We have here two huge RAID6 systems, that are connected through a 10G Ethernet switch. They are equally in size.
On these RAID6 we create LVs euqally in size and want them to be parts of a RAID1:

 RAID6    RAID6
monosan  duosan
  LV       LV             (multiple of)
    \     /     10GE
     RAID1
       |        10GE
      aoe
       |        1GE
     server               (multiple of)

>  What is preventing the md layer from doing what you want is that a heck of a lot of code would have to be written,

But could you tell me what in the current »design« prevents us to resize the LVs and the RAID1 on top?
When I understand it correctly the RAID information are written at the end of each accessible partition/LV.
What is different on an extended LV from a partition that is replaced with a bigger one?

[...]
Thanks
Lars

==================================

The drawing was helpful, as I interpreted your architecture differently.  I thought you had something like
     RAID6
    /  \
   /    \
RAID1    RAID1

Still, you could get into a great deal of inefficiency.   What are the block/chunk sizes in RAID6, RAID1, and your file system, and how is the journaling set up?  
How many disks are in the RAID1, and what is the average I/O size that your application generates.  

Lets' say you do a 4KB WRITE in the file system.  Factor in all of the above and see how many I/Os get generated on each disk.  If you are write-intensive, then consider how much less I/O will be generated if you reduced the sizes of the 2 RAID6 by 2 disks each, and added a RAID1 on each system, and used the RAID1 for the journal.  The Journal is write-intensive, and RAID1 is best architecture for a journal.  Also you can set correct and matching block sizes for the Journaling.  RAID6 has a huge penalty on writes, so the more writes you can move from the RAID6 md, the better.  

As for the question about resizing, I was under impression you built it differently, so forget that response. There is nothing in your architecture that will prevent you from resizing.

David

��.n��������+%������w��{.n�����{����w��ܨ}���Ơz�j:+v�����w����ޙ��&�)ߡ�a����z�ޗ���ݢj��w�f