one raidz2 or two raidz ?

[Adam Gold]

I have what I fear is a blatant noob question. What are the pros and
crons of (let's assume no more than a total of 8 disks):

(i) a raidz-2 pool with x disks
(ii) a raidz pool with two raidz vdevs each comprising x/2 disks

I'm thinking it may be that (i) is safer from a redundancy point of
view while (ii) performs better. I'm be very interested to hear other
people's thoughts though. Thanks in advance.

[Steve Costaras]

With the same 8 drives, a pool with a vdev of a single raid-z2 with 8 drives would have higher MTTF_DL (mean time to failure/data loss) than a pool comprised of a 2 vdevs of 4-drive raidz.

However the two pool with two vdevs of raidz will have greater write performance (~2x that of the single vdev of raidz2) assuming random writes.

The greater write performance is not due to the raid type but due to the fact you have multiple vdevs. This is why for large san's you create many vdevs/raid groups and stripe them together to increase performance.

-----Original Message-----
Sent: Friday, June 22, 2012 05:15 AM
To: 'zfs-discuss'
Subject: [zfs-discuss] one raidz2 or two raidz ?
I have what I fear is a blatant noob question. What are the pros and
(i) a raidz-2 pool with x disks
(ii) a raidz pool with two raidz vdevs each comprising x/2 disks
I'm thinking it may be that (i) is safer from a redundancy point of
view while (ii) performs better. I'm be very interested to hear other
people's thoughts though. Thanks in advance.

[Gordan Bobic]

Post by Steve Costaras
With the same 8 drives, a pool with a vdev of a single raid-z2 with 8 drives would have higher MTTF_DL (mean time to failure/data loss) than a pool comprised of a 2 vdevs of 4-drive raidz.
However the two pool with two vdevs of raidz will have greater write performance (~2x that of the single vdev of raidz2) assuming random writes.

On traditional RAID, yes, but I thought ZFS specifically overcomes this
by using variable size stripes, and since it always writes a complete
stripe, it never has to do the read-modify-write process that kills
performance of parity RAIDs.

Post by Steve Costaras
The greater write performance is not due to the raid type but due to the fact you have multiple vdevs. This is why for large san's you create many vdevs/raid groups and stripe them together to increase performance.

Can you explain why multiple vdevs improve write performance?

Gordan

[Gordan Bobic]

Post by Adam Gold
I have what I fear is a blatant noob question. What are the pros and
(i) a raidz-2 pool with x disks
(ii) a raidz pool with two raidz vdevs each comprising x/2 disks
I'm thinking it may be that (i) is safer from a redundancy point of
view while (ii) performs better. I'm be very interested to hear other
people's thoughts though. Thanks in advance.

Well, for a start, RAIDZ2 is going to be safer than 2xRAIDZ because with
RAIDZ2 you can lose any 2 disks. With RAIDZ1 you can only lose 1 disk in
each stripe.

In terms of performance, it depends on your use case. If you have many
small files (especially if you have disks with 4KB sectors), smaller
arrays _might_ work better.

Also consider things like resilvering time and probability of a read
error during resilvering (which would cause data loss).

Search through the archives, I'm sure issues with size and RAID levels
have been explained before.

Gordan

[Steve Costaras]

In ZFS, even though you have dynamic stripe widths & stripe sizes (up to 128K down to ashift size) you are in a situation where you're dependent on the write size as your stripe is going to be broken down to the number of disks you have keeping in mind that each vdev has it's own parity (for parity raids). So for 8 drives you can have 4 data drives, (raidz2 *2), 6 data (raidz *2) or raidz2*1). for small arrays like this, you have a better chance to engage all drives, now with lots of drives, and say you're writing out a small file especially if you set your zfs stripe size to 128K. For mixed data (what a file server would have, opposed to say a database) this gets complicated. For a 1MiB file with ashift=12 (4Kib) you would be in the range of 8 - 256 data disks (depending on 4Kib - 128Kib dynamic stripe). Now with an ODD number of data drives you start running into another set of issues. So you're partially correct, it's not that it's not trying to write a 'partial' stripe but the number of drives used can overlap in requests which cause the same issue as each drive can only handle 1 request at a time (after merging/ordering) but it has the same end effect in how many iops you can do with a random workload.

-----Original Message-----
Sent: Friday, June 22, 2012 05:30 AM
Subject: Re: [zfs-discuss] one raidz2 or two raidz ?

Post by Steve Costaras
With the same 8 drives, a pool with a vdev of a single raid-z2 with 8 drives would have higher MTTF_DL (mean time to failure/data loss) than a pool comprised of a 2 vdevs of 4-drive raidz.
However the two pool with two vdevs of raidz will have greater write performance (~2x that of the single vdev of raidz2) assuming random writes.

On traditional RAID, yes, but I thought ZFS specifically overcomes this
by using variable size stripes, and since it always writes a complete
stripe, it never has to do the read-modify-write process that kills
performance of parity RAIDs.

Post by Steve Costaras
The greater write performance is not due to the raid type but due to the fact you have multiple vdevs. This is why for large san's you create many vdevs/raid groups and stripe them together to increase performance.

Can you explain why multiple vdevs improve write performance?
Gordan