[zfs-discuss] Good/recommended storage controller for ZFS on typical Supermicro box

Discussion:

David Bruzos

2015-02-24 20:27:05 UTC

Hello all:
What storage controller(s) would make good choices for SATA disks on a typical Supermicro 24 drive server? I currently have a couple of 3ware 9750 controllers, but they don't support JBOD mode, they only support "simgle" disk mode.

I want to know what controllers deliver reasonable performance, while allowing ZFS to do what it was designed to do.

Thanks all!
--
David Bruzos (Systems Administrator)
Jacksonville Port Authority
2831 Talleyrand Ave.
Jacksonville, FL 32206
Cell: (904) 625-0969
Office: (904) 357-3069
Email: ***@jaxport.com

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

Nigel Henderson

2015-02-24 20:32:43 UTC

Permalink

Not sure if these would "suit" but I use a couple of IBM m1015's flashed to IT mode.

Post by David Bruzos
What storage controller(s) would make good choices for SATA disks on a
typical Supermicro 24 drive server? I currently have a couple of 3ware
9750 controllers, but they don't support JBOD mode, they only support
"simgle" disk mode.
I want to know what controllers deliver reasonable performance, while
allowing ZFS to do what it was designed to do.
Thanks all!
--
David Bruzos (Systems Administrator)
Jacksonville Port Authority
2831 Talleyrand Ave.
Jacksonville, FL 32206
Cell: (904) 625-0969
Office: (904) 357-3069
To unsubscribe from this group and stop receiving emails from it, send

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

Colin Faber

2015-02-24 21:30:06 UTC

Permalink

Same, this works really well. The only catch 22 is having a motherboard
that can flash them.

http://www.unixgr.com/hacking-the-ibm-m1015-a-k-a-lsi-9640-8i-raid-controller/
step by step if you choose to go this route.

Post by Nigel Henderson
Not sure if these would "suit" but I use a couple of IBM m1015's flashed to IT mode.
On 24 February 2015 20:27:05 GMT+00:00, David Bruzos <

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
To unsubscribe from this group and stop receiving emails from it, send an

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

Uncle Stoatwarbler

2015-02-24 21:34:46 UTC

Permalink

Post by Colin Faber
Same, this works really well. The only catch 22 is having a motherboard
that can flash them.
http://www.unixgr.com/hacking-the-ibm-m1015-a-k-a-lsi-9640-8i-raid-controller/
step by step if you choose to go this route.

The megaraid tools have a linux port, so you don't have to boot into
freedos if you don't want to.

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

Colin Faber

2015-02-24 21:36:17 UTC

Permalink

Yeah, in this case, I know that the dos megaraid tools work correctly, as
this process is a big fickle to begin with =)

Post by Colin Faber

Post by Colin Faber
Same, this works really well. The only catch 22 is having a motherboard
that can flash them.

http://www.unixgr.com/hacking-the-ibm-m1015-a-k-a-lsi-9640-8i-raid-controller/

Post by Colin Faber
step by step if you choose to go this route.

The megaraid tools have a linux port, so you don't have to boot into
freedos if you don't want to.
To unsubscribe from this group and stop receiving emails from it, send an

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

Gordan Bobic

2015-02-24 21:13:31 UTC

Permalink

3ware is passably OK. They don't quite expose the disks raw, but it's close
enough. And if you install the 3ware tools, they will create the device
nodes you can use to get SMART data out.
Personally I prefer plain vanilla SATA controllers, but it's difficult to
get a lot of ports unless you use PMPs. Not that PMPs are a problem, but
they aren't that cheap. A 2nd hand 8-24 port SAS card on ebay is often
cheaper.

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

Uncle Stoatwarbler

2015-02-24 21:27:23 UTC

Permalink

Post by Gordan Bobic
3ware is passably OK.

Beware of the 9500 range. They drop commands and lockup.

awful things.

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

John Drescher

2015-02-24 21:33:33 UTC

Permalink

I would look at LSI based HBAs. LSI 2208 is a good place to start:

http://www.servethehome.com/lsi-sas-2208-raid-controller-information-listing/

John

Post by David Bruzos
What storage controller(s) would make good choices for SATA disks on a typical Supermicro 24 drive server? I currently have a couple of 3ware 9750 controllers, but they don't support JBOD mode, they only support "simgle" disk mode.
I want to know what controllers deliver reasonable performance, while allowing ZFS to do what it was designed to do.
Thanks all!
--
David Bruzos (Systems Administrator)
Jacksonville Port Authority
2831 Talleyrand Ave.
Jacksonville, FL 32206
Cell: (904) 625-0969
Office: (904) 357-3069

--
John M. Drescher

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

Schlacta, Christ

2015-02-24 21:36:25 UTC

Permalink

for platter based storage, look for anything based on an LSI chip. Nothing
at newegg right now is a bad choice. Literally. If you can't shop newegg,
use it as a reference. Use the newer SAS3 and PCIE3 chips if you need
SSDi, otherwise, anything is sufficient and exceptional for ZFS.

Post by John Drescher
http://www.servethehome.com/lsi-sas-2208-raid-controller-information-listing/
John

Post by David Bruzos
What storage controller(s) would make good choices for SATA disks on a

typical Supermicro 24 drive server? I currently have a couple of 3ware
9750 controllers, but they don't support JBOD mode, they only support
"simgle" disk mode.

Post by David Bruzos
I want to know what controllers deliver reasonable performance, while

allowing ZFS to do what it was designed to do.

Post by David Bruzos
Thanks all!
--
David Bruzos (Systems Administrator)
Jacksonville Port Authority
2831 Talleyrand Ave.
Jacksonville, FL 32206
Cell: (904) 625-0969
Office: (904) 357-3069
To unsubscribe from this group and stop receiving emails from it, send

--
John M. Drescher
To unsubscribe from this group and stop receiving emails from it, send an

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

Gordan Bobic

2015-02-24 21:39:53 UTC

Permalink

Post by Schlacta, Christ
for platter based storage, look for anything based on an LSI chip.
Nothing at newegg right now is a bad choice. Literally. If you can't shop
newegg, use it as a reference. Use the newer SAS3 and PCIE3 chips if you
need SSDi, otherwise, anything is sufficient and exceptional for ZFS.

Post by John Drescher
http://www.servethehome.com/lsi-sas-2208-raid-controller-information-listing/
John

Post by David Bruzos
What storage controller(s) would make good choices for SATA disks on a

typical Supermicro 24 drive server? I currently have a couple of 3ware
9750 controllers, but they don't support JBOD mode, they only support
"simgle" disk mode.

Post by David Bruzos
I want to know what controllers deliver reasonable performance, while

allowing ZFS to do what it was designed to do.

--
John M. Drescher
To unsubscribe from this group and stop receiving emails from it, send an

To unsubscribe from this group and stop receiving emails from it, send an

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

Alex Gardiner

2015-02-24 21:46:42 UTC

Permalink

I’ve been running the LSI 9211-4i with a Supermicro 826TQ chassis for some time now.

Once flashed with the IT Firmware, things behave as expected and the performance seems really good - but I don’t have much to compare it to!

I'd avoid older LSI cards that are based on PCI-X chipsets and rely on a phantom PCI-X to PCIe bridge to make them work. I've found them to be very hit and miss as to whether they will work properly on various motherboards. They'll either work fine or not at all.
for platter based storage, look for anything based on an LSI chip. Nothing at newegg right now is a bad choice. Literally. If you can't shop newegg, use it as a reference. Use the newer SAS3 and PCIE3 chips if you need SSDi, otherwise, anything is sufficient and exceptional for ZFS.
http://www.servethehome.com/lsi-sas-2208-raid-controller-information-listing/
John

Post by David Bruzos
What storage controller(s) would make good choices for SATA disks on a typical Supermicro 24 drive server? I currently have a couple of 3ware 9750 controllers, but they don't support JBOD mode, they only support "simgle" disk mode.
I want to know what controllers deliver reasonable performance, while allowing ZFS to do what it was designed to do.
Thanks all!
--
David Bruzos (Systems Administrator)
Jacksonville Port Authority
2831 Talleyrand Ave.
Jacksonville, FL 32206
Cell: (904) 625-0969
Office: (904) 357-3069

--
John M. Drescher

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

Nigel Henderson

2015-02-25 08:16:51 UTC

Permalink

In response to Colin F.
Thanks for pointing that out. I had to create a EUFI bootable USB drive,
copying all the firmware files over and do the "flash" from there. booting
into a normal environment would not allow me to update the cards. Took me
ages to figure out what was going on, thought I had a duff motherboard or a
duff M1015. If I remember correctly (not at home right now) I used a USB
creator program called "rufus" (https://rufus.akeo.ie/) to create the
bootable USB stick.

Iâve been running the LSI 9211-4i with a Supermicro 826TQ chassis for some
time now.
Once flashed with the IT Firmware, things behave as expected and the
performance seems really good - but I donât have much to compare it to!

Post by Gordan Bobic
I'd avoid older LSI cards that are based on PCI-X chipsets and rely on a

phantom PCI-X to PCIe bridge to make them work. I've found them to be very
hit and miss as to whether they will work properly on various motherboards.
They'll either work fine or not at all.

Post by Gordan Bobic
for platter based storage, look for anything based on an LSI chip.

Nothing at newegg right now is a bad choice. Literally. If you can't shop
newegg, use it as a reference. Use the newer SAS3 and PCIE3 chips if you
need SSDi, otherwise, anything is sufficient and exceptional for ZFS.
http://www.servethehome.com/lsi-sas-2208-raid-controller-information-listing/

Post by Gordan Bobic
John

Post by David Bruzos
What storage controller(s) would make good choices for SATA disks on a

typical Supermicro 24 drive server? I currently have a couple of 3ware
9750 controllers, but they don't support JBOD mode, they only support
"simgle" disk mode.

Post by Gordan Bobic

Post by David Bruzos
I want to know what controllers deliver reasonable performance, while

allowing ZFS to do what it was designed to do.

Post by Gordan Bobic

--
John M. Drescher
To unsubscribe from this group and stop receiving emails from it, send
To unsubscribe from this group and stop receiving emails from it, send
To unsubscribe from this group and stop receiving emails from it, send

To unsubscribe from this group and stop receiving emails from it, send an

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

David Bruzos

2015-02-25 12:32:47 UTC

Permalink

Thanks all for the very quick responses! I'm taking a look at the LSI 2208 based controllers and will probably go with them.
--
David Bruzos (Systems Administrator)
Jacksonville Port Authority
2831 Talleyrand Ave.
Jacksonville, FL 32206
Cell: (904) 625-0969
Office: (904) 357-3069

Post by Nigel Henderson
In response to Colin F.
Thanks for pointing that out. I had to create a EUFI bootable USB drive, copying all the firmware files over and do the "flash" from there. booting into a normal environment would not allow me to update the cards. Took me ages to figure out what was going on, thought I had a duff motherboard or a duff M1015. If I remember correctly (not at home right now) I used a USB creator program called "rufus" (https://rufus.akeo.ie/) to create the bootable USB stick.
I???ve been running the LSI 9211-4i with a Supermicro 826TQ chassis for some time now.
Once flashed with the IT Firmware, things behave as expected and the performance seems really good - but I don???t have much to compare it to!

Post by David Bruzos
What storage controller(s) would make good choices for SATA disks on a typical Supermicro 24 drive server? I currently have a couple of 3ware 9750 controllers, but they don't support JBOD mode, they only support "simgle" disk mode.
I want to know what controllers deliver reasonable performance, while allowing ZFS to do what it was designed to do.
Thanks all!
--
David Bruzos (Systems Administrator)
Jacksonville Port Authority
2831 Talleyrand Ave.
Jacksonville, FL 32206
Cell: (904) 625-0969
Office: (904) 357-3069

--
John M. Drescher

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

Benjamin Smith

2015-02-26 02:24:25 UTC

Permalink

We recently tested a very similar build with excellent results. We even used a
Supermicro Mobo but in a Norco case. See the thread called "Enterprise scale
ZFS on a budget". I know you said SATA but SAS + expander cables just might be
the ticket for you.

We're using CentOS 6.

Ben

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.

Bryn Hughes

2015-02-26 19:02:31 UTC

Permalink

Post by Benjamin Smith

I've had very good luck with the Norco RPC-4224 as a chassis and the
HighPoint RocketRaid 2760A. The RocketRaid card isn't actually a RAID card
at all, it's really just a SAS HBA with an integrated PCI-e switch and 3 x
8-port Marvell SAS controllers. It works perfectly with the mvsas driver
included in the Linux kernel, no need to compile drivers or anything messy
like that. It makes for a really nice clean solution with the Norco
chassis as there are only 6 SAS cables internally, no rats nest of wires.

Works perfectly with SATA devices, no SATA or SAS expanders needed with my
setup so it's a very simple clean solution.

I'm on Ubuntu 14.04 currently.

Bryn

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.