I'm putting together a RAID5 array for a Gentoo system, and I'm going to
need an additional IDE controller since there's only 2 controllers onboard.

Low cost and Linux compatible are the requirements.
Any suggestions?

Most of the controllers I'm seeing have some sort of RAID functionality
built-in. Can these type of controllers be used as a standard IDE
controller, ignoring the RAID functionality? Ones that have come up
in my searches are:
Several SIL0680-based cards
Highpoint Rocket100
Several Promise-based cards (I think these seem to use different chips
for different models)



Thanks
-WD

In article <L4ICc.155149$DG4.45950@fe2.columbus.rr.com> (Thu, 24 Jun 2004
21:54:51 +0000), Will Dormann wrote:

> I'm putting together a RAID5 array for a Gentoo system
>
> Low cost and Linux compatible are the requirements.
> Any suggestions?

Intel and 3Ware make hardware RAID controllers.

> Most of the controllers I'm seeing have some sort of RAID functionality
> built-in.
> [...]
> Highpoint Rocket100

The Highpoint 374 isn't a RAID controller and I don't think any of their
products are actually RAID controllers. The 374 can be used as a
standard IDE adapter because that's what it is; I've had no problem with
mine.

> Several Promise-based cards (I think these seem to use different chips
> for different models)

Promise also makes non-RAID controllers.

If you use either of the two above, you'll be using software RAID. (In
itself, this is neither good nor bad, and it may or may not fill your
requirements.)

Hamilcar Barca wrote:


> If you use either of the two above, you'll be using software RAID. (In
> itself, this is neither good nor bad, and it may or may not fill your
> requirements.)


Yes, software RAID is the plan. It's going to be a NAS box, so
performance isn't really important at all. (Since the network is
likely to be the bottleneck)


-WD

> > If you use either of the two above, you'll be using software RAID. (In
> > itself, this is neither good nor bad, and it may or may not fill your
> > requirements.)
>
> Yes, software RAID is the plan. It's going to be a NAS box, so
> performance isn't really important at all. (Since the network is
> likely to be the bottleneck)

Actually, software RAID can often out-perform hardware RAID, especially if
you're talking about RAID 5. Your host CPU can churn out parity at an
astounding rate. Even relatively modest CPU's by today's standards can
outperform some $600 RAID cards with ease.

steve

Steve Wolfe wrote:

>>>If you use either of the two above, you'll be using software RAID. (In
>>>itself, this is neither good nor bad, and it may or may not fill your
>>>requirements.)
>>
>>Yes, software RAID is the plan. It's going to be a NAS box, so
>>performance isn't really important at all. (Since the network is
>>likely to be the bottleneck)
>
>
> Actually, software RAID can often out-perform hardware RAID, especially if
> you're talking about RAID 5. Your host CPU can churn out parity at an
> astounding rate. Even relatively modest CPU's by today's standards can
> outperform some $600 RAID cards with ease.

That's good to hear. I've done a dry run of creating the NAS with
VMWare, and it worked quite well. The true test will be to actually do
it with my spare machine.

I picked RAID5 to get the most available storage, while still keeping
redundancy. But needing 3 drives means that I'll need an extra IDE
controller.

The very cheapest controller I could find is this one:
http://www.koutech.com/proddetail.asp?linenumber=105
It's about $15 at Newegg. But I can't seem to determine if it's
supported by Linux. I believe it uses a SiliconImage chip, but the
Koutech website doesn't list Linux as a supported OS. (Though it does
list Linux for its other cards)


-WD

On Fri, 25 Jun 2004 15:38:27 -0600, Steve Wolfe wrote:

>> > If you use either of the two above, you'll be using software RAID. (In
>> > itself, this is neither good nor bad, and it may or may not fill your
>> > requirements.)
>>
>> Yes, software RAID is the plan. It's going to be a NAS box, so
>> performance isn't really important at all. (Since the network is
>> likely to be the bottleneck)
>
> Actually, software RAID can often out-perform hardware RAID, especially if
> you're talking about RAID 5. Your host CPU can churn out parity at an
> astounding rate. Even relatively modest CPU's by today's standards can
> outperform some $600 RAID cards with ease.

I'm running software RAID5 with 5 slices and 1 hotspare. It's not the CPU
that kills you, it's the disk I/O. Depending on what you're writing, you
might end up doing 5x the amount of I/O that you're trying to do. I think
what happens is something like: read old dir/parity info, read old partial
data, write old+new data, write new dir/parity info. I'm not sure what
else it's doing, but I measured about 5x the amount of I/O, and my disk
channels were "pinned" (aka redlined!). Apparently if you can tune the
size of the stripe, it can work better? In any case, RAID5 works much
better on reading. You might even get some situations (similar to
RAID0) where you're reading in parallel from several slices/stripes and
get better than 1x disk speed. So, I've come to the conclusion that
software RAID5 is best for situations where you rarely/sparsely write, and
occasionally/often read. That's my situation: a code repository for
reference, not high performance but valuable data.

The good RAID5 hardware gets it's performance by having a large number of
channels all reading/writing to disks simultaneously, as well as battery
backed RAM cache/buffers.

For situations where you write a lot more, I would recommend RAID1 (aka
mirroring). There you only multiply your writes only by 2x (instead of 5x
I/O). Reading is also considerably improved (if software is smart enough)
since it can choose to read the data from either disk, depending on the
head position and whatever other factors the drivers use for optimization.

As for network being the bottleneck: probably, if you have very fast
disks. That might change if you take into account the 5x factor. Also,
software RAID cannot rely on battery backed up cache/buffer, so it might
always try to flush out buffers, instead of taking its time with deferred
(lazy?) writes. You might want to investigate.

Also consider your UPS. RAID with corrupted writes is trashed. I don't
think you can recover like from a failed disk. Consider also that you're
vulnerable during the hotspare reconstruction period. Another disk failure
during that interval, and you're dead! RAID is not replacement for backups!

Like they say: YMMV.

--
Juhan Leemet
Logicognosis, Inc.

> For situations where you write a lot more, I would recommend RAID1 (aka
> mirroring). There you only multiply your writes only by 2x (instead of 5x
> I/O). Reading is also considerably improved (if software is smart enough)
> since it can choose to read the data from either disk, depending on the
> head position and whatever other factors the drivers use for optimization.

I took some SCSI drives a while back, and had a very hard time getting any
significantly better write performance out of RAID 1 than I did from RAID 5:

http://www.codon.com/docs/software-RAID.html

> As for network being the bottleneck: probably, if you have very fast
> disks.

Unless he's got gigabit networking, the network *will* be the bottleneck
unless he's got extremely slow disks. With 100mbit ethernet only handling
10-11 megaBYTES realistic throughput per second on a full-duplex, switched
connection, it doesn't take much disk at all to keep up. And even with
gigabit controllers and a gigabit switch, it takes a very potent machine
indeed to handle *file sharing* at gigabit speeds, even if you've got an
incredibly fast disk array.

> Also consider your UPS. RAID with corrupted writes is trashed. I don't
> think you can recover like from a failed disk.

I don't see why it wouldn't be just another inconsistency in the array
that gets handled in a resync, as far as the md device goes. As for the
file system, it gets fsck'd, or the journal recovered.

> RAID is not replacement for backups!

Very true!

steve

On Sat, 26 Jun 2004 18:48:16 -0600, Steve Wolfe wrote:
>> For situations where you write a lot more, I would recommend RAID1 (aka
>> mirroring). There you only multiply your writes only by 2x (instead of 5x
>> I/O)...

> I took some SCSI drives a while back, and had a very hard time getting
> any significantly better write performance out of RAID 1 than I did from
> RAID 5:
>
> http://www.codon.com/docs/software-RAID.html

Very interesting. I'm also surprised that your RAID5 writing is as fast
the RAID 0+1 and 1+0. Some of your bars show even higher?!? There should
be extra reads/writes in there for the parity information in RAID5. From
my crude anecdotal numbers, I would have expected a 4/5 or so decrease.

I admit that I did my tests a while ago on old used Sun gear that only had
wide 68-pin SCSI and 5 x SEAGATE ST118202LC disks. They would seem to be
similar to your IBM UltraStars? I'm not familiar with the Adaptec 2940
U2W. Is that one channel per board? Two? Transfer rate? So you had 2 disks
on each 2940? I assume you used all 4 disks for all tests?

I had a look at the OP and it wasn't clear to me whether he was
building a brand new system with the latest gear, or implementing on a
more pedestrian platform. I guess my advice pertains to older gear (e.g.
my own or similar uses), and not particularly "tuned", at that.

My measurement was from running a "tar cf -|tar xf -" kind of transfer
from NFS server to local RAID5. It was the RAID5 that limited transfers. I
have not tried any bonnie or bonnie++ tests on it. Maybe I should.

Which Linux O/S and release were you using? Which RAID software?

Thanks for the URL, and the "data points".

>> As for network being the bottleneck: probably, if you have very fast
>> disks.
>
> Unless he's got gigabit networking, the network *will* be the bottleneck
> unless he's got extremely slow disks. With 100mbit ethernet only
> handling 10-11 megaBYTES realistic throughput per second on a
> full-duplex, switched connection, it doesn't take much disk at all...

Yes, according to your numbers. In my case, with 100MHz ethernet and a
single SCSI chain on wide SCSI the disk system was the bottleneck.
Obviously, it is platform dependent. Room for tuning also.

BTW, I get suspicious of high I/O numbers. It used to be that people got
fooled by disk I/O tests because they forgot about the disk block buffers
used by *nix. They ended up basically testing memory bandwidth, and not
true disk bandwidth. I don't think that's what you're doing (-s 1024, that's 1GB,
right? vs. 256MB memory). You have to be writing many times your memory size,
to make sure that the (dynamic) disk block buffer cache does not confound
things. Might be fun to try again with -s 2047? Do you get same results?

Also, I wonder about the amount of parallelism in the bonnie tests. If
there are multiples of 3 or 4 (similar to numbers of disks?), I wonder if
you might see resultant artifacts. I think bonnie was originally designed
to test 1 disk on 1 CPU, but I might be wrong. Any extensions to multiple
disk systems and multiple CPUs might progressively add distortions? They
explicitly warn about interpretations of the "efficiency" measure.

Might have been fun to do RAID1 for a comparison, checkpoint. That should
result in exactly the same write performance as "single" (two writes
going in parallel), with 2x read (alternate reads like RAID0?)? Perhaps a
tiny bit more, if there is any intelligent read-ahead going on? Thoughts?

>> ...RAID with corrupted writes is trashed...
>> I don't think you can recover like from a failed disk.
>
> I don't see why it wouldn't be just another inconsistency in the array
> that gets handled in a resync, as far as the md device goes. As for the
> file system, it gets fsck'd, or the journal recovered.

I'll admit that was conjecture on my part, based on the fact that stuff
you're writing is "merged" with previous stuff on the disks. Depending on
where the fault(s) occur, your damage might be variable. I think if you
damage writes going to more than 1 disk, you're (somewhat?) screwed? RAID5
allows for 1 drive to fail, but not multiples. I don't think it can handle
"arbitrary inconsistencies". If only one write to one disk gets trashed,
then I agree fsck/journal will clean it up, but you might lose more than
just the most recent write. You might also lose some of the old data (in
the same RAID block size?) that was "merged". For more than 1 write? I
don't know. Complicated! Too much trouble to try to setup a test scenario.

p.s. Can you boot from your RAID5? or do you boot from normal disk devices
and then later mount your RAID5 file systems?

--
Juhan Leemet
Logicognosis, Inc.

Juhan Leemet wrote:


> I had a look at the OP and it wasn't clear to me whether he was
> building a brand new system with the latest gear, or implementing on a
> more pedestrian platform. I guess my advice pertains to older gear (e.g.
> my own or similar uses), and not particularly "tuned", at that.

My goal is to create the cheapest possible NAS box using an existing
K6-III 450 machine, and 3 new hard drives.

Performance is not important. Protection against any single drive
failure and availability of maximum available storage space is. It
will be used to store my MP3s, videos that I've archived from my MythTV
machine, and Acronis TrueImage backups. Maybe storage of DVD
works-in-progress.


-WD

Will Dormann wrote:
>
> My goal is to create the cheapest possible NAS box using an existing
> K6-III 450 machine, and 3 new hard drives.
>
> Performance is not important. Protection against any single drive
> failure and availability of maximum available storage space is.

That's the very goal of RAID5.

--
Ben M.