SEO

I was trying to set up a simple mirror (ie, software raid 1) under
OpenBSD 3.8 using CCD and the generic kernel, mirroring the entirety of
two one-partition disks. This should have been bone-simple but it was
pretty frustrating. The various net HOWTOs on how to set up a mirror
with ccd/ccdconfig were less than fully informative (and in some places
just plain wrong). And it doesn't help when mis-specifying the
parameters to simple commands like newfs will give you instant kernel
panics and crash your system hard...

I'm not sure why this area of the system's not as robust as everything
else in OpenBSD, but in case it helps others, here's what I found I had
to do to not get the usual failures and error messages and to make it
work. (Google bait follows: " inappropriate file type or format /
CCDIOCSET / error 22 on component 0 / device not configured / write
error ")

My scheme:

0) Obviously, get the bios to recognize the two disks (my fault in this
case, since I had to find the semi-hidden setting to turn on the
secondary IDE channel in the BIOS). They should both be seen by the
kernel at boot. FWIW, I've got two Seagate 40GB ide drives, wd0 and
wd1.

1) fdisk -i wd0 (ditto for wd1)
Writes a default partition table on each drive with one slice taking up
the whole drive.

2) disklabel -e -d wd0 (ditto for wd1)
This gets you an "a" partition on each diskthat takes up the whole disk
(minus the first cylinder). Change the "fstype" field in this first
partition from "unused" to "4.2BSD". Ignore the "c" partition that
represents the whole physical disk.

3) newfs wd0a (ditto for wd1a)
Make an identical file system on each drive *separately*, not on the
combined mirror. (Don't ask me why, all I know is this what made it
work for me.) Use whatever newfs options appeal to you, as long as both
drives are configured identically.

4) edit /etc/ccd.conf:
Mine reads:
ccd0 16 CCDF_MIRROR /dev/wd0a /dev/wd1a
(No, I don't know what the interleave factor of "16" does for a mirror
set, either. I just left it alone.)

5) ccdconfig -C
This creates your mirror. Check it's there with ccdconfig -v -g.
In my case, I got a new disk, /dev/ccd0c, for my troubles.

6) mount the mirror set and edit /etc/fstab so it's permanent:
For me, this was done with "mount /dev/ccd0c /archive". My final
/etc/fstab entry looks like:
/dev/ccd0c /samba ffs rw,nodev,nosuid,noexec 1 2
(of course you may have different mount options than these...this is my
archive backup disk set so I don't expect to be putting executables
there)

Hope this helps, but if not post your questions to the group, not to me
- all I know about the subject I've just written here. Good luck.

-jon-

Jonathan Rogers <thatseattleguy@gmail.com> wrote:
> I was trying to set up a simple mirror (ie, software raid 1) under
> OpenBSD 3.8 using CCD and the generic kernel, mirroring the entirety of
> two one-partition disks. This should have been bone-simple but it was
> pretty frustrating. The various net HOWTOs on how to set up a mirror
> with ccd/ccdconfig were less than fully informative (and in some places
> just plain wrong). And it doesn't help when mis-specifying the
> parameters to simple commands like newfs will give you instant kernel
> panics and crash your system hard...

Indeed, OpenBSD is not forgiving when it comes to messing around with
filesystems and drives.

> I'm not sure why this area of the system's not as robust as everything
> else in OpenBSD, but in case it helps others, here's what I found I had
> to do to not get the usual failures and error messages and to make it
> work. (Google bait follows: " inappropriate file type or format /
> CCDIOCSET / error 22 on component 0 / device not configured / write
> error ")
>
> My scheme:
>
> 0) Obviously, get the bios to recognize the two disks (my fault in this
> case, since I had to find the semi-hidden setting to turn on the
> secondary IDE channel in the BIOS). They should both be seen by the
> kernel at boot. FWIW, I've got two Seagate 40GB ide drives, wd0 and
> wd1.
>
> 1) fdisk -i wd0 (ditto for wd1)
> Writes a default partition table on each drive with one slice taking up
> the whole drive.

Okay, better make sure there's nothing on it.

> 2) disklabel -e -d wd0 (ditto for wd1)
> This gets you an "a" partition on each diskthat takes up the whole disk
> (minus the first cylinder). Change the "fstype" field in this first
> partition from "unused" to "4.2BSD". Ignore the "c" partition that
> represents the whole physical disk.

You should be using ccd for ccd devices, see ccd(4) (on a 3.8 system, I
assume it'll be similar for older systems).

Okay, it will not help much - but it's cleaner, and better for forward
compatibility.

> 3) newfs wd0a (ditto for wd1a)
> Make an identical file system on each drive *separately*, not on the
> combined mirror. (Don't ask me why, all I know is this what made it
> work for me.) Use whatever newfs options appeal to you, as long as both
> drives are configured identically.

Should not be necessary. Looks like voodoo to me.

> 4) edit /etc/ccd.conf:
> Mine reads:
> ccd0 16 CCDF_MIRROR /dev/wd0a /dev/wd1a
> (No, I don't know what the interleave factor of "16" does for a mirror
> set, either. I just left it alone.)
>
> 5) ccdconfig -C
> This creates your mirror. Check it's there with ccdconfig -v -g.
> In my case, I got a new disk, /dev/ccd0c, for my troubles.
>
> 6) mount the mirror set and edit /etc/fstab so it's permanent:
> For me, this was done with "mount /dev/ccd0c /archive". My final
> /etc/fstab entry looks like:
> /dev/ccd0c /samba ffs rw,nodev,nosuid,noexec 1 2
> (of course you may have different mount options than these...this is my
> archive backup disk set so I don't expect to be putting executables
> there)
>
> Hope this helps, but if not post your questions to the group, not to me
> - all I know about the subject I've just written here. Good luck.

Now, there's just been a lengthy thread on misc@openbsd.org where some
very knowledgeable fellows, including the RAIDframe maintainer, told
someone who had also just set up a CCD mirror and posted instructions
similar to yours that a) HOWTOs are evil and b) ccd mirroring will eat
your data (so use RAIDframe).

I tend to agree with a) here - your instructions are not wrong, but not
always correct either, and may tempt other to just type them without
understanding them. Additionally, b) is very, very important if your
data is important enough to mirror - it's likely to be important enough
to mirror right, too.

Joachim

>> 3) newfs wd0a (ditto for wd1a)
> >Make an identical file system on each drive *separately*, not on the
> >combined mirror. (Don't ask me why, all I know is this what made it work for me.)

>Should not be necessary. Looks like voodoo to me.

I agree it's voodoo - but without setting up the file systems in
advance on the separate drives, when I tried to do the newfs on the ccd
mirrored drive all I got was "error 22 ioctl (CCDIOCSET) on component
0". This was the only way I found to make it work.

>someone who had also just set up a CCD mirror and posted instructions
>similar to yours that a) HOWTOs are evil and b) ccd mirroring will eat
>your data (so use RAIDframe).
>...if your data is important enough to mirror - it's likely to be important enough
>to mirror right, too.

No argument there. I was avoiding raidframe only because I'm an OpenBSD
newbie (w/lots of Linux experience) and I was trying to find a solution
that would still allow me to use the stock/generic kernel for now.
After some more experience I'll look at switching to raidframe.

Certainly the kernel panics I got in the initial failed attempts to
make ccd work don't promote a sense of high confidence and
well-being....

thanks =jon=

Jonathan Rogers <thatseattleguy@gmail.com> wrote:
> Joachim wrote:
>>someone who had also just set up a CCD mirror and posted instructions
>>similar to yours that a) HOWTOs are evil and b) ccd mirroring will eat
>>your data (so use RAIDframe). ...if your data is important enough to
>>mirror - it's likely to be important enough to mirror right, too.
>
> No argument there. I was avoiding raidframe only because I'm an OpenBSD
> newbie (w/lots of Linux experience) and I was trying to find a solution
> that would still allow me to use the stock/generic kernel for now.
> After some more experience I'll look at switching to raidframe.
>
> Certainly the kernel panics I got in the initial failed attempts to
> make ccd work don't promote a sense of high confidence and
> well-being....

Yes, I have observed the same. RAIDframe is not much better there.

That said, RAIDframe has saved my behind at least once when a disk
crashed. Recovery was not totally painless (the disk confused the heck
out of the IDE bus, taking down the other attached disk as well, so it
was not immediately obvious which one had failed), but ultimately very
succesful.

Also, it's very stable once you set it up and stop poking it - I've had
quite a few problems and panics setting it up, though.

That being said, raid(4) has instructions for creating a custom kernel.
I use this one for single-processor PCs with RAID:

include "arch/i386/conf/GENERIC"
pseudo-device raid 4 # RAIDframe
option RAID_AUTOCONFIG

Where 'option RAID_AUTOCONFIG' is only necessary if you want root on
RAID.

Instructions for compiling a new kernel are in the FAQ, 5.3. Take the
opportunity to upgrade to -stable while you're there. It's very
painless, though it's best to do it while not using the machine or to
run it with nice, as it produces quite a bit of system load.

Joachim