Hello,

I am just taking a look at our file system configuration and to me, it
doesn't look very well layed out. Can someone point out how this is
flawed (if it is) and possibly how it should be arranged. Please note
that i'm not qualified to fully understand all of this.

The idea behind this two raid5 setup is that we have redunancy and
performance. Two raid 5 sets are used so that more than one disk can
fail without causing problems (i assume). It also may have to do
with the controllers and how they are configured (each array assigned
to a different controller).

If anyone can offer a technically critique of this setup, that would
be great. I can already tell that this setup is causing hdisk4 to be
written too more than hdisk3, but is there any other reasons that this
setup potentially causes problems? (lack of redundancy, major
performance decrease, etc etc).

thx

-Adam

We have a vg called datavg that contains two raid 5 disk sets. There
are only raw partitions, 20 of them.

# lsvg -l datavg
datavg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT
POINT
pick0 raw 16 16 1 open/syncd N/A
pick1 raw 16 16 1 open/syncd N/A
pick2 raw 16 16 1 open/syncd N/A
pick3 raw 16 16 1 open/syncd N/A
pick4 raw 16 16 1 open/syncd N/A
pick5 raw 16 16 1 open/syncd N/A
pick6 raw 16 16 1 open/syncd N/A
pick7 raw 16 16 1 open/syncd N/A
pick8 raw 16 16 1 open/syncd N/A
pick9 raw 16 16 1 open/syncd N/A
pick10 raw 16 16 2 open/syncd N/A
pick11 raw 16 16 2 open/syncd N/A
pick12 raw 16 16 2 open/syncd N/A
pick13 raw 16 16 2 open/syncd N/A
pick14 raw 16 16 2 open/syncd N/A
pick15 raw 16 16 2 open/syncd N/A
pick16 raw 16 16 2 open/syncd N/A
pick17 raw 16 16 1 open/syncd N/A
pick18 raw 16 16 2 open/syncd N/A
pick19 raw 14 14 1 open/syncd N/A

# lsvg -p datavg
datavg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE
DISTRIBUTION
hdisk3 active 159 0
00..00..00..00..00
hdisk4 active 159 0
00..00..00..00..00
#

The raid5 sets are hdisk3 and hdisk4. These volumes (20) are allocated
to the disksets. Some of these volumes are actually on both of the
raid5 sets.

ex.

# lspv -l hdisk4 | sort
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
hdisk4:
pick1 16 16 00..16..00..00..00 N/A
pick10 8 8 00..01..00..00..07 N/A
pick11 1 1 00..01..00..00..00 N/A
pick12 1 1 00..01..00..00..00 N/A
pick13 1 1 00..01..00..00..00 N/A
pick14 1 1 00..01..00..00..00 N/A
pick15 1 1 00..01..00..00..00 N/A
pick16 1 1 00..01..00..00..00 N/A
pick18 1 1 00..01..00..00..00 N/A
pick2 16 16 00..01..15..00..00 N/A
pick3 16 16 00..01..15..00..00 N/A
pick4 16 16 14..01..01..00..00 N/A
pick5 16 16 15..01..00..00..00 N/A
pick6 16 16 03..01..00..12..00 N/A
pick7 16 16 00..01..00..15..00 N/A
pick8 16 16 00..01..00..05..10 N/A
pick9 16 16 00..01..00..00..15 N/A

# lspv -l hdisk3 | sort
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
hdisk3:
pick0 16 16 00..16..00..00..00 N/A
pick10 8 8 00..08..00..00..00 N/A
pick11 15 15 00..06..09..00..00 N/A
pick12 15 15 00..00..15..00..00 N/A
pick13 15 15 08..00..07..00..00 N/A
pick14 15 15 15..00..00..00..00 N/A
pick15 15 15 09..00..00..06..00 N/A
pick16 15 15 00..00..00..15..00 N/A
pick17 16 16 00..01..00..11..04 N/A
pick18 15 15 00..00..00..00..15 N/A
pick19 14 14 00..01..00..00..13 N/A

most lv's seem to be soley on hdisk4 but there are few that are only
on hdisk3 but some are spread accross both.

ex:

# lslv -l pick18
pick18:N/A
PV COPIES IN BAND DISTRIBUTION
hdisk4 001:000:000 100% 000:001:000:000:000
hdisk3 015:000:000 0% 000:000:000:000:015

# lslv -m pick18
pick18:N/A
LP PP1 PV1
0001 0049 hdisk4
0002 0132 hdisk3
0003 0133 hdisk3
0004 0134 hdisk3
0005 0135 hdisk3
0006 0136 hdisk3
0007 0137 hdisk3
0008 0138 hdisk3
0009 0139 hdisk3
0010 0140 hdisk3
0011 0141 hdisk3
0012 0142 hdisk3
0013 0143 hdisk3
0014 0144 hdisk3
0015 0145 hdisk3
0016 0146 hdisk3

1st critique would be that these are not filesystems, they are raw LVs


Do you have a specific performance issue ? or is this just a general
whats up with my systems type thing ?

one thing to consider is parralelism, i.e. you have 2 sets of raid 5
over differing controllers, you could spread your LVs across both
disks and therefore both controllers...

you also want to use filemon to see which are you most heavliy hit LVs
and concentrate on those 1st, or make a list of higest --> lowest
utilised and this will be used in the reorgvg command later.

Check the Inter Policy and make sure its maximum: This one isnt -->

lslv lvname | grep INTER-POLICY
INTER-POLICY: minimum RELOCATABLE: yes


change it with -->
chlv -e x lvname

Then reorg the volume group with reorgvg.. check out the man pages -->

http://publib16.boulder.ibm.com/doc_...gd/2365c86.htm
http://publib16.boulder.ibm.com/doc_...s4/reorgvg.htm

"The reorgvg command reorganizes the placement of allocated physical
partitions
within the VolumeGroup, according to the allocation characteristics of
each
logical volume. Use the LogicalVolume parameter to reorganize specific
logical
volumes; highest priority is given to the first logical volume name in
the
LogicalVolume parameter list and lowest priority is given to the last
logical
volume in the parameter list."

You may also want to look at queue_depth and max_coalesce for raid5
disks... search this news group for hits on tuning, you can ramp up
the queue_depth based on the number of disks the raid5 disk consists
of etc...

Just a bit of info, but reading the tuning guide or the perf
certification redbook will give you much more of an insight etc..

http://publib16.boulder.ibm.com/doc_...ftungdtfrm.htm

http://publib-b.boulder.ibm.com/Redb...6184.html?Open

HTH
Mark Taylor