Hello people,

Can someone give us some advice. Currently our mirrorset is remirroring
very slow. It is one 4 TB and one 1 TB volume on a iscsi san.

4 TB (2 concatenated 2 TB san volumes)
1 TB (1 san volume)

At this rate the sync will be ready in about two weeks :-)

I have checked the netowork settings, and everything looks ok. Now I saw
at sunsolve a document describing the "Increasing the
md_resync_bufsz".

http://sunsolve.sun.com/search/docum...ey=1-9-81236-1
Can this help us ?
Or does anyone else can give us a hint why this is so slow ?
(sar output below)
Greetings... Richard


SunOS server1 5.10 Generic_125100-10 sun4u 08/11/2007

01:18:22 device %busy avque r+w/s blks/s avwait avserv

01:18:23 iscsi_se 97 2.7 253 24930 0.1 10.5
iscsi_se 17 0.2 14 1816 0.0 11.8
iscsi_se 2 0.0 20 331 0.0 1.1
iscsi_se 94 0.9 25 1892 0.1 38.4
iscsi_se 80 0.8 13 1695 0.0 60.3
iscsi_se 2 0.0 20 331 0.0 0.8
md0 0 0.0 0 0 0.0 0.0
md1 0 0.0 0 0 0.0 0.0
md2 0 0.0 0 0 0.0 0.0
md3 0 0.0 0 0 0.0 0.0
md4 0 0.0 0 0 0.0 0.0
md5 0 0.0 0 0 0.0 0.0
md6 0 0.0 0 0 0.0 0.0
md7 0 0.0 0 0 0.0 0.0
md8 0 0.0 0 0 0.0 0.0
md9 100 3.8 285 26838 0.0 13.2
md10 98 2.8 272 25264 0.0 10.4
md11 95 1.0 44 2223 0.0 21.7
md12 100 1.0 27 3512 0.0 36.4
md13 20 0.2 14 1817 0.0 14.0
md14 80 0.8 13 1695 0.0 60.4
md20 0 0.0 0 0 0.0 0.0
ssd0.isc 0 0.0 20 331 0.0 0.0
ssd1.isc 0 0.0 13 1695 0.0 0.0
ssd2.isc 0 0.0 25 1892 0.0 0.0
ssd3.isc 0 0.0 20 331 0.0 0.0
ssd4.isc 0 0.0 14 1817 0.0 0.0
ssd5.isc 0 0.0 253 24933 0.0 0.0
nfs1 0 0.0 0 0 0.0 0.0
sd0 0 0.0 0 0 0.0 0.0
sd0,a 0 0.0 0 0 0.0 0.0
sd0,b 0 0.0 0 0 0.0 0.0
sd0,c 0 0.0 0 0 0.0 0.0
sd0,d 0 0.0 0 0 0.0 0.0
sd0,h 0 0.0 0 0 0.0 0.0
sd3 0 0.0 0 0 0.0 0.0
sd3,a 0 0.0 0 0 0.0 0.0
sd3,b 0 0.0 0 0 0.0 0.0
sd3,c 0 0.0 0 0 0.0 0.0
sd3,d 0 0.0 0 0 0.0 0.0
sd3,h 0 0.0 0 0 0.0 0.0
sd15 0 0.0 0 0 0.0 0.0
sd15,a 0 0.0 0 0 0.0 0.0
sd15,h 0 0.0 0 0 0.0 0.0
sd18 0 0.0 0 0 0.0 0.0
sd18,a 0 0.0 0 0 0.0 0.0
sd18,h 0 0.0 0 0 0.0 0.0
ssd0 2 0.0 20 331 0.0 0.9
ssd0,a 2 0.0 20 331 0.0 0.9
ssd0,h 0 0.0 0 0 0.0 0.0
ssd1 80 0.8 13 1695 0.0 60.4
ssd1,a 80 0.8 13 1695 0.0 60.4
ssd1,c 0 0.0 0 0 0.0 0.0
ssd2 94 0.9 25 1892 0.0 38.5
ssd2,a 94 0.9 25 1892 0.0 38.5
ssd2,h 0 0.0 0 0 0.0 0.0
ssd3 2 0.0 20 331 0.0 1.2
ssd3,a 2 0.0 20 331 0.0 1.2
ssd3,h 0 0.0 0 0 0.0 0.0
ssd4 20 0.2 14 1817 0.0 14.0
ssd4,a 20 0.2 14 1817 0.0 14.0
ssd4,c 0 0.0 0 0 0.0 0.0
ssd5 98 2.8 253 24933 0.0 11.1
ssd5,a 98 2.8 253 24933 0.0 11.1
ssd5,h 0 0.0 0 0 0.0 0.0
ohci0,bu 0 0.0 0 0 0.0 0.0
ohci0,ct 0 0.0 0 0 0.0 0.0
ohci0,in 0 0.0 0 0 0.0 0.0
ohci0,is 0 0.0 0 0 0.0 0.0
ohci0,to 0 0.0 0 0 0.0 0.0

R.Smits wrote:
> Hello people,
>
> Can someone give us some advice. Currently our mirrorset is remirroring
> very slow. It is one 4 TB and one 1 TB volume on a iscsi san.
>
> 4 TB (2 concatenated 2 TB san volumes)
> 1 TB (1 san volume)
>
> At this rate the sync will be ready in about two weeks :-)
>
> I have checked the netowork settings, and everything looks ok. Now I saw
> at sunsolve a document describing the "Increasing the
> md_resync_bufsz".
>
> http://sunsolve.sun.com/search/docum...ey=1-9-81236-1
> Can this help us ?
> Or does anyone else can give us a hint why this is so slow ?
> (sar output below)
> Greetings... Richard
>
>
> SunOS server1 5.10 Generic_125100-10 sun4u 08/11/2007
>
> 01:18:22 device %busy avque r+w/s blks/s avwait avserv
>
> 01:18:23 iscsi_se 97 2.7 253 24930 0.1 10.5
> iscsi_se 17 0.2 14 1816 0.0 11.8
> iscsi_se 2 0.0 20 331 0.0 1.1
> iscsi_se 94 0.9 25 1892 0.1 38.4
> iscsi_se 80 0.8 13 1695 0.0 60.3
> iscsi_se 2 0.0 20 331 0.0 0.8
> md0 0 0.0 0 0 0.0 0.0
> md1 0 0.0 0 0 0.0 0.0
> md2 0 0.0 0 0 0.0 0.0
> md3 0 0.0 0 0 0.0 0.0
> md4 0 0.0 0 0 0.0 0.0
> md5 0 0.0 0 0 0.0 0.0
> md6 0 0.0 0 0 0.0 0.0
> md7 0 0.0 0 0 0.0 0.0
> md8 0 0.0 0 0 0.0 0.0
> md9 100 3.8 285 26838 0.0 13.2
> md10 98 2.8 272 25264 0.0 10.4
> md11 95 1.0 44 2223 0.0 21.7
> md12 100 1.0 27 3512 0.0 36.4
> md13 20 0.2 14 1817 0.0 14.0
> md14 80 0.8 13 1695 0.0 60.4
> md20 0 0.0 0 0 0.0 0.0
> ssd0.isc 0 0.0 20 331 0.0 0.0
> ssd1.isc 0 0.0 13 1695 0.0 0.0
> ssd2.isc 0 0.0 25 1892 0.0 0.0
> ssd3.isc 0 0.0 20 331 0.0 0.0
> ssd4.isc 0 0.0 14 1817 0.0 0.0
> ssd5.isc 0 0.0 253 24933 0.0 0.0
> nfs1 0 0.0 0 0 0.0 0.0
> sd0 0 0.0 0 0 0.0 0.0
> sd0,a 0 0.0 0 0 0.0 0.0
> sd0,b 0 0.0 0 0 0.0 0.0
> sd0,c 0 0.0 0 0 0.0 0.0
> sd0,d 0 0.0 0 0 0.0 0.0
> sd0,h 0 0.0 0 0 0.0 0.0
> sd3 0 0.0 0 0 0.0 0.0
> sd3,a 0 0.0 0 0 0.0 0.0
> sd3,b 0 0.0 0 0 0.0 0.0
> sd3,c 0 0.0 0 0 0.0 0.0
> sd3,d 0 0.0 0 0 0.0 0.0
> sd3,h 0 0.0 0 0 0.0 0.0
> sd15 0 0.0 0 0 0.0 0.0
> sd15,a 0 0.0 0 0 0.0 0.0
> sd15,h 0 0.0 0 0 0.0 0.0
> sd18 0 0.0 0 0 0.0 0.0
> sd18,a 0 0.0 0 0 0.0 0.0
> sd18,h 0 0.0 0 0 0.0 0.0
> ssd0 2 0.0 20 331 0.0 0.9
> ssd0,a 2 0.0 20 331 0.0 0.9
> ssd0,h 0 0.0 0 0 0.0 0.0
> ssd1 80 0.8 13 1695 0.0 60.4
> ssd1,a 80 0.8 13 1695 0.0 60.4
> ssd1,c 0 0.0 0 0 0.0 0.0
> ssd2 94 0.9 25 1892 0.0 38.5
> ssd2,a 94 0.9 25 1892 0.0 38.5
> ssd2,h 0 0.0 0 0 0.0 0.0
> ssd3 2 0.0 20 331 0.0 1.2
> ssd3,a 2 0.0 20 331 0.0 1.2
> ssd3,h 0 0.0 0 0 0.0 0.0
> ssd4 20 0.2 14 1817 0.0 14.0
> ssd4,a 20 0.2 14 1817 0.0 14.0
> ssd4,c 0 0.0 0 0 0.0 0.0
> ssd5 98 2.8 253 24933 0.0 11.1
> ssd5,a 98 2.8 253 24933 0.0 11.1
> ssd5,h 0 0.0 0 0 0.0 0.0
> ohci0,bu 0 0.0 0 0 0.0 0.0
> ohci0,ct 0 0.0 0 0 0.0 0.0
> ohci0,in 0 0.0 0 0 0.0 0.0
> ohci0,is 0 0.0 0 0 0.0 0.0
> ohci0,to 0 0.0 0 0 0.0 0.0

Additional info :

This is after 1 1/2 days the remirroring began.

d12: Mirror
Submirror 0: d13
State: Okay
Submirror 1: d14
State: Resyncing
Resync in progress: 11 % done
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 2097056000 blocks (999 GB)

d9: Mirror
Submirror 0: d10
State: Okay
Submirror 1: d11
State: Resyncing
Resync in progress: 3.1 % done
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 8472461312 blocks (3.9 TB)

In article <lt6dnUqDEOAMbyHbRVnyvwA@infopact.nl>,
"R.Smits" <smitsr@gmail.com> writes:
> Hello people,
>
> Can someone give us some advice. Currently our mirrorset is remirroring
> very slow. It is one 4 TB and one 1 TB volume on a iscsi san.
>
> 4 TB (2 concatenated 2 TB san volumes)
> 1 TB (1 san volume)
>
> At this rate the sync will be ready in about two weeks :-)
>
> I have checked the netowork settings, and everything looks ok. Now I saw
> at sunsolve a document describing the "Increasing the
> md_resync_bufsz".
>
> http://sunsolve.sun.com/search/docum...ey=1-9-81236-1
> Can this help us ?
> Or does anyone else can give us a hint why this is so slow ?

It's deliberately slow, so it doesn't impact normal operation
of the system. The resync speed was probably decided upon long
before there were TB storage devices.

--
Andrew Gabriel
[email address is not usable -- followup in the newsgroup]

R.Smits wrote:
> Hello people,
>
> Can someone give us some advice. Currently our mirrorset is remirroring
> very slow. It is one 4 TB and one 1 TB volume on a iscsi san.
>
[CUT...]

what kind of interfaces you're using for iSCSI?

Regards,
Blazej