Henrik Goldman wrote:
> Following my other Sun 280R questions I've had this week I reached a
> state where the system is up and stable.
> I installed it now with Solaris 8 (plain install, Generic_108528-29)
> but comparing with our older Ultra5 I don't see the 3-4 times speed
> increase that I had hoped for.
> We mainly use this system for compilation of c++ code using g++ and the
> specs being 2x 750 MHz with 2GB ram should outperform our Ultra 5 with
> 400 MHz and 512 MB easily.
>
> While I've only done a bit of early testing I noticed that the disks are
> pretty slow. The system comes with 2 Seagate 36GB FC-AL disks but
> currently we only use one of them.
>
> Maybe I'm missing some magic patches compared to a base install that
> would help?
>
> Thanks.
>
> -- Henrik

A modest tuning with no draw backs in /etc/system:
* Double the Solaris 8 thresholds for ufs throttling
set ufs:ufs_HW=786432
set ufs:ufs_LW=524288

Reboot or immediately activate with
echo ufs_HW/W 786432 | adb -kw
echo ufs_LW/W 524288 | adb -kw

For resilience, remount you disk with UFS logging:
In /etc/vfstab put the word logging in the options column.
Reboot or immediately activate with
mount -o remount,logging <mount-point>

UFS logging can also improve speed in some situations
(but decrease speed in others).

Disk mirroring with Disk Suite will further increase
resilience and read performance.

Install glm patch 109885-20.
Reboot to activate.

--
Michael Tosch @ hp : com

On Apr 13, 7:01 pm, "Henrik Goldman" <h...@x-formation.com> wrote:

>
> However I see clearly now that the disk is more slow then the old 20 gb IDE
> disk that was in the Ultra 5 machine we're replacing. Some unit tests done
> by our code write a lot of files to the disk and then read them again. This
> has to happen within 2 seconds but takes 3.9 seconds on the 280R where as it
> never failed on the Ultra 5. Maybe I should look into options with write
> cache?
>

That's just not right. There are crappier disk systems than the U5/
U10 had but you'd be hard put to find one, and the 280R certainly is
not an example. You have some horrible misconfiguration or are
misidentifying the problem. Use some disk benchmark (bonnie?) to
establish what the performance problem really is (and in particular to
establish that it's not I/O).

Do you have logging turned on on the filesystems? This can make a huge
difference to metadata-heavy loads (lots of tiny short-lived files)
often generated by compilations. I don't think logging became the
default until 10, so unless the mounts specify it you don't have it,
and you should.

On 2008-04-14, Tim Bradshaw <tfb+google@tfeb.org> wrote:
> That's just not right. There are crappier disk systems than the U5/
> U10 had but you'd be hard put to find one, and the 280R certainly is
> not an example. You have some horrible misconfiguration or are
> misidentifying the problem. Use some disk benchmark (bonnie?) to
> establish what the performance problem really is (and in particular to
> establish that it's not I/O).

I'll second that. I recommend to run bonnie [1] on your machines to have
a comparison. Here are the results of a simple bonnie run on our 280R with
two processors at 900 MHz, 4 GB memory, using a Seagate ST373405 SCSI disk
(73.4GB, 10000 RPM, PN 390-0071) with UFS logging, and running Solaris 9:

-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine GB M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU /sec %CPU
Fire 280R 5 23.3 96.6 32.5 23.4 11.1 11.6 22.5 93.8 63.7 33.4 78 2.7
Fire 280R 10 23.4 97.4 31.9 23.5 12.0 12.6 20.0 83.7 22.7 12.4 31 2.4

We do no longer have any Ultra 5s around. Instead I took a Sun Blade
100 with a 502 MHz processor, 1152 MB memory, using a Seagate ST315310A
(15.3GB, 7200 RPM Ultra ATA/100, PN 370-4154) with UFS logging, running
Solaris 10. I would not expect this box to be slower than a Ultra 5:

-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine GB M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU /sec %CPU
Blade 100 1 13.6 93.3 22.8 66.7 6.3 22.5 13.3 92.2 39.1 63.8 87 5.9
Blade 100 2 14.2 94.2 27.8 73.9 6.9 24.2 13.0 91.6 37.6 70.9 49 5.2

Andreas.

[1] http://code.google.com/p/bonnie-64/

Hi Michael,

Thanks for your suggestions. The changes you suggested has eliminated all
the previous speed problems. Now everything works so much faster.

However now with the installation of this patch (or at least thats what I
believe is the cause) logins to the machine takes like forever. Before I
started messing around with this everything was fairly quick. Now it can
take up to 1 minute before I login. It just hangs after entering password.

-- Henrik

>
> That's just not right. There are crappier disk systems than the U5/
> U10 had but you'd be hard put to find one, and the 280R certainly is
> not an example. You have some horrible misconfiguration or are
> misidentifying the problem. Use some disk benchmark (bonnie?) to
> establish what the performance problem really is (and in particular to
> establish that it's not I/O).

Actually it's a plain standard install of Sol 8 which causes this.
Misconfiguration was done by Sun in this case.

> Do you have logging turned on on the filesystems? This can make a huge
> difference to metadata-heavy loads (lots of tiny short-lived files)
> often generated by compilations. I don't think logging became the
> default until 10, so unless the mounts specify it you don't have it,
> and you should.

Now I've turned it on and it has eliminated those problems. Thanks also for
suggesting this.

-- Henrik

On Apr 14, 9:34 pm, "Henrik Goldman" <h...@x-formation.com> wrote:

> Now I've turned it on and it has eliminated those problems. Thanks also for
> suggesting this.

I suspect fairly strongly it was logging not the other changes that
made the difference actually. It kind of is the case that logging
should have been the default for much longer than it has, but I
suspect a lot of people would have had large tantrums if it was.

> Machine GB M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU /sec
> %CPU
> Fire 280R 5 23.3 96.6 32.5 23.4 11.1 11.6 22.5 93.8 63.7 33.4 78
> 2.7
> Fire 280R 10 23.4 97.4 31.9 23.5 12.0 12.6 20.0 83.7 22.7 12.4 31
> 2.4

It's so strange why your performance is so much better than ours. Maybe your
HD is simply faster:

-------Sequential Output-------- ---Sequential
Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per
Char- --Block--- --Seeks---
Machine GB M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU /sec
%CPU
0 14.7 53.2 31.7 14.6 26.8 22.8 23.8 100.0 200.7 100.4 3509
38.6

-- Henrik

On 2008-04-20, Henrik Goldman <hg@x-formation.com> wrote:
>> Machine GB M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU /sec
>> %CPU
>> Fire 280R 5 23.3 96.6 32.5 23.4 11.1 11.6 22.5 93.8 63.7 33.4 78
>> 2.7
>> Fire 280R 10 23.4 97.4 31.9 23.5 12.0 12.6 20.0 83.7 22.7 12.4 31
>> 2.4
>
> It's so strange why your performance is so much better than ours. Maybe your
> HD is simply faster:
>
> -------Sequential Output-------- ---Sequential
> Input-- --Random--
> -Per Char- --Block--- -Rewrite-- -Per
> Char- --Block--- --Seeks---
> Machine GB M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU /sec
> %CPU
> 0 14.7 53.2 31.7 14.6 26.8 22.8 23.8 100.0 200.7 100.4 3509
> 38.6

You have just run bonnie with the default options, i.e. with a file size
of 100 MB. As long as the file size is significantly less than your memory
size, you are testing the general processing speed of your system but
not your hard disks. You can see this also easily from the %CPU column
which tells that with 100.4% your sequential block-wise reading test was
CPU-bound. You should run bonnie with a file size that is larger or even
a multiple of the available memory to measure your disk performance.

On our Sun Fire 280R, we get following results with bonnie default options:

-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine GB M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU /sec %CPU
Fire 280R 0 23.8 95.7 39.6 23.7 42.6 23.9 26.0 98.6 486.3 102.1 23280 98.9

You wouldn't get 486.3 M/sec from our disks at our 280R

Andreas.

Henrik Goldman wrote:
>> If you can and the code fits try compile in /tmp/...
>> see if its faster in memory.
>
> How can I see if /tmp is a ram disk or if it's a physical disk?
> From df -k I get:
>
> bash-3.2$ df -k
> Filesystem kbytes used avail capacity Mounted on
> /dev/dsk/c1t0d0s0 32944868 2140235 30475185 7% /
> /proc 0 0 0 0% /proc
> fd 0 0 0 0% /dev/fd
> mnttab 0 0 0 0% /etc/mnttab
> swap 3720128 16 3720112 1% /var/run
> swap 3744136 24024 3720112 1% /tmp
>
/tmp is memory up to the free limit, the its on swap. you have 3gb
memory free.
$ vmstat 5 30
shows swap & free memory

>> Solaris make and gnu make has flags for parallell make.
>
> Already in-use :-)
>
>> # format -e can enable write cache for the scsi disks on the 280R.
>> But dont know if its recommended without UPS and no ZFS in solaris 8.
>
> Once the machine hits the data center it will be backed by 2 diesel
> generators... so power is not my worst fear.
> Furthermore it's just a compilation machine and thus no important data
> on it.
>
write cache enabling usually speeds up mirroring disks 10 times.

> I think I should see about the ram disk idea and if that doesn't help
> look for enabling this write cache.
>
> -- Henrik
/tmp/... is ram until you use it all,
then it continues on the swap partition.
/Jorgen