SEO

We have a Sun Blade 2000 with 16GB of memory running Solaris as
supplied by Sun. It seems to limit the working set size (reported by
top) to 8GB for a single process, although it will allocate up to
13.9GB to that process. Is this a limitation of the HW/OS or is there
a tunable configuration parameter we can change to allow a larger
wroking set? We really need a large working set - it is ok if only one
large job executes. The application is a statistical package called
Stata.

Daniel Feenberg
feenberg isat nber dotte org

drfremove@nber.org (Daniel Feenberg) writes:

>We have a Sun Blade 2000 with 16GB of memory running Solaris as
>supplied by Sun. It seems to limit the working set size (reported by
>top) to 8GB for a single process, although it will allocate up to
>13.9GB to that process. Is this a limitation of the HW/OS or is there
>a tunable configuration parameter we can change to allow a larger
>wroking set? We really need a large working set - it is ok if only one
>large job executes. The application is a statistical package called
>Stata.

You need to establish what other bits of the system using memory;
or do you still have a lot of free memory?

Are you doing a lot of I/O?

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

drfremove@nber.org (Daniel Feenberg) wrote in message news:<41e0307b.0407110639.6a13084f@posting.google. com>...
> We have a Sun Blade 2000 with 16GB of memory running Solaris as
> supplied by Sun. It seems to limit the working set size (reported by
> top) to 8GB for a single process, although it will allocate up to
> 13.9GB to that process. Is this a limitation of the HW/OS or is there
> a tunable configuration parameter we can change to allow a larger
> wroking set? We really need a large working set - it is ok if only one
> large job executes. The application is a statistical package called
> Stata.
>
> Daniel Feenberg
> feenberg isat nber dotte org

You need a seconf cpu, the memory management unit is on the cpi-die,
the SPARC III cpu's max addressable RAM is 8 GB.

CU
Klaus Grote
DNS GmbH

Casper H.S. Dik <Casper.Dik@Sun.COM> wrote in message news:<40f15867$0$93324$e4fe514c@news.xs4all.nl>...
> drfremove@nber.org (Daniel Feenberg) writes:
>
> >We have a Sun Blade 2000 with 16GB of memory running Solaris as
> >supplied by Sun. It seems to limit the working set size (reported by
> >top) to 8GB for a single process, although it will allocate up to
> >13.9GB to that process. Is this a limitation of the HW/OS or is there
> >a tunable configuration parameter we can change to allow a larger
> >wroking set? We really need a large working set - it is ok if only one
> >large job executes. The application is a statistical package called
> >Stata.
>
> You need to establish what other bits of the system using memory;
> or do you still have a lot of free memory?
>
> Are you doing a lot of I/O?
>
> Casper


Apart from some small daemons there is nothing else running. The
application does no I/O except at the start and finish. All during the
run TOP shows a working set size of "8GB" and "13.9GB" for "memory
allocated". From my knowledge of the program I would have expected the
working set to be about 13.8GB. This led me to suspect there was a
switch somewhere that regulated the maximum working set size.

klaus.grote@web.de (Klaus Grote) writes:

>drfremove@nber.org (Daniel Feenberg) wrote in message news:<41e0307b.0407110639.6a13084f@posting.google. com>...
>> We have a Sun Blade 2000 with 16GB of memory running Solaris as
>> supplied by Sun. It seems to limit the working set size (reported by
>> top) to 8GB for a single process, although it will allocate up to
>> 13.9GB to that process. Is this a limitation of the HW/OS or is there
>> a tunable configuration parameter we can change to allow a larger
>> wroking set? We really need a large working set - it is ok if only one
>> large job executes. The application is a statistical package called
>> Stata.
>>
>> Daniel Feenberg
>> feenberg isat nber dotte org

>You need a seconf cpu, the memory management unit is on the cpi-die,
>the SPARC III cpu's max addressable RAM is 8 GB.

Not quite; I think it is 16GB; also, if you have memory you can't address,
it won't show up in the kernel.

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

drfremove@nber.org (Daniel Feenberg) writes:

>Apart from some small daemons there is nothing else running. The
>application does no I/O except at the start and finish. All during the
>run TOP shows a working set size of "8GB" and "13.9GB" for "memory
>allocated". From my knowledge of the program I would have expected the
>working set to be about 13.8GB. This led me to suspect there was a
>switch somewhere that regulated the maximum working set size.

What does top show as free memory?

Is the application paging a lot or is it using all of the
CPU? (If it's running a CPU flat out, then it simply doesn't
need more memory)

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

klaus.grote@web.de (Klaus Grote) wrote in message news:<93bac7d6.0407112312.55a8baff@posting.google. com>...
> drfremove@nber.org (Daniel Feenberg) wrote in message news:<41e0307b.0407110639.6a13084f@posting.google. com>...
> > We have a Sun Blade 2000 with 16GB of memory running Solaris as
> > supplied by Sun. It seems to limit the working set size (reported by
> > top) to 8GB for a single process, although it will allocate up to
> > 13.9GB to that process. Is this a limitation of the HW/OS or is there
> > a tunable configuration parameter we can change to allow a larger
> > wroking set? We really need a large working set - it is ok if only one
> > large job executes. The application is a statistical package called
> > Stata.
> >
> > Daniel Feenberg
> > feenberg isat nber dotte org
>
> You need a seconf cpu, the memory management unit is on the cpi-die,
> the SPARC III cpu's max addressable RAM is 8 GB.
>
> CU
> Klaus Grote
> DNS GmbH

We have 2 CPUs installed. I can't tell from your comment what
significance that is. Does this mean that the Blade 2000 can't
efficiently run a program with a working set of more than 8GB?

Daniel Feenberg

drfremove@nber.org (Daniel Feenberg) writes:

>We have 2 CPUs installed. I can't tell from your comment what
>significance that is. Does this mean that the Blade 2000 can't
>efficiently run a program with a working set of more than 8GB?

The Sun Blade 2000 can address 16GB with one CPU.

The UltraSPARC III and later CPUs have built-in memory controllers;
each of these controllers can address 4 banks of 4GB each, for
a maximum of 16GB of memory.

But this is not a limitation of the amount of memory a single
process can use; the CPU will just go elsewhere to access the
other memory (on the bigger systems, that is)

The Sun Blade 1000/2000 will only use the memory controller of CPU 0
so the number of CPUs installed does not matter for the amount
of memory used.

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

Casper H.S. Dik <Casper.Dik@Sun.COM> wrote in message news:<40f28514$0$34762$e4fe514c@news.xs4all.nl>...
> drfremove@nber.org (Daniel Feenberg) writes:
>
> >Apart from some small daemons there is nothing else running. The
> >application does no I/O except at the start and finish. All during the
> >run TOP shows a working set size of "8GB" and "13.9GB" for "memory
> >allocated". From my knowledge of the program I would have expected the
> >working set to be about 13.8GB. This led me to suspect there was a
> >switch somewhere that regulated the maximum working set size.
>
> What does top show as free memory?
>
> Is the application paging a lot or is it using all of the
> CPU? (If it's running a CPU flat out, then it simply doesn't
> need more memory)
>
> Casper

Here are the first few lines of top (nothing else is using much
memory):

load averages: 1.00, 1.00, 1.01 nber6 15:14:52
62 processes: 59 sleeping, 1 stopped, 2 on cpu
CPU states: 48.8% idle, 49.9% user, 1.3% kernel, 0.0% iowait, 0.0%
swap
Memory: 16.0G real, 5.4G free, 13.6G swap in use, 115M swap free

PID USERNAME THR PR NCE SIZE RES STATE TIME FLTS CPU
COMMAND
573 gmiller 1 0 19 13.6G 8.0G cpu00 116.6H 0 49.97%
stata-se
9932 root 1 59 0 4608K 2920K sleep 0:00 0 0.02% sshd
9938 root 1 59 0 2800K 1920K cpu01 0:00 0 0.01% top


Whenever we try to start up a new process(for instance when trying to
ssh in), we get a message like this in /var/adm/messages:

Jul 12 15:09:22 nber6 genunix: [ID 470503 kern.warning] WARNING:
Sorry, no swap
space to grow stack for pid 9929 (sshd)

Here is the output of swap command:

# /usr/sbin/swap -s
total: 8418784k bytes allocated + 5855088k reserved = 14273872k used,
118032k
available

We don't think it is merely a coincidence that the working set is 8.0
GB, especially in view of the inability to spawn a new process, and
the error
messages in the log.

Thanks for any help.

Daniel Feenberg

drfremove@nber.org (Daniel Feenberg) writes:

>Here are the first few lines of top (nothing else is using much
>memory):

>load averages: 1.00, 1.00, 1.01 nber6 15:14:52
>62 processes: 59 sleeping, 1 stopped, 2 on cpu
>CPU states: 48.8% idle, 49.9% user, 1.3% kernel, 0.0% iowait, 0.0%
>swap
>Memory: 16.0G real, 5.4G free, 13.6G swap in use, 115M swap free

So plenty of free memory.

> PID USERNAME THR PR NCE SIZE RES STATE TIME FLTS CPU
>COMMAND
> 573 gmiller 1 0 19 13.6G 8.0G cpu00 116.6H 0 49.97%
>stata-se
> 9932 root 1 59 0 4608K 2920K sleep 0:00 0 0.02% sshd
> 9938 root 1 59 0 2800K 1920K cpu01 0:00 0 0.01% top

Using near 100% of one CPU.

This looks like the process has a resident set size it is comfortable
with; however, the remaining swap space is rather low and that is
an issue (not so much for the process in question but for the
system itself).

>Here is the output of swap command:

># /usr/sbin/swap -s
>total: 8418784k bytes allocated + 5855088k reserved = 14273872k used,
>118032k
>available

>We don't think it is merely a coincidence that the working set is 8.0
>GB, especially in view of the inability to spawn a new process, and
>the error messages in the log.

It is a conincedence, I think, because the process has taken 13.6GB
of swap so it doesn't need more. I do think that for some rason your
system needs more swap; it has "reserved" 6GB for something (typically
forked processes, or mmap's of large parts of anonymous memory
as yet unused), but it is running happily with an RSS of 8GB.

If the process was finding pages being stolen by the kernel and trying
to get a larger RSS, you would find it using a lot *less* CPU because it
would be blocked on page faults continuously.

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.