SEO

I have a 32-bit SHARED_MAGIC application that needs to allocate as much
shared memory as it can. I understand the total of all 32-bit shared
memory is limited to 1.75Gb on HP-UX 11.00 (unless I use SHMEM_MAGIC).
Assume I have the shmmax and my swap area big enough to actually allow
allocation of the 1.75Gb.

If I literally allocate as much shared memory as is available (maybe in
multiple segments, to overcome the 1Gb quadrant limit), it will give it
to me, but I gather other 32-bit applications that want to map shared
libraries won't be able to execute. As our application is long-lived,
this would be a problem!

I'm trying to find out the minimum "spare" shared memory the
application should leave on the system to ensure everything *else* can
still run. I can see that this depends on what else I want to run(!),
but if it's just a matter of the usual chatter of shell scripts and
editors and so on, what sort of ballpark are we talking? 10s of Mb? or
100s? More? How could I determine the safe limit?

Cheers,
-nick

nick.battle@gmail.com wrote:
> I have a 32-bit SHARED_MAGIC application that needs to allocate as much
> shared memory as it can. I understand the total of all 32-bit shared
> memory is limited to 1.75Gb on HP-UX 11.00 (unless I use SHMEM_MAGIC).
> Assume I have the shmmax and my swap area big enough to actually allow
> allocation of the 1.75Gb.
>
> If I literally allocate as much shared memory as is available (maybe in
> multiple segments, to overcome the 1Gb quadrant limit), it will give it
> to me, but I gather other 32-bit applications that want to map shared
> libraries won't be able to execute. As our application is long-lived,
> this would be a problem!
>
> I'm trying to find out the minimum "spare" shared memory the
> application should leave on the system to ensure everything *else* can
> still run. I can see that this depends on what else I want to run(!),
> but if it's just a matter of the usual chatter of shell scripts and
> editors and so on, what sort of ballpark are we talking? 10s of Mb? or
> 100s? More? How could I determine the safe limit?

Nick --

(I said some of this over on ITRC, but no reason not to repeat myself).

I can't comment on the typical shared library/shared object requirements
on your average box (given that doing development/testing is hardly
likely to have the same workloads... we're either mostly-idle or
stress testing for exhaustion). I would think, however, that you'd
have a difficult time determining a safe limit with accuracy -- simply
because you could never be positive that all the applications in
question had all their dynamic libraries loaded up.... if user foo's
emacs needed to load up the "Butter your Toast" module (emacs does
include the Toaster+AutoCAD module as well as the OS and Web Browser
now, right? ), your carefully calculated estimates are voided.

Your best bet would be to put your application in a Memory Window where
if possible you share Q2. That would allow you 2Gb of shared memory
within your window before you use any from the Globally shared Q4
(which is where shared libraries start). Even if you can't share Q2,
if you can start your app, set up your segment(s) as 1.5Gb or whatnot
[assuming that there's some library already mapped in Q4, which is a
pretty safe bet usually] even if you exhaust the global Q4 then the
"normal" applications running in the Global Window will still be able
to map their libraries in q3 (the fallback). This means that your
app (and any in a different window) wouldn't be able to map those
libraries... but if your app is already running and you've reached
your steady state, you won't care. (Oh, be sure that your app isn't
creating the Shared Memory object with MAP_GLOBAL/IPC_GLOBAL or
this is irrelevant... if you ask for Global, you get Q4 first).

Of course -- the best option would be to just recompile your app to
be 64-bit and have a couple of Tb to play with instead.

Don

> I said some of this over on ITRC

Well, I repeated the question here, so let me repeat my thanks for your
reply.

I've just seen that you've also answered by 2nd question here: that
shlibs do fall back to Q3 if Q4 is full. That means I don't have a
problem (though I can't access 100% of the possible memory, but I knew
that anyway).

A 64-bit rewrite would be wonderful, but it's rather old code and I'm
afraid the effort would be considerable. We might be able to do
selective rewrites ofthe bits that manage this large data area (it's a
lookup table, as you might have guessed).

Cheers,
-nick

Incidentally, if we did use Memory Windows, the easiest thing for us to
do would be to split the application and Oracle (9i) into their own
windows - Oracle is a large user of shared memory in Q3.

But does that work: can an application be the client of an Oracle
database running in a different memory window? (If the communication
between the two relies on shared memory, this might be a problem...?).

Cheers,
-nick

nick.battle@gmail.com wrote:
> Incidentally, if we did use Memory Windows, the easiest thing for us to
> do would be to split the application and Oracle (9i) into their own
> windows - Oracle is a large user of shared memory in Q3.
>
> But does that work: can an application be the client of an Oracle
> database running in a different memory window? (If the communication
> between the two relies on shared memory, this might be a problem...?).

If they literally rely on attaching to each other's shared memory - then
no, that wouldn't work to have them in different windows. Whichever
app created the new shared object would succeed, but the other app
would fail to attach.

(Unfortunately, I don't know the details of how Oracle handles their
client communication...).

If they create *new* segments to communicate between the client and
the database server, and said segments are created using IPC_GLOBAL,
then things would be okay. But you'd have to check with Oracle to
see if that's the way things work. (Oracle supply you with any
HP-UX documentation? A quick search on their site with "HP-UX
memory windows" apparently liked "Windows" more than the other
terms, which wasn't helpful).

Don