Hi!

I have a question concerning the memory management of HP-UX 10.20.
I tried to monitor (via top) how much memory my process uses during its
work.
With top I was able to see that the memory usage increases, but i wasn't
able to notice any decrease of it.
My first idea was that my program has a memory leak, so I wrote a small
test-program that allocates 65k of memory and freed it afterwards to
see what top is showing.
During the run of the program I also saw only an increase of the memory
usage.

Can anyone tell me, why HP-UX 10.20 is not freeing the memory or why
top is not showing any freed memory?

Kind regards, Andreas Schorn.

PS: The test programm:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(int argc, char *argv[]) {
void * pMem;

printf("Wait 15 s!\n");
fflush(stdout);
sleep(15);

pMem = malloc(65000);
printf("65000 Bytes allocated. Wait 15 s!\n");
fflush(stdout);
sleep(15);

free(pMem);

printf("65000 Bytes freed. Wating!\n");
fflush(stdout);

while(true) sleep(60);
}

Andreas Schorn <andreas.schorn@aixo.com> wrote:
> Hi!

> I have a question concerning the memory management of HP-UX 10.20.
> I tried to monitor (via top) how much memory my process uses during its
> work.
> With top I was able to see that the memory usage increases, but i wasn't
> able to notice any decrease of it.
> My first idea was that my program has a memory leak, so I wrote a small
> test-program that allocates 65k of memory and freed it afterwards to
> see what top is showing.
> During the run of the program I also saw only an increase of the memory
> usage.

> Can anyone tell me, why HP-UX 10.20 is not freeing the memory or why
> top is not showing any freed memory?

Because free() doesn't actually shrink the heap.
It does cache the memory - so you should be re-using it on the
next malloc call, though.

[This always surprises me when I'm poking around too, frankly].

If you want my best guess why it's this way, it would
be because all the allocations come out of a single private
chunk of memory (when you use brk/sbrk, which malloc does).

So think of the following case:

You malloc 50 bytes, getting address X. (heap grows by 50 from start)
You malloc 500 bytes, getting address X+50. (heap grows by 500)
You malloc 128 bytes, getting address X+550 (heap grows by 128)

Now -- you free the second allocation. How? You can't just
shrink the heap -- you'd blow away the third allocation as
well. You can't "re-order" the third allocation to start at
X+50 -- the user already has a virtual address for it somewhere
else. So what you do (in the best case) is wait until the third
allocation goes away -- and in the mean time any *new* allocations
that fit within that 500 bytes get whatever you can use from
that 500 bytes instead of making the heap grow again.

Now, the question in my mind is why the library folks don't
shrink the heap in the case when you are clear to do so (i.e.
no un-freed allocations until the end of the heap)... and that
I really don't know and haven't had the opportunity to ask.
Maybe it's simpler to assume the common case is that you
can't shrink the heap -- and for all I know that may well be
the common case. Maybe (like other dynamic allocators [including
the kernel's, in fact]) it assumes that if you needed the memory,
you may well need it again soon and it's better to hang on to
it to optimize subsequent mallocs. Can't say for sure, honestly.

If you're really, really on the wire when it comes to memory
consumption and want to be sure every byte is returned to the
system when you work - you could use private non-file mmaps
to manage your own data allocation -- those are completely
returned to the system when you munmap(). There's no guarantee
that any shared libraries you link in wouldn't be doing the
usual malloc()/free() calls, though - so that's not going to
be a total solution. (If you're curious as to why this is
different -- it is because the data object of the process is
exactly that, one big object of which the heap is part. Each
mmap is a separate object -- so in the example above, you'd
have 3 distinct objects which could be freed to the system
arbitrarily instead of one chunk of memory ending at X+678)

In the code below, add another pass to re-allocate the memory
after the free. You'll notice the virtual size stays the same.

Don


> Kind regards, Andreas Schorn.

> PS: The test programm:

> #include <stdio.h>
> #include <stdlib.h>
> #include <unistd.h>

> int main(int argc, char *argv[]) {
> void * pMem;

> printf("Wait 15 s!\n");
> fflush(stdout);
> sleep(15);

> pMem = malloc(65000);
> printf("65000 Bytes allocated. Wait 15 s!\n");
> fflush(stdout);
> sleep(15);

> free(pMem);

> printf("65000 Bytes freed. Wating!\n");
> fflush(stdout);

> while(true) sleep(60);
> }

--
Quidquid latine dictum sit, altum sonatur.