SEO

(Please neglect first post, see this post only)
Hi All,
I have some questions related to shared libraries on AIX:
1. Is they are any good way to know order in which shared libraries
are loaded on AIX. On SUN, I can get this information using
LD_DEBUG=files... I need this information because I am assuming that
with runtime linking enabled, symbols are searched in libraries in
order in which they are loaded.
2. When are shared libraries that are attached to 0xd segment
removed? I see shared libs loaded by prog (genkld) even 2 hrs after
prog stopped. I tried rm *.so too.
3.If I want to get lazyloading behavior AND make sure that all
references to given symbol bind to same definition, I figured I need
to use
-blazy -G -bexpall when building libraries specify all dependent
libraries.
-blazy -brtllib -bnosymbolic -bexpall when linking main (since -brtl
destorys lazyloading).
This worked if I have interposed symbols among shared libraries
only. If I had data symbol defined in main executable and multiple
libraries, in all libraries it uses same definition while in main it
uses local definition.
Is this correct thing to do if I want to get lazyloading+RTL (all
references bind to same definition). I cannot use -brtl as that
prevents lazyloading.
I also noticed that data section of one of my libraries was on 0X2
segment even though 0xF is not full and library permissions are 775.
Ideas?
5.Is heap segment on 32 bit AIX address space(0x3-0xB) available for
loading shared libraries if oxd,ox2 segments are full? If so, are the
text segments of shared libraries loaded in this heap space shared
among processes? If not, is it possible to use mmap to acheive this?
Can 0x2,0X3-OXB be used for loading both text and data segments of
shared libraries? Can I use -bmaxdata for changing this heap size?
6.Are segments reserved for shared memory services 0xC,0XE?
7.Whats the correct way to record runtime path. -L records runtime
path too. I noticed that if I recorded absolute paths with -L,
different versions of same library can be loaded (which are in
different paths) and so -L is safest thing to use in that sense make
sure my own local copy of library is used even if another process
already loaded a library with same name at different path.
8. As on SUN's is it good convention to use libname.so for shared
libraries or should it be put in a .a.
9. And finally with -blazy if I don't specify dependencies when build
a .so, I see a runtime error that library couldn't be loaded.Fatal:
could not load module libd.so()(errno 8), even though all dependencies
needed are recorded in main. So it a MUST to make libraries
self-contained for lazyloading to work.
Thanks,
Sunil.

sunil wrote:

> (Please neglect first post, see this post only)
> Hi All,
> I have some questions related to shared libraries on AIX:
> 1. Is they are any good way to know order in which shared libraries
> are loaded on AIX.

Depth first.

> On SUN, I can get this information using
> LD_DEBUG=files... I need this information because I am assuming that
> with runtime linking enabled, symbols are searched in libraries in
> order in which they are loaded.

Actually, no. With RTL, the breadth-first load order is computed
and used when searching for symbols.

> 2. When are shared libraries that are attached to 0xd segment
> removed?

They aren't. slibclean or a reboot is required. Removing the file
doesn't work, since the module is still "in use" by the system.

> Is this correct thing to do if I want to get lazyloading+RTL (all
> references bind to same definition). I cannot use -brtl as that
> prevents lazyloading.

I think you need to give this up. If RTL is defined as negating
lazy loading, then it does so for a reason. Your efforts
notwithstanding, there is no support for combining the two
techniques. Frankly, lazy loading only buys you something if
(1) your module does not export any data symbols, and (2) there
is some (significant) likelihood that the module will never get
loaded. Since you mention data symbols in your modules, lazy
loading is not going to happen for those modules, period.

> I also noticed that data section of one of my libraries was on 0X2
> segment even though 0xF is not full and library permissions are 775.
> Ideas?

There are rules governing where modules go. For details, read
http://www-106.ibm.com/developerwork...s/loader1.html


> 5.Is heap segment on 32 bit AIX address space(0x3-0xB) available for
> loading shared libraries if oxd,ox2 segments are full? If so, are the
> text segments of shared libraries loaded in this heap space shared
> among processes?

Read the above-referenced tutorial.

> If not, is it possible to use mmap to acheive this?

No. There is no support for dynamic loading via mmap/etc.

> Can 0x2,0X3-OXB be used for loading both text and data segments of
> shared libraries? Can I use -bmaxdata for changing this heap size?

The loader takes care of this for you. The -bmaxdata option governs
the total possible size of the heap.

> 6.Are segments reserved for shared memory services 0xC,0XE?

On AIX 5.2 there's more flexibility in how segments are used.
Read the current documentation on the [Very] Large Progam Support
Overview:
http://www16.boulder.ibm.com/pseries...htm#a179c11c5d

> 7. Whats the correct way to record runtime path. -L records runtime
> path too. I noticed that if I recorded absolute paths with -L,
> different versions of same library can be loaded (which are in
> different paths) and so -L is safest thing to use in that sense make
> sure my own local copy of library is used even if another process
> already loaded a library with same name at different path.

Not sure what the question is, but look up -blibpath:

> 8. As on SUN's is it good convention to use libname.so for shared
> libraries or should it be put in a .a.

Whatever you want. Current versions of the linker provide the
-blibsuff:<suffix> option so you can turn -lfoo into libfoo.<suffix>
with anything you like. With RTL, .so is built in.

Since AIX doesn't do .so versioning, we suggest .a, new names for
members, and using the F_LOADONLY flag in the file header (ldedit
on AIX 5.2) of older members that shouldn't be used for linking,
only for runtime. In any event, the filename doesn't matter;
only it's contents.

> 9. And finally with -blazy if I don't specify dependencies when build
> a .so, I see a runtime error that library couldn't be loaded.Fatal:
> could not load module libd.so()(errno 8), even though all dependencies
> needed are recorded in main.

Recorded, or actually exported? You may need to use -bexpall or
-bexpfull when building your main app, as the linker only automatically
exports symbols (with -brtl) that is _knows_ need to be exported.
If you don't include a dependent, the linker isn't going to know
what is required. Use dump -HTv to review the loader symbol table.

Lots of information in
http://www-106.ibm.com/developerwork...dfs/aix_ll.pdf
--
Gary R. Hook / AIX PartnerWorld for Developers / These opinions are MINE
__________________________________________________ ______________________

Thanks for reply.
"Gary R. Hook" <nospam@nospammers.net> wrote in message news:<8q8mc.5557$PB5.3226@newssvr24.news.prodigy.c om>...
> > 1. Is they are any good way to know order in which shared libraries
> > are loaded on AIX.
>
> Depth first.
I was asking more in terms of lazyloading, since library order then
is determined by execution path. It would be nice if there is some
tool/runtime linker flags to examine this.
> Actually, no. With RTL, the breadth-first load order is computed
> and used when searching for symbols.
With deps for main at root the tree?
> I think you need to give this up. If RTL is defined as negating
> lazy loading, then it does so for a reason.
There is no documentation anywhere which says that conceptually these
two cannot be combined. But as you mentioned, its not buying me much
so I am giving it up.
> There are rules governing where modules go. For details, read
> http://www-106.ibm.com/developerwork...s/loader1.html
> On AIX 5.2 there's more flexibility in how segments are used.
> Read the current documentation on the [Very] Large Progam Support
> Overview:
http://www16.boulder.ibm.com/pseries...htm#a179c11c5d
I read these links and I have following questions:
1. Why does document say that for Large Address model, we have only 2
segs left for shared mem services? We can set bmaxdata to maximum of
0xA (2 GB) and that leaves us with 1 GB (4 segs) after taking out a GB
for 0x1,0x2,0xD,0xF.
2. With VLModel do we have all addresses from 0x3-0xC,0xE with
-bmaxdata:0XD0000000? Now the most important question. I have shared
libs that eat around 2.8GB of VM space(data+text). So can I use VL
model? and all these shared libs will be loaded privately right? (in
0x3-0xC,0xE and part of 0x2) .Is there any way to make them shared
among processes.
3. Document says with VL model when we specify -bmaxdata>0xAFFFFFFF,
a program will not use globally shared libraries- disastrous So
0xD,0XF can be reused for loading my own process private shared libs?
> Recorded, or actually exported? You may need to use -bexpall or
> -bexpfull when building your main app, as the linker only automatically
I used that when building main and also all libraries. But i recorded
depenency only in main and not in library. In other words, libd.so
depends on libb.so. I I record -lb -ld only in main and don't record
anything when building d.so, w/o lazyloading and with RTL works fine.
But w lazyloading+RTL causes crash.
(All that follows is without lazyloading and with Run Time Linking)
I am assuming its not a must to record dependencies when I am building
libraries. What are implications of not recording dependencies? One
thing is it seems to be a good idea w.r.t where libraries are loaded
since I don't need to make sure I have space for a library and its
dependents in same segment - so this may cause better usage of memory.
Any bad effects?
Thanks,
Sunil.

sunil wrote:

> Thanks for reply.
> "Gary R. Hook" <nospam@nospammers.net> wrote in message news:<8q8mc.5557$PB5.3226@newssvr24.news.prodigy.c om>...
>
>>> 1. Is they are any good way to know order in which shared libraries
>>>are loaded on AIX.
>>
>>Depth first.
>
> I was asking more in terms of lazyloading, since library order then
> is determined by execution path. It would be nice if there is some
> tool/runtime linker flags to examine this.

DOesn't matter. The only thing that lazy loading does is defer
the loading of a module that the linker already knew about. There's
no symbol lookup involved, only the actual inclusion of a specific
module in the process. I suppose it would be worth my time to
gen-up an example to explore the combination of LL and RTL in
certain circumstances. Since there's no runtime determination
of which module to use, however, what you see in the dump -Tv
output is what you get.

>>Actually, no. With RTL, the breadth-first load order is computed
>>and used when searching for symbols.
>
> With deps for main at root the tree?

Yes. Start with the main app, dump -Hv, add each listed module
to the list, then for each of those, examine their dependents and
add them to the list, then for each of _those_ do the same, etc.
For modules loaded dynamically, if the named module is new, it's
added to the list, then it's dependents are examined in the same
fashion.

>>I think you need to give this up. If RTL is defined as negating
>>lazy loading, then it does so for a reason.
>
> There is no documentation anywhere which says that conceptually these
> two cannot be combined. But as you mentioned, its not buying me much
> so I am giving it up.

Again, I might should check this further. My comments are based
upon my memory of the events on 1996 when we wrote all of this
code.

>>There are rules governing where modules go. For details, read
>>http://www-106.ibm.com/developerwork...s/loader1.html
>>On AIX 5.2 there's more flexibility in how segments are used.
>>Read the current documentation on the [Very] Large Progam Support
>>Overview:
>
> http://www16.boulder.ibm.com/pseries...htm#a179c11c5d
> I read these links and I have following questions:
> 1. Why does document say that for Large Address model, we have only 2
> segs left for shared mem services? We can set bmaxdata to maximum of
> 0xA (2 GB) and that leaves us with 1 GB (4 segs) after taking out a GB
> for 0x1,0x2,0xD,0xF.

The maximum heap size is 8 segs, 0x3 - 0xA, for a normal maxdata
program. 0xD/0xF are shared lib segs. That leaves 0xB/C/E, although
clearly not contiguous. Seg 0 is the kernel, so it's unusable, too.
Seg 1 text, seg 2 stack. All accounted for.

> 2. With VLModel do we have all addresses from 0x3-0xC,0xE with
> -bmaxdata:0XD0000000? Now the most important question. I have shared
> libs that eat around 2.8GB of VM space(data+text). So can I use VL
> model? and all these shared libs will be loaded privately right? (in
> 0x3-0xC,0xE and part of 0x2) .Is there any way to make them shared
> among processes.

Um, if you specify a maxdata value larger than 2 GB, you lose the public
shared lib segs. The remaining segs are available to you.

The shared lib segs are fixed in size; if you exceed the aggregate 512
MB, your modules are loaded privately. If you use the VL model, all
modules are loaded privately. No sharing. No, you can't change this.

> 3. Document says with VL model when we specify -bmaxdata>0xAFFFFFFF,
> a program will not use globally shared libraries- disastrous So
> 0xD,0XF can be reused for loading my own process private shared libs?

Yes; see above.

>>Recorded, or actually exported? You may need to use -bexpall or
>>-bexpfull when building your main app, as the linker only automatically
>
> I used that when building main and also all libraries. But i recorded
> depenency only in main and not in library. In other words, libd.so
> depends on libb.so. I I record -lb -ld only in main and don't record
> anything when building d.so, w/o lazyloading and with RTL works fine.

Good; it should.

> But w lazyloading+RTL causes crash.

I think this is where the problem is, and what I need to test.

> (All that follows is without lazyloading and with Run Time Linking)
> I am assuming its not a must to record dependencies when I am building
> libraries. What are implications of not recording dependencies? One
> thing is it seems to be a good idea w.r.t where libraries are loaded
> since I don't need to make sure I have space for a library and its
> dependents in same segment - so this may cause better usage of memory.
> Any bad effects?

If you don't use LL, you just have to ensure that any RTL references
are resolved at exec or load time, i.e. by the time the module is
brought into the process. Symbol resolution happens at exec/load,
not at symbol reference. How you get those modules into the process
is up to you, but it's perfectly adequate (using RTL) to use main
alone to list all the required dependents.

--
Gary R. Hook / AIX PartnerWorld for Developers / These opinions are MINE
__________________________________________________ ______________________