Unix Technical Forum

Newbie Memory Mapped IO Question

This is a discussion on Newbie Memory Mapped IO Question within the AIX Operating System forums, part of the Unix Operating Systems category; --> Hi All, I've written an ETL tool and I have recently introduced memory mapped IO on 'unix' to reduce ...


Go Back   Unix Technical Forum > Unix Operating Systems > AIX Operating System

Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 01-05-2008, 07:06 AM
Peter Nolan
 
Posts: n/a
Default Newbie Memory Mapped IO Question

Hi All,
I've written an ETL tool and I have recently introduced memory mapped
IO on 'unix' to reduce memory usage and to make it extremely scalable.

The advice I got for solaris was to put the memory mapped io files into
/tmp. However, I have an AIX client trying out the software and they
tell me they want to keep the size of /tmp to a minimum and that my
memory mapped io files should be written elsewhere as defined by a
parameter passed to the tool.

However, the Solaris guys tell me /tmp actually means something to the
OS to say that all files in /tmp are memory mapped files and high
access rate files.

So, I was wondering.....does /tmp have any special meaning to AIX?
Should memory mapped io files go into /tmp or are the AIX guys at my
client correct in saying /tmp does not actually mean anything about the
files themselves to the OS?

I have tested the software pointing the memory mapped files to just
another file system and it seems to work fine and is amazingly fast so
I am sure that the files remain in memory even though they are not in
/tmp...

However, I am something of a novice at AIX as an operating system. So I
was just wondering....

Thanks

Peter Nolan
www.peternolan.com

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 01-05-2008, 07:06 AM
Mark Taylor
 
Posts: n/a
Default Re: Newbie Memory Mapped IO Question

AKAIK /tmp holds no special meaning for memory mapped files in AIX.
Siebel on AIX memory maps files from the /siebel bespoke filesystem, so
theres part of your answer.

Also when I done the AIX Kernel Internals course, there is no mention
of /tmp being special .. its just a filesystem on an LV ..

HTH
Mark Taylor

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 01-05-2008, 07:06 AM
Mark Taylor
 
Posts: n/a
Default Re: Newbie Memory Mapped IO Question

AFAIK /tmp holds no special meaning for memory mapped files in AIX.
Siebel on AIX memory maps files from the /siebel bespoke filesystem, so
theres part of your answer.

Also when I done the AIX Kernel Internals course, there is no mention
of /tmp being special .. its just a filesystem on an LV ..

HTH
Mark Taylor

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 01-05-2008, 07:06 AM
Michael Kraemer
 
Posts: n/a
Default Re: Newbie Memory Mapped IO Question

In article <1122884684.107079.181440@g49g2000cwa.googlegroups .com>, "Peter
Nolan" <peter@peternolan.com> writes:
> The advice I got for solaris was to put the memory mapped io files into
> /tmp. However, I have an AIX client trying out the software and they
> tell me they want to keep the size of /tmp to a minimum and that my
> memory mapped io files should be written elsewhere as defined by a
> parameter passed to the tool.
>
> However, the Solaris guys tell me /tmp actually means something to the
> OS to say that all files in /tmp are memory mapped files and high
> access rate files.
>
> So, I was wondering.....does /tmp have any special meaning to AIX?


not that I know of, except that you might be in trouble if it's full :-)
It might play a role that usually /tmp is on a local disk,
as opposed to other filesystems which might be nfs-mounted.
Thus heavy mmap()'ing might cause heavy network traffic in some cases.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5 (permalink)  
Old 01-05-2008, 07:06 AM
Adrian Bridgett
 
Posts: n/a
Default Re: Newbie Memory Mapped IO Question

normally on solaris /tmp is a ramdisk and hence very fast. On AIX it's
normally just a normal FS. I wonder if that is what they mean to say?

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6 (permalink)  
Old 01-05-2008, 07:06 AM
Simon Marchese
 
Posts: n/a
Default Re: Newbie Memory Mapped IO Question

Adrian Bridgett wrote:
> normally on solaris /tmp is a ramdisk and hence very fast. On AIX it's
> normally just a normal FS. I wonder if that is what they mean to say?
>


Reckon so. In particular, note that because it is not backed by disk, on
Solaris /tmp is wiped at reboot. So if you want you data to be
persistent, /tmp is not the place to put it on Solaris.

AIX does support ram-only filesystems. Look at
http://www.tek-tips.com/faqs.cfm?fid=475.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #7 (permalink)  
Old 01-05-2008, 07:07 AM
Kamal R. Prasad
 
Posts: n/a
Default Re: Newbie Memory Mapped IO Question

"However, the Solaris guys tell me /tmp actually means something to the

OS to say that all files in /tmp are memory mapped files and high
access rate files. "

For UNIC systems, /tmp indicates temporary storage that is not
guaranteed to persist after a reboot. Some unix'es clean up /tmp -but
AIX doesn't.
The default setting in AIX is to create a small disk-based filesystem
for /tmp, and the rest of the directories are hosted by larger
disk-based filesystems. Off hand, I can say that /tmp/probably doesn't
get journalled. if you are using jfs2.
If your customer wants the files in another directory and you want to
improve performance of that filesystem, you can change attributes of
the filesystem using chfs command.

regards
-kamal

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #8 (permalink)  
Old 01-05-2008, 07:07 AM
Peter Nolan
 
Posts: n/a
Default Re: Newbie Memory Mapped IO Question

Hi All,
thanks for the feedback....much appreciated....

The files I am talking about are deleted at the end of the batch run
and if the machine is rebooted it will re-create the files
anyway......hopefully it will not be re-booted in the middle of our
processing!! (LOL)...

Thanks for the tips on RAMDISK....maybe that is it with Solaris....all
I was told from my solaris expert was 'put the files in /tmp and it
will go fast'...so I did (LOL)...my unix knowledge is relatively
limited...

We've been running the software on AIX for the last 5 days now and it
runs very well......we might try out the ramdisk and see if it goes any
faster!!

Best Regards

Peter
www.peternolan.com

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #9 (permalink)  
Old 01-05-2008, 07:19 AM
Jose Pina Coelho
 
Posts: n/a
Default Re: Newbie Memory Mapped IO Question

"Kamal R. Prasad" <kamalp@acm.org> wrote in news:1122967993.413203.230090
@g43g2000cwa.googlegroups.com:

> "However, the Solaris guys tell me /tmp actually means something to the
>
> OS to say that all files in /tmp are memory mapped files and high
> access rate files. "
>
> For UNIC systems, /tmp indicates temporary storage that is not
> guaranteed to persist after a reboot. Some unix'es clean up /tmp -but
> AIX doesn't.

Solaris usualy mounts /tmp as a ramfs, hence it's lost on reboot.

Some unices clean it on boot, others don't.

AIX has a nice utility to do cleanups called skulker, it cleans any files
that haven't been modified over 24 hours.


> The default setting in AIX is to create a small disk-based filesystem
> for /tmp, and the rest of the directories are hosted by larger
> disk-based filesystems. Off hand, I can say that /tmp/probably doesn't
> get journalled.

All AIX filesystems are journaled by default (and you really want to keep
it that way).

--
Doing AIX support was the most monty-pythonesque
activity available at the time.
Eagerly awaiting my thin chocolat mint.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #10 (permalink)  
Old 01-05-2008, 07:19 AM
Jose Pina Coelho
 
Posts: n/a
Default Re: Newbie Memory Mapped IO Question

"Peter Nolan" <peter@peternolan.com> wrote in news:1123081888.811265.139580
@g14g2000cwa.googlegroups.com:

> We've been running the software on AIX for the last 5 days now and it
> runs very well......we might try out the ramdisk and see if it goes any
> faster!!

Don't forget that the ramdisk will use up memory, so increase the swapspace
by the same amount as the total size of the ramdisks.


--
Doing AIX support was the most monty-pythonesque
activity available at the time.
Eagerly awaiting my thin chocolat mint.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 05:16 AM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com