SEO

We have an application that runs the AIX command "sync". When I do a ps
-ef there can be as many as eighteen syncs runnning at any one time. In
some cases they can take over 60 seconds to run. We have the syncd
running every 60 seconds which is started in the normal way on bootup.
The developers say that they need to run sync to make sure the data is
written to disk before the next part of the process is executed.
Here is an example
/ ->vmstat 10 2; ps -ef | grep sync | egrep -v "grep|syncd"
System Configuration: lcpu=4 mem=12288MB
kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
16 5 2062629 999 0 8 11 97 280 0 2865 82664 4252 21 40 29
10
13 2 2060066 3185 0 27 5 21 46 0 2643 60088 3216 31 46 13
10
eod 577596 9531582 0 15:14:22 - 0:00 sync
eod 1601690 13017324 0 15:14:25 - 0:00 sync
eod 2302016 6500598 0 15:14:15 - 0:00 [sync]
eod 2801908 1278180 0 15:14:27 - 0:00 sync
eod 8077322 13471800 0 15:14:20 - 0:00 sync
eod 10608760 12238876 0 15:14:16 - 0:00 [sync]
eod 13254798 20996098 0 15:14:16 - 0:00 [sync]
lm223 13770950 9249014 0 15:14:19 pts/1042 0:00 [sync]
eod 21049420 1089594 0 15:14:28 - 0:00 sync
eod 23068802 23552058 0 15:14:19 - 0:00 [sync]
eod 24158244 2125826 0 15:14:17 - 0:00 [sync]
/ ->
As you can see sys is higher than usr and it always is when there are
several syncs running. The r is 13 and the box has 4 processors. pi and
po quite often reach over 100. The box was not sized for as many users
as it has now ie the user population has grown. I think the box has
more than one performance bottleneck however it is the sync issue that
I am posting this message aboutl.


I have pointed the following quote from the ibm documentation of the
sync command to the developers

"Note: The writing, although scheduled, is not necessarily complete
upon return from the sync subroutine."

The code is old and regarded as legacy. I feel the sync's may have at
one time been necessary on, maybe tyranosaurus-unix but on AIX 5.2 it
is un-necessary and misguided as the sync may not have completed before
control is passed back to the calling code anyway.

I have requested that they look into removing the sync from their code.


What do you guys think? Particularly about having as many as 18 syncs
running at one time.


Cheers
Bozo

bozo wrote:
> We have an application that runs the AIX command "sync". When I do a ps
> -ef there can be as many as eighteen syncs runnning at any one time. In
> some cases they can take over 60 seconds to run. We have the syncd
> running every 60 seconds which is started in the normal way on bootup.
> The developers say that they need to run sync to make sure the data is
> written to disk before the next part of the process is executed.

Ignorance is a terrible thing. I wonder how much ignorance earns
annually on the open market?


If data being written to a file needs to be committed before moving
on, the fsync() function will be much more lightweight. Not to
mention appropriate. Calling out to the sync command to hose up
the entire system is extremely inefficient, especially when it's
clear that much *much* effort is being duplicated.

Worst case: change the sync daemon to perform updates more often
(every N seconds) and change the code to sleep( N ) whenever it
needs to pause while waiting on the disk.

If performance and disk commits are that critical, they should be
opening and writing to a raw device, not going through the file
system layers.

Thanks for the reply, we had 80 syncs running earlier today the box is
slowly seizing up and the application provider still say the problem is
the operating system. How many times have I encountered this attitude
in my career? Too many to count but I know what to do, nail them with
hard evidence and back them into a corner.
Thanks again for the sanity check Gary.
Bozo


Gary R. Hook wrote:
> bozo wrote:
> > We have an application that runs the AIX command "sync". When I do a ps
> > -ef there can be as many as eighteen syncs runnning at any one time. In
> > some cases they can take over 60 seconds to run. We have the syncd
> > running every 60 seconds which is started in the normal way on bootup.
> > The developers say that they need to run sync to make sure the data is
> > written to disk before the next part of the process is executed.
>
> Ignorance is a terrible thing. I wonder how much ignorance earns
> annually on the open market?
>
>
> If data being written to a file needs to be committed before moving
> on, the fsync() function will be much more lightweight. Not to
> mention appropriate. Calling out to the sync command to hose up
> the entire system is extremely inefficient, especially when it's
> clear that much *much* effort is being duplicated.
>
> Worst case: change the sync daemon to perform updates more often
> (every N seconds) and change the code to sleep( N ) whenever it
> needs to pause while waiting on the disk.
>
> If performance and disk commits are that critical, they should be
> opening and writing to a raw device, not going through the file
> system layers.