SEO

I have a interesting problem that has been causing me headaches for
many months now, and unfortunately, even patching right up to the
current recommended cluster does not resolve things (as of May 2005).

General symptoms:
* After (generally) several weeks of uptime, a Solaris 8 machine will
gradually start intermittently refusing network connections (both tcp
and udp), culminating eventually in a complete system halt (not even
accessable from the console). A power-cycle is required to bring things
back to normal.

Causitive factors:
* An application that pulls back binary log data from the Solaris audit
subsystem (by using the auditsvc() system call, attached to a pipe),
passes it through praudit to convert log data to text format, and then
bounces it to a remote syslog server via UDP.

As mentioned in the title, the problem doesn't occur on Solaris
5/6/7/9/10, and only appears after several million audit events are
generated (and therefore, several million UDP packets are sent).
The application that pulls back the data from auditsvc() is not hogging
significant quantities of memory, and has no noticable leaks, or CPU
usage overheads. (Even if it did, it's a user-space program, and
shouldn't be able to do anything nasty to kernel components like the
network subsystem anyway..)

I also know that the application WAS working on Solaris 8, prior to a
particular patch release being applied (not certain which one,
unfortunately).

At this stage, I have a feeling that there may be a memory leak, or
buffer overflow, somewhere in the Solaris 8 network layer, when faced
with sending a large amount of data over UDP. I'm not certain whether
there's also a tie-in with the audit subsystem also - but
unfortunately, the problem shows up so slowly (but still predictably),
that it's hard to pin down much more information at this stage.

Has anyone else noticed problems of this sort on solaris 8, that may or
may not be related to UDP transmission? Do you remember which patch /
cluster broke things?

Many thanks,

Leigh.