Hey guys,
I have a general question regarding syslog and its imposed usage on
the CPU/Mem/IO especially around scheduled cronjobs. We're currently
logging close to around 1.5-2gb of logs daily. One of the logs grows to
about 700-900mb daily on average. And, at least before we make the
changes, messages has duplicating the same entries (so
/var/log/messages 700-900mb+).
The concern we have is if any log entries might be dropping, or if
it's not even close to a possibility that they are dropping. One of my
coworkers expressed this concern, especially when crons are kicked off
- we have all the basic crons running daily: anacron, logrotate,
slocate, etc. Should we be concerned about the system not being able to
cache & buffer everything to the point where it starts dropping
information?
The hardware setup is pretty beefy as it is, so I wouldn't suspect
anything. But I wanted to make sure. My limited knowledge of the server
we're running:

Fedora Core 5 (2.6.15-1.2054_FC5smp)
3ghz Pentium 4 (I believe hyperthreading)
1gb RAM (DDR I'm suspecting - not sure of the bus speed)
Hardware Raid 5 setup over SATA (probably 4x 160gb)

Any ideas?
If this is sufficient enough for what we're doing now, is there a
'threshold' at which syslog, if ever, would drop its logging?

As of yet, I haven't noticed any major issues where there are gaps in
the logs. But I just wanted to find out if this is something to be
concerned about in the near-future as we expect our logging to
increase.

In comp.os.linux.setup jplee3@gmail.com:
> Hey guys,
> I have a general question regarding syslog and its imposed usage on
> the CPU/Mem/IO especially around scheduled cronjobs. We're currently
> logging close to around 1.5-2gb of logs daily. One of the logs grows to
> about 700-900mb daily on average. And, at least before we make the
> changes, messages has duplicating the same entries (so
> /var/log/messages 700-900mb+).
> The concern we have is if any log entries might be dropping, or if

Why would you guess? Your feelings have zero significance, at
least install/run 'sar' on the system and check how the system
behaves at times you suspect problems after running for a few
days. Simply rotate more often if there are problems or just
don't log that much crap.

[..]

> Fedora Core 5 (2.6.15-1.2054_FC5smp)

I'd strongly suggest to install patches, preferable trough yum!

> 3ghz Pentium 4 (I believe hyperthreading)

Why would you believe? 'cat /proc/cpuinfo' should tell.

> 1gb RAM (DDR I'm suspecting - not sure of the bus speed)

'dmidecode' might know and it should be somewhere in /proc or/and
/sys, check the kernel docs about the matter.

> Hardware Raid 5 setup over SATA (probably 4x 160gb)

Install/use the tool for your hw raid controller to check.

Good luck

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 81: Please excuse me, I have to circuit an AC
line through my head to get this database working.

Michael Heiming wrote:
> In comp.os.linux.setup jplee3@gmail.com:
> > Hey guys,
> > I have a general question regarding syslog and its imposed usage on
> > the CPU/Mem/IO especially around scheduled cronjobs. We're currently
> > logging close to around 1.5-2gb of logs daily. One of the logs grows to
> > about 700-900mb daily on average. And, at least before we make the
> > changes, messages has duplicating the same entries (so
> > /var/log/messages 700-900mb+).
> > The concern we have is if any log entries might be dropping, or if
>
> Why would you guess? Your feelings have zero significance, at
> least install/run 'sar' on the system and check how the system
> behaves at times you suspect problems after running for a few
> days. Simply rotate more often if there are problems or just
> don't log that much crap.
I don't think 'sar' is installed on the system. Not sure if they'll be
open to installing stuff on it - they basically don't want to touch the
server (reason explained below).
Regarding how much "crap" we're logging - that's out of my hands as
well. I can only suggest and hope that they log less as I continue to
filter out the 'unnecessary' log entries that don't really need to be
recorded.

>
> [..]
>
> > Fedora Core 5 (2.6.15-1.2054_FC5smp)
>
> I'd strongly suggest to install patches, preferable trough yum!
>
The disadvantage about where I work is that it [previously] was a 100%
Microsoft shop and the IT guys are extremely sensitive and paranoid
about touching almost *anything* on this system because they feel they
don't have the 'expertise'
I will recommend to them that they at least run yum, apt-get or smart
(or SOMETHING) to get things to where they should be.

> > 3ghz Pentium 4 (I believe hyperthreading)
>
> Why would you believe? 'cat /proc/cpuinfo' should tell.
Does /proc/cpuinfo explicitly say "hyperthreading" or "HT" anywhere? Or
is it supposed to?

For example (on my machine at home):
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) Pentium(R) 4 CPU 3.00GHz
stepping : 9
cpu MHz : 3006.884
cache size : 512 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid xtpr
bogomips : 5963.77

processor : 1
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) Pentium(R) 4 CPU 3.00GHz
stepping : 9
cpu MHz : 3006.884
cache size : 512 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid xtpr
bogomips : 5996.54

- this is definitely an HT machine, and it just shows that there are 2
processors. But it doesn't say "HT" or "hyperthreading" anywhere is
there something else that obviously points towards it being HT? Sorry
in advance if there is - I'm not as familiar with Unix hardware
diagnosis.


>
> > 1gb RAM (DDR I'm suspecting - not sure of the bus speed)
>
> 'dmidecode' might know and it should be somewhere in /proc or/and
> /sys, check the kernel docs about the matter.
>
Like sars, it doesn't seem like dmidecode is a utility that is
installed on this system. But I'll recommend that these two programs be
installed.

> > Hardware Raid 5 setup over SATA (probably 4x 160gb)
>
> Install/use the tool for your hw raid controller to check.
>
Since the IT guys are paranoid about EVERYTHING, the system is pretty
much locked down. I don't have root access and the server is locked
away in a small server room that I've only been in once so far.

> Good luck
>
Thanks - this is definitely a work-in-progress... I can only hope
that they'll open up the access at least a little more. I'm pretty new
to the company, so I suppose it will take some time.

> --
> Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
> mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
> #bofh excuse 81: Please excuse me, I have to circuit an AC
> line through my head to get this database working.

In comp.os.linux.setup jplee3@gmail.com:

> Michael Heiming wrote:
>> In comp.os.linux.setup jplee3@gmail.com:

[ use sar to check systems load/possible performance problems ]

> I don't think 'sar' is installed on the system. Not sure if they'll be
> open to installing stuff on it - they basically don't want to touch the
> server (reason explained below).

Thinking will not answer the question to you, use the tools on
your system to check if it is installed (as rpm) 'rpm -q
sysstat' (sar comes with the sysstat package on RH).

[..]

>> > Fedora Core 5 (2.6.15-1.2054_FC5smp)

>> I'd strongly suggest to install patches, preferable trough yum!

> The disadvantage about where I work is that it [previously] was a 100%
> Microsoft shop and the IT guys are extremely sensitive and paranoid
> about touching almost *anything* on this system because they feel they
> don't have the 'expertise'

They shouldn't be in front of a production server with this
attitude. As you can see from running Fedora Core on a production
server, you want some enterprise distro or clone on such a
system. Alone the long term availability of patches and support
demand it!

[..]

>> > 3ghz Pentium 4 (I believe hyperthreading)

>> Why would you believe? 'cat /proc/cpuinfo' should tell.

> Does /proc/cpuinfo explicitly say "hyperthreading" or "HT" anywhere? Or
> is it supposed to?

> For example (on my machine at home):
> physical id : 0

> physical id : 0
[..]

> - this is definitely an HT machine, and it just shows that there are 2
> processors. But it doesn't say "HT" or "hyperthreading" anywhere is
> there something else that obviously points towards it being HT? Sorry
> in advance if there is - I'm not as familiar with Unix hardware
> diagnosis.

It does.

>> > 1gb RAM (DDR I'm suspecting - not sure of the bus speed)

>> 'dmidecode' might know and it should be somewhere in /proc or/and
>> /sys, check the kernel docs about the matter.

> Like sars, it doesn't seem like dmidecode is a utility that is
> installed on this system. But I'll recommend that these two programs be
> installed.

Should come with the 'kernel-utils' package, a must have on a server.

>> > Hardware Raid 5 setup over SATA (probably 4x 160gb)

>> Install/use the tool for your hw raid controller to check.

> Since the IT guys are paranoid about EVERYTHING, the system is pretty
> much locked down. I don't have root access and the server is locked
> away in a small server room that I've only been in once so far.

How would you ever get aware of a broken disk if you don't run
some script using the tool in question from cron that informs you
about the problem? How would you be able to check if an exchanged
disk syncs cleanly to enable redundancy again?

RAID doesn't make any sense if you just run until it breaks. The
summed up cluelessness is really outstanding.

Good luck

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 205: Quantum dynamics are affecting the transistors

Michael Heiming wrote:
> In comp.os.linux.setup jplee3@gmail.com:
>
> > Michael Heiming wrote:
> >> In comp.os.linux.setup jplee3@gmail.com:
>
> [ use sar to check systems load/possible performance problems ]
>
> > I don't think 'sar' is installed on the system. Not sure if they'll be
> > open to installing stuff on it - they basically don't want to touch the
> > server (reason explained below).
>
> Thinking will not answer the question to you, use the tools on
> your system to check if it is installed (as rpm) 'rpm -q
> sysstat' (sar comes with the sysstat package on RH).
>
Well, I wasn't here over the weekend [when I first read your response],
which is why I said "I think," but about 1 minute before writing this
sentence [as I'm in the office now], I checked and the util is not
installed.

> [..]
>
> >> > Fedora Core 5 (2.6.15-1.2054_FC5smp)
>
> >> I'd strongly suggest to install patches, preferable trough yum!
>
> > The disadvantage about where I work is that it [previously] was a 100%
> > Microsoft shop and the IT guys are extremely sensitive and paranoid
> > about touching almost *anything* on this system because they feel they
> > don't have the 'expertise'
>
> They shouldn't be in front of a production server with this
> attitude. As you can see from running Fedora Core on a production
> server, you want some enterprise distro or clone on such a
> system. Alone the long term availability of patches and support
> demand it!
>
Well, they initially had RHEL3 installed and for some reason decided to
go with FC5 - I'm not sure what they were thinking. The IT dept here is
growing, and not many people know a lot in terms of Unix (myself
included). So as I said, this is a work-in-progress; a huge one.

> [..]
>
> >> > 3ghz Pentium 4 (I believe hyperthreading)
>
> >> Why would you believe? 'cat /proc/cpuinfo' should tell.
>
> > Does /proc/cpuinfo explicitly say "hyperthreading" or "HT" anywhere? Or
> > is it supposed to?
>
> > For example (on my machine at home):
> > physical id : 0
>
> > physical id : 0
> [..]
>
> > - this is definitely an HT machine, and it just shows that there are 2
> > processors. But it doesn't say "HT" or "hyperthreading" anywhere is
> > there something else that obviously points towards it being HT? Sorry
> > in advance if there is - I'm not as familiar with Unix hardware
> > diagnosis.
>
> It does.
>
> >> > 1gb RAM (DDR I'm suspecting - not sure of the bus speed)
>
> >> 'dmidecode' might know and it should be somewhere in /proc or/and
> >> /sys, check the kernel docs about the matter.
>
> > Like sars, it doesn't seem like dmidecode is a utility that is
> > installed on this system. But I'll recommend that these two programs be
> > installed.
>
> Should come with the 'kernel-utils' package, a must have on a server.
>
They do have this but I can't run it as they've only given permissions
to the root user on /dev/mem

> >> > Hardware Raid 5 setup over SATA (probably 4x 160gb)
>
> >> Install/use the tool for your hw raid controller to check.
>
> > Since the IT guys are paranoid about EVERYTHING, the system is pretty
> > much locked down. I don't have root access and the server is locked
> > away in a small server room that I've only been in once so far.
>
> How would you ever get aware of a broken disk if you don't run
> some script using the tool in question from cron that informs you
> about the problem? How would you be able to check if an exchanged
> disk syncs cleanly to enable redundancy again?
>
> RAID doesn't make any sense if you just run until it breaks. The
> summed up cluelessness is really outstanding.
>
Sorry, I'm fairly new to Unix/Linux administration. So obviously, there
are many things I don't know, and many things I need to learn. I guess
this is part of the process...

> Good luck
>
> --
> Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
> mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
> #bofh excuse 205: Quantum dynamics are affecting the transistors

In comp.os.linux.setup jplee3@gmail.com:
> Michael Heiming wrote:
>> In comp.os.linux.setup jplee3@gmail.com:
>> > Michael Heiming wrote:
>> >> In comp.os.linux.setup jplee3@gmail.com:

[ use sar to check systems load/possible performance problems ]

>> >> 'dmidecode' might know and it should be somewhere in /proc or/and
>> >> /sys, check the kernel docs about the matter.

>> > Like sars, it doesn't seem like dmidecode is a utility that
>> > is installed on this system. But I'll recommend that these
>> > two programs be installed.

>> Should come with the 'kernel-utils' package, a must have on a server.

> They do have this but I can't run it as they've only given
> permissions to the root user on /dev/mem

Why do you care about the problems at all if you don't have root
permissions? It doesn't seem you job at all?

[..]

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 149: Dew on the telephone lines.