SEO

> Not to beat a dead horse, but do we really want to force folks to be
> parsing logs for performance monitoring? Especially if that log parsing
> is just going to result in data being inserted into a table anyway?
>
> I know there's concern about performance of the stats system and maybe
> that needs to be addressed, but pushing users to log parsing is a lot of
> extra effort, non-standard, likely to be overlooked, and doesn't play
> well with other tools. It also conflicts with all the existing
> statistics framework.


One thing that doesn't seemed to be being looked at it is the cost of
logging. Logging is very expensive. I don't know if it is more expensive
than the stats system, but you can cut your tps in half by having any
level of verbose logging on.

Yes that can be offset by pushing the logging to another spindle, and
being careful about what you are logging but still.

Either way, we are taking the hit, it is just a matter of where. IMO it
would be better to have the information in the database where it makes
sense, than pushing out to a log that:

A. Will likely be forgotten
B. Is only accessible if you have shell access to the machine (not as
common as all of us would like to think)

Sincerely,

Joshua D. Drake


--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/


---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

On Sat, 12 May 2007, Joshua D. Drake wrote:

> One thing that doesn't seemed to be being looked at it is the cost of
> logging.

If any of this executed at something like the query level, sure, that
would be real important. The majority of the logging I suggested here is
of things that happen at checkpoint time. The presumption is that the
overhead of that is considerably greater than writing a log line or even
five.

The least intensive event I would like to be loggable is when a new WAL
segment is created and cleared, which is again a pretty small bit of log
compared to the 16MB write. I wouldn't mind seeing that exposed under
pg_stats instead, just had more interesting things to statify first.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

On Sat, 2007-12-05 at 14:26 -0700, Joshua D. Drake wrote:
> Either way, we are taking the hit, it is just a matter of where. IMO it
> would be better to have the information in the database where it makes
> sense, than pushing out to a log

If performance monitoring information is provided as a database object,
what would the right interface be? IMHO the problem with cleanly
presenting monitoring information within a normal database system is
that this sort of data is fundamentally dynamic and continuous: to
determine how the performance of the system changes over time, you need
to repeatedly rescan the table/view/SRF and recompute your analysis
essentially from scratch. Trying to get even simple information like
"queries per second" from pg_stat_activity is an example of how this can
be painful.

<plug>
BTW, if the system included the concept of a continuous data *stream* as
a kind of database object, this problem would be much more tractable
In fact, there is some code in a version of TelegraphCQ that exposes
various information about the runtime state of the system as a set of
system-defined data streams -- like any other stream, users could then
use those streams in arbitrary queries.
</plug>

-Neil



---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend