SEO

Sorry if this gets through more than once, I seem to be having email
difficulties...

On Tue, 5 Sep 2006, Jeremy Drake wrote:

> I noticed when I was working on a patch quite a while back that there are
> no regression tests for large object support. I know, large objects
> are not the most sexy part of the code-base, and I think they tend to be
> ignored/forgotten most of the time. Which IMHO is all the more reason
> they should have some regression tests. Otherwise, if someone managed to
> break them somehow, it is quite likely not to be noticed for quite some
> time.
>
> So in this vein, I have recently found myself with some free time, and a
> desire to contribute something, and decided this would be the perfect
> place to get my feet wet without stepping on any toes.
>
> I guess what I should ask is, would a patch to add a test for large
> objects to the regression suite be well received? And, is there any
> advice for how to go about making these tests?
>
> I am considering, and I think that in order to get a real test of the
> large objects, I would need to load data into a large object which would
> be sufficient to be loaded into more than one block (large object blocks
> were 1 or 2K IIRC) so that the block boundary case could be tested. Is
> there any precedent on where to grab such a large chunk of data from? I
> was thinking about using an excerpt from a public domain text such as Moby
> Dick, but on second thought binary data may be better to test things with.
>
> My current efforts, and probably the preliminary portion of the final
> test, involves loading a small amount (less than one block) of text into a
> large object inline from a sql script and calling the various functions
> against it to verify that they do what they should. In the course of
> doing so, I find that it is necessary to stash certain values across
> statements (large object ids, large object 'handles'), and so far I am
> using a temporary table to store these. Is this reasonable, or is there a
> cleaner way to do that?
>
>

--
Never make anything simple and efficient when a way can be found to
make it complex and wonderful.

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Jeremy Drake <psql@jdrake.com> writes:
> I just tried using the \lo_import command in a regression test, and I
> think I figured out why this will not work:
> ...
> Yes, that's the large object OID in the output there, and it is different
> each run (as I expect).

Right. I'd suggest temporarily setting ECHO off to hide the
unpredictable part of the output. There are similar measures taken in
many of the contrib tests.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

On Sun, 24 Sep 2006, Jeremy Drake wrote:

> On Thu, 21 Sep 2006, Tom Lane wrote:
>
> > I suggest that instead of testing the server-side lo_import/lo_export
> > functions, perhaps you could test the psql equivalents and write and
> > read a file in psql's working directory.
<snip>
> In the mean time, I will alter the test to also test the psql backslash
> commands based on how the copy equivalents are tested, since I had
> forgotten them and they need to be tested also.

I just tried using the \lo_import command in a regression test, and I
think I figured out why this will not work:

$ make check
....
largeobject ... FAILED
....

$ cat regression.diffs
*** ./expected/largeobject.out Sun Sep 24 19:55:25 2006
--- ./results/largeobject.out Sun Sep 24 19:55:58 2006
***************
*** 188,194 ****
(1 row)

\lo_import 'results/lotest.txt'
! lo_import 31138
\set newloid :LASTOID
-- This is a hack to test that export/import are reversible
-- This uses knowledge about the inner workings of large object
mechanism
--- 188,194 ----
(1 row)

\lo_import 'results/lotest.txt'
! lo_import 31199
\set newloid :LASTOID
-- This is a hack to test that export/import are reversible
-- This uses knowledge about the inner workings of large object
mechanism

================================================== ====================


Yes, that's the large object OID in the output there, and it is different
each run (as I expect). If you look at src/bin/psql/large_obj.c line 192,
you see:

fprintf(pset.queryFout, "lo_import %u\n", loid);

Which is executed unconditionally whenever the lo_import is successful.
While in a normal circumstance, it is quite necessary to know the loid,
since it does change each call, in this case it serves to break the diffs,
and so I guess it is impossible to use the \lo_import command in a
regression test.


--
The first time, it's a KLUDGE!
The second, a trick.
Later, it's a well-established technique!
-- Mike Broido, Intermetrics

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

On Mon, 25 Sep 2006, Tom Lane wrote:

> Jeremy Drake <psql@jdrake.com> writes:
> > I just tried using the \lo_import command in a regression test, and I
> > think I figured out why this will not work:
> > ...
> > Yes, that's the large object OID in the output there, and it is different
> > each run (as I expect).
>
> Right. I'd suggest temporarily setting ECHO off to hide the
> unpredictable part of the output. There are similar measures taken in
> many of the contrib tests.

I tried this:
jeremyd=# \set QUIET
jeremyd=# \set ECHO off
jeremyd=# BEGIN;
jeremyd=# \lo_import results/lotest.txt
lo_import 84951
jeremyd=# ROLLBACK;

From what I could tell in the code, the message is printed regardless of
setting.

It looks like the large_obj.c output is missing much of the output
settings handling which is in the PrintQueryStatus function in common.c,
such as handling quiet mode, and html output. I will try to dig around
and try to put together a patch to make it respect the settings like other
commands...

--
"You are old," said the youth, "and your programs don't run,
And there isn't one language you like;
Yet of useful suggestions for help you have none --
Have you thought about taking a hike?"

"Since I never write programs," his father replied,
"Every language looks equally bad;
Yet the people keep paying to read all my books
And don't realize that they've been had."

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings