SEO

Hi!

Now we (Oleg and me) are working on moving tsearch into core.

Pls, review suggested syntax. Comments, suggestions, objections will be appreciated.

1) parser operation (pg_ts_parser table)
CREATE PARSER prsname (
START = funcname,
GETTOKEN = funcname,
END = funcname,
LEXTYPES = funcname
[ , HEADLINE = funcname ]
);

DROP PARSER [IF EXISTS] prsname [ CASCADE | RESTRICT ];
ALTER PARSER prsname RENAME TO newprsname;
COMMENT ON PARSER IS text;

2) dictionaries (pg_ts_dict)

CREATE DICTIONARY dictname (
INIT = funcname,
LEXIZE = funcname,
OPT = text,
);

--create new dictionary as already existed but with different
-- options for example
CREATE DICTIONARY dictname [(
[ INIT = funcname ]
[ , LEXIZE = funcname ]
[ , OPT = text ]
)] LIKE template_dictname;

DROP DICTINARY [IF EXISTS] dictname [ CASCADE | RESTRICT ];
ALTER DICTIONARY dictname RENAME TO newdictname;
ALTER DICTIONARY dictname SET OPT=text;
COMMENT ON DICTIONARY IS text;

3) configuration (pg_ts_cfg [,pg_ts_cfgmap])
CREATE TSEARCH CONFIGURATION cfgname (
PARSER = prsname
[, LOCALE = localename]
);

--create new configuration and optionally copies
--map of lexeme's type to dictionaries
CREATE TSEARCH CONFIGURATION cfgname [(
LOCALE = localename
)] LIKE template_cfg [WITH MAP];

DROP TSEARCH CONFIGURATION [IF EXISTS] cfgname [ CASCADE | RESTRICT ];
ALTER TSEARCH CONFIGURATION cfgname RENAME TO newcfgname;
ALTER TSEARCH CONFIGURATION cfgname SET LOCALE=localename;
ALTER TSEARCH CONFIGURATION cfgname SET PARSER=prsname;
COMMENT ON TSEARCH CONFIGURATION IS text;

4) operate mapping lexemes to list of dictionary
CREATE TSEARCH MAPPING ON cfgname FOR lexemetypename USE dictname1[, dictname2
[..] ];
DROP TSEARCH MAPPING [IF EXISTS] ON cfgname FOR lexemetypename;
ALTER TSEARCH MAPPING ON cfgname FOR lexemetypename USE dictname1[, dictname2
[..] ];


Next, tsearch's configuration will be readable by psql backslashed command (F
means fulltext):
\dF - list of configurations
\dF PATTERN - describe configuration with used parser and lexeme's mapping
\dFd - list of dictionaries
\dFd PATTERN - describe dictionary
\dFp - parser's list
\dFp PATETRN - describe parser



--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/


--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Teodor Sigaev <teodor@sigaev.ru> writes:
> Now we (Oleg and me) are working on moving tsearch into core.
> Pls, review suggested syntax. Comments, suggestions, objections will be appreciated.

Is it really necessary to invent a batch of special-purpose commands?
Seems like this will add some thousands of lines of code and no actual
new functionality; not to mention loss of backwards compatibility for
existing tsearch2 users.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Hmm, IMHO, it's needed for consistent interface: nobody adds new column to table
by editing pg_class & pg_attribute, nobody looks for description of table by
selection values from system table.


Tom Lane wrote:
> Teodor Sigaev <teodor@sigaev.ru> writes:
>> Now we (Oleg and me) are working on moving tsearch into core.
>> Pls, review suggested syntax. Comments, suggestions, objections will be appreciated.
>
> Is it really necessary to invent a batch of special-purpose commands?
> Seems like this will add some thousands of lines of code and no actual
> new functionality; not to mention loss of backwards compatibility for
> existing tsearch2 users.
>
> regards, tom lane

--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Teodor Sigaev wrote:
> Hmm, IMHO, it's needed for consistent interface: nobody adds new
> column to table by editing pg_class & pg_attribute, nobody looks for
> description of table by selection values from system table.
>
>
> Tom Lane wrote:
>> Teodor Sigaev <teodor@sigaev.ru> writes:
>>> Now we (Oleg and me) are working on moving tsearch into core.
>>> Pls, review suggested syntax. Comments, suggestions, objections will
>>> be appreciated.
>>
>> Is it really necessary to invent a batch of special-purpose commands?
>> Seems like this will add some thousands of lines of code and no actual
>> new functionality; not to mention loss of backwards compatibility for
>> existing tsearch2 users.
>>
>>
>

Thousands of lines seems a high estimate, but maybe I'm wrong. I guess
an alternative would be to do this in some builtin functions, but that
seems a tad unclean.

I am also a bit concerned that the names of the proposed objects
(parser, dictionary) don't convey their purpose adequately. Maybe
TS_DICTIONARY and TS_PARSER might be better if we in fact need to name them.

cheers

andrew


---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

On Fri, 17 Nov 2006, Andrew Dunstan wrote:

> Teodor Sigaev wrote:
>> Hmm, IMHO, it's needed for consistent interface: nobody adds new column to
>> table by editing pg_class & pg_attribute, nobody looks for description of
>> table by selection values from system table.
>>
>>
>> Tom Lane wrote:
>>> Teodor Sigaev <teodor@sigaev.ru> writes:
>>>> Now we (Oleg and me) are working on moving tsearch into core.
>>>> Pls, review suggested syntax. Comments, suggestions, objections will be
>>>> appreciated.
>>>
>>> Is it really necessary to invent a batch of special-purpose commands?
>>> Seems like this will add some thousands of lines of code and no actual
>>> new functionality; not to mention loss of backwards compatibility for
>>> existing tsearch2 users.
>>>
>>>
>>
>
> Thousands of lines seems a high estimate, but maybe I'm wrong. I guess an
> alternative would be to do this in some builtin functions, but that seems a
> tad unclean.

As Teodor already wrote we want to be consistent with the current interface to
system catalog, as long as full text search is going to the pg core.
We don't invent anything new, we just extending current user's interface
to support full text search.

>
> I am also a bit concerned that the names of the proposed objects (parser,
> dictionary) don't convey their purpose adequately. Maybe TS_DICTIONARY and
> TS_PARSER might be better if we in fact need to name them.

this looks reasonable to me.

>
> cheers
>
> andrew
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly
>

Regards,
Oleg
__________________________________________________ ___________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Oleg Bartunov wrote:
> On Fri, 17 Nov 2006, Andrew Dunstan wrote:

> >I am also a bit concerned that the names of the proposed objects (parser,
> >dictionary) don't convey their purpose adequately. Maybe TS_DICTIONARY and
> >TS_PARSER might be better if we in fact need to name them.
>
> this looks reasonable to me.

Huh, but we don't use keywords with ugly abbreviations and underscores.
How about "FULLTEXT DICTIONARY" and "FULLTEXT PARSER"? (Using
"FULLTEXT" instead of "FULL TEXT" means you don't created common
reserved words, and furthermore you don't collide with an existing type
name.)

I also think the "thousands of lines" is an exaggeration :-) The
grammar should take a couple dozen at most. The rest of the code would
go to their own files.

We should also take the opportunity to discuss new keywords for the XML
support -- will we use new grammar, or functions?

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Alvaro Herrera wrote:
> We should also take the opportunity to discuss new keywords for the
> XML support -- will we use new grammar, or functions?

The XML stuff is defined in the SQL standard and there are existing
implementations, so any nonstandard syntax is going to be significantly
less useful. (The other problem is that you can't implement most of
the stuff in functions anyway.)

I don't see any comparable arguments about this full-text search stuff.
In particular I don't see any arguments why a change would necessary at
all, including why moving to core would be necessary in the first
place.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Alvaro Herrera wrote:
> Oleg Bartunov wrote:
>
>> On Fri, 17 Nov 2006, Andrew Dunstan wrote:
>>
>
>
>>> I am also a bit concerned that the names of the proposed objects (parser,
>>> dictionary) don't convey their purpose adequately. Maybe TS_DICTIONARY and
>>> TS_PARSER might be better if we in fact need to name them.
>>>
>> this looks reasonable to me.
>>
>
> Huh, but we don't use keywords with ugly abbreviations and underscores.
> How about "FULLTEXT DICTIONARY" and "FULLTEXT PARSER"? (Using
> "FULLTEXT" instead of "FULL TEXT" means you don't created common
> reserved words, and furthermore you don't collide with an existing type
> name.)
>

good point. this works for me.

>
> We should also take the opportunity to discuss new keywords for the XML
> support -- will we use new grammar, or functions?
>
>

Well, it will have to be keywords if we want to be able to do anything
like the spec, IIRC.

cheers

andrew


---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Peter Eisentraut <peter_e@gmx.net> writes:
> I don't see any comparable arguments about this full-text search stuff.
> In particular I don't see any arguments why a change would necessary at
> all, including why moving to core would be necessary in the first
> place.

AFAIR the only argument in favor of that is basically a marketing one:
users perceive a feature as more real, or more supported, if it's in
core. I don't find this argument especially compelling myself.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

On Fri, 17 Nov 2006, Tom Lane wrote:

> Peter Eisentraut <peter_e@gmx.net> writes:
> > I don't see any comparable arguments about this full-text search stuff.
> > In particular I don't see any arguments why a change would necessary at
> > all, including why moving to core would be necessary in the first
> > place.
>
> AFAIR the only argument in favor of that is basically a marketing one:
> users perceive a feature as more real, or more supported, if it's in
> core. I don't find this argument especially compelling myself.

I am currently in the position that my hosting provider is apprehensive
about installing modules in contrib because they believe they are less
secure. They cited (real or imagined) "security holes" as the reason they
would not install tsearch2, or any other contrib module. This leaves me
without any fulltext indexing option, as it requires a superuser to
install. I have currently worked around this by running my own postgres
instance from my home directory, as they provide shell access and allow
running background processes, but I was really happy when I heard that
tsearch2 was going to be integrated into core in 8.3.

I think I would settle for some sort of assurance somewhere by someone who
sounds authoritative that the contrib modules are not less secure than
postgres core, and are fully supported by the developers. I think if I
could point them at that, I may be able to convince them that it is safe.

>
> regards, tom lane
>


---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match