A thesaurus dictionary (sometimes abbreviated as TZ) is a collection of words that includes information about the relationships of words and phrases, i.e., broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, related terms, etc.
\nBasically a thesaurus dictionary replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. PostgreSQL's current implementation of the thesaurus dictionary is an extension of the synonym dictionary with added phrase support. A thesaurus dictionary requires a configuration file of the following format:
\n# this is a comment\nsample word(s) : indexed word(s)\nmore sample word(s) : more indexed word(s)\n...\n\n
where the colon (:) symbol acts as a delimiter between a phrase and its replacement.
A thesaurus dictionary uses a subdictionary (which is specified in the dictionary's configuration) to normalize the input text before checking for phrase matches. It is only possible to select one subdictionary. An error is reported if the subdictionary fails to recognize a word. In that case, you should remove the use of the word or teach the subdictionary about it. You can place an asterisk (*) at the beginning of an indexed word to skip applying the subdictionary to it, but all sample words must be known to the subdictionary.
The thesaurus dictionary chooses the longest match if there are multiple phrases matching the input, and ties are broken by using the last definition.
\nSpecific stop words recognized by the subdictionary cannot be specified; instead use ? to mark the location where any stop word can appear. For example, assuming that a and the are stop words according to the subdictionary:
? one ? two : swsw\n\n
matches a one the two and the one a two; both would be replaced by swsw.
Since a thesaurus dictionary has the capability to recognize phrases it must remember its state and interact with the parser. A thesaurus dictionary uses these assignments to check if it should handle the next word or stop accumulation. The thesaurus dictionary must be configured carefully. For example, if the thesaurus dictionary is assigned to handle only the asciiword token, then a thesaurus dictionary definition like one 7 will not work since token type uint is not assigned to the thesaurus dictionary.
Thesauruses are used during indexing so any change in the thesaurus dictionary's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stopwords does not force reindexing.
\nTo define a new thesaurus dictionary, use the thesaurus template. For example:
CREATE TEXT SEARCH DICTIONARY thesaurus_simple (\n TEMPLATE = thesaurus,\n DictFile = mythesaurus,\n Dictionary = pg_catalog.english_stem\n);\n\n
Here:
\nthesaurus_simple is the new dictionary's name
mythesaurus is the base name of the thesaurus configuration file. (Its full name will be $SHAREDIR/tsearch_data/mythesaurus.ths, where $SHAREDIR means the installation shared-data directory.)
pg_catalog.english_stem is the subdictionary (here, a Snowball English stemmer) to use for thesaurus normalization. Notice that the subdictionary will have its own configuration (for example, stop words), which is not shown here.
Now it is possible to bind the thesaurus dictionary thesaurus_simple to the desired token types in a configuration, for example:
ALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_simple;\n
Consider a simple astronomical thesaurus thesaurus_astro, which contains some astronomical word combinations:
supernovae stars : sn\ncrab nebulae : crab\n\n
Below we create a dictionary and bind some token types to an astronomical thesaurus and English stemmer:
\nCREATE TEXT SEARCH DICTIONARY thesaurus_astro (\n TEMPLATE = thesaurus,\n DictFile = thesaurus_astro,\n Dictionary = english_stem\n);\n\nALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_astro, english_stem;\n\n
Now we can see how it works. ts_lexize is not very useful for testing a thesaurus, because it treats its input as a single token. Instead we can use plainto_tsquery and to_tsvector which will break their input strings into multiple tokens:
SELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------\n 'sn'\n\nSELECT to_tsvector('supernova star');\n to_tsvector\n-------------\n 'sn':1\n\nIn principle, one can use to_tsquery if you quote the argument:
SELECT to_tsquery('''supernova star''');\n to_tsquery\n------------\n 'sn'\n\nNotice that supernova star matches supernovae stars in thesaurus_astro because we specified the english_stem stemmer in the thesaurus definition. The stemmer removed the e and s.
To index the original phrase as well as the substitute, just include it in the right-hand part of the definition:
\nsupernovae stars : sn supernovae stars\n\nSELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------------------\n 'sn' & 'supernova' & 'star'\nA thesaurus dictionary (sometimes abbreviated as TZ) is a collection of words that includes information about the relationships of words and phrases, i.e., broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, related terms, etc.
\nBasically a thesaurus dictionary replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. PostgreSQL's current implementation of the thesaurus dictionary is an extension of the synonym dictionary with added phrase support. A thesaurus dictionary requires a configuration file of the following format:
\n# this is a comment\nsample word(s) : indexed word(s)\nmore sample word(s) : more indexed word(s)\n...\n\n
where the colon (:) symbol acts as a delimiter between a phrase and its replacement.
A thesaurus dictionary uses a subdictionary (which is specified in the dictionary's configuration) to normalize the input text before checking for phrase matches. It is only possible to select one subdictionary. An error is reported if the subdictionary fails to recognize a word. In that case, you should remove the use of the word or teach the subdictionary about it. You can place an asterisk (*) at the beginning of an indexed word to skip applying the subdictionary to it, but all sample words must be known to the subdictionary.
The thesaurus dictionary chooses the longest match if there are multiple phrases matching the input, and ties are broken by using the last definition.
\nSpecific stop words recognized by the subdictionary cannot be specified; instead use ? to mark the location where any stop word can appear. For example, assuming that a and the are stop words according to the subdictionary:
? one ? two : swsw\n\n
matches a one the two and the one a two; both would be replaced by swsw.
Since a thesaurus dictionary has the capability to recognize phrases it must remember its state and interact with the parser. A thesaurus dictionary uses these assignments to check if it should handle the next word or stop accumulation. The thesaurus dictionary must be configured carefully. For example, if the thesaurus dictionary is assigned to handle only the asciiword token, then a thesaurus dictionary definition like one 7 will not work since token type uint is not assigned to the thesaurus dictionary.
Thesauruses are used during indexing so any change in the thesaurus dictionary's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stopwords does not force reindexing.
\nTo define a new thesaurus dictionary, use the thesaurus template. For example:
CREATE TEXT SEARCH DICTIONARY thesaurus_simple (\n TEMPLATE = thesaurus,\n DictFile = mythesaurus,\n Dictionary = pg_catalog.english_stem\n);\n\n
Here:
\nthesaurus_simple is the new dictionary's name
mythesaurus is the base name of the thesaurus configuration file. (Its full name will be $SHAREDIR/tsearch_data/mythesaurus.ths, where $SHAREDIR means the installation shared-data directory.)
pg_catalog.english_stem is the subdictionary (here, a Snowball English stemmer) to use for thesaurus normalization. Notice that the subdictionary will have its own configuration (for example, stop words), which is not shown here.
Now it is possible to bind the thesaurus dictionary thesaurus_simple to the desired token types in a configuration, for example:
ALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_simple;\n\n
Consider a simple astronomical thesaurus thesaurus_astro, which contains some astronomical word combinations:
supernovae stars : sn\ncrab nebulae : crab\n\n
Below we create a dictionary and bind some token types to an astronomical thesaurus and English stemmer:
\nCREATE TEXT SEARCH DICTIONARY thesaurus_astro (\n TEMPLATE = thesaurus,\n DictFile = thesaurus_astro,\n Dictionary = english_stem\n);\n\nALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_astro, english_stem;\n\n
Now we can see how it works. ts_lexize is not very useful for testing a thesaurus, because it treats its input as a single token. Instead we can use plainto_tsquery and to_tsvector which will break their input strings into multiple tokens:
SELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------\n 'sn'\n\nSELECT to_tsvector('supernova star');\n to_tsvector\n-------------\n 'sn':1\n\nIn principle, one can use to_tsquery if you quote the argument:
SELECT to_tsquery('''supernova star''');\n to_tsquery\n------------\n 'sn'\n\nNotice that supernova star matches supernovae stars in thesaurus_astro because we specified the english_stem stemmer in the thesaurus definition. The stemmer removed the e and s.
To index the original phrase as well as the substitute, just include it in the right-hand part of the definition:
\nsupernovae stars : sn supernovae stars\n\nSELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------------------\n 'sn' & 'supernova' & 'star'\n\nA thesaurus dictionary (sometimes abbreviated as TZ) is a collection of words that includes information about the relationships of words and phrases, i.e., broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, related terms, etc.
\nBasically a thesaurus dictionary replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. PostgreSQL's current implementation of the thesaurus dictionary is an extension of the synonym dictionary with added phrase support. A thesaurus dictionary requires a configuration file of the following format:
\n# this is a comment\nsample word(s) : indexed word(s)\nmore sample word(s) : more indexed word(s)\n...\n\n
where the colon (:) symbol acts as a delimiter between a phrase and its replacement.
A thesaurus dictionary uses a subdictionary (which is specified in the dictionary's configuration) to normalize the input text before checking for phrase matches. It is only possible to select one subdictionary. An error is reported if the subdictionary fails to recognize a word. In that case, you should remove the use of the word or teach the subdictionary about it. You can place an asterisk (*) at the beginning of an indexed word to skip applying the subdictionary to it, but all sample words must be known to the subdictionary.
The thesaurus dictionary chooses the longest match if there are multiple phrases matching the input, and ties are broken by using the last definition.
\nSpecific stop words recognized by the subdictionary cannot be specified; instead use ? to mark the location where any stop word can appear. For example, assuming that a and the are stop words according to the subdictionary:
? one ? two : swsw\n\n
matches a one the two and the one a two; both would be replaced by swsw.
Since a thesaurus dictionary has the capability to recognize phrases it must remember its state and interact with the parser. A thesaurus dictionary uses these assignments to check if it should handle the next word or stop accumulation. The thesaurus dictionary must be configured carefully. For example, if the thesaurus dictionary is assigned to handle only the asciiword token, then a thesaurus dictionary definition like one 7 will not work since token type uint is not assigned to the thesaurus dictionary.
Thesauruses are used during indexing so any change in the thesaurus dictionary's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stopwords does not force reindexing.
\nTo define a new thesaurus dictionary, use the thesaurus template. For example:
CREATE TEXT SEARCH DICTIONARY thesaurus_simple (\n TEMPLATE = thesaurus,\n DictFile = mythesaurus,\n Dictionary = pg_catalog.english_stem\n);\n\n
Here:
\nthesaurus_simple is the new dictionary's name
mythesaurus is the base name of the thesaurus configuration file. (Its full name will be $SHAREDIR/tsearch_data/mythesaurus.ths, where $SHAREDIR means the installation shared-data directory.)
pg_catalog.english_stem is the subdictionary (here, a Snowball English stemmer) to use for thesaurus normalization. Notice that the subdictionary will have its own configuration (for example, stop words), which is not shown here.
Now it is possible to bind the thesaurus dictionary thesaurus_simple to the desired token types in a configuration, for example:
ALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_simple;\n\n
Consider a simple astronomical thesaurus thesaurus_astro, which contains some astronomical word combinations:
supernovae stars : sn\ncrab nebulae : crab\n\n
Below we create a dictionary and bind some token types to an astronomical thesaurus and English stemmer:
\nCREATE TEXT SEARCH DICTIONARY thesaurus_astro (\n TEMPLATE = thesaurus,\n DictFile = thesaurus_astro,\n Dictionary = english_stem\n);\n\nALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_astro, english_stem;\n\n
Now we can see how it works. ts_lexize is not very useful for testing a thesaurus, because it treats its input as a single token. Instead we can use plainto_tsquery and to_tsvector which will break their input strings into multiple tokens:
SELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------\n 'sn'\n\nSELECT to_tsvector('supernova star');\n to_tsvector\n-------------\n 'sn':1\n\nIn principle, one can use to_tsquery if you quote the argument:
SELECT to_tsquery('''supernova star''');\n to_tsquery\n------------\n 'sn'\n\nNotice that supernova star matches supernovae stars in thesaurus_astro because we specified the english_stem stemmer in the thesaurus definition. The stemmer removed the e and s.
To index the original phrase as well as the substitute, just include it in the right-hand part of the definition:
\nsupernovae stars : sn supernovae stars\n\nSELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------------------\n 'sn' & 'supernova' & 'star'\n\nA thesaurus dictionary (sometimes abbreviated as TZ) is a collection of words that includes information about the relationships of words and phrases, i.e., broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, related terms, etc.
\nBasically a thesaurus dictionary replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. PostgreSQL's current implementation of the thesaurus dictionary is an extension of the synonym dictionary with added phrase support. A thesaurus dictionary requires a configuration file of the following format:
\n# this is a comment\nsample word(s) : indexed word(s)\nmore sample word(s) : more indexed word(s)\n...\n\n
where the colon (:) symbol acts as a delimiter between a phrase and its replacement.
A thesaurus dictionary uses a subdictionary (which is specified in the dictionary's configuration) to normalize the input text before checking for phrase matches. It is only possible to select one subdictionary. An error is reported if the subdictionary fails to recognize a word. In that case, you should remove the use of the word or teach the subdictionary about it. You can place an asterisk (*) at the beginning of an indexed word to skip applying the subdictionary to it, but all sample words must be known to the subdictionary.
The thesaurus dictionary chooses the longest match if there are multiple phrases matching the input, and ties are broken by using the last definition.
\nSpecific stop words recognized by the subdictionary cannot be specified; instead use ? to mark the location where any stop word can appear. For example, assuming that a and the are stop words according to the subdictionary:
? one ? two : swsw\n\n
matches a one the two and the one a two; both would be replaced by swsw.
Since a thesaurus dictionary has the capability to recognize phrases it must remember its state and interact with the parser. A thesaurus dictionary uses these assignments to check if it should handle the next word or stop accumulation. The thesaurus dictionary must be configured carefully. For example, if the thesaurus dictionary is assigned to handle only the asciiword token, then a thesaurus dictionary definition like one 7 will not work since token type uint is not assigned to the thesaurus dictionary.
Thesauruses are used during indexing so any change in the thesaurus dictionary's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stopwords does not force reindexing.
\nTo define a new thesaurus dictionary, use the thesaurus template. For example:
CREATE TEXT SEARCH DICTIONARY thesaurus_simple (\n TEMPLATE = thesaurus,\n DictFile = mythesaurus,\n Dictionary = pg_catalog.english_stem\n);\n\n
Here:
\nthesaurus_simple is the new dictionary's name
mythesaurus is the base name of the thesaurus configuration file. (Its full name will be $SHAREDIR/tsearch_data/mythesaurus.ths, where $SHAREDIR means the installation shared-data directory.)
pg_catalog.english_stem is the subdictionary (here, a Snowball English stemmer) to use for thesaurus normalization. Notice that the subdictionary will have its own configuration (for example, stop words), which is not shown here.
Now it is possible to bind the thesaurus dictionary thesaurus_simple to the desired token types in a configuration, for example:
ALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_simple;\n\n
Consider a simple astronomical thesaurus thesaurus_astro, which contains some astronomical word combinations:
supernovae stars : sn\ncrab nebulae : crab\n\n
Below we create a dictionary and bind some token types to an astronomical thesaurus and English stemmer:
\nCREATE TEXT SEARCH DICTIONARY thesaurus_astro (\n TEMPLATE = thesaurus,\n DictFile = thesaurus_astro,\n Dictionary = english_stem\n);\n\nALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_astro, english_stem;\n\n
Now we can see how it works. ts_lexize is not very useful for testing a thesaurus, because it treats its input as a single token. Instead we can use plainto_tsquery and to_tsvector which will break their input strings into multiple tokens:
SELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------\n 'sn'\n\nSELECT to_tsvector('supernova star');\n to_tsvector\n-------------\n 'sn':1\n\nIn principle, one can use to_tsquery if you quote the argument:
SELECT to_tsquery('''supernova star''');\n to_tsquery\n------------\n 'sn'\n\nNotice that supernova star matches supernovae stars in thesaurus_astro because we specified the english_stem stemmer in the thesaurus definition. The stemmer removed the e and s.
To index the original phrase as well as the substitute, just include it in the right-hand part of the definition:
\nsupernovae stars : sn supernovae stars\n\nSELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------------------\n 'sn' & 'supernova' & 'star'\n\nA thesaurus dictionary (sometimes abbreviated as TZ) is a collection of words that includes information about the relationships of words and phrases, i.e., broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, related terms, etc.
\nBasically a thesaurus dictionary replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. PostgreSQL's current implementation of the thesaurus dictionary is an extension of the synonym dictionary with added phrase support. A thesaurus dictionary requires a configuration file of the following format:
\n# this is a comment\nsample word(s) : indexed word(s)\nmore sample word(s) : more indexed word(s)\n...\n\n
where the colon (:) symbol acts as a delimiter between a phrase and its replacement.
A thesaurus dictionary uses a subdictionary (which is specified in the dictionary's configuration) to normalize the input text before checking for phrase matches. It is only possible to select one subdictionary. An error is reported if the subdictionary fails to recognize a word. In that case, you should remove the use of the word or teach the subdictionary about it. You can place an asterisk (*) at the beginning of an indexed word to skip applying the subdictionary to it, but all sample words must be known to the subdictionary.
The thesaurus dictionary chooses the longest match if there are multiple phrases matching the input, and ties are broken by using the last definition.
\nSpecific stop words recognized by the subdictionary cannot be specified; instead use ? to mark the location where any stop word can appear. For example, assuming that a and the are stop words according to the subdictionary:
? one ? two : swsw\n\n
matches a one the two and the one a two; both would be replaced by swsw.
Since a thesaurus dictionary has the capability to recognize phrases it must remember its state and interact with the parser. A thesaurus dictionary uses these assignments to check if it should handle the next word or stop accumulation. The thesaurus dictionary must be configured carefully. For example, if the thesaurus dictionary is assigned to handle only the asciiword token, then a thesaurus dictionary definition like one 7 will not work since token type uint is not assigned to the thesaurus dictionary.
Thesauruses are used during indexing so any change in the thesaurus dictionary's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stopwords does not force reindexing.
\nTo define a new thesaurus dictionary, use the thesaurus template. For example:
CREATE TEXT SEARCH DICTIONARY thesaurus_simple (\n TEMPLATE = thesaurus,\n DictFile = mythesaurus,\n Dictionary = pg_catalog.english_stem\n);\n\n
Here:
\nthesaurus_simple is the new dictionary's name
mythesaurus is the base name of the thesaurus configuration file. (Its full name will be $SHAREDIR/tsearch_data/mythesaurus.ths, where $SHAREDIR means the installation shared-data directory.)
pg_catalog.english_stem is the subdictionary (here, a Snowball English stemmer) to use for thesaurus normalization. Notice that the subdictionary will have its own configuration (for example, stop words), which is not shown here.
Now it is possible to bind the thesaurus dictionary thesaurus_simple to the desired token types in a configuration, for example:
ALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_simple;\n\n
Consider a simple astronomical thesaurus thesaurus_astro, which contains some astronomical word combinations:
supernovae stars : sn\ncrab nebulae : crab\n\n
Below we create a dictionary and bind some token types to an astronomical thesaurus and English stemmer:
\nCREATE TEXT SEARCH DICTIONARY thesaurus_astro (\n TEMPLATE = thesaurus,\n DictFile = thesaurus_astro,\n Dictionary = english_stem\n);\n\nALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_astro, english_stem;\n\n
Now we can see how it works. ts_lexize is not very useful for testing a thesaurus, because it treats its input as a single token. Instead we can use plainto_tsquery and to_tsvector which will break their input strings into multiple tokens:
SELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------\n 'sn'\n\nSELECT to_tsvector('supernova star');\n to_tsvector\n-------------\n 'sn':1\n\nIn principle, one can use to_tsquery if you quote the argument:
SELECT to_tsquery('''supernova star''');\n to_tsquery\n------------\n 'sn'\n\nNotice that supernova star matches supernovae stars in thesaurus_astro because we specified the english_stem stemmer in the thesaurus definition. The stemmer removed the e and s.
To index the original phrase as well as the substitute, just include it in the right-hand part of the definition:
\nsupernovae stars : sn supernovae stars\n\nSELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------------------\n 'sn' & 'supernova' & 'star'\n\nA thesaurus dictionary (sometimes abbreviated as TZ) is a collection of words that includes information about the relationships of words and phrases, i.e., broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, related terms, etc.
\nBasically a thesaurus dictionary replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. PostgreSQL's current implementation of the thesaurus dictionary is an extension of the synonym dictionary with added phrase support. A thesaurus dictionary requires a configuration file of the following format:
\n# this is a comment\nsample word(s) : indexed word(s)\nmore sample word(s) : more indexed word(s)\n...\n\n
where the colon (:) symbol acts as a delimiter between a phrase and its replacement.
A thesaurus dictionary uses a subdictionary (which is specified in the dictionary's configuration) to normalize the input text before checking for phrase matches. It is only possible to select one subdictionary. An error is reported if the subdictionary fails to recognize a word. In that case, you should remove the use of the word or teach the subdictionary about it. You can place an asterisk (*) at the beginning of an indexed word to skip applying the subdictionary to it, but all sample words must be known to the subdictionary.
The thesaurus dictionary chooses the longest match if there are multiple phrases matching the input, and ties are broken by using the last definition.
\nSpecific stop words recognized by the subdictionary cannot be specified; instead use ? to mark the location where any stop word can appear. For example, assuming that a and the are stop words according to the subdictionary:
? one ? two : swsw\n\n
matches a one the two and the one a two; both would be replaced by swsw.
Since a thesaurus dictionary has the capability to recognize phrases it must remember its state and interact with the parser. A thesaurus dictionary uses these assignments to check if it should handle the next word or stop accumulation. The thesaurus dictionary must be configured carefully. For example, if the thesaurus dictionary is assigned to handle only the asciiword token, then a thesaurus dictionary definition like one 7 will not work since token type uint is not assigned to the thesaurus dictionary.
Thesauruses are used during indexing so any change in the thesaurus dictionary's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stopwords does not force reindexing.
\nTo define a new thesaurus dictionary, use the thesaurus template. For example:
CREATE TEXT SEARCH DICTIONARY thesaurus_simple (\n TEMPLATE = thesaurus,\n DictFile = mythesaurus,\n Dictionary = pg_catalog.english_stem\n);\n\n
Here:
\nthesaurus_simple is the new dictionary's name
mythesaurus is the base name of the thesaurus configuration file. (Its full name will be $SHAREDIR/tsearch_data/mythesaurus.ths, where $SHAREDIR means the installation shared-data directory.)
pg_catalog.english_stem is the subdictionary (here, a Snowball English stemmer) to use for thesaurus normalization. Notice that the subdictionary will have its own configuration (for example, stop words), which is not shown here.
Now it is possible to bind the thesaurus dictionary thesaurus_simple to the desired token types in a configuration, for example:
ALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_simple;\n\n
Consider a simple astronomical thesaurus thesaurus_astro, which contains some astronomical word combinations:
supernovae stars : sn\ncrab nebulae : crab\n\n
Below we create a dictionary and bind some token types to an astronomical thesaurus and English stemmer:
\nCREATE TEXT SEARCH DICTIONARY thesaurus_astro (\n TEMPLATE = thesaurus,\n DictFile = thesaurus_astro,\n Dictionary = english_stem\n);\n\nALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_astro, english_stem;\n\n
Now we can see how it works. ts_lexize is not very useful for testing a thesaurus, because it treats its input as a single token. Instead we can use plainto_tsquery and to_tsvector which will break their input strings into multiple tokens:
SELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------\n 'sn'\n\nSELECT to_tsvector('supernova star');\n to_tsvector\n-------------\n 'sn':1\n\nIn principle, one can use to_tsquery if you quote the argument:
SELECT to_tsquery('''supernova star''');\n to_tsquery\n------------\n 'sn'\n\nNotice that supernova star matches supernovae stars in thesaurus_astro because we specified the english_stem stemmer in the thesaurus definition. The stemmer removed the e and s.
To index the original phrase as well as the substitute, just include it in the right-hand part of the definition:
\nsupernovae stars : sn supernovae stars\n\nSELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------------------\n 'sn' & 'supernova' & 'star'\n\nA thesaurus dictionary (sometimes abbreviated as TZ) is a collection of words that includes information about the relationships of words and phrases, i.e., broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, related terms, etc.
\nBasically a thesaurus dictionary replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. PostgreSQL's current implementation of the thesaurus dictionary is an extension of the synonym dictionary with added phrase support. A thesaurus dictionary requires a configuration file of the following format:
\n# this is a comment\nsample word(s) : indexed word(s)\nmore sample word(s) : more indexed word(s)\n...\n\n
where the colon (:) symbol acts as a delimiter between a phrase and its replacement.
A thesaurus dictionary uses a subdictionary (which is specified in the dictionary's configuration) to normalize the input text before checking for phrase matches. It is only possible to select one subdictionary. An error is reported if the subdictionary fails to recognize a word. In that case, you should remove the use of the word or teach the subdictionary about it. You can place an asterisk (*) at the beginning of an indexed word to skip applying the subdictionary to it, but all sample words must be known to the subdictionary.
The thesaurus dictionary chooses the longest match if there are multiple phrases matching the input, and ties are broken by using the last definition.
\nSpecific stop words recognized by the subdictionary cannot be specified; instead use ? to mark the location where any stop word can appear. For example, assuming that a and the are stop words according to the subdictionary:
? one ? two : swsw\n\n
matches a one the two and the one a two; both would be replaced by swsw.
Since a thesaurus dictionary has the capability to recognize phrases it must remember its state and interact with the parser. A thesaurus dictionary uses these assignments to check if it should handle the next word or stop accumulation. The thesaurus dictionary must be configured carefully. For example, if the thesaurus dictionary is assigned to handle only the asciiword token, then a thesaurus dictionary definition like one 7 will not work since token type uint is not assigned to the thesaurus dictionary.
Thesauruses are used during indexing so any change in the thesaurus dictionary's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stopwords does not force reindexing.
\nTo define a new thesaurus dictionary, use the thesaurus template. For example:
CREATE TEXT SEARCH DICTIONARY thesaurus_simple (\n TEMPLATE = thesaurus,\n DictFile = mythesaurus,\n Dictionary = pg_catalog.english_stem\n);\n\n
Here:
\nthesaurus_simple is the new dictionary's name
mythesaurus is the base name of the thesaurus configuration file. (Its full name will be $SHAREDIR/tsearch_data/mythesaurus.ths, where $SHAREDIR means the installation shared-data directory.)
pg_catalog.english_stem is the subdictionary (here, a Snowball English stemmer) to use for thesaurus normalization. Notice that the subdictionary will have its own configuration (for example, stop words), which is not shown here.
Now it is possible to bind the thesaurus dictionary thesaurus_simple to the desired token types in a configuration, for example:
ALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_simple;\n\n
Consider a simple astronomical thesaurus thesaurus_astro, which contains some astronomical word combinations:
supernovae stars : sn\ncrab nebulae : crab\n\n
Below we create a dictionary and bind some token types to an astronomical thesaurus and English stemmer:
\nCREATE TEXT SEARCH DICTIONARY thesaurus_astro (\n TEMPLATE = thesaurus,\n DictFile = thesaurus_astro,\n Dictionary = english_stem\n);\n\nALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_astro, english_stem;\n\n
Now we can see how it works. ts_lexize is not very useful for testing a thesaurus, because it treats its input as a single token. Instead we can use plainto_tsquery and to_tsvector which will break their input strings into multiple tokens:
SELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------\n 'sn'\n\nSELECT to_tsvector('supernova star');\n to_tsvector\n-------------\n 'sn':1\n\nIn principle, one can use to_tsquery if you quote the argument:
SELECT to_tsquery('''supernova star''');\n to_tsquery\n------------\n 'sn'\n\nNotice that supernova star matches supernovae stars in thesaurus_astro because we specified the english_stem stemmer in the thesaurus definition. The stemmer removed the e and s.
To index the original phrase as well as the substitute, just include it in the right-hand part of the definition:
\nsupernovae stars : sn supernovae stars\n\nSELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------------------\n 'sn' & 'supernova' & 'star'\n\nA thesaurus dictionary (sometimes abbreviated as TZ) is a collection of words that includes information about the relationships of words and phrases, i.e., broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, related terms, etc.
\nBasically a thesaurus dictionary replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. PostgreSQL's current implementation of the thesaurus dictionary is an extension of the synonym dictionary with added phrase support. A thesaurus dictionary requires a configuration file of the following format:
\n# this is a comment\nsample word(s) : indexed word(s)\nmore sample word(s) : more indexed word(s)\n...\n\n
where the colon (:) symbol acts as a delimiter between a phrase and its replacement.
A thesaurus dictionary uses a subdictionary (which is specified in the dictionary's configuration) to normalize the input text before checking for phrase matches. It is only possible to select one subdictionary. An error is reported if the subdictionary fails to recognize a word. In that case, you should remove the use of the word or teach the subdictionary about it. You can place an asterisk (*) at the beginning of an indexed word to skip applying the subdictionary to it, but all sample words must be known to the subdictionary.
The thesaurus dictionary chooses the longest match if there are multiple phrases matching the input, and ties are broken by using the last definition.
\nSpecific stop words recognized by the subdictionary cannot be specified; instead use ? to mark the location where any stop word can appear. For example, assuming that a and the are stop words according to the subdictionary:
? one ? two : swsw\n\n
matches a one the two and the one a two; both would be replaced by swsw.
Since a thesaurus dictionary has the capability to recognize phrases it must remember its state and interact with the parser. A thesaurus dictionary uses these assignments to check if it should handle the next word or stop accumulation. The thesaurus dictionary must be configured carefully. For example, if the thesaurus dictionary is assigned to handle only the asciiword token, then a thesaurus dictionary definition like one 7 will not work since token type uint is not assigned to the thesaurus dictionary.
Thesauruses are used during indexing so any change in the thesaurus dictionary's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stopwords does not force reindexing.
\nTo define a new thesaurus dictionary, use the thesaurus template. For example:
CREATE TEXT SEARCH DICTIONARY thesaurus_simple (\n TEMPLATE = thesaurus,\n DictFile = mythesaurus,\n Dictionary = pg_catalog.english_stem\n);\n\n
Here:
\nthesaurus_simple is the new dictionary's name
mythesaurus is the base name of the thesaurus configuration file. (Its full name will be $SHAREDIR/tsearch_data/mythesaurus.ths, where $SHAREDIR means the installation shared-data directory.)
pg_catalog.english_stem is the subdictionary (here, a Snowball English stemmer) to use for thesaurus normalization. Notice that the subdictionary will have its own configuration (for example, stop words), which is not shown here.
Now it is possible to bind the thesaurus dictionary thesaurus_simple to the desired token types in a configuration, for example:
ALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_simple;\n\n
Consider a simple astronomical thesaurus thesaurus_astro, which contains some astronomical word combinations:
supernovae stars : sn\ncrab nebulae : crab\n\n
Below we create a dictionary and bind some token types to an astronomical thesaurus and English stemmer:
\nCREATE TEXT SEARCH DICTIONARY thesaurus_astro (\n TEMPLATE = thesaurus,\n DictFile = thesaurus_astro,\n Dictionary = english_stem\n);\n\nALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_astro, english_stem;\n\n
Now we can see how it works. ts_lexize is not very useful for testing a thesaurus, because it treats its input as a single token. Instead we can use plainto_tsquery and to_tsvector which will break their input strings into multiple tokens:
SELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------\n 'sn'\n\nSELECT to_tsvector('supernova star');\n to_tsvector\n-------------\n 'sn':1\n\nIn principle, one can use to_tsquery if you quote the argument:
SELECT to_tsquery('''supernova star''');\n to_tsquery\n------------\n 'sn'\n\nNotice that supernova star matches supernovae stars in thesaurus_astro because we specified the english_stem stemmer in the thesaurus definition. The stemmer removed the e and s.
To index the original phrase as well as the substitute, just include it in the right-hand part of the definition:
\nsupernovae stars : sn supernovae stars\n\nSELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------------------\n 'sn' & 'supernova' & 'star'\n\nA thesaurus dictionary (sometimes abbreviated as TZ) is a collection of words that includes information about the relationships of words and phrases, i.e., broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, related terms, etc.
\nBasically a thesaurus dictionary replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. PostgreSQL's current implementation of the thesaurus dictionary is an extension of the synonym dictionary with added phrase support. A thesaurus dictionary requires a configuration file of the following format:
\n# this is a comment\nsample word(s) : indexed word(s)\nmore sample word(s) : more indexed word(s)\n...\n\n
where the colon (:) symbol acts as a delimiter between a phrase and its replacement.
A thesaurus dictionary uses a subdictionary (which is specified in the dictionary's configuration) to normalize the input text before checking for phrase matches. It is only possible to select one subdictionary. An error is reported if the subdictionary fails to recognize a word. In that case, you should remove the use of the word or teach the subdictionary about it. You can place an asterisk (*) at the beginning of an indexed word to skip applying the subdictionary to it, but all sample words must be known to the subdictionary.
The thesaurus dictionary chooses the longest match if there are multiple phrases matching the input, and ties are broken by using the last definition.
\nSpecific stop words recognized by the subdictionary cannot be specified; instead use ? to mark the location where any stop word can appear. For example, assuming that a and the are stop words according to the subdictionary:
? one ? two : swsw\n\n
matches a one the two and the one a two; both would be replaced by swsw.
Since a thesaurus dictionary has the capability to recognize phrases it must remember its state and interact with the parser. A thesaurus dictionary uses these assignments to check if it should handle the next word or stop accumulation. The thesaurus dictionary must be configured carefully. For example, if the thesaurus dictionary is assigned to handle only the asciiword token, then a thesaurus dictionary definition like one 7 will not work since token type uint is not assigned to the thesaurus dictionary.
Thesauruses are used during indexing so any change in the thesaurus dictionary's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stopwords does not force reindexing.
\nTo define a new thesaurus dictionary, use the thesaurus template. For example:
CREATE TEXT SEARCH DICTIONARY thesaurus_simple (\n TEMPLATE = thesaurus,\n DictFile = mythesaurus,\n Dictionary = pg_catalog.english_stem\n);\n\n
Here:
\nthesaurus_simple is the new dictionary's name
mythesaurus is the base name of the thesaurus configuration file. (Its full name will be $SHAREDIR/tsearch_data/mythesaurus.ths, where $SHAREDIR means the installation shared-data directory.)
pg_catalog.english_stem is the subdictionary (here, a Snowball English stemmer) to use for thesaurus normalization. Notice that the subdictionary will have its own configuration (for example, stop words), which is not shown here.
Now it is possible to bind the thesaurus dictionary thesaurus_simple to the desired token types in a configuration, for example:
ALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_simple;\n\n
Consider a simple astronomical thesaurus thesaurus_astro, which contains some astronomical word combinations:
supernovae stars : sn\ncrab nebulae : crab\n\n
Below we create a dictionary and bind some token types to an astronomical thesaurus and English stemmer:
\nCREATE TEXT SEARCH DICTIONARY thesaurus_astro (\n TEMPLATE = thesaurus,\n DictFile = thesaurus_astro,\n Dictionary = english_stem\n);\n\nALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_astro, english_stem;\n\n
Now we can see how it works. ts_lexize is not very useful for testing a thesaurus, because it treats its input as a single token. Instead we can use plainto_tsquery and to_tsvector which will break their input strings into multiple tokens:
SELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------\n 'sn'\n\nSELECT to_tsvector('supernova star');\n to_tsvector\n-------------\n 'sn':1\n\nIn principle, one can use to_tsquery if you quote the argument:
SELECT to_tsquery('''supernova star''');\n to_tsquery\n------------\n 'sn'\n\nNotice that supernova star matches supernovae stars in thesaurus_astro because we specified the english_stem stemmer in the thesaurus definition. The stemmer removed the e and s.
To index the original phrase as well as the substitute, just include it in the right-hand part of the definition:
\nsupernovae stars : sn supernovae stars\n\nSELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------------------\n 'sn' & 'supernova' & 'star'\n\nA thesaurus dictionary (sometimes abbreviated as TZ) is a collection of words that includes information about the relationships of words and phrases, i.e., broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, related terms, etc.
\nBasically a thesaurus dictionary replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. PostgreSQL's current implementation of the thesaurus dictionary is an extension of the synonym dictionary with added phrase support. A thesaurus dictionary requires a configuration file of the following format:
\n# this is a comment\nsample word(s) : indexed word(s)\nmore sample word(s) : more indexed word(s)\n...\n\n
where the colon (:) symbol acts as a delimiter between a phrase and its replacement.
A thesaurus dictionary uses a subdictionary (which is specified in the dictionary's configuration) to normalize the input text before checking for phrase matches. It is only possible to select one subdictionary. An error is reported if the subdictionary fails to recognize a word. In that case, you should remove the use of the word or teach the subdictionary about it. You can place an asterisk (*) at the beginning of an indexed word to skip applying the subdictionary to it, but all sample words must be known to the subdictionary.
The thesaurus dictionary chooses the longest match if there are multiple phrases matching the input, and ties are broken by using the last definition.
\nSpecific stop words recognized by the subdictionary cannot be specified; instead use ? to mark the location where any stop word can appear. For example, assuming that a and the are stop words according to the subdictionary:
? one ? two : swsw\n\n
matches a one the two and the one a two; both would be replaced by swsw.
Since a thesaurus dictionary has the capability to recognize phrases it must remember its state and interact with the parser. A thesaurus dictionary uses these assignments to check if it should handle the next word or stop accumulation. The thesaurus dictionary must be configured carefully. For example, if the thesaurus dictionary is assigned to handle only the asciiword token, then a thesaurus dictionary definition like one 7 will not work since token type uint is not assigned to the thesaurus dictionary.
Thesauruses are used during indexing so any change in the thesaurus dictionary's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stopwords does not force reindexing.
\nTo define a new thesaurus dictionary, use the thesaurus template. For example:
CREATE TEXT SEARCH DICTIONARY thesaurus_simple (\n TEMPLATE = thesaurus,\n DictFile = mythesaurus,\n Dictionary = pg_catalog.english_stem\n);\n\n
Here:
\nthesaurus_simple is the new dictionary's name
mythesaurus is the base name of the thesaurus configuration file. (Its full name will be $SHAREDIR/tsearch_data/mythesaurus.ths, where $SHAREDIR means the installation shared-data directory.)
pg_catalog.english_stem is the subdictionary (here, a Snowball English stemmer) to use for thesaurus normalization. Notice that the subdictionary will have its own configuration (for example, stop words), which is not shown here.
Now it is possible to bind the thesaurus dictionary thesaurus_simple to the desired token types in a configuration, for example:
ALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_simple;\n\n
Consider a simple astronomical thesaurus thesaurus_astro, which contains some astronomical word combinations:
supernovae stars : sn\ncrab nebulae : crab\n\n
Below we create a dictionary and bind some token types to an astronomical thesaurus and English stemmer:
\nCREATE TEXT SEARCH DICTIONARY thesaurus_astro (\n TEMPLATE = thesaurus,\n DictFile = thesaurus_astro,\n Dictionary = english_stem\n);\n\nALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_astro, english_stem;\n\n
Now we can see how it works. ts_lexize is not very useful for testing a thesaurus, because it treats its input as a single token. Instead we can use plainto_tsquery and to_tsvector which will break their input strings into multiple tokens:
SELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------\n 'sn'\n\nSELECT to_tsvector('supernova star');\n to_tsvector\n-------------\n 'sn':1\n\nIn principle, one can use to_tsquery if you quote the argument:
SELECT to_tsquery('''supernova star''');\n to_tsquery\n------------\n 'sn'\n\nNotice that supernova star matches supernovae stars in thesaurus_astro because we specified the english_stem stemmer in the thesaurus definition. The stemmer removed the e and s.
To index the original phrase as well as the substitute, just include it in the right-hand part of the definition:
\nsupernovae stars : sn supernovae stars\n\nSELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------------------\n 'sn' & 'supernova' & 'star'\n\nA thesaurus dictionary (sometimes abbreviated as TZ) is a collection of words that includes information about the relationships of words and phrases, i.e., broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, related terms, etc.
\nBasically a thesaurus dictionary replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. PostgreSQL's current implementation of the thesaurus dictionary is an extension of the synonym dictionary with added phrase support. A thesaurus dictionary requires a configuration file of the following format:
\n# this is a comment\nsample word(s) : indexed word(s)\nmore sample word(s) : more indexed word(s)\n...\n\n
where the colon (:) symbol acts as a delimiter between a phrase and its replacement.
A thesaurus dictionary uses a subdictionary (which is specified in the dictionary's configuration) to normalize the input text before checking for phrase matches. It is only possible to select one subdictionary. An error is reported if the subdictionary fails to recognize a word. In that case, you should remove the use of the word or teach the subdictionary about it. You can place an asterisk (*) at the beginning of an indexed word to skip applying the subdictionary to it, but all sample words must be known to the subdictionary.
The thesaurus dictionary chooses the longest match if there are multiple phrases matching the input, and ties are broken by using the last definition.
\nSpecific stop words recognized by the subdictionary cannot be specified; instead use ? to mark the location where any stop word can appear. For example, assuming that a and the are stop words according to the subdictionary:
? one ? two : swsw\n\n
matches a one the two and the one a two; both would be replaced by swsw.
Since a thesaurus dictionary has the capability to recognize phrases it must remember its state and interact with the parser. A thesaurus dictionary uses these assignments to check if it should handle the next word or stop accumulation. The thesaurus dictionary must be configured carefully. For example, if the thesaurus dictionary is assigned to handle only the asciiword token, then a thesaurus dictionary definition like one 7 will not work since token type uint is not assigned to the thesaurus dictionary.
Thesauruses are used during indexing so any change in the thesaurus dictionary's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stopwords does not force reindexing.
\nTo define a new thesaurus dictionary, use the thesaurus template. For example:
CREATE TEXT SEARCH DICTIONARY thesaurus_simple (\n TEMPLATE = thesaurus,\n DictFile = mythesaurus,\n Dictionary = pg_catalog.english_stem\n);\n\n
Here:
\nthesaurus_simple is the new dictionary's name
mythesaurus is the base name of the thesaurus configuration file. (Its full name will be $SHAREDIR/tsearch_data/mythesaurus.ths, where $SHAREDIR means the installation shared-data directory.)
pg_catalog.english_stem is the subdictionary (here, a Snowball English stemmer) to use for thesaurus normalization. Notice that the subdictionary will have its own configuration (for example, stop words), which is not shown here.
Now it is possible to bind the thesaurus dictionary thesaurus_simple to the desired token types in a configuration, for example:
ALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_simple;\n\n
Consider a simple astronomical thesaurus thesaurus_astro, which contains some astronomical word combinations:
supernovae stars : sn\ncrab nebulae : crab\n\n
Below we create a dictionary and bind some token types to an astronomical thesaurus and English stemmer:
\nCREATE TEXT SEARCH DICTIONARY thesaurus_astro (\n TEMPLATE = thesaurus,\n DictFile = thesaurus_astro,\n Dictionary = english_stem\n);\n\nALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_astro, english_stem;\n\n
Now we can see how it works. ts_lexize is not very useful for testing a thesaurus, because it treats its input as a single token. Instead we can use plainto_tsquery and to_tsvector which will break their input strings into multiple tokens:
SELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------\n 'sn'\n\nSELECT to_tsvector('supernova star');\n to_tsvector\n-------------\n 'sn':1\n\nIn principle, one can use to_tsquery if you quote the argument:
SELECT to_tsquery('''supernova star''');\n to_tsquery\n------------\n 'sn'\n\nNotice that supernova star matches supernovae stars in thesaurus_astro because we specified the english_stem stemmer in the thesaurus definition. The stemmer removed the e and s.
To index the original phrase as well as the substitute, just include it in the right-hand part of the definition:
\nsupernovae stars : sn supernovae stars\n\nSELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------------------\n 'sn' & 'supernova' & 'star'\n\nA thesaurus dictionary (sometimes abbreviated as TZ) is a collection of words that includes information about the relationships of words and phrases, i.e., broader terms (BT), narrower terms (NT), preferred terms, non-preferred terms, related terms, etc.
\nBasically a thesaurus dictionary replaces all non-preferred terms by one preferred term and, optionally, preserves the original terms for indexing as well. PostgreSQL's current implementation of the thesaurus dictionary is an extension of the synonym dictionary with added phrase support. A thesaurus dictionary requires a configuration file of the following format:
\n# this is a comment\nsample word(s) : indexed word(s)\nmore sample word(s) : more indexed word(s)\n...\n\n
where the colon (:) symbol acts as a delimiter between a phrase and its replacement.
A thesaurus dictionary uses a subdictionary (which is specified in the dictionary's configuration) to normalize the input text before checking for phrase matches. It is only possible to select one subdictionary. An error is reported if the subdictionary fails to recognize a word. In that case, you should remove the use of the word or teach the subdictionary about it. You can place an asterisk (*) at the beginning of an indexed word to skip applying the subdictionary to it, but all sample words must be known to the subdictionary.
The thesaurus dictionary chooses the longest match if there are multiple phrases matching the input, and ties are broken by using the last definition.
\nSpecific stop words recognized by the subdictionary cannot be specified; instead use ? to mark the location where any stop word can appear. For example, assuming that a and the are stop words according to the subdictionary:
? one ? two : swsw\n\n
matches a one the two and the one a two; both would be replaced by swsw.
Since a thesaurus dictionary has the capability to recognize phrases it must remember its state and interact with the parser. A thesaurus dictionary uses these assignments to check if it should handle the next word or stop accumulation. The thesaurus dictionary must be configured carefully. For example, if the thesaurus dictionary is assigned to handle only the asciiword token, then a thesaurus dictionary definition like one 7 will not work since token type uint is not assigned to the thesaurus dictionary.
Thesauruses are used during indexing so any change in the thesaurus dictionary's parameters requires reindexing. For most other dictionary types, small changes such as adding or removing stopwords does not force reindexing.
\nTo define a new thesaurus dictionary, use the thesaurus template. For example:
CREATE TEXT SEARCH DICTIONARY thesaurus_simple (\n TEMPLATE = thesaurus,\n DictFile = mythesaurus,\n Dictionary = pg_catalog.english_stem\n);\n\n
Here:
\nthesaurus_simple is the new dictionary's name
mythesaurus is the base name of the thesaurus configuration file. (Its full name will be $SHAREDIR/tsearch_data/mythesaurus.ths, where $SHAREDIR means the installation shared-data directory.)
pg_catalog.english_stem is the subdictionary (here, a Snowball English stemmer) to use for thesaurus normalization. Notice that the subdictionary will have its own configuration (for example, stop words), which is not shown here.
Now it is possible to bind the thesaurus dictionary thesaurus_simple to the desired token types in a configuration, for example:
ALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_simple;\n\n
Consider a simple astronomical thesaurus thesaurus_astro, which contains some astronomical word combinations:
supernovae stars : sn\ncrab nebulae : crab\n\n
Below we create a dictionary and bind some token types to an astronomical thesaurus and English stemmer:
\nCREATE TEXT SEARCH DICTIONARY thesaurus_astro (\n TEMPLATE = thesaurus,\n DictFile = thesaurus_astro,\n Dictionary = english_stem\n);\n\nALTER TEXT SEARCH CONFIGURATION russian\n ALTER MAPPING FOR asciiword, asciihword, hword_asciipart\n WITH thesaurus_astro, english_stem;\n\n
Now we can see how it works. ts_lexize is not very useful for testing a thesaurus, because it treats its input as a single token. Instead we can use plainto_tsquery and to_tsvector which will break their input strings into multiple tokens:
SELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------\n 'sn'\n\nSELECT to_tsvector('supernova star');\n to_tsvector\n-------------\n 'sn':1\n\nIn principle, one can use to_tsquery if you quote the argument:
SELECT to_tsquery('''supernova star''');\n to_tsquery\n------------\n 'sn'\n\nNotice that supernova star matches supernovae stars in thesaurus_astro because we specified the english_stem stemmer in the thesaurus definition. The stemmer removed the e and s.
To index the original phrase as well as the substitute, just include it in the right-hand part of the definition:
\nsupernovae stars : sn supernovae stars\n\nSELECT plainto_tsquery('supernova star');\n plainto_tsquery\n-----------------------------\n 'sn' & 'supernova' & 'star'\n\n