Tokenizers for full-text search
pgext.cloud · Source code · License text · Control file
| Package | pg_tokenizer |
|---|---|
| Version | 0.1.1 |
| PostgreSQL versions | 14 · 15 · 16 · 17 · 18 |
| Repository | PIGSTY |
| Extension type | Preloaded |
| Requires CREATE EXTENSION | Yes |
| Requires preload | Yes |
| Trusted | No |
| Relocatable | No |
| Shared libraries | pg_tokenizer |
| Schema | tokenizer_catalog |
| Stars | 45 |
| Latest commit | 2025-09-29 |
| Latest release | 2025-09-29 |
| Catalogue updated | 2026-08-30 |
| pgroonga | Use Groonga as index, fast full text search platform for all languages! |
| pg_jieba | Chinese full-text search parser based on cppjieba |
| pg_cjk_parser | CJK bigram parser derived from PostgreSQL full-text search |
| zhparser | a parser for full-text search of Chinese |
| pg_bigm | create 2-gram (bigram) index for faster full text search. |
| pg_tiktoken | tiktoken tokenizer for use with OpenAI models in postgres |
| pg_tiktoken_c | Fast tiktoken BPE tokenizer for PostgreSQL implemented in C |
| unaccent | text search dictionary that removes accents |
| dict_xsyn | text search dictionary template for extended synonym processing |
| dict_int | text search dictionary template for integers |
| hunspell_cs_cz | Czech Hunspell Dictionary |
| pg_kazsearch | Kazakh full-text search extension for PostgreSQL |