Update to latest Snowball sources.
authorTom Lane <tgl@sss.pgh.pa.us>
Wed, 19 Feb 2025 02:13:46 +0000 (21:13 -0500)
committerTom Lane <tgl@sss.pgh.pa.us>
Wed, 19 Feb 2025 02:13:54 +0000 (21:13 -0500)
commitb464e51ab32fbf09cf5d9c911a8e26f491ad1f44
tree0fa15dd6477c1995919f63efe9e38b652a3ab25b
parent71d02dc478d574c75bd0af82cec774c7b9059a61
Update to latest Snowball sources.

It's been some time since we did this, partly because the upstream
snowball project hasn't formally tagged a new release since 2021.
The main motivation for doing it now is to absorb a bug fix
(their commit e322673a841d9abd69994ae8cd20e191090b6ef4), which
prevents a null pointer dereference crash if SN_create_env() gets
a malloc failure at just the wrong point.  We'll patch the back
branches with only that change, but we might as well do the full
sync dance on HEAD.

Aside from a bunch of mostly-minor tweaks to existing stemmers, this
update adds a new stemmer for Estonian.  It also removes the existing
stemmer for Romanian using ISO-8859-2 encoding.  Upstream apparently
concluded that ISO-8859-2 doesn't provide an adequate representation
of some Romanian characters, and the UTF-8 implementation should be
used instead.

While at it, update the README's instructions for doing a sync,
which have not been adjusted during the addition of meson tooling.

Thanks to Maksim Korotkov for discovering the null-pointer
bug and submitting the fix to upstream snowball.

Reported-by: Maksim Korotkov <m.korotkov@postgrespro.ru>
Discussion: https://postgr.es/m/1d1a46-67ab1000-21-80c451@83151435
61 files changed:
doc/src/sgml/textsearch.sgml
src/backend/snowball/Makefile
src/backend/snowball/README
src/backend/snowball/dict_snowball.c
src/backend/snowball/libstemmer/api.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_basque.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_catalan.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_danish.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_dutch.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_english.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_finnish.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_french.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_german.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_indonesian.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_irish.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_italian.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_norwegian.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_porter.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_portuguese.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_spanish.c
src/backend/snowball/libstemmer/stem_ISO_8859_1_swedish.c
src/backend/snowball/libstemmer/stem_ISO_8859_2_hungarian.c
src/backend/snowball/libstemmer/stem_ISO_8859_2_romanian.c [deleted file]
src/backend/snowball/libstemmer/stem_KOI8_R_russian.c
src/backend/snowball/libstemmer/stem_UTF_8_arabic.c
src/backend/snowball/libstemmer/stem_UTF_8_armenian.c
src/backend/snowball/libstemmer/stem_UTF_8_basque.c
src/backend/snowball/libstemmer/stem_UTF_8_catalan.c
src/backend/snowball/libstemmer/stem_UTF_8_danish.c
src/backend/snowball/libstemmer/stem_UTF_8_dutch.c
src/backend/snowball/libstemmer/stem_UTF_8_english.c
src/backend/snowball/libstemmer/stem_UTF_8_estonian.c [new file with mode: 0644]
src/backend/snowball/libstemmer/stem_UTF_8_finnish.c
src/backend/snowball/libstemmer/stem_UTF_8_french.c
src/backend/snowball/libstemmer/stem_UTF_8_german.c
src/backend/snowball/libstemmer/stem_UTF_8_greek.c
src/backend/snowball/libstemmer/stem_UTF_8_hindi.c
src/backend/snowball/libstemmer/stem_UTF_8_hungarian.c
src/backend/snowball/libstemmer/stem_UTF_8_indonesian.c
src/backend/snowball/libstemmer/stem_UTF_8_irish.c
src/backend/snowball/libstemmer/stem_UTF_8_italian.c
src/backend/snowball/libstemmer/stem_UTF_8_lithuanian.c
src/backend/snowball/libstemmer/stem_UTF_8_nepali.c
src/backend/snowball/libstemmer/stem_UTF_8_norwegian.c
src/backend/snowball/libstemmer/stem_UTF_8_porter.c
src/backend/snowball/libstemmer/stem_UTF_8_portuguese.c
src/backend/snowball/libstemmer/stem_UTF_8_romanian.c
src/backend/snowball/libstemmer/stem_UTF_8_russian.c
src/backend/snowball/libstemmer/stem_UTF_8_serbian.c
src/backend/snowball/libstemmer/stem_UTF_8_spanish.c
src/backend/snowball/libstemmer/stem_UTF_8_swedish.c
src/backend/snowball/libstemmer/stem_UTF_8_tamil.c
src/backend/snowball/libstemmer/stem_UTF_8_turkish.c
src/backend/snowball/libstemmer/stem_UTF_8_yiddish.c
src/backend/snowball/libstemmer/utilities.c
src/backend/snowball/meson.build
src/backend/snowball/snowball_create.pl
src/bin/initdb/initdb.c
src/include/snowball/libstemmer/header.h
src/include/snowball/libstemmer/stem_ISO_8859_2_romanian.h [deleted file]
src/include/snowball/libstemmer/stem_UTF_8_estonian.h [new file with mode: 0644]