Add direct conversion routines between EUC_TW and Big5.
authorHeikki Linnakangas <heikki.linnakangas@iki.fi>
Thu, 28 Jan 2021 12:53:03 +0000 (14:53 +0200)
committerHeikki Linnakangas <heikki.linnakangas@iki.fi>
Thu, 28 Jan 2021 12:53:03 +0000 (14:53 +0200)
commit6c5576075b0f93f2235ac8a82290fe3b6e82300d
treeca90df578b121b9f8249767d9b318b4e49957ae1
parentb80e10638e36b9d2f0b39170c613837af2ca2aac
Add direct conversion routines between EUC_TW and Big5.

Conversions between EUC_TW and Big5 were previously implemented by
converting the whole input to MIC first, and then from MIC to the target
encoding. Implement functions to convert directly between the two.

The reason to do this now is that I'm working on a patch that will change
the conversion function signature so that if the input is invalid, we
convert as much as we can and return the number of bytes successfully
converted. That's not possible if we use an intermediary format, because
if an error happens in the intermediary -> final conversion, we lose track
of the location of the invalid character in the original input. Avoiding
the intermediate step makes the conversions faster, too.

Reviewed-by: John Naylor
Discussion: https://www.postgresql.org/message-id/b9e3167f-f84b-7aa4-5738-be578a4db924%40iki.fi
src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c