Peter Eisentraut [Mon, 4 Jul 2022 06:06:05 +0000 (08:06 +0200)]
Fix for change timeline field of IDENTIFY_SYSTEM to int8
Amendment to
ec40f3422412cfdc140b5d3f67db7fd2dac0f1e2: We also need to
change the way the datum is supplied to int8. Otherwise, the value is
still cut off as an int4, and it will crash on 32-bit platforms.
Peter Eisentraut [Mon, 4 Jul 2022 05:25:26 +0000 (07:25 +0200)]
Change timeline field of IDENTIFY_SYSTEM to int8
It was int4, but in the other replication commands, timelines are
returned as int8.
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/
7e4fdbdc-699c-4cd0-115d-
fb78a957fc22@enterprisedb.com
Peter Eisentraut [Mon, 4 Jul 2022 05:25:26 +0000 (07:25 +0200)]
Fix attlen in RowDescription of BASE_BACKUP response
Should be 8 for int8, not -1.
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/
7e4fdbdc-699c-4cd0-115d-
fb78a957fc22@enterprisedb.com
Tom Lane [Mon, 4 Jul 2022 01:38:32 +0000 (21:38 -0400)]
Use a short socket directory path in pg_upgrade testing.
Several buildfarm members are failing the pg_upgrade test in
REL_15_STABLE, though the identical test is fine in HEAD.
On thorntail it's possible to see that the problem is an
overlength socket path name, and I bet the same is true
on the others.
The normally-started postmasters used in the test are already
set up with short socket directory paths, but we neglected to
tell pg_upgrade itself to do likewise when starting child
postmasters, and indeed it seems to be explicitly selecting
the test directory instead.
Back-patch to v15 where the current test script was introduced.
(The previous script might have the same issue, because I don't
see any use of -s/--socketdir in it either; but we've had no
complaints, so leave it alone for now.)
Discussion: https://postgr.es/m/
1410025.
1656890531@sss.pgh.pa.us
Andrew Dunstan [Sun, 3 Jul 2022 21:08:25 +0000 (17:08 -0400)]
Remove %error-verbose directive from jsonpath parser
None of the other bison parsers contains this directive, and it gives
rise to some unfortunate and impenetrable messages, so just remove it.
Backpatch to release 12, where it was introduced.
Per gripe from Erik Rijkers
Discussion: https://postgr.es/m/
ba069ce2-a98f-dc70-dc17-
2ccf2a9bf7c7@xs4all.nl
Tom Lane [Sun, 3 Jul 2022 20:49:12 +0000 (16:49 -0400)]
Allow makeaclitem() to accept multiple privilege names.
Interpret its privileges argument as a comma-separated list of
privilege names, as in has_table_privilege and other functions.
This is actually net less code, since the support routine to
parse that already exists, and we can drop convert_priv_string()
which had no other use-case.
Robins Tharakan
Discussion: https://postgr.es/m/
e5a05dc54ba64408b3dd260171c1abaf@EX13D05UWC001.ant.amazon.com
Noah Misch [Sun, 3 Jul 2022 20:07:17 +0000 (13:07 -0700)]
Add Windows file version information to libpq/test programs.
Back-patch to v15, the first version to install these programs.
Discussion: https://postgr.es/m/
20220501080706.GA1542365@rfd.leadboat.com
Tom Lane [Sun, 3 Jul 2022 19:27:27 +0000 (15:27 -0400)]
Simplify tab completion of extension versions, redux.
After commit
662dbe2c8, psql tab completion didn't conveniently
support the case of "ALTER EXTENSION foo UPDATE". It'd always
add "TO", which is fine if you want to specify a target version
but not if you don't ... and surely the latter is the much more
common case.
To fix, remove "TO" from the initially offered completion; you now
need to press TAB one additional time to get that. We won't try to
duplicate the old behavior of attempting initial completion on the
target version along with TO. It's too squirrelly to get the quoting
right, and this is such an infrequent usage that it doesn't seem worth
expending a lot of effort and special code on.
Noted by Noah Misch. Back-patch to v15.
Discussion: https://postgr.es/m/
20220703083217.GB2476530@rfd.leadboat.com
Peter Eisentraut [Sun, 3 Jul 2022 18:11:05 +0000 (20:11 +0200)]
Remove redundant null pointer checks before PQclear and PQconninfoFree
These functions already had the free()-like behavior of handling null
pointers as a no-op. But it wasn't documented, so add it explicitly
to the documentation, too.
Discussion: https://www.postgresql.org/message-id/flat/
dac5d2d0-98f5-94d9-8e69-
46da2413593d%40enterprisedb.com
Peter Eisentraut [Thu, 16 Jun 2022 19:50:56 +0000 (21:50 +0200)]
Remove redundant null pointer checks before free()
Per applicable standards, free() with a null pointer is a no-op.
Systems that don't observe that are ancient and no longer relevant.
Some PostgreSQL code already required this behavior, so this change
does not introduce any new requirements, just makes the code more
consistent.
Discussion: https://www.postgresql.org/message-id/flat/
dac5d2d0-98f5-94d9-8e69-
46da2413593d%40enterprisedb.com
Peter Eisentraut [Fri, 17 Jun 2022 09:51:38 +0000 (11:51 +0200)]
Remove redundant null pointer checks before pg_free()
These are especially useless because the whole point of pg_free() was
to do that very check before calling free().
pg_free() could be removed altogether, but I'm keeping it here to keep
the API consistent.
Discussion: https://www.postgresql.org/message-id/flat/
dac5d2d0-98f5-94d9-8e69-
46da2413593d%40enterprisedb.com
Noah Misch [Sun, 3 Jul 2022 04:03:19 +0000 (21:03 -0700)]
Fix previous commit's ecpg_clocale for ppc Darwin.
Per buildfarm member prairiedog, this platform rejects uninitialized
global variables in shared libraries. Back-patch to v10, like the
addition of the variable.
Reviewed by Tom Lane.
Discussion: https://postgr.es/m/
20220703030619.GB2378460@rfd.leadboat.com
Noah Misch [Sat, 2 Jul 2022 20:00:30 +0000 (13:00 -0700)]
ecpglib: call newlocale() once per process.
ecpglib has been calling it once per SQL query and once per EXEC SQL GET
DESCRIPTOR. Instead, if newlocale() has not succeeded before, call it
while establishing a connection. This mitigates three problems:
- If newlocale() failed in EXEC SQL GET DESCRIPTOR, the command silently
proceeded without the intended locale change.
- On AIX, each newlocale()+freelocale() cycle leaked memory.
- newlocale() CPU usage may have been nontrivial.
Fail the connection attempt if newlocale() fails. Rearrange
ecpg_do_prologue() to validate the connection before its uselocale().
The sort of program that may regress is one running in an environment
where newlocale() fails. If that program establishes connections
without running SQL statements, it will stop working in response to this
change. I'm betting against the importance of such an ECPG use case.
Most SQL execution (any using ECPGdo()) has long required newlocale()
success, so there's little a connection could do without newlocale().
Back-patch to v10 (all supported versions).
Reviewed by Tom Lane. Reported by Guillaume Lelarge.
Discussion: https://postgr.es/m/
20220101074055.GA54621@rfd.leadboat.com
Jeff Davis [Sat, 2 Jul 2022 18:20:45 +0000 (11:20 -0700)]
Emit debug message when executing extension script.
Allows extension authors to more easily debug problems related to the
sequence of update scripts that are executed.
Discussion: https://postgr.es/m/
5636a7534a4833884172fe4369d825b26170b3cc.camel%40j-davis.com
Reviewed-by: Peter Eisentraut, Nathan Bossart
Peter Eisentraut [Sat, 2 Jul 2022 10:33:07 +0000 (12:33 +0200)]
Reformat some node comments
Reformat some comments in node field definitions to avoid long lines.
This makes room for per-field annotations in a future patch to
generate node support functions automatically.
Discussion: https://www.postgresql.org/message-id/
c5906b07-220a-a3d4-8ff3-
8ee593009424@enterprisedb.com
Thomas Munro [Sat, 2 Jul 2022 04:06:47 +0000 (16:06 +1200)]
Default to dynamic_shared_memory_type=sysv on Solaris.
POSIX shm_open() can sleep for a long time and fail spuriously because
of contention on an internal lock file on Solaris (and presumably
illumos). Commit
389869af fixed the main problem with this, namely that
we could crash, but it's now clear that "posix" is not a good default.
Therefore, choose "sysv" at initdb time on Solaris and illumos. Other
choices are still available by editing the postgresql.conf file.
Back-patch only to 15, because contention is much less likely further
back, and it doesn't seem like a good idea to change this in released
branches. This should clear up the failures on build farm animal
margay.
Discussion: https://postgr.es/m/CA%2BhUKGKqKrCV5xKWfh9rnm%3Do%3DDwZLTLtnsj_XpUi9g5%3DV%2B9oyg%40mail.gmail.com
Michael Paquier [Sat, 2 Jul 2022 03:30:45 +0000 (12:30 +0900)]
Introduce pg_attribute_nonnull(...)
pg_attribute_nonnull(...) can be used to generate compiler warnings
when a function is called with the specified arguments set to NULL, as
per an idea from Andres Freund. An empty argument list indicates that
no pointer arguments can be NULL. pg_attribute_nonnull() only works for
compilers that support the nonnull function attribute. If nonnull is
not supported, pg_attribute_nonnull() has no effect.
As a beginning, this commit uses it for the DefineCustomXXXVariable()
functions to generate warnings when the "name" and "value" arguments are
set to NULL. This will likely be expanded to other places in the
future, where it makes sense.
Author: Nathan Bossart
Reviewed by: Michael Paquier, Tom Lane
Discussion: https://postgr.es/m/
20220525061739.ur7x535vtzyzkmqo@alap3.anarazel.de
Noah Misch [Sat, 2 Jul 2022 01:27:50 +0000 (18:27 -0700)]
Replace PGISOLATIONTIMEOUT with 2 * PG_TEST_TIMEOUT_DEFAULT.
Now that the more-generic variable exists, use it.
Discussion: https://postgr.es/m/
20220219024136.GA3670392@rfd.leadboat.com
Noah Misch [Sat, 2 Jul 2022 01:27:18 +0000 (18:27 -0700)]
Fix race condition in t/028_pitr_timelines.pl.
Per buildfarm members sungazer and mylodon. Back-patch to v15, which
introduced this test.
Discussion: https://postgr.es/m/
20220627070457.GA2176699@rfd.leadboat.com
Tom Lane [Fri, 1 Jul 2022 22:39:30 +0000 (18:39 -0400)]
Remove no-longer-used parameter for create_groupingsets_path().
numGroups is unused since commit
b5635948a; let's get rid of it.
XueJing Zhao, reviewed by Richard Guo
Discussion: https://postgr.es/m/DM6PR05MB64923CC8B63A2CAF3B2E5D47B7AD9@DM6PR05MB6492.namprd05.prod.outlook.com
Peter Eisentraut [Fri, 1 Jul 2022 14:41:46 +0000 (16:41 +0200)]
Add missing GETTEXT_FLAGS entry
Peter Eisentraut [Fri, 1 Jul 2022 08:51:45 +0000 (10:51 +0200)]
Add construct_array_builtin, deconstruct_array_builtin
There were many calls to construct_array() and deconstruct_array() for
built-in types, for example, when dealing with system catalog columns.
These all hardcoded the type attributes necessary to pass to these
functions.
To simplify this a bit, add construct_array_builtin(),
deconstruct_array_builtin() as wrappers that centralize this hardcoded
knowledge. This simplifies many call sites and reduces the amount of
hardcoded stuff that is spread around.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/
2914356f-9e5f-8c59-2995-
5997fc48bcba%40enterprisedb.com
Fujii Masao [Fri, 1 Jul 2022 05:19:52 +0000 (14:19 +0900)]
doc: Add index entry for MERGE SQL command.
Back-patch to v15, where MERGE command was supported.
Author: Fujii Masao
Reviewed-by: Alvaro Herrera
Discussion: https://postgr.es/m/
4e7a4b38-c34a-ce2c-c71e-
832d2b768b29@oss.nttdata.com
John Naylor [Fri, 1 Jul 2022 04:41:36 +0000 (11:41 +0700)]
Clarify that pg_dump takes ACCESS SHARE lock
Add link to the description of lock levels to avoid confusing "shared locks"
with SHARE locks.
Florin Irion
Reviewed-by: Álvaro Herrera, Tom Lane, and Nathan Bossart
Discussion: https://www.postgresql.org/message-id/flat/
d0f30cc2-3c76-1d43-f291-
7c4b2872d653@gmail.com
Thomas Munro [Fri, 1 Jul 2022 00:05:52 +0000 (12:05 +1200)]
Harden dsm_impl.c against unexpected EEXIST.
Previously, we trusted the OS not to report EEXIST unless we'd passed in
IPC_CREAT | IPC_EXCL or O_CREAT | O_EXCL, as appropriate. Solaris's
shm_open() can in fact do that, causing us to crash because we didn't
ereport and then we blithely assumed the mapping was successful.
Let's treat EEXIST just like any other error, unless we're actually
trying to create a new segment. This applies to shm_open(), where this
behavior has been seen, and also to the equivalent operations for our
sysv and mmap modes just on principle.
Based on the underlying reason for the error, namely contention on a
lock file managed by Solaris librt for each distinct name, this problem
is only likely to happen on 15 and later, because the new shared memory
stats system produces shm_open() calls for the same path from
potentially large numbers of backends concurrently during
authentication. Earlier releases only shared memory segments between a
small number of parallel workers under one Gather node. You could
probably hit it if you tried hard enough though, and we should have been
more defensive in the first place. Therefore, back-patch to all
supported releases.
Per build farm animal margay. This isn't the end of the story, though,
it just changes random crashes into random "File exists" errors; more
work needed for a green build farm.
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGKqKrCV5xKWfh9rnm%3Do%3DDwZLTLtnsj_XpUi9g5%3DV%2B9oyg%40mail.gmail.com
Michael Paquier [Fri, 1 Jul 2022 00:37:17 +0000 (09:37 +0900)]
Fix code comments still referring to pg_start/stop_backup()
pg_start_backup() and pg_stop_backup() have been respectively renamed to
pg_backup_start() and pg_backup_stop() as of
39969e2, but a few comments
did not get the call.
Reviewed-by: Kyotaro Horiguchi, David Steele
Discussion: https://postgr.es/m/YrqGlj1+4DF3dbZ/@paquier.xyz
Michael Paquier [Fri, 1 Jul 2022 00:13:57 +0000 (09:13 +0900)]
Refactor the TAP test of auto_explain
Previously, the tests were structured so as all the queries whose plans
are checked run first, followed by pattern checks using the full set of
server logs. This can be problematic when extending the tests, as this
increases query plan overlaps, where two tests finish by having similar
plan outputs potentially invalidating the tests wanted.
The tests are refactored so as log content matches are checked in
isolation of each query run, by grabbing the position of the server logs
before running each query whose plan is generated in the logs. This
avoids issues when extending the tests, something that would become a
larger problem with a follow-up patch that adds a new GUC in
auto_explain to control the size of the each parameter logged.
Author: Dagfinn Ilmari Mannsåker
Discussion: https://postgr.es/m/87ee09mohb.fsf@wibble.ilmari.org
Peter Eisentraut [Thu, 30 Jun 2022 22:16:38 +0000 (00:16 +0200)]
Change some unnecessary MemSet calls
MemSet() with a value other than 0 just falls back to memset(), so the
indirection is unnecessary if the value is constant and not 0. Since
there is some interest in getting rid of MemSet(), this gets some easy
cases out of the way. (There are a few MemSet() calls that I didn't
change to maintain the consistency with their surrounding code.)
Discussion: https://www.postgresql.org/message-id/flat/CAEudQApCeq4JjW1BdnwU=m=-DvG5WyUik0Yfn3p6UNphiHjj+w@mail.gmail.com
Peter Eisentraut [Thu, 30 Jun 2022 18:10:13 +0000 (19:10 +0100)]
Avoid unnecessary MemSet call
The variable in question was changed from a struct to a pointer some
time ago (
77947c51c08). Using MemSet to zero it still works but is
obviously unidiomatic and confusing, so change it to a straight
assignment.
Author: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAEudQApCeq4JjW1BdnwU=m=-DvG5WyUik0Yfn3p6UNphiHjj+w@mail.gmail.com
Tom Lane [Thu, 30 Jun 2022 15:26:40 +0000 (11:26 -0400)]
Stamp HEAD as 16devel.
Let the hacking begin ...
Tom Lane [Thu, 30 Jun 2022 15:08:12 +0000 (11:08 -0400)]
Add
82d0ffae3 to git-blame-ignore-revs.
Tom Lane [Thu, 30 Jun 2022 15:03:03 +0000 (11:03 -0400)]
pgindent run prior to branching v15.
pgperltidy and reformat-dat-files too. Not many changes.
Bruce Momjian [Tue, 28 Jun 2022 20:35:32 +0000 (16:35 -0400)]
relnotes: update item about public schema permission change
Also mention the bootstrap superuser.
Reported-by: Noah Misch
Discussion: https://postgr.es/m/
20220628063719.GA2208244@rfd.leadboat.com
Tom Lane [Mon, 27 Jun 2022 20:11:56 +0000 (16:11 -0400)]
Stamp 15beta2.
Magnus Hagander [Mon, 27 Jun 2022 17:43:27 +0000 (19:43 +0200)]
Remove outdated recommendation for manual VACUUM
We have had a working and tunable autovacuum
for at least a decade now, so remove the recommendation to
manually vacuum tables at least every night.
Autovacuum is now also triggered by INSERTs, so we can also
remove the recommendation to run VACUUM (ANALYZE) after lots
of INSERTs or DELETEs.
Instead, suggest using autovacuum by moving the respective
paragraph up to where the importance of VACUUM is emphasized.
Author: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-By: Magnus Hagander, Peter Geoghegan
Discussion: https://postgr.es/m/
6f5e3da98fec14640f389d7b84c3b413833697f4.camel@cybertec.at
Peter Eisentraut [Mon, 27 Jun 2022 10:53:05 +0000 (12:53 +0200)]
PL/Python: Update guide to alternative expected files
plpython_unicode_3.out was already removed a long time ago, so it
being listed here was very out of date.
plpython_types_3.out was removed with the Python 2 removal.
Peter Eisentraut [Mon, 27 Jun 2022 07:06:44 +0000 (09:06 +0200)]
pg_upgrade: Fix version comparison for global ICU support
Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/
20220625151930.GH22452@telsasoft.com
Michael Paquier [Mon, 27 Jun 2022 06:33:39 +0000 (15:33 +0900)]
doc: Mention attribute "inherited" in pg_stats_ext[_exprs]
These have been added in
269b532, but the documentation did not reflect
that.
Author: Noriyoshi Shinoda
Reviewed-by: Justin Pryzby
Discussion: https://postgr.es/m/DM4PR84MB17349C4E7D88A68264C18AF3EED69@DM4PR84MB1734.NAMPRD84.PROD.OUTLOOK.COM
Peter Eisentraut [Mon, 27 Jun 2022 06:19:02 +0000 (08:19 +0200)]
Translation updates
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash:
46c120873f1e906cc8dab74d8d756417e1b367f6
Heikki Linnakangas [Mon, 27 Jun 2022 05:21:08 +0000 (08:21 +0300)]
Fix visibility check when XID is committed in CLOG but not in procarray.
TransactionIdIsInProgress had a fast path to return 'false' if the
single-item CLOG cache said that the transaction was known to be
committed. However, that was wrong, because a transaction is first
marked as committed in the CLOG but doesn't become visible to others
until it has removed its XID from the proc array. That could lead to an
error:
ERROR: t_xmin is uncommitted in tuple to be updated
or for an UPDATE to go ahead without blocking, before the previous
UPDATE on the same row was made visible.
The window is usually very short, but synchronous replication makes it
much wider, because the wait for synchronous replica happens in that
window.
Another thing that makes it hard to hit is that it's hard to get such
a commit-in-progress transaction into the single item CLOG cache.
Normally, if you call TransactionIdIsInProgress on such a transaction,
it determines that the XID is in progress without checking the CLOG
and without populating the cache. One way to prime the cache is to
explicitly call pg_xact_status() on the XID. Another way is to use a
lot of subtransactions, so that the subxid cache in the proc array is
overflown, making TransactionIdIsInProgress rely on pg_subtrans and
CLOG checks.
This has been broken ever since it was introduced in 2008, but the race
condition is very hard to hit, especially without synchronous
replication. There were a couple of reports of the error starting from
summer 2021, but no one was able to find the root cause then.
TransactionIdIsKnownCompleted() is now unused. In 'master', remove it,
but I left it in place in backbranches in case it's used by extensions.
Also change pg_xact_status() to check TransactionIdIsInProgress().
Previously, it only checked the CLOG, and returned "committed" before
the transaction was actually made visible to other queries. Note that
this also means that you cannot use pg_xact_status() to reproduce the
bug anymore, even if the code wasn't fixed.
Report and analysis by Konstantin Knizhnik. Patch by Simon Riggs, with
the pg_xact_status() change added by me.
Author: Simon Riggs
Reviewed-by: Andres Freund
Discussion: https://www.postgresql.org/message-id/flat/
4da7913d-398c-e2ad-d777-
f752cf7f0bbb%40garret.ru
Thomas Munro [Sun, 26 Jun 2022 22:30:15 +0000 (10:30 +1200)]
Fix relptr's encoding of the base address.
Previously, we encoded both NULL and the first byte at the base address
as 0. That confusion led to the assertion in commit
e07d4ddc, which
failed when min_dynamic_shared_memory was used. Give them distinct
encodings, by switching to 1-based offsets for non-NULL pointers. Also
improve macro hygiene in passing (missing/misplaced parentheses), and
remove open-coded access to the raw offset value from freepage.c/h.
Although
e07d4ddc was back-patched to 10, the only code that actually
makes use of relptr at the base address arrived in
84b1c63a, so no need
to back-patch further than 14 for now.
Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/
20220519193839.GT19626%40telsasoft.com
Tom Lane [Sun, 26 Jun 2022 12:58:05 +0000 (08:58 -0400)]
Harden range_table_mutator() against null RangeTblEntry.subquery.
Commit
64919aaab made pull_up_simple_subquery set rte->subquery = NULL
after doing the deed, so that we don't waste cycles copying a
now-useless subquery tree around. This turns out to create a core dump
hazard in range_table_mutator, which supposes that that field is never
NULL. Apparently none of our own code invokes query_tree_mutator or
range_table_mutator on the top Query after subquery pullup; but it
wouldn't be surprising if outside code does, and anyway I'm working
on a v16 patch that will need it.
We can fix this cleanly by just getting rid of the special-case
handling of this field and treating it more like all the rest.
I think the special case might be left over from a time when
QTW_DONT_COPY_QUERY was the default behavior, but that was eons ago.
Thanks to Dean Rasheed for review.
Discussion: https://postgr.es/m/545569.
1656107045@sss.pgh.pa.us
Thomas Munro [Sat, 25 Jun 2022 22:40:06 +0000 (10:40 +1200)]
Don't trust signalfd() on illumos.
Since commit
6a2a70a02, we've used signalfd() to receive latch wakeups
when building with WAIT_USE_EPOLL (default for Linux and illumos), and
our traditional self-pipe when falling back to WAIT_USE_POLL (default
for other Unixes with neither epoll() nor kqueue()).
Unexplained hangs and kernel panics have been reported on illumos
systems, apparently linked to this use of signalfd(), leading illumos
users and build farm members to have to define WAIT_USE_POLL explicitly
as a work-around. A bug report exists at
https://www.illumos.org/issues/13700 but no fix is available yet.
Let's provide a way for illumos users to go back to self-pipes with
epoll(), like releases before 14, and choose that by default. No change
for Linux users. To help with development/debugging, macros
WAIT_USE_{EPOLL,POLL} and WAIT_USE_{SIGNALFD,SELF_PIPE} can be defined
explicitly to override the defaults.
Back-patch to 14, where we started using signalfd().
Reported-by: Japin Li <japinli@hotmail.com>
Reported-by: Olaf Bohlen <olbohlen@eenfach.de> (off-list)
Reviewed-by: Japin Li <japinli@hotmail.com>
Discussion: https://postgr.es/m/MEYP282MB1669C8D88F0997354C2313C1B6CA9%40MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
Noah Misch [Sat, 25 Jun 2022 16:07:41 +0000 (09:07 -0700)]
CREATE INDEX: use the original userid for more ACL checks.
Commit
a117cebd638dd02e5c2e791c25e43745f233111b used the original userid
for ACL checks located directly in DefineIndex(), but it still adopted
the table owner userid for more ACL checks than intended. That broke
dump/reload of indexes that refer to an operator class, collation, or
exclusion operator in a schema other than "public" or "pg_catalog".
Back-patch to v10 (all supported versions), like the earlier commit.
Nathan Bossart and Noah Misch
Discussion: https://postgr.es/m/
f8a4105f076544c180a87ef0c4822352@stmuk.bayern.de
Noah Misch [Sat, 25 Jun 2022 16:07:41 +0000 (09:07 -0700)]
Use --no-unlogged-table-data in t/027_stream_regress.pl.
This removes the need to drop unlogged relations in the src/test/regress
suite, like commit
dec8ad367e46180f826d5b6dc820fbecba1b71d2 did.
Reviewed by Thomas Munro.
Discussion: https://postgr.es/m/39945.
1650895508@sss.pgh.pa.us
Peter Eisentraut [Fri, 24 Jun 2022 11:03:59 +0000 (13:03 +0200)]
Remove unportable test
The ssl test "IPv4 host with CIDR mask does not match" apparently has
a portability problem. Some operating systems don't reject the host
name specification "192.0.2.1/32" as an IP address, and that is then
later rejected when the SNI is set, which results in a different error
message that the test is supposed to verify.
The value of the test has been questioned in the discussion, and it
was suggested that removing it would be an acceptable fix, so that's
what this is doing.
Reported-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Bug: #17522
Discussion: https://www.postgresql.org/message-id/flat/17522-
bfcd5c603b5f4daa%40postgresql.org
Amit Kapila [Fri, 24 Jun 2022 03:51:24 +0000 (09:21 +0530)]
Fix intermetent test failure in 028_row_filter.pl.
The test was not waiting for the subscriber's data synchronization to
happen after refreshing the publication on the subscriber side. This leads
subscriber's apply worker to skip applying the changes on the
corresponding relation which results in a test failure.
Reported-by: Hou Zhijie, as per buildfarm
Author: Hou Zhijie
Reviewed-by: Masahiko Sawada, Amit Kapila
Discussion: https://postgr.es/m/OS0PR01MB5716A69496A8E2F2E155DB8D94B59@OS0PR01MB5716.jpnprd01.prod.outlook.com
Amit Kapila [Fri, 24 Jun 2022 03:07:26 +0000 (08:37 +0530)]
Doc: clarify that tablesync ignores publish operation.
This patch documents that the initial data synchronization (tablesync) for
logical replication does not take into account the publication 'publish'
parameter when copying the existing table data.
Author: Peter Smith
Reviewed-by: Shi yu, Euler Taveira, Robert Haas, Amit Kapila
Discussion: https://postgr.es/m/CAHut+PtbfALjFpS2MkrvQ+wWQKByP7CNh9RtFta-r=BHEU3S3w@mail.gmail.com
Peter Eisentraut [Thu, 23 Jun 2022 20:40:52 +0000 (22:40 +0200)]
Fix missing newline at end of message
Michael Paquier [Thu, 23 Jun 2022 07:42:27 +0000 (16:42 +0900)]
Fix typo in pg_publication.c
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+PuV2XXjC4spHXy_EOhpD6MDrmmDMWnVJLYpd1_P=2+mJw@mail.gmail.com
Peter Eisentraut [Thu, 23 Jun 2022 06:59:27 +0000 (08:59 +0200)]
PL/Tcl: Don't link with -lc explicitly
It has been reported that PL/Tcl built on macOS with GCC >=11 crashes.
The reason is that there is a hash_search() function in the operating
system's libraries, and that ends up being called instead of the one
in postgres. This has something to do with how the linker resolves
references between the various possibilities it has been given, and
somehow something changed that it is now picking that one in this
configuration.
We found that removing the -lc from the link command line fixes this
problem. The -lc was introduced a long time ago in commit
e3909672f12e0ddf3e202b824fda068ad2195ef2, and we think the reasons
might be obsolete, so we decided that we'll try to just remove it and
see if any problems arise.
Discussion: https://www.postgresql.org/message-id/flat/
a78c847a-4f79-9286-be99-
e819e9e4139e%40enterprisedb.com
Amit Kapila [Thu, 23 Jun 2022 03:53:46 +0000 (09:23 +0530)]
Fix memory leak due to LogicalRepRelMapEntry.attrmap.
When rebuilding the relation mapping on subscribers, we were not releasing
the attribute mapping's memory which was no longer required.
The attribute mapping used in logical tuple conversion was refactored in
PG13 (by commit
e1551f96e6) but we forgot to update the related code that
frees the attribute map.
Author: Hou Zhijie
Reviewed-by: Amit Langote, Amit Kapila, Shi yu
Backpatch-through: 10, where it was introduced
Discussion: https://postgr.es/m/OSZPR01MB6310F46CD425A967E4AEF736FDA49@OSZPR01MB6310.jpnprd01.prod.outlook.com
Michael Paquier [Thu, 23 Jun 2022 01:49:20 +0000 (10:49 +0900)]
Fix two issues with HEADER MATCH in COPY
072132f0 used the attnum offset to access the raw_fields array when
checking that the attribute names of the header and of the relation
match, leading to incorrect results or even crashes if the attribute
numbers of a relation are changed, like on a dropped attribute. This
fixes the logic to use the correct attribute names for the header
matching requirements.
Also, this commit disallows HEADER MATCH in COPY TO as there is no
validation that can be done in this case.
The tests are expanded for HEADER MATCH with COPY FROM and dropped
columns, with cases where a relation has a dropped and re-added column,
as well as a reduced set of columns.
Author: Julien Rouhaud
Reviewed-by: Peter Eisentraut, Michael Paquier
Discussion: https://postgr.es/m/
20220607154744.vvmitnqhyxrne5ms@jrouhaud
Andres Freund [Wed, 22 Jun 2022 23:50:14 +0000 (16:50 -0700)]
pgstat: Mention pgstat_replslot.c in pgstat.c.
Oversight, by me, in commit
5891c7a8ed8.
Author: "Drouvot, Bertrand" <bdrouvot@amazon.com>
Discussion: https://postgr.es/m/
bd58e027-6598-57a2-679b-
d576d17bfaa9@amazon.com
Bruce Momjian [Wed, 22 Jun 2022 20:59:54 +0000 (16:59 -0400)]
doc: improve wording of plpgsql RAISE format text
Reported-by: pg@kirasoft.com
Discussion: https://postgr.es/m/
165455351426.573551.
7050474465030525109@wrigleys.postgresql.org
Backpatch-through: 10
Bruce Momjian [Wed, 22 Jun 2022 18:33:26 +0000 (14:33 -0400)]
doc: clarify wording about phantom reads
Reported-by: akhilhello@gmail.com
Discussion: https://postgr.es/m/
165222922369.669.
10475917322916060899@wrigleys.postgresql.org
Backpatch-through: 10
Robert Haas [Wed, 22 Jun 2022 17:11:49 +0000 (13:11 -0400)]
amcheck: Fix incorrect use of VARATT_IS_COMPRESSED.
The macro is being applied to a TOAST pointer, not a varlena header.
Therefore the use of VARATT_IS_COMPRESSED() is wrong. We can check
VARATT_EXTERNAL_IS_COMPRESSED(), but then we don't need the length
check that follows.
Report and fix by Kyotaro Horiguchi.
Discussion: http://postgr.es/m/
20220517.162719.
1671558681467343711.horikyota.ntt@gmail.com
Peter Eisentraut [Wed, 22 Jun 2022 12:24:48 +0000 (14:24 +0200)]
doc: Clean up title case use
Tom Lane [Tue, 21 Jun 2022 16:04:11 +0000 (12:04 -0400)]
Simplify tab completion of extension versions.
Second thoughts about
9cd43f6cb: given that we're staying bug-compatible
with the old behavior of using double not single quotes for extension
versions, we can simplify this completion code by pretending that
extension versions *are* identifiers, and not using VERBATIM. Then
_complete_from_query() will think that the query results are identifiers
in need of quoting, and we end up with the same behavior as before.
This doesn't work for Query_for_list_of_available_extension_versions_with_TO,
but let's just drop that: there is no other place where we handle
multi-keyword phrases that way, and it doesn't seem very desirable here
either. Handle completion of "UPDATE TO" in our more usual pattern.
Discussion: https://postgr.es/m/CAMkU=1yV+egSYrzWvbDY8VZ6bKEMrKbzxr-HTuiHi+wDgSUMgA@mail.gmail.com
Amit Kapila [Tue, 21 Jun 2022 10:09:35 +0000 (15:39 +0530)]
Fix stale values in partition map entries on subscribers.
We build the partition map entries on subscribers while applying the
changes for update/delete on partitions. The component relation in each
entry is closed after its use so we need to update it on successive use of
cache entries.
This problem was there since the original commit
f1ac27bfda that
introduced this code but we didn't notice it till the recent commit
26b3455afa started to use the component relation of partition map cache
entry.
Reported-by: Tom Lane, as per buildfarm
Author: Amit Langote, Hou Zhijie
Reviewed-by: Amit Kapila, Shi Yu
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/OSZPR01MB6310F46CD425A967E4AEF736FDA49@OSZPR01MB6310.jpnprd01.prod.outlook.com
Peter Eisentraut [Tue, 21 Jun 2022 09:21:04 +0000 (11:21 +0200)]
doc: Make some synopsis punctuation consistent
Amit Kapila [Tue, 21 Jun 2022 02:37:43 +0000 (08:07 +0530)]
Fix partition table's REPLICA IDENTITY checking on the subscriber.
In logical replication, we will check if the target table on the
subscriber is updatable by comparing the replica identity of the table on
the publisher with the table on the subscriber. When the target table is a
partitioned table, we only check its replica identity but not for the
partition tables. This leads to assertion failure while applying changes
for update/delete as we expect those to succeed only when the
corresponding partition table has a primary key or has a replica
identity defined.
Fix it by checking the replica identity of the partition table while
applying changes.
Reported-by: Shi Yu
Author: Shi Yu, Hou Zhijie
Reviewed-by: Amit Langote, Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/OSZPR01MB6310F46CD425A967E4AEF736FDA49@OSZPR01MB6310.jpnprd01.prod.outlook.com
Peter Eisentraut [Mon, 20 Jun 2022 12:33:41 +0000 (14:33 +0200)]
Fix file references inside some SGML comments
In some cases, the comments mentioning their own file or another file
were not correct.
Author: Josh Soref <jsoref@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CACZqfqCyhE-voJUrUkc3GQuDakfJKx1ViiCdY7B8tQwyL6727g%40mail.gmail.com
Bruce Momjian [Sun, 19 Jun 2022 22:05:27 +0000 (18:05 -0400)]
relnotes: simplify wording of pg_basebackup compression item
Peter Eisentraut [Sun, 19 Jun 2022 15:39:50 +0000 (17:39 +0200)]
Message and documentation refinements
Tom Lane [Sat, 18 Jun 2022 23:45:38 +0000 (19:45 -0400)]
Fix busted tab completion of extension versions.
In
02b8048ba I (tgl) got rid of the need for most tab-completion queries
to return pre-quoted identifiers. But I over-hastily removed the
quote_ident call from Query_for_list_of_available_extension_versions*
too; those still need it, because what is returned isn't an identifier
at all and will (almost?) always need quoting.
Arguably we should use quote_literal here instead. But quote_ident
works too and people may be used to that behavior, so stick with it.
In passing, fix inconsistent omission of schema-qualification in
Query_for_list_of_encodings. That's not a security issue per our
current guidelines, but it ought to be like the rest.
Jeff Janes
Discussion: https://postgr.es/m/CAMkU=1yV+egSYrzWvbDY8VZ6bKEMrKbzxr-HTuiHi+wDgSUMgA@mail.gmail.com
Michael Paquier [Fri, 17 Jun 2022 00:03:07 +0000 (09:03 +0900)]
doc: Do s/int/integer/ to describe the type of some GUC parameters
Three parameters have been using "int" rather than "integer" to describe
their type:
auth_delay.milliseconds
max_logical_replication_workers
pg_prewarm.autoprewarm_interval
This is inconsistent with any other integer GUCs listed in the docs
(148, as far as I can see).
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+Pv6X5T-veN2abUDUvBxZm+SSm-9otfi3LZPGyOc6u6hiA@mail.gmail.com
Tomas Vondra [Thu, 16 Jun 2022 13:02:48 +0000 (15:02 +0200)]
Revert changes in HOT handling of BRIN indexes
This reverts commits
5753d4ee32 and
fe60b67250 that modified HOT to
ignore BRIN indexes. The commit message for
5753d4ee32 claims that:
When determining whether an index update may be skipped by using
HOT, we can ignore attributes indexed only by BRIN indexes. There
are no index pointers to individual tuples in BRIN, and the page
range summary will be updated anyway as it relies on visibility
info.
This is partially incorrect - it's true BRIN indexes don't point to
individual tuples, so HOT chains are not an issue, but the visibitlity
info is not sufficient to keep the index up to date. This can easily
result in corrupted indexes, as demonstrated in the hackers thread.
This does not mean relaxing the HOT restrictions for BRIN is a lost
cause, but it needs to handle the two aspects (allowing HOT chains and
updating the page range summaries) as separate. But that requires a
major changes, and it's too late for that in the current dev cycle.
Reported-by: Tomas Vondra
Discussion: https://postgr.es/m/
05ebcb44-f383-86e3-4f31-
0a97a55634cf@enterprisedb.com
Michael Paquier [Thu, 16 Jun 2022 08:21:04 +0000 (17:21 +0900)]
Fix comment in regression tests for large objects
The values assigned to INV_WRITE and INV_READ were reversed in the
tests, which would be confusing when writing tests specific to read or
write operations on LOs.
Author: Yugo Nagata
Discussion: https://postgr.es/m/
20220527153028.
61a4608f66abcd026fd3806f@sraoss.co.jp
Amit Kapila [Thu, 16 Jun 2022 03:15:07 +0000 (08:45 +0530)]
Fix data inconsistency between publisher and subscriber.
We were not updating the partition map cache in the subscriber even when
the corresponding remote rel is changed. Due to this data was getting
incorrectly replicated for partition tables after the publisher has
changed the table schema.
Fix it by resetting the required entries in the partition map cache after
receiving a new relation mapping from the publisher.
Reported-by: Shi Yu
Author: Shi Yu, Hou Zhijie
Reviewed-by: Amit Langote, Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/OSZPR01MB6310F46CD425A967E4AEF736FDA49@OSZPR01MB6310.jpnprd01.prod.outlook.com
Alvaro Herrera [Wed, 15 Jun 2022 14:54:38 +0000 (16:54 +0200)]
Link to MVCC docs in MERGE docs
In addition, this moves the new paragraph in the MVCC page upwards, for
a more consistent flow; some minor markup mistakes, style issues and
typos are fixed too.
Per comments from Justin Pryzby.
Discussion: https://postgr.es/m/
20220511163350.GL19626@telsasoft.com
Michael Paquier [Wed, 15 Jun 2022 06:36:16 +0000 (15:36 +0900)]
Re-add locally-generated files in pg_upgrade's .gitignore and Makefile
This reverts the changes to pg_upgrade's Makefile and .gitignore done in
15b6d21. The TAP tests run in isolation, executing pg_upgrade in
tmp_check/ in the build directory so as any files created in the
execution path (reindex_hash.sql and delete_old_cluster.{sh,bat}) are
never in the tree, so entries are not necessary in this case. However,
not having these impacts the cleanliness of the code tree when running
./pg_upgrade directly from src/bin/pg_upgrade/.
This commit adds back to .gitignore all the files generated in the
execution path, and the Makefile rule to clean them up if they exist.
Per gripe from Tom Lane.
Discussion: https://postgr.es/m/90595.
1655227384@sss.pgh.pa.us
Amit Kapila [Wed, 15 Jun 2022 04:22:12 +0000 (09:52 +0530)]
Fix cache look-up failures while applying changes in logical replication.
While building a new attrmap which maps partition attribute numbers to
remoterel's, we incorrectly update the map for dropped column attributes.
Later, it caused cache look-up failure when we tried to use the map to
fetch the information about attributes.
This also fixes the partition map cache invalidation which was using the
wrong type cast to fetch the entry. We were using stale partition map
entry after invalidation which leads to the assertion or cache look-up
failure.
Reported-by: Shi Yu
Author: Hou Zhijie, Shi Yu
Reviewed-by: Amit Langote, Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/OSZPR01MB6310F46CD425A967E4AEF736FDA49@OSZPR01MB6310.jpnprd01.prod.outlook.com
Michael Paquier [Wed, 15 Jun 2022 02:24:52 +0000 (11:24 +0900)]
Tweak behavior of psql --single-transaction depending on ON_ERROR_STOP
This commit, in completion of
157f873, forces a ROLLBACK for
--single-transaction only when ON_ERROR_STOP is used when one of the
steps defined by -f/-c fails. Hence, COMMIT is always used when
ON_ERROR_STOP is not set, ignoring the status code of the last action
taken in the set of switches specified by -c/-f (previously ROLLBACK
would have been issued even without ON_ERROR_STOP if the last step
failed, while COMMIT was issued if a step in-between failed as long as
the last step succeeded, leading to more inconsistency).
While on it, this adds much more test coverage in this area when not
using ON_ERROR_STOP with multiple switch patterns involving -c and -f
for query files, single queries and slash commands.
The behavior of ON_ERROR_STOP is arguably a bug, but there was no much
support for a backpatch to force a ROLLBACK on a step failure, so this
change is done only on HEAD for now.
Per discussion with Tom Lane and Kyotaro Horiguchi.
Discussion: https://postgr.es/m/Yqbc8bAdwnP02na4@paquier.xyz
Tom Lane [Tue, 14 Jun 2022 22:16:46 +0000 (18:16 -0400)]
Avoid ecpglib core dump with out-of-order operations.
If an application executed operations like EXEC SQL PREPARE
without having first established a database connection, it could
get a core dump instead of the expected clean failure. This
occurred because we did "pthread_getspecific(actual_connection_key)"
without ever having initialized the TSD key actual_connection_key.
The results of that are probably platform-specific, but at least
on Linux it often leads to a crash.
To fix, add calls to ecpg_pthreads_init() in the code paths that
might use actual_connection_key uninitialized. It's harmless
(and hopefully inexpensive) to do that more than once.
Per bug #17514 from Okano Naoki. The problem's ancient, so
back-patch to all supported branches.
Discussion: https://postgr.es/m/17514-
edd4fad547c5692c@postgresql.org
Tom Lane [Tue, 14 Jun 2022 21:47:09 +0000 (17:47 -0400)]
Doc: clarify the default collation behavior of domains.
The previous wording was "the underlying data type's default collation
is used", which is wrong or at least misleading. The domain inherits
the base type's collation behavior, which if "default" actually can
mean that we use some non-default collation obtained from elsewhere.
Per complaint from Jian He.
Discussion: https://postgr.es/m/CACJufxHMR8_4WooDPjjvEdaxB2hQ5a49qthci8fpKP0MKemVRQ@mail.gmail.com
Tom Lane [Mon, 13 Jun 2022 18:28:05 +0000 (14:28 -0400)]
pg_upgrade: further tweaking of make_outputdirs().
Use the same error message for all cases of pathname overrun,
since users aren't going to much care which one was too long.
Add missing newline to said error (as pg_upgrade's version
of pg_fatal requires that).
Add pathname overrun checks for the individual log files,
not just the directories.
Remove initial newline in log files; the new scheme here
guarantees that we'll never be appending to an old file.
Kyotaro Horiguchi and Tom Lane
Discussion: https://postgr.es/m/
20220613.120551.
729848632120189555.horikyota.ntt@gmail.com
Andrew Dunstan [Sun, 12 Jun 2022 13:17:17 +0000 (09:17 -0400)]
Make subscription tests pass with log_error_verbosity=verbose
Recent additions to the subscription tests check for log entries, but
fail to account for the possible presence of an SQL errror code, which
happens if log_error_verbosity is set to 'verbose'. Add this into the
regular expressions that are checked for.
Bruce Momjian [Sat, 11 Jun 2022 19:28:20 +0000 (15:28 -0400)]
relnotes: update word wrapping
Bruce Momjian [Sat, 11 Jun 2022 18:59:18 +0000 (14:59 -0400)]
relnotes: update AS OF date
Bruce Momjian [Sat, 11 Jun 2022 18:37:50 +0000 (14:37 -0400)]
relnotes: add links to the release notes, indent, word wrap
Bruce Momjian [Fri, 10 Jun 2022 17:40:50 +0000 (13:40 -0400)]
relnotes: add markup of text
Doc link additions coming
Tom Lane [Fri, 10 Jun 2022 14:35:57 +0000 (10:35 -0400)]
Un-break whole-row Vars referencing domain-over-composite types.
In commit
ec62cb0aa, I foolishly replaced ExecEvalWholeRowVar's
lookup_rowtype_tupdesc_domain call with just lookup_rowtype_tupdesc,
because I didn't see how a domain could be involved there, and
there were no regression test cases to jog my memory. But the
existing code was correct, so revert that change and add a test
case showing why it's necessary. (Note: per comment in struct
DatumTupleFields, it is correct to produce an output tuple that's
labeled with the base composite type, not the domain; hence just
blindly looking through the domain is correct here.)
Per bug #17515 from Dan Kubb. Back-patch to v11 where domains over
composites became a thing.
Discussion: https://postgr.es/m/17515-
a24737438363aca0@postgresql.org
Peter Eisentraut [Fri, 10 Jun 2022 04:05:08 +0000 (06:05 +0200)]
Fix collation of JSON_TABLE output columns
The output columns of JSON_TABLE should have the collations of their
data type. The existing implementation sets the default collation if
the type is collatable.
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://www.postgresql.org/message-id/flat/
9d75ce67-0121-5050-5bec-
bf5009db55ce%40enterprisedb.com
Etsuro Fujita [Thu, 9 Jun 2022 10:30:00 +0000 (19:30 +0900)]
Improve comments for trivial_subqueryscan().
This function can be called from mark_async_capable_plan(), a helper
function for create_append_plan(), before set_subqueryscan_references(),
to determine the triviality of a SubqueryScan that is a child of an
Append plan node, which is done before doing finalize_plan() on the
SubqueryScan (if necessary) and set_plan_references() on the subplan,
unlike when called from set_subqueryscan_references(). The reason why
this is safe wouldn't be that obvious, so add comments explaining this.
Follow-up for commit
c2bb02bc2.
Reviewed by Zhihong Yu.
Discussion: https://postgr.es/m/CAPmGK17%2BGiJBthC6va7%2B9n6t75e-M1N0U18YB2G1B%2BE5OdrNTA%40mail.gmail.com
Peter Eisentraut [Thu, 9 Jun 2022 06:49:13 +0000 (08:49 +0200)]
psql: Show notices immediately (again)
The new show-all-results feature in psql (
7844c9918) went out of its
way to show notices next to the results of the statements (in a
multi-statement string) that caused them. This also had the
consequence that notices for a single statement were not shown until
after the statement had executed, instead of right away. After some
discussion, it seems very difficult to satisfy both of these goals, so
here we are giving up on the first goal and just show the notices as
we get them. This restores the pre-
7844c9918 behavior for notices.
Reported-by: Alastair McKinley <a.mckinley@analyticsengines.com>
Author: Fabien COELHO <coelho@cri.ensmp.fr>
Discussion: https://www.postgresql.org/message-id/flat/PAXPR02MB760039506C87A2083AD85575E3DA9%40PAXPR02MB7600.eurprd02.prod.outlook.com
Tom Lane [Wed, 8 Jun 2022 17:26:18 +0000 (13:26 -0400)]
Be more careful about GucSource for internally-driven GUC settings.
The original advice for hard-wired SetConfigOption calls was to use
PGC_S_OVERRIDE, particularly for PGC_INTERNAL GUCs. However,
that's really overkill for PGC_INTERNAL GUCs, since there is no
possibility that we need to override a user-provided setting.
Instead use PGC_S_DYNAMIC_DEFAULT in most places, so that the
value will appear with source = 'default' in pg_settings and thereby
not be shown by psql's new \dconfig command. The one exception is
that when changing in_hot_standby in a hot-standby session, we still
use PGC_S_OVERRIDE, because people felt that seeing that in \dconfig
would be a good thing.
Similarly use PGC_S_DYNAMIC_DEFAULT for the auto-tune value of
wal_buffers (if possible, that is if wal_buffers wasn't explicitly
set to -1), and for the typical 2MB value of max_stack_depth.
In combination these changes remove four not-very-interesting
entries from the typical output of \dconfig, all of which people
fingered as "why is that showing up?" in the discussion thread.
Discussion: https://postgr.es/m/
3118455.
1649267333@sss.pgh.pa.us
Tom Lane [Wed, 8 Jun 2022 16:01:51 +0000 (12:01 -0400)]
Doc: copy-edit "jsonb Indexing" section.
The patch introducing jsonpath dropped a para about that between
two related examples, and didn't bother updating the introductory
sentences that it falsified. The grammar was pretty shaky as well.
Peter Eisentraut [Wed, 8 Jun 2022 13:35:44 +0000 (15:35 +0200)]
Put option listing back into alphabetical order
Michael Paquier [Wed, 8 Jun 2022 02:24:06 +0000 (11:24 +0900)]
Fix portability issue in TAP tests of psql for locales
Some locales use a comma as decimal separator (like Czech or French),
and psql's 001_basic.pl for \timing was not able to handle that
properly. This fixes the matching regexes to be able to handle both
comma and dot as possible decimal separators, as per a suggestion from
Andrew Dunstan.
psql tests were the only place with such a portability issue
(check-world passed here with a forced LANG/LANGUAGE). These tests are
new as of
c0280bc, so there is no need for a backpatch.
Reported-by: Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRBz8iQmd2aOaCLvO-rJY6vZr-h6Q0qvV0J+yb78J7uiaA@mail.gmail.com
Michael Paquier [Wed, 8 Jun 2022 01:53:01 +0000 (10:53 +0900)]
Restructure pg_upgrade output directories for better idempotence
38bfae3 has moved the contents written to files by pg_upgrade under a
new directory called pg_upgrade_output.d/ located in the new cluster's
data folder, and it used a simple structure made of two subdirectories
leading to a fixed structure: log/ and dump/. This design has made
weaker pg_upgrade on repeated calls, as we could get failures when
creating one or more of those directories, while potentially losing the
logs of a previous run (logs are retained automatically on failure, and
cleaned up on success unless --retain is specified). So a user would
need to clean up pg_upgrade_output.d/ as an extra step for any repeated
calls of pg_upgrade. The most common scenario here is --check followed
by the actual upgrade, but one could see a failure when specifying an
incorrect input argument value. Removing entirely the logs would have
the disadvantage of removing all the past information, even if --retain
was specified at some past step.
This result is annoying for a lot of users and automated upgrade flows.
So, rather than requiring a manual removal of pg_upgrade_output.d/, this
redesigns the set of output directories in a more dynamic way, based on
a suggestion from Tom Lane and Daniel Gustafsson. pg_upgrade_output.d/
is still the base path, but a second directory level is added, mostly
named after an ISO-8601-formatted timestamp (in short human-readable,
with milliseconds appended to the name to avoid any conflicts). The
logs and dumps are saved within the same subdirectories as previously,
as of log/ and dump/, but these are located inside the subdirectory
named after the timestamp.
The logs of a given run are removed only after a successful run if
--retain is not used, and pg_upgrade_output.d/ is kept if there are any
logs from a previous run. Note that previously, pg_upgrade would have
kept the logs even after a successful --check but that was inconsistent
compared to the case without --check when using --retain. The code in
charge of the removal of the output directories is now refactored into a
single routine.
Two TAP tests are added with some --check commands (one failure case and
one success case), to look after the issue fixed here. Note that the
tests had to be tweaked a bit to fit with the new directory structure so
as it can find any logs generated on failure. This is still going to
require a change in the buildfarm client for the case where pg_upgrade
is tested without the TAP test, though, but I'll tackle that with a
separate patch where needed.
Reported-by: Tushar Ahuja
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson, Justin Pryzby
Discussion: https://postgr.es/m/
77e6ecaa-2785-97aa-f229-
4b6e047cbd2b@enterprisedb.com
David Rowley [Wed, 8 Jun 2022 00:39:09 +0000 (12:39 +1200)]
Harden Memoization code against broken data types
Bug #17512 highlighted that a suitably broken data type could cause the
backend to crash if either the hash function or equality function were in
someway non-deterministic based on their input values. Such a data type
could cause a crash of the backend due to some code which assumes that
we'll always find a hash table entry corresponding to an item in the
Memoize LRU list.
Here we remove the assumption that we'll always find the entry
corresponding to the given LRU list item and add run-time checks to verify
we have found the given item in the cache.
This is not a fix for bug #17512, but it will turn the crash reported by
that bug report into an internal ERROR.
Reported-by: Ales Zeleny
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/CAApHDvpxFSTwvoYWT7kmFVSZ9zLAeHb=S9vrz=RExMgSkQNWqw@mail.gmail.com
Backpatch-through: 14, where Memoize was added.
Tom Lane [Tue, 7 Jun 2022 19:34:30 +0000 (15:34 -0400)]
Fix off-by-one loop termination condition in pg_stat_get_subscription().
pg_stat_get_subscription scanned one more LogicalRepWorker array entry
than is really allocated. In the worst case this could lead to SIGSEGV,
if the LogicalRepCtx data structure is near the end of shared memory.
That seems quite unlikely though (thanks to the ordering of calls in
CreateSharedMemoryAndSemaphores) and we've heard no field reports of it.
A more likely misbehavior is one row of garbage data in the function's
result, but even that is not real likely because of the check that the
pid field matches some live backend.
Report and fix by Kuntal Ghosh. This bug is old, so back-patch
to all supported branches.
Discussion: https://postgr.es/m/CAGz5QCJykEDzW6jQK6Yz7Qh_PMtD=95de_7QoocbVR2Qy8hWZA@mail.gmail.com
Tom Lane [Mon, 6 Jun 2022 15:26:57 +0000 (11:26 -0400)]
Don't fail on libpq-generated error reports in pg_amcheck.
An error PGresult generated by libpq itself, such as a report of
connection loss, won't have broken-down error fields.
should_processing_continue() blithely assumed that
PG_DIAG_SEVERITY_NONLOCALIZED would always be present, and would
dump core if it wasn't.
Per grepping to see if
6d157e7cb's mistake was repeated elsewhere.
Tom Lane [Mon, 6 Jun 2022 15:20:21 +0000 (11:20 -0400)]
Don't fail on libpq-generated error reports in ecpg_raise_backend().
An error PGresult generated by libpq itself, such as a report of
connection loss, won't have broken-down error fields.
ecpg_raise_backend() blithely assumed that PG_DIAG_MESSAGE_PRIMARY
would always be present, and would end up passing a NULL string
pointer to snprintf when it isn't. That would typically crash
before
3779ac62d, and it would fail to provide a useful error report
in any case. Best practice is to substitute PQerrorMessage(conn)
in such cases, so do that.
Per bug #17421 from Masayuki Hirose. Back-patch to all supported
branches.
Discussion: https://postgr.es/m/17421-
790ff887e3188874@postgresql.org
Michael Paquier [Mon, 6 Jun 2022 02:05:59 +0000 (11:05 +0900)]
Fix psql's single transaction mode on client-side errors with -c/-f switches
psql --single-transaction is able to handle multiple -c and -f switches
in a single transaction since
d5563d7d, but this had the surprising
behavior of forcing a transaction COMMIT even if psql failed with an
error in the client (for example incorrect path given to \copy), which
would generate an error, but still commit any changes that were already
applied in the backend. This commit makes the behavior more consistent,
by enforcing a transaction ROLLBACK if any commands fail, both
client-side and backend-side, so as no changes are applied if one error
happens in any of them.
Some tests are added on HEAD to provide some coverage about all that.
Backend-side errors are unreliable as IPC::Run can complain on SIGPIPE
if psql quits before reading a query result, but that should work
properly in the case where any errors come from psql itself, which is
what the original report is about.
Reported-by: Christoph Berg
Author: Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/17504-
76b68018e130415e@postgresql.org
Backpatch-through: 10
Tom Lane [Sat, 4 Jun 2022 15:54:05 +0000 (11:54 -0400)]
Automatically count the number of output lines in psql/help.c.
The hard-wired PageOutput arguments in usage() and sibling functions
have been a perennial maintenance gotcha, and there's no reason to
think we'll ever get any better about that. Let's get rid of those
magic constants by constructing the output in a buffer where we can
count the newlines before calling PageOutput. (Perhaps this is
microscopically slower; but none of these functions are performance
critical, and anyway we might well be buying back all the cost by
avoiding having to pass most of the data through snprintf.c. I could
not detect any speed difference in a desultory check.) This also
gets rid of the need to assume that platform-specific variations in
the output are insignificant.
While at it, make the code shorter and more abstract by inventing
helper macros HELP0() and HELPN() to encapsulate the specific
output actions being invoked.
Discussion: https://postgr.es/m/365160.
1654289490@sss.pgh.pa.us
Michael Paquier [Sat, 4 Jun 2022 03:16:52 +0000 (12:16 +0900)]
Force run of pg_upgrade in the build directory in its TAP test
TAP tests are run from their own directory in the source tree, and in a
VPATH build the execution of the pg_upgrade command was leaving behind a
file in the source tree, that should be left untouched. In order to
avoid this issue, the test moves to PostgreSQL::Test::Utils::tmp_check,
so as any files generated by pg_upgrade do not impact the source tree,
but the build tree. This has as nice side-effect to make unnessary the
presence of such files in pg_upgrade's .gitignore and Makefile. This
strategy is similar to psql's test 010_tab_completion.pl, though the
reasons behind this choice are different.
In passing, fix one misleading test name that was added by
99f6f19.
Per discussion with Peter Eisentraut, Andrew Dunstan, Tom Lane, Andres
Freund and myself.
Discussion: https://postgr.es/m/
f80ace33-11fb-1cd3-20f8-
98f51d151088@enterprisedb.com
Tom Lane [Fri, 3 Jun 2022 19:49:36 +0000 (15:49 -0400)]
Improve psql \?'s description of large-object-related commands.
Provide a gloss of which command does what, as all other backslash
commands have. Put the large-object command section into a more
considered spot in the list.
In passing, update the output-lines count in helpVariables()
(oversight in
7844c9918, looks like).
Thibaud Walkowiak, reviewed by Nathan Bossart and myself
Discussion: https://postgr.es/m/
43f0439c-df3e-a045-ac99-
af33523cc2d4@dalibo.com
Tom Lane [Fri, 3 Jun 2022 17:54:53 +0000 (13:54 -0400)]
Doc: improve example for intarray's uniq() function.
The previous entry invited confusion between what uniq() does
by itself and what it does when combined with sort(). The latter
usage is pretty useful so we should show it, but add an additional
example to clarify the results of uniq() alone.
Per suggestion from Martin Kalcher. Back-patch to v13, where
we switched to formatting that supports multiple examples.
Discussion: https://postgr.es/m/
165407884456.573551.
8779012279828726162@wrigleys.postgresql.org