Tom Lane [Thu, 12 May 2022 16:42:29 +0000 (12:42 -0400)]
libpq: drop pending pipelined commands in pqDropConnection().
The original coding did this in pqDropServerData(), which seems
fairly backwards. Pending commands are more like already-queued
output data, which is dropped in pqDropConnection(). Moving the
operation means that we clear the command queue immediately upon
detecting connection drop, which improves the sanity of subsequent
behavior. In particular this eliminates duplicated error message
text as a consequence of code added in
b15f25446, which supposed
that a nonempty command queue must mean the prior operation is
still active.
There might be an argument for backpatching this to v14; but as with
b15f25446, I'm unsure about interactions with
618c16707. For now,
given the lack of complaints about v14's behavior, leave it alone.
Per report from Peter Eisentraut.
Discussion: https://postgr.es/m/
de57761c-b99b-3435-b0a6-
474c72b1149a@enterprisedb.com
Andres Freund [Thu, 12 May 2022 16:10:03 +0000 (09:10 -0700)]
Mark a few 'bbsink' related functions / variables static.
Discussion: https://postgr.es/m/
20220506234924.6mxxotl3xl63db3l@alap3.anarazel.de
Tom Lane [Thu, 12 May 2022 15:31:46 +0000 (11:31 -0400)]
Make pull_var_clause() handle GroupingFuncs exactly like Aggrefs.
This follows in the footsteps of commit
2591ee8ec by removing one more
ill-advised shortcut from planning of GroupingFuncs. It's true that
we don't intend to execute the argument expression(s) at runtime, but
we still have to process any Vars appearing within them, or we risk
failure at setrefs.c time (or more fundamentally, in EXPLAIN trying
to print such an expression). Vars in upper plan nodes have to have
referents in the next plan level, whether we ever execute 'em or not.
Per bug #17479 from Michael J. Sullivan. Back-patch to all supported
branches.
Richard Guo
Discussion: https://postgr.es/m/17479-
6260deceaf0ad304@postgresql.org
Bruce Momjian [Thu, 12 May 2022 14:11:55 +0000 (10:11 -0400)]
relnotes: adjust several logical replication items and FK text
Reported-by: Amit Langote, Amit Kapila, Takamichi Osumi
Discussion: https://postgr.es/m/CAA4eK1KdAho_fNkYRe=o=XpxWs_TG4O0V8CJBAvTG8HamWt6OQ@mail.gmail.com
Etsuro Fujita [Thu, 12 May 2022 08:30:00 +0000 (17:30 +0900)]
postgres_fdw: Update comments in make_new_connection().
Expand the comment about the parallel_commit option to mention that the
default is false.
Also, since the comment about alteration of the keep_connections option,
which was located above the expanded comment, holds true for the
parallel_commit option, rewrite it to reflect this, and move it to after
the expanded comment.
Follow-up for commit
04e706d42.
Discussion: https://postgr.es/m/CAPmGK16Kg2Bf90sqzcZ4YM5cN_G-4h7wFUS01qQpqNB%2B2BG5_w%40mail.gmail.com
Michael Paquier [Thu, 12 May 2022 06:08:45 +0000 (15:08 +0900)]
Add some missing PGDLLIMPORT markings
Three variables in pqsignal.h (UnBlockSig, BlockSig and StartupBlockSig)
were not marked with PGDLLIMPORT, as they are declared in a way that
prevents mark_pgdllimport.pl to detect them. These variables are
redefined in a style more consistent with the other headers, allowing
the script to find and mark them.
PGDLLIMPORT was missing for __pg_log_level in logging.h, so add it
back. The marking got accidentally removed in
9a374b77, just after its
addition in
8ec5694.
While on it, add a comment in mark_pgdllimport.pl explaining what are
the arguments needed by the script (aka a list of header paths).
Reported-by: Andres Freund
Discussion: https://postgr.es/m/
20220506234924.6mxxotl3xl63db3l@alap3.anarazel.de
Bruce Momjian [Thu, 12 May 2022 02:46:36 +0000 (22:46 -0400)]
relnotes: mention non-exclusive backup mode was deprecated
Reported-by: Ian Lawrence Barwick
Discussion: https://postgr.es/m/CAB8KJ=j9VhsFXuN+xv5qzxQV7gj-6TKqqwVLSgz5zU7F8nHCYw@mail.gmail.com
John Naylor [Thu, 12 May 2022 02:18:21 +0000 (09:18 +0700)]
Use correct datum macros in more tuplesort specialization functions.
Also clarify that ApplySignedSortComparator() is not built on 32-bit machines.
Folow-up to
c90c16591
Reviewed-by: David Rowley
Discussion: https://www.postgresql.org/message-id/CAFBsxsFmt4_JUP8XgSJqwaAS9a9s8K8_PvMu%3Dj%3DDfwU%3D8QjNPw%40mail.gmail.com
Bruce Momjian [Thu, 12 May 2022 02:19:12 +0000 (22:19 -0400)]
relnotes: add author to in-memory sorts item
Reported-by: Thomas Munro
Discussion: https://postgr.es/m/CA+hUKGKha50-H2ye-9x3PJWU5wBq_MPc2rmVKM1xJ0tL+ntROw@mail.gmail.com
Bruce Momjian [Thu, 12 May 2022 02:01:46 +0000 (22:01 -0400)]
relnotes: update for non-exclusive backup mode removal
Reported-by: Ian Lawrence Barwick
Discussion: https://postgr.es/m/CAB8KJ=jmLBMAmxjcew12-yntb7m3FH-8A2+Qb7+RybZp1w9GOg@mail.gmail.com
Bruce Momjian [Thu, 12 May 2022 00:55:01 +0000 (20:55 -0400)]
relnote: improve sorting entries
Reported-by: David Rowley
Discussion: https://postgr.es/m/CAApHDvrToHFvL-W03vSi6T50uScyhrdUGzTtcuLAjdjtRF8FLA@mail.gmail.com
Robert Haas [Wed, 11 May 2022 19:27:33 +0000 (15:27 -0400)]
Remove non-functional code for unloading loadable modules.
The code for unloading a library has been commented-out for over 12
years, ever since commit
602a9ef5a7c60151e10293ae3c4bb3fbb0132d03, and we're
no closer to supporting it now than we were back then.
Nathan Bossart, reviewed by Michael Paquier and by me.
Discussion: http://postgr.es/m/Ynsc9bRL1caUSBSE@paquier.xyz
Bruce Momjian [Wed, 11 May 2022 15:28:29 +0000 (11:28 -0400)]
relnotes: adjustments from Álvaro Herrera
Reported-by: Álvaro Herrera
Discussion: https://postgr.es/m/
202205111412.n7xgu3eppins@alvherre.pgsql
Bruce Momjian [Wed, 11 May 2022 14:45:17 +0000 (10:45 -0400)]
relnotes: update foreign key partition and add sort items
Reported-by: David Rowley, Amit Langote
Discussion: https://postgr.es/m/CA+HiwqFcEdbNinfK94aTgBJKmtkYGdtkPnjt3b0fVzKi+WZ+KA@mail.gmail.com
Michael Paquier [Wed, 11 May 2022 06:38:55 +0000 (15:38 +0900)]
Fix typos and grammar in code and test comments
This fixes the grammar of some comments in a couple of tests (SQL and
TAP), and in some C files.
Author: Justin Pryzby
Discussion: https://postgr.es/m/
20220511020334.GH19626@telsasoft.com
Thomas Munro [Wed, 11 May 2022 06:03:03 +0000 (18:03 +1200)]
Add logging for excessive ProcSignalBarrier waits.
To enable diagnosis of systems that are not processing ProcSignalBarrier
requests promptly, add a LOG message every 5 seconds if we seem to be
wedged. Although you could already see this state as a wait event in
pg_stat_activity, the log message also shows the PID of the process that
is preventing progress.
Also add DEBUG1 logging around the whole wait loop.
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/CA%2BTgmoYJ03r5359gQutRGP9BtigYCg3_UskcmnVjBf-QO3-0pQ%40mail.gmail.com
Amit Kapila [Wed, 11 May 2022 05:41:44 +0000 (11:11 +0530)]
Fix the logical replication timeout during large transactions.
The problem is that we don't send keep-alive messages for a long time
while processing large transactions during logical replication where we
don't send any data of such transactions. This can happen when the table
modified in the transaction is not published or because all the changes
got filtered. We do try to send the keep_alive if necessary at the end of
the transaction (via WalSndWriteData()) but by that time the
subscriber-side can timeout and exit.
To fix this we try to send the keepalive message if required after
processing certain threshold of changes.
Reported-by: Fabrice Chapuis
Author: Wang wei and Amit Kapila
Reviewed By: Masahiko Sawada, Euler Taveira, Hou Zhijie, Hayato Kuroda
Backpatch-through: 10
Discussion: https://postgr.es/m/CAA5-nLARN7-3SLU_QUxfy510pmrYK6JJb=bk3hcgemAM_pAv+w@mail.gmail.com
Michael Paquier [Wed, 11 May 2022 05:21:06 +0000 (14:21 +0900)]
Silence extra logging when using "postgres -C" on runtime-computed GUCs
Presently, the server may emit a variety of log messages when inspecting
a runtime-computed GUC, mostly in the shape of one LOG message with the
default configuration, related to the startup sequence launched as such
GUCs require a load of the control file and of external shared
libraries.
For example, the server will always emit a "database system is shut
down" LOG (unless the user has set log_min_messages higher than LOG),
which is an annoying behavior as "postgres -C" is expected to only emit
in its output the parameter value we are looking for. The parameter
value is sent to stdout, while the logs are sent to stderr so we could
recommend to use a redirection, but there was not much love for this
workaround either.
To avoid such extra log messages, per discussion, this change sets
log_min_messages to FATAL internally when -C is used on a
runtime-computed GUC (even if set to PANIC in postgresql.conf). At
FATAL, the user will still receive messages explaining why a GUC value
cannot be inspected, and will know if the command is attempted on a
server already running, something not supported yet for a
runtime-computed GUC.
Reported-by: Magnus Hagander, Bruce Momjian
Author: Nathan Bossart, Michael Paquier
Discussion: https://postgr.es/m/Yni6ZHkGotUU+RSf@paquier.xyz
Peter Eisentraut [Wed, 11 May 2022 04:16:21 +0000 (06:16 +0200)]
Add missing source files to nls.mk
Bruce Momjian [Wed, 11 May 2022 03:41:26 +0000 (23:41 -0400)]
relnotes: more adjustments
Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/
20220511013117.GG19626@telsasoft.com
Bruce Momjian [Wed, 11 May 2022 02:10:45 +0000 (22:10 -0400)]
relnotes: logical replication permissions checked by subscrib.
Reported-by: Mark Dilger
Discussion: https://postgr.es/m/
F3D44160-F41A-4F2E-AED5-
5C15B6903AC4@enterprisedb.com
Bruce Momjian [Wed, 11 May 2022 01:57:11 +0000 (21:57 -0400)]
relnotes: adjustments
Remove SSL item backpatched (Jonathan S. Katz), adjust logical
replication item (Mark Dilger), adjust --compress wording (Justin
Pryzby).
Discussion: https://postgr.es/m/
20220511012854.GF19626@telsasoft.com
Michael Paquier [Wed, 11 May 2022 01:21:52 +0000 (10:21 +0900)]
Improve setup of environment values for commands in MSVC's vcregress.pl
The current setup assumes that commands for lz4, zstd and gzip always
exist by default if not enforced by a user's environment. However,
vcpkg, as one example, installs libraries but no binaries, so this
default setup to assume that a command should always be present would
cause failures. This commit improves the detection of such external
commands as follows:
* If a ENV value is available, trust the environment/user and use it.
* If a ENV value is not available, check its execution by looking in the
current PATH, by launching a simple "$command --version" (that should be
portable enough).
** On execution failure, ignore ENV{command}.
** On execution success, set ENV{command} = "$command".
Note that this new rule applies to gzip, lz4 and zstd but not tar that
we assume will always exist. Those commands are set up in the
environment only when using bincheck and taptest. The CI includes all
those commands and I have checked that their setup is correct there. I
have also tested this change in a MSVC environment where we have none of
those commands.
While on it, remove the references to lz4 from the documentation and
vcregress.pl in ~v13. --with-lz4 has been added in v14~ so there is no
point to have this information in these older branches.
Reported-by: Andrew Dunstan
Reviewed-by: Andrew Dunstan
Discussion: https://postgr.es/m/
14402151-376b-a57a-6d0c-
10ad12608e12@dunslane.net
Backpatch-through: 10
David Rowley [Tue, 10 May 2022 23:38:13 +0000 (11:38 +1200)]
Fix some incorrect preprocessor tests in tuplesort specializations
697492434 added 3 new quicksort specialization functions for common
datatypes.
That commit was not very consistent in how it would determine if we're
compiling for 32-bit or 64-bit machines. It would sometimes use
USE_FLOAT8_BYVAL and at other times check if SIZEOF_DATUM == 8. This
could cause theoretical problems due to the way USE_FLOAT8_BYVAL is now
defined based on SIZEOF_VOID_P >= 8. If pointers for some reason were
ever larger than 8-bytes then we'd end up doing 32-bit comparisons
mistakenly. Let's just always check SIZEOF_DATUM >= 8.
It also seems that ssup_datum_signed_cmp is just never used on 32-bit
builds, so let's just ifdef that out to make sure we never accidentally
use that comparison function on such machines. This also allows us to
ifdef out 1 of the 3 new specialization quicksort functions in 32-bit
builds which seems to shrink down the binary by over 4KB on my machine.
In passing, also add the missing DatumGetInt32() / DatumGetInt64() macros
in the comparison functions.
Discussion: https://postgr.es/m/CAApHDvqcQExRhtRa9hJrJB_5egs3SUfOcutP3m+3HO8A+fZTPA@mail.gmail.com
Reviewed-by: John Naylor
Tom Lane [Tue, 10 May 2022 22:42:02 +0000 (18:42 -0400)]
configure: don't probe for libldap_r if libldap is 2.5 or newer.
In OpenLDAP 2.5 and later, libldap itself is always thread-safe and
there's never a libldap_r. Our existing coding dealt with that
by assuming it wouldn't find libldap_r if libldap is thread-safe.
But that rule fails to cope if there are multiple OpenLDAP versions
visible, as is likely to be the case on macOS in particular. We'd
end up using shiny new libldap in the backend and a hoary libldap_r
in libpq.
Instead, once we've found libldap, check if it's >= 2.5 (by
probing for a function introduced then) and don't bother looking
for libldap_r if so. While one can imagine library setups that
this'd still give the wrong answer for, they seem unlikely to
occur in practice.
Per report from Peter Eisentraut. Back-patch to all supported branches.
Discussion: https://postgr.es/m/
fedacd7c-2a38-25c9-e7ff-
dea549d0e979@enterprisedb.com
Bruce Momjian [Tue, 10 May 2022 21:49:51 +0000 (17:49 -0400)]
relnotes: adjust sections for various items
Also improve postgres_fdw.application_name
Reported-by: Justin Pryzby, Tatsuo Ishii
Diagnosed-by: 20220510210235.GD19626@telsasoft.com
Bruce Momjian [Tue, 10 May 2022 20:47:48 +0000 (16:47 -0400)]
relnote: add ARM64 spinlock item
Reported-by: Jonathan Katz
Discussion: per private email
Bruce Momjian [Tue, 10 May 2022 20:34:11 +0000 (16:34 -0400)]
relnotes: remove sequence replication and update 'postgres -C'
Reported-by: Nathan Bossart, Jonathan Katz
Discussion: https://postgr.es/m/
20220510194456.GA3716556@nathanxps13
Bruce Momjian [Tue, 10 May 2022 20:05:12 +0000 (16:05 -0400)]
relnote: extensive updates
Reported-by: Erik Rijkers, Justin Pryzby
Discussion: https://postgr.es/m/
20220510180935.GB19626@telsasoft.com
Peter Eisentraut [Tue, 10 May 2022 19:15:56 +0000 (21:15 +0200)]
Formatting and punctuation improvements in sample configuration files
Peter Eisentraut [Tue, 10 May 2022 18:57:37 +0000 (20:57 +0200)]
Remove some tabs in SQL code in C string literals
This is not handled uniformly throughout the code, but at least nearby
code can be consistent.
Peter Eisentraut [Tue, 10 May 2022 18:31:13 +0000 (20:31 +0200)]
doc: Update SQL keywords for SQL:2016 technical corrigenda
Several keywords were missing or misclassified in the original
SQL:2016 standard. This has been corrected in later technical
corrigenda. This change updates the PostgreSQL documentation
accordingly.
This also fixes a mistake in
606948b058d: The keywords JSON_SCALAR and
JSON_SERIALIZE added there are not from SQL:2016 but from future
SQL:202x, so they don't belong in that list yet.
(
606948b058d also added JSON to the reserved list, which is what the
corrigendum also does, but failed to remove it from the nonreserved
list.)
Bruce Momjian [Tue, 10 May 2022 16:51:25 +0000 (12:51 -0400)]
relnotes: "training" -> "trailing"
Reported-by: Geoff Winkless
Discussion: https://postgr.es/m/CAEzk6fdF_J4jqicLz=FZ6R1u2EjpEtmzD55tFjVbxw-O-kR1=w@mail.gmail.com
Bruce Momjian [Tue, 10 May 2022 15:35:28 +0000 (11:35 -0400)]
doc: first draft of PG 15 release notes
Michael Paquier [Tue, 10 May 2022 02:31:31 +0000 (11:31 +0900)]
Fix several issues with the TAP tests of pg_upgrade
This commit addresses the following issues in the TAP tests of
pg_upgrade, introduced in
322becb:
- Remove --port and --host for commands that already rely on a node's
environment PGHOST and PGPORT.
- Switch from run_log() to command_ok(), as all the commands executed in
the tests should succeed.
- Change EXTRA_REGRESS_OPTS to make it count as a shell fragment (fixing
s/OPT/OPTS on a way), to be compatible with the various Makefiles using
it as well as 027_stream_regress.pl in the recovery tests. The command
built for the execution the pg_regress command is reformatted, while on
it, to map with the recovery test doing the same thing (we should
refactor and consolidate that in the future, perhaps).
- Re-add the test for database names stressing the behavior of
backslashes with double quotes, mostly here for Windows.
Tests doable with the upgrade across different major versions still work
the same way.
Reported-by: Noah Misch
Discussion: https://postgr.es/m/
20220502042718.GB1565149@rfd.leadboat.com
Tom Lane [Mon, 9 May 2022 18:15:37 +0000 (14:15 -0400)]
Fix core dump in transformValuesClause when there are no columns.
The parser code that transformed VALUES from row-oriented to
column-oriented lists failed if there were zero columns.
You can't write that straightforwardly (though probably you
should be able to), but the case can be reached by expanding
a "tab.*" reference to a zero-column table.
Per bug #17477 from Wang Ke. Back-patch to all supported branches.
Discussion: https://postgr.es/m/17477-
0af3c6ac6b0a6ae0@postgresql.org
Tom Lane [Mon, 9 May 2022 15:02:36 +0000 (11:02 -0400)]
Revert "Disallow infinite endpoints in generate_series() for timestamps."
This reverts commit
eafdf9de06e9b60168f5e47cedcfceecdc6d4b5f
and its back-branch counterparts. Corey Huinker pointed out that
we'd discussed this exact change back in 2016 and rejected it,
on the grounds that there's at least one usage pattern with LIMIT
where an infinite endpoint can usefully be used. Perhaps that
argument needs to be re-litigated, but there's no time left before
our back-branch releases. To keep our options open, restore the
status quo ante; if we do end up deciding to change things, waiting
one more quarter won't hurt anything.
Rather than just doing a straight revert, I added a new test case
demonstrating the usage with LIMIT. That'll at least remind us of
the issue if we forget again.
Discussion: https://postgr.es/m/
3603504.
1652068977@sss.pgh.pa.us
Discussion: https://postgr.es/m/CADkLM=dzw0Pvdqp5yWKxMd+VmNkAMhG=4ku7GnCZxebWnzmz3Q@mail.gmail.com
Noah Misch [Mon, 9 May 2022 15:35:08 +0000 (08:35 -0700)]
In REFRESH MATERIALIZED VIEW, set user ID before running user code.
It intended to, but did not, achieve this. Adopt the new standard of
setting user ID just after locking the relation. Back-patch to v10 (all
supported versions).
Reviewed by Simon Riggs. Reported by Alvaro Herrera.
Security: CVE-2022-1552
Noah Misch [Mon, 9 May 2022 15:35:08 +0000 (08:35 -0700)]
Make relation-enumerating operations be security-restricted operations.
When a feature enumerates relations and runs functions associated with
all found relations, the feature's user shall not need to trust every
user having permission to create objects. BRIN-specific functionality
in autovacuum neglected to account for this, as did pg_amcheck and
CLUSTER. An attacker having permission to create non-temp objects in at
least one schema could execute arbitrary SQL functions under the
identity of the bootstrap superuser. CREATE INDEX (not a
relation-enumerating operation) and REINDEX protected themselves too
late. This change extends to the non-enumerating amcheck interface.
Back-patch to v10 (all supported versions).
Sergey Shinderuk, reviewed (in earlier versions) by Alexander Lakhin.
Reported by Alexander Lakhin.
Security: CVE-2022-1552
Peter Eisentraut [Mon, 9 May 2022 05:17:08 +0000 (07:17 +0200)]
Add missing source files to nls.mk
Michael Paquier [Sun, 8 May 2022 23:39:59 +0000 (08:39 +0900)]
Fix control file update done in restartpoints still running after promotion
If a cluster is promoted (aka the control file shows a state different
than DB_IN_ARCHIVE_RECOVERY) while CreateRestartPoint() is still
processing, this function could miss an update of the control file for
"checkPoint" and "checkPointCopy" but still do the recycling and/or
removal of the past WAL segments, assuming that the to-be-updated LSN
values should be used as reference points for the cleanup. This causes
a follow-up restart attempting crash recovery to fail with a PANIC on a
missing checkpoint record if the end-of-recovery checkpoint triggered by
the promotion did not complete while the cluster abruptly stopped or
crashed before the completion of this checkpoint. The PANIC would be
caused by the redo LSN referred in the control file as located in a
segment already gone, recycled by the previous restartpoint with
"checkPoint" out-of-sync in the control file.
This commit fixes the update of the control file during restartpoints so
as "checkPoint" and "checkPointCopy" are updated even if the cluster has
been promoted while a restartpoint is running, to be on par with the set
of WAL segments actually recycled in the end of CreateRestartPoint().
This problem exists in all the stable branches. However, commit
7ff23c6, by removing the last call of CreateCheckPoint() from the
startup process, has made this bug much easier to reason about as
concurrent checkpoints are not possible anymore. No backpatch is done
yet, mostly out of caution from me as a point release is close by, but
we need to think harder about the case of concurrent checkpoints at
promotion if the bgwriter is not considered as running by the startup
process in ~v14, so this change is done only on HEAD for the moment.
Reported-by: Fujii Masao, Rui Zhao
Author: Kyotaro Horiguchi
Reviewed-by: Nathan Bossart, Michael Paquier
Discussion: https://postgr.es/m/
20220316.102444.
2193181487576617583.horikyota.ntt@gmail.com
Thomas Munro [Sun, 8 May 2022 04:54:09 +0000 (16:54 +1200)]
Fix race in 032_relfilenode_reuse.pl.
Add wait_for_catchup() call to the test added by commit
e2f65f42. Per
slow build farm animal grison.
Also fix a comment.
Discussion: https://postgr.es/m/CA%2BhUKGLJ2Vy8hVQmnYotmTaEKZK0%3D-GcXgNAgcHzArZvtS4L_g%40mail.gmail.com
Noah Misch [Sat, 7 May 2022 07:33:15 +0000 (00:33 -0700)]
Under has_wal_read_bug, skip contrib/bloom/t/001_wal.pl.
Per buildfarm members snapper and kittiwake. Back-patch to v10 (all
supported versions).
Discussion: https://postgr.es/m/
20220116210241.GC756210@rfd.leadboat.com
Thomas Munro [Sat, 7 May 2022 03:19:52 +0000 (15:19 +1200)]
Fix old-fd issues using global barriers everywhere.
Commits
4eb21763 and
b74e94dc introduced a way to force every backend to
close all relation files, to fix an ancient Windows-only bug.
This commit extends that behavior to all operating systems and adds
a couple of extra barrier points, to fix a totally different class of
bug: the reuse of relfilenodes in scenarios that have no other kind of
cache invalidation to prevent file descriptor mix-ups.
In all releases, data corruption could occur when you moved a database
to another tablespace and then back again. Despite that, no back-patch
for now as the infrastructure required is too new and invasive. In
master only, since commit
aa010514, it could also happen when using
CREATE DATABASE with a user-supplied OID or via pg_upgrade.
Author: Andres Freund <andres@anarazel.de>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/
20220209220004.kb3dgtn2x2k2gtdm%40alap3.anarazel.de
Thomas Munro [Sat, 7 May 2022 04:19:42 +0000 (16:19 +1200)]
Rethink PROCSIGNAL_BARRIER_SMGRRELEASE.
With sufficiently bad luck, it was possible for IssuePendingWritebacks()
to reopen a file after we'd processed PROCSIGNAL_BARRIER_SMGRRELEASE and
before the file was unlinked by some other backend. That left a small
hole in commit
4eb21763's plan to fix all spurious errors from DROP
TABLESPACE and similar on Windows.
Fix by closing md.c's segments, instead of just closing fd.c's
descriptors, and then teaching smgrwriteback() not to open files that
aren't already open.
Reported-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/
20220209220004.kb3dgtn2x2k2gtdm%40alap3.anarazel.de
Robert Haas [Fri, 6 May 2022 13:24:06 +0000 (09:24 -0400)]
Fix misleading comments about background worker registration.
Since
6bc8ef0b7f1f1df3998745a66e1790e27424aa0c, the maximum number
of backends can't change as background workers are registered, but
these comments still reflect the way things worked prior to that.
Also, per recent discussion, some modules call SetConfigOption()
from _PG_init(). It's not entirely clear to me whether we want to
regard that as a fully supported operation, but since we know it's
a thing that happens, it at least deserves a mention in the comments,
so add that.
Nathan Bossart, reviewed by Anton A. Melnikov
Discussion: http://postgr.es/m/
20220419154658.GA2487941@nathanxps13
Daniel Gustafsson [Fri, 6 May 2022 12:41:36 +0000 (14:41 +0200)]
pgcrypto: remove questionmark from error message
The PXE_CIPHER_INIT error is used to report initialization errors, so
appending a questionmark to the error isn't entirely accurate (using a
space before the questionmark doubly so).
Discussion: https://postgr.es/m/
C89D932C-501E-4473-9750-
638CFCD9095E@yesql.se
Daniel Gustafsson [Fri, 6 May 2022 12:41:33 +0000 (14:41 +0200)]
pgcrypto: report init errors as PXE_CIPHER_INIT
Report OpenSSL errors during initialization as PXE_CIPHER_INIT since
that's just what they were, and not generic unknown errors. This also
removes the last users of the generic error, and thus it can be removed.
Discussion: http://postgr.es/m/
C89D932C-501E-4473-9750-
638CFCD9095E@yesql.se
Daniel Gustafsson [Fri, 6 May 2022 12:41:31 +0000 (14:41 +0200)]
Clear the OpenSSL error queue before cryptohash operations
Setting up an EVP context for ciphers banned under FIPS generate
two OpenSSL errors in the queue, and as we only consume one from
the queue the other is at the head for the next invocation:
postgres=# select md5('foo');
ERROR: could not compute MD5 hash: unsupported
postgres=# select md5('foo');
ERROR: could not compute MD5 hash: initialization error
Clearing the error queue when creating the context ensures that
we don't pull in an error from an earlier operation.
Discussion: https://postgr.es/m/
C89D932C-501E-4473-9750-
638CFCD9095E@yesql.se
Michael Paquier [Fri, 6 May 2022 11:01:15 +0000 (20:01 +0900)]
Fix typo in origin.c
Introduced in
5aa2350.
Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+PsuWz6_7aCmivNU5FahgQxDUTQtc3+__XnWkBzQcfn43w@mail.gmail.com
Peter Eisentraut [Fri, 6 May 2022 07:17:38 +0000 (09:17 +0200)]
Update SQL features
Update a few items that have become supported or mostly supported but
weren't updated at the time.
Peter Eisentraut [Fri, 6 May 2022 07:14:15 +0000 (09:14 +0200)]
Fix some whitespace in documentation markup
Peter Eisentraut [Fri, 6 May 2022 07:07:14 +0000 (09:07 +0200)]
Tom Lane [Thu, 5 May 2022 18:54:53 +0000 (14:54 -0400)]
Update time zone data files to tzdata release 2022a.
DST law changes in Palestine. Historical corrections for
Chile and Ukraine.
Andres Freund [Wed, 4 May 2022 19:50:38 +0000 (12:50 -0700)]
Fix timing issue in deadlock recovery conflict test.
Per buildfarm members longfin and skink.
Discussion: https://postgr.es/m/
20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de
Backpatch: 10-
Tom Lane [Wed, 4 May 2022 18:44:40 +0000 (14:44 -0400)]
Fix rowcount estimate for SubqueryScan that's under a Gather.
SubqueryScan was always getting labeled with a rowcount estimate
appropriate for non-parallel cases. However, nodes that are
underneath a Gather should be treated as processing only one
worker's share of the rows, whether the particular node is explicitly
parallel-aware or not. Most non-scan-level node types get this
right automatically because they base their rowcount estimate on
that of their input sub-Path(s). But SubqueryScan didn't do that,
instead using the whole-relation rowcount estimate as if it were
a non-parallel-aware scan node. If there is a parallel-aware node
below the SubqueryScan, this is wrong, and it results in inflating
the cost estimates for nodes above the SubqueryScan, which can cause
us to not choose a parallel plan, or choose a silly one --- as indeed
is visible in the one regression test whose results change with this
patch. (Although that plan tree appears to contain no SubqueryScans,
there were some in it before setrefs.c deleted them.)
To fix, use path->subpath->rows not baserel->tuples as the number
of input tuples we'll process. This requires estimating the quals'
selectivity afresh, which is slightly annoying; but it shouldn't
really add much cost thanks to the caching done in RestrictInfo.
This is pretty clearly a bug fix, but I'll refrain from back-patching
as people might not appreciate plan choices changing in stable branches.
The fact that it took us this long to identify the bug suggests that
it's not a major problem.
Per report from bucoo, though this is not his proposed patch.
Discussion: https://postgr.es/m/
202204121457159307248@sohu.com
Peter Eisentraut [Wed, 4 May 2022 15:36:31 +0000 (17:36 +0200)]
Remove JsonPathSpec typedef
It doesn't seem very useful, and it's a bit in the way of the planned
node support automation.
Discussion: https://www.postgresql.org/message-id/
202204191140.3wsbevfhqmu3@alvherre.pgsql
Peter Eisentraut [Wed, 4 May 2022 15:34:22 +0000 (17:34 +0200)]
Add missing enum tag in enum used in nodes
Similar to
983bdc4fac492a99bb8ab5a471ca7437139e5cf6.
Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/
202204191140.3wsbevfhqmu3@alvherre.pgsql
Peter Eisentraut [Wed, 4 May 2022 11:33:59 +0000 (13:33 +0200)]
Simplify configure test
The test for lz4.h used AC_CHECK_HEADERS, but nothing was using the
resulting symbol HAVE_LZ4_H. Change this to use AC_CHECK_HEADER
instead. This was probably an oversight, seeing that the nearby
similar tests do this correctly.
Daniel Gustafsson [Wed, 4 May 2022 12:15:25 +0000 (14:15 +0200)]
Rename libpq test programs with libpq_ prefix
The testclient and uri-regress programs in the libpq test suite had
quite generic names which didn't convey much meaning. Since they are
installed as part of the MSVC test runs, ensure that their purpose
is a little bit clearer by renaming with a libpq_ prefix. While at
it rename uri-regress to uri_regress to avoid mixing dash and under-
score in the same filename.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/
20220501080706.GA1542365@rfd.leadboat.com
Peter Eisentraut [Wed, 4 May 2022 05:57:39 +0000 (07:57 +0200)]
Fix incorrect format placeholders
Andres Freund [Tue, 3 May 2022 01:25:00 +0000 (18:25 -0700)]
Fix possibility of self-deadlock in ResolveRecoveryConflictWithBufferPin().
The tests added in
9f8a050f68d failed nearly reliably on FreeBSD in CI, and
occasionally on the buildfarm. That turns out to be caused not by a bug in the
test, but by a longstanding bug in recovery conflict handling.
The standby timeout handler, used by ResolveRecoveryConflictWithBufferPin(),
executed SendRecoveryConflictWithBufferPin() inside a signal handler. A bad
idea, because the deadlock timeout handler (or a spurious latch set) could
have interrupted ProcWaitForSignal(). If unlucky that could cause a
self-deadlock on ProcArrayLock, if the deadlock check is in
SendRecoveryConflictWithBufferPin()->CancelDBBackends().
To fix, set a flag in StandbyTimeoutHandler(), and check the flag in
ResolveRecoveryConflictWithBufferPin().
Subsequently the recovery conflict tests will be backpatched.
Discussion: https://postgr.es/m/
20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de
Backpatch: 10-
Andres Freund [Tue, 3 May 2022 00:19:11 +0000 (17:19 -0700)]
Add tests for recovery deadlock conflicts.
The recovery conflict tests added in
9f8a050f68d surfaced a bug in the
interaction between buffer pin and deadlock recovery conflicts. To make sure
that the bugfix won't break deadlock conflict detection, add a test for that
scenario.
031_recovery_conflict.pl will later be backpatched, with this included.
Discussion: https://postgr.es/m/
20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de
Michael Paquier [Mon, 2 May 2022 11:16:19 +0000 (20:16 +0900)]
basebackup_to_shell: Add missing MarkGUCPrefixReserved()
Oversight in
c6306db24, as per a requirement from
88103567. All the
other modules in the tree, be they in contrib/ or src/test/modules/,
already do that.
Author: Bharath Rupireddy
Discussion: https://postgr.es/m/CALj2ACUy7q_KwSMda+2SHPSWep32tNUM8cXGRS3=-Vfodo9OUg@mail.gmail.com
Etsuro Fujita [Mon, 2 May 2022 07:45:00 +0000 (16:45 +0900)]
Fix typo in comment.
Jeff Davis [Sat, 30 Apr 2022 15:28:33 +0000 (08:28 -0700)]
pg_walinspect: fix case where flush LSN is in the middle of a record.
Instability in the test for pg_walinspect revealed that
pg_get_wal_records_info_till_end_of_wal(x) would try to decode all the
records with a start LSN earlier than the flush LSN, even though that
might include a partial record at the end of the range. In that case,
read_local_xlog_page_no_wait() would return NULL when it tried to read
past the flush LSN, which would be interpreted as an error by the
caller. That caused a test failure only on a BF animal that had been
restarted recently, but could be expected to happen in the wild quite
easily depending on the alignment of various parameters.
Fix by using private data in read_local_xlog_page_no_wait() to signal
end-of-wal to the caller, so that it can be properly distinguished
from a real error.
Discussion: https://postgr.es/m/Ymd/e5eeZMNAkrXo%40paquier.xyz
Discussion: https://postgr.es/m/111657.
1650910309@sss.pgh.pa.us
Authors: Thomas Munro, Bharath Rupireddy.
Tom Lane [Sat, 30 Apr 2022 15:54:28 +0000 (11:54 -0400)]
Tighten enforcement of variable CONSTANT markings in plpgsql.
I noticed that plpgsql would allow assignment of a new value to a
variable even when that variable is marked CONSTANT, if the variable
is used as an output parameter in CALL or is a refcursor variable
that OPEN assigns a new value to. Fix these oversights.
In the CALL case, the check has to be done at runtime because we
cannot know at parse time which parameters are OUT parameters.
For OPEN, it seems best to likewise enforce at runtime because
then we needn't throw error if the variable has a nonnull value
(since OPEN will only try to overwrite a null value).
Although this is surely a bug fix, no back-patch: it seems unlikely
that anyone would thank us for breaking formerly-working code in
minor releases.
Discussion: https://postgr.es/m/214453.
1651182729@sss.pgh.pa.us
Andrew Dunstan [Fri, 29 Apr 2022 13:01:05 +0000 (09:01 -0400)]
Claim SQL standard compliance for SQL/JSON features
Discussion: https://postgr.es/m/
d03d809c-d0fb-fd6a-1476-
d6dc18ec940e@dunslane.net
Andrew Dunstan [Thu, 28 Apr 2022 19:28:20 +0000 (15:28 -0400)]
Fix JSON_OBJECTAGG uniquefying bug
Commit
f4fb45d15c contained a bug in removing items with null values when
unique keys are required, where the leading items that are sorted
contained such values. Fix that and add a test for it.
Discussion: https://postgr.es/m/CAJA4AWQ_XbSmsNbW226UqNyRLJ+wb=iQkQMj77cQyoNkqtf=2Q@mail.gmail.com
Etsuro Fujita [Thu, 28 Apr 2022 06:15:00 +0000 (15:15 +0900)]
Disable asynchronous execution if using gating Result nodes.
mark_async_capable_plan(), which is called from create_append_plan() to
determine whether subplans are async-capable, failed to take into
account that the given subplan created from a given subpath might
include a gating Result node if the subpath is a SubqueryScanPath or
ForeignPath, causing a segmentation fault there when the subplan created
from a SubqueryScanPath includes the Result node, or causing
ExecAsyncRequest() to throw an error about an unrecognized node type
when the subplan created from a ForeignPath includes the Result node,
because in the latter case the Result node was unintentionally
considered as async-capable, but we don't currently support executing
Result nodes asynchronously. Fix by modifying mark_async_capable_plan()
to disable asynchronous execution in such cases. Also, adjust code in
the ProjectionPath case in mark_async_capable_plan(), for consistency
with other cases, and adjust/improve comments there.
is_async_capable_path() added in commit
27e1f1456, which was rewritten
to mark_async_capable_plan() in a later commit, has the same issue,
causing the error at execution mentioned above, so back-patch to v14
where the aforesaid commit went in.
Per report from Justin Pryzby.
Etsuro Fujita, reviewed by Zhihong Yu and Justin Pryzby.
Discussion: https://postgr.es/m/
20220408124338.GK24419%40telsasoft.com
Michael Paquier [Thu, 28 Apr 2022 04:08:16 +0000 (13:08 +0900)]
Revert recent changes with durable_rename_excl()
This reverts commits
2c902bb and
ccfbd92. Per buildfarm members
kestrel, rorqual and calliphoridae, the assertions checking that a TLI
history file should not exist when created by a WAL receiver have been
failing, and switching to durable_rename() over durable_rename_excl()
would cause the newest TLI history file to overwrite the existing one.
We need to think harder about such cases, so revert the new logic for
now.
Note that all the failures have been reported in the test
025_stuck_on_old_timeline.
Discussion: https://postgr.es/m/511362.
1651116498@sss.pgh.pa.us
John Naylor [Thu, 28 Apr 2022 02:27:32 +0000 (09:27 +0700)]
Fix SQL syntax in comment in logical/worker.c
Euler Taveira
Disussion: https://www.postgresql.org/message-id/
25f95189-eef8-43c4-9d7b-
419b651963c8%40www.fastmail.com
Michael Paquier [Thu, 28 Apr 2022 02:10:40 +0000 (11:10 +0900)]
Remove durable_rename_excl()
ccfbd92 has replaced all existing in-core callers of this function in
favor of durable_rename(). durable_rename_excl() is by nature unsafe on
crashes happening at the wrong time, so just remove it.
Author: Nathan Bossart
Reviewed-by: Robert Haas, Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/
20220407182954.GA1231544@nathanxps13
Michael Paquier [Thu, 28 Apr 2022 01:11:45 +0000 (10:11 +0900)]
Replace existing durable_rename_excl() calls with durable_rename()
durable_rename_excl() attempts to avoid overwriting any existing files
by using link() and unlink(), falling back to rename() on some platforms
(e.g., Windows where link() followed by unlink() is not concurrent-safe,
see
909b449). Most callers of durable_rename_excl() use it just in case
there is an existing file, but it happens that for all of them we never
expect a target file to exist (WAL segment recycling, creation of
timeline history file and basic_archive).
basic_archive used durable_rename_excl() to avoid overwriting an archive
concurrently created by another server. Now, there is a stat() call to
avoid overwriting an existing archive a couple of lines above, so note
that this change opens a small TOCTOU window in this module between the
stat() call and durable_rename().
Furthermore, as mentioned in the top comment of durable_rename_excl(),
this routine can result in multiple hard links to the same file and data
corruption, with two or more links to the same file in pg_wal/ if a
crash happens before the unlink() call during WAL recycling.
Specifically, this would produce links to the same file for the current
WAL file and the next one because the half-recycled WAL file was
re-recycled during crash recovery of a follow-up cluster restart.
This change replaces all calls to durable_rename_excl() with
durable_rename(). This removes the protection against accidentally
overwriting an existing file, but some platforms are already living
without it, and all those code paths never expect an existing file (a
couple of assertions are added to check after that, in case).
This is a bug fix, but knowing the unlikeliness of the problem involving
one of more crashes at an exceptionally bad moment, no backpatch is
done. This could be revisited in the future.
Author: Nathan Bossart
Reviewed-by: Robert Haas, Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/
20220407182954.GA1231544@nathanxps13
Peter Eisentraut [Wed, 27 Apr 2022 07:49:10 +0000 (09:49 +0200)]
Fix incorrect format placeholders
Peter Eisentraut [Wed, 27 Apr 2022 07:15:09 +0000 (09:15 +0200)]
Handle NULL fields in WRITE_INDEX_ARRAY
Unlike existing WRITE_*_ARRAY macros, WRITE_INDEX_ARRAY needs to
handle the case that the field is NULL. We already have the
convention to print NULL fields as "<>", so we do that here as well.
There is currently no corresponding read function for this, so reading
this back in is not implemented, but it could be if needed.
Reported-by: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/CAMbWs4-LN%3DbF8f9eU2R94dJtF54DfDvBq%2BovqHnOQqbinYDrUw%40mail.gmail.com
Michael Paquier [Tue, 26 Apr 2022 05:24:13 +0000 (14:24 +0900)]
Fix typo in pg_walinspect.c
Spotted while looking at the surroundings, introduced by
2258e76.
Michael Paquier [Tue, 26 Apr 2022 04:41:17 +0000 (13:41 +0900)]
Add some isolation tests for CLUSTER
This commit adds two isolation tests for CLUSTER, using:
- A normal table, making sure that CLUSTER blocks and completes if the
table is locked by a concurrent session.
- A partitioned table with a partition owned by a different user. If
the partitioned table is locked by a concurrent session, CLUSTER on the
partitioned table should block. If the partition owned by a different
user is locked, CLUSTER on its partitioned table should complete and
skip the partition.
3f19e17 has added an early check to ignore such a
partition with a SQL regression test, but this was not checking that
CLUSTER should not block.
Discussion: https://postgr.es/m/YlqveniXn9AI6RFZ@paquier.xyz
Andrew Dunstan [Mon, 25 Apr 2022 19:02:13 +0000 (15:02 -0400)]
Inhibit mingw CRT's auto-globbing of command line arguments
For some reason by default the mingw C Runtime takes it upon itself to
expand program arguments that look like shell globbing characters. That
has caused much scratching of heads and mis-attribution of the causes of
some TAP test failures, so stop doing that.
This removes an inconsistency with Windows binaries built with MSVC,
which have no such behaviour.
Per suggestion from Noah Misch.
Backpatch to all live branches.
Discussion: https://postgr.es/m/
20220423025927.GA1274057@rfd.leadboat.com
Alvaro Herrera [Mon, 25 Apr 2022 13:48:13 +0000 (15:48 +0200)]
Drop unlogged table after test is done
Another test is constructed on top of regression tests, which does not
work correctly with unlogged tables. For now, cope with that by making
sure no unlogged table is left behind.
Per buildfarm pink after
4fb5c794e586.
Alvaro Herrera [Mon, 25 Apr 2022 13:00:49 +0000 (15:00 +0200)]
Cover brin/gin/gist/spgist ambuildempty routines in regression tests
Changing some TEMP or permanent tables to UNLOGGED is sufficient to
invoke these ambuildempty routines, which were all not uncovered by any
tests. These changes do not otherwise affect the test suite.
Author: Amul Sul <sulamul@gmail.com>
Discussion: https://postgr.es/m/CAAJ_b95nneRCLM-=qELEdgCYSk6W_++-C+Q_t+wH3SW-hF50iw@mail.gmail.com
Alvaro Herrera [Mon, 25 Apr 2022 08:32:13 +0000 (10:32 +0200)]
Always pfree strings returned by GetDatabasePath
Several places didn't do it, and in many cases it didn't matter because
it would be a small allocation in a short-lived context; but other
places may accumulate a few (for example, in CreateDatabaseUsingFileCopy,
one per tablespace). In most databases this is highly unlikely to be
very serious either, but it seems better to make the code consistent in
case there's future copy-and-paste.
The only case of actual concern seems to be the aforementioned routine,
which is new with commit
9c08aea6a309, so there's no need to backpatch.
As pointed out by Coverity.
Tom Lane [Sat, 23 Apr 2022 16:16:12 +0000 (12:16 -0400)]
Fix incautious CTE matching in rewriteSearchAndCycle().
This function looks for a reference to the recursive WITH CTE,
but it checked only the CTE name not ctelevelsup, so that it could
seize on a lower CTE that happened to have the same name. This
would result in planner failures later, either weird errors such as
"could not find attribute 2 in subquery targetlist", or crashes
or assertion failures. The code also merely Assert'ed that it found
a matching entry, which is not guaranteed at all by the parser.
Per bugs #17320 and #17318 from Zhiyong Wu.
Thanks to Kyotaro Horiguchi for investigation.
Discussion: https://postgr.es/m/17320-
70e37868182512ab@postgresql.org
Discussion: https://postgr.es/m/17318-
2eb65a3a611d2368@postgresql.org
Noah Misch [Sat, 23 Apr 2022 03:20:11 +0000 (20:20 -0700)]
Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.
Today's test case detected alignment problems only when executing on
AIX. This change lets popular platforms detect the same problems.
Reviewed by Masahiko Sawada.
Discussion: https://postgr.es/m/
20220415072601.GG862547@rfd.leadboat.com
Robert Haas [Fri, 22 Apr 2022 20:16:52 +0000 (16:16 -0400)]
Remove some recently-added pg_dump test cases.
Commit
d2d35479796c3510e249d6fc72adbd5df918efbf included a pretty
extensive set of test cases, and some of them don't work on all
of our Windows machines. This happens because IPC::Run expands
its arguments as shell globs on a few machines, but doesn't on most
of the buildfarm. It might be good to fix that problem systematically
somehow, but in the meantime, there are enough test cases for this
commit that it seems OK to just remove the ones that are failing.
Discussion: http://postgr.es/m/
3a190754-b2b0-d02b-dcfd-
4ec1610ffbcb@dunslane.net
Discussion: http://postgr.es/m/CA+TgmoYRGUcFBy6VgN0+Pn4f6Wv=2H0HZLuPHqSy6VC8Ba7vdg@mail.gmail.com
Peter Eisentraut [Fri, 22 Apr 2022 09:19:17 +0000 (11:19 +0200)]
doc: Add links to tables
Formal tables should generally have an xref in the text that points to
them. Add them here.
David Rowley [Fri, 22 Apr 2022 04:02:15 +0000 (16:02 +1200)]
Fix performance regression in tuplesort specializations
697492434 added 3 new qsort specialization functions aimed to improve the
performance of sorting many of the common pass-by-value data types when
they're the leading or only sort key.
Unfortunately, that has caused a performance regression when sorting
datasets where many of the values being compared were equal. What was
happening here was that we were falling back to the standard sort
comparison function to handle tiebreaks. When the two given Datums
compared equally we would incur both the overhead of an indirect function
call to the standard comparer to perform the tiebreak and also the
standard comparer function would go and compare the leading key needlessly
all over again.
Here improve the situation in the 3 new comparison functions. We now
return 0 directly when the two Datums compare equally and we're performing
a 1-key sort.
Here we don't do anything to help the multi-key sort case where the
leading key uses one of the sort specializations functions. On testing
this case, even when the leading key's values are all equal, there
appeared to be no performance regression. Let's leave it up to future
work to optimize that case so that the tiebreak function no longer
re-compares the leading key over again.
Another possible fix for this would have been to add 3 additional sort
specialization functions to handle single-key sorts for these
pass-by-value types. The reason we didn't do that here is that we may
deem some other sort specialization to be more useful than single-key
sorts. It may be impractical to have sort specialization functions for
every single combination of what may be useful and it was already decided
that further analysis into which ones are the most useful would be delayed
until the v16 cycle. Let's not let this regression force our hand into
trying to make that decision for v15.
Author: David Rowley
Reviewed-by: John Naylor
Discussion: https://postgr.es/m/CA+hUKGJRbzaAOUtBUcjF5hLtaSHnJUqXmtiaLEoi53zeWSizeA@mail.gmail.com
Tom Lane [Thu, 21 Apr 2022 21:58:52 +0000 (17:58 -0400)]
Remove inadequate assertion check in CTE inlining.
inline_cte() expected to find exactly as many references to the
target CTE as its cterefcount indicates. While that should be
accurate for the tree as emitted by the parser, there are some
optimizations that occur upstream of here that could falsify it,
notably removal of unused subquery output expressions.
Trying to make the accounting 100% accurate seems expensive and
doomed to future breakage. It's not really worth it, because
all this code is protecting is downstream assumptions that every
referenced CTE has a plan. Let's convert those assertions to
regular test-and-elog just in case there's some actual problem,
and then drop the failing assertion.
Per report from Tomas Vondra (thanks also to Richard Guo for
analysis). Back-patch to v12 where the faulty code came in.
Discussion: https://postgr.es/m/
29196a1e-ed47-c7ca-9be2-
b1c636816183@enterprisedb.com
Tom Lane [Thu, 21 Apr 2022 21:12:49 +0000 (17:12 -0400)]
Fix missed cases in libpq's error handling.
Commit
618c16707 invented an "error_result" flag in PGconn, which
intends to represent the state that we have an error condition and
need to build a PGRES_FATAL_ERROR PGresult from the message text in
conn->errorMessage, but have not yet done so. (Postponing construction
of the error object simplifies dealing with out-of-memory conditions
and with concatenation of messages for multiple errors.) For nearly all
purposes, this "virtual" PGresult object should act the same as if it
were already materialized. But a couple of places in fe-protocol3.c
didn't get that memo, and were only testing conn->result as they used
to, without also checking conn->error_result.
In hopes of reducing the probability of similar mistakes in future,
I invented a pgHavePendingResult() macro that includes both tests.
Per report from Peter Eisentraut.
Discussion: https://postgr.es/m/
b52277b9-fa66-b027-4a37-
fb8989c73ff8@enterprisedb.com
Tom Lane [Thu, 21 Apr 2022 20:23:12 +0000 (16:23 -0400)]
Rethink method for assigning OIDs to the template0 and postgres DBs.
Commit
aa0105141 assigned fixed OIDs to template0 and postgres
in a very ad-hoc way. Notably, instead of teaching Catalog.pm
about these OIDs, the unused_oids script was just hacked to
not show them as unused. That's problematic since, for example,
duplicate_oids wouldn't report any future conflict. Hence,
invent a macro DECLARE_OID_DEFINING_MACRO() that can be used to
define an OID that is known to Catalog.pm and will participate
in duplicate-detection as well as renumbering by renumber_oids.pl.
(We don't anticipate renumbering these particular OIDs, but we
might as well build out all the Catalog.pm infrastructure while
we're here.)
Another issue is that
aa0105141 neglected to touch IsPinnedObject,
with the result that it now claimed template0 and postgres are
pinned. The right thing to do there seems to be to teach it that
no database is pinned, since in fact DROP DATABASE doesn't check
for pinned-ness (and at least for these cases, that is an
intentional choice). It's not clear whether this wrong answer
had any visible effect, but perhaps it could have resulted in
erroneous management of dependency entries.
In passing, rename the TemplateDbOid macro to Template1DbOid
to reduce confusion (likely we should have done that way back
when we invented template0, but we didn't), and rename the
OID macros for template0 and postgres to have a similar style.
There are no changes to postgres.bki here, so no need for a
catversion bump.
Discussion: https://postgr.es/m/
2935358.
1650479692@sss.pgh.pa.us
Alvaro Herrera [Thu, 21 Apr 2022 17:12:21 +0000 (19:12 +0200)]
Standardize references to Zstandard as <productname>
Some places used ZSTD, which isn't widely used anywhere. Use ZSTD only
to refer to the environment variable; use zstd (all lowercase) to refer
to the utility.
Per complaint from Justin Pryzby.
Discussion: https://postgr.es/m/
20220414003301.GT26620@telsasoft.com
Alvaro Herrera [Thu, 21 Apr 2022 16:57:40 +0000 (18:57 +0200)]
CREATE PUBLICATION ref: Minor tweaks to row filters
Prompted by a complaint from Justin Pryzby.
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/
20220414003301.GT26620@telsasoft.com
Tom Lane [Thu, 21 Apr 2022 16:02:23 +0000 (12:02 -0400)]
Use DECLARE_TOAST_WITH_MACRO() to simplify toast-table declarations.
This is needed so that renumber_oids.pl can handle renumbering
shared catalog declarations, which need to provide C macros for
the OIDs of the shared toast table and index. The previous
method of writing a C macro separately was error-prone anyway.
Also teach renumber_oids.pl about DECLARE_UNIQUE_INDEX_PKEY,
as we missed doing when inventing that macro.
There are no changes to postgres.bki here, so no need for a
catversion bump.
Discussion: https://postgr.es/m/
2995325.
1650487527@sss.pgh.pa.us
Etsuro Fujita [Thu, 21 Apr 2022 06:30:00 +0000 (15:30 +0900)]
postgres_fdw: Disable batch insert when BEFORE ROW INSERT triggers exist.
Previously, we allowed this, but such triggers might query the table to
insert into and act differently if the tuples that have already been
processed and prepared for insertion are not there, so disable it in
such cases.
Back-patch to v14 where batch insert was added.
Discussion: https://postgr.es/m/CAPmGK16_uPqsmgK0-LpLSUk54_BoK13bPrhxhfjSoSTVz414hA%40mail.gmail.com
Peter Geoghegan [Thu, 21 Apr 2022 01:29:02 +0000 (18:29 -0700)]
vacuumlazy.c: MultiXactIds are MXIDs, not XMIDs.
Oversights in commits
0b018fab and
f3c15cbe.
Peter Geoghegan [Thu, 21 Apr 2022 00:17:43 +0000 (17:17 -0700)]
Fix CLUSTER tuplesorts on abbreviated expressions.
CLUSTER sort won't use the datum1 SortTuple field when clustering
against an index whose leading key is an expression. This makes it
unsafe to use the abbreviated keys optimization, which was missed by the
logic that sets up SortSupport state. Affected tuplesorts output tuples
in a completely bogus order as a result (the wrong SortSupport based
comparator was used for the leading attribute).
This issue is similar to the bug fixed on the master branch by recent
commit
cc58eecc5d. But it's a far older issue, that dates back to the
introduction of the abbreviated keys optimization by commit
4ea51cdfe8.
Backpatch to all supported versions.
Author: Peter Geoghegan <pg@bowt.ie>
Author: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CA+hUKG+bA+bmwD36_oDxAoLrCwZjVtST2fqe=b4=qZcmU7u89A@mail.gmail.com
Backpatch: 10-
Tom Lane [Wed, 20 Apr 2022 22:08:15 +0000 (18:08 -0400)]
Disallow infinite endpoints in generate_series() for timestamps.
Such cases will lead to infinite loops, so they're of no practical
value. The numeric variant of generate_series() already threw error
for this, so borrow its message wording.
Per report from Richard Wesley. Back-patch to all supported branches.
Discussion: https://postgr.es/m/
91B44E7B-68D5-448F-95C8-
B4B3B0F5DEAF@duckdblabs.com
Robert Haas [Wed, 20 Apr 2022 15:02:35 +0000 (11:02 -0400)]
Allow db.schema.table patterns, but complain about random garbage.
psql, pg_dump, and pg_amcheck share code to process object name
patterns like 'foo*.bar*' to match all tables with names starting in
'bar' that are in schemas starting with 'foo'. Before v14, any number
of extra name parts were silently ignored, so a command line '\d
foo.bar.baz.bletch.quux' was interpreted as '\d bletch.quux'. In v14,
as a result of commit
2c8726c4b0a496608919d1f78a5abc8c9b6e0868, we
instead treated this as a request for table quux in a schema named
'foo.bar.baz.bletch'. That caused problems for people like Justin
Pryzby who were accustomed to copying strings of the form
db.schema.table from messages generated by PostgreSQL itself and using
them as arguments to \d.
Accordingly, revise things so that if an object name pattern contains
more parts than we're expecting, we throw an error, unless there's
exactly one extra part and it matches the current database name.
That way, thisdb.myschema.mytable is accepted as meaning just
myschema.mytable, but otherdb.myschema.mytable is an error, and so
is some.random.garbage.myschema.mytable.
Mark Dilger, per report from Justin Pryzby and discussion among
various people.
Discussion: https://www.postgresql.org/message-id/
20211013165426.GD27491%40telsasoft.com
Tom Lane [Wed, 20 Apr 2022 15:04:28 +0000 (11:04 -0400)]
Remove trailing whitespace from *.sgml files.
Historically we've been lax about this, but seeing that we're not
lax in C files, there doesn't seem to be a good reason to be so
in the documentation. Remove the existing occurrences (mostly
though not entirely in copied-n-pasted psql output), and modify
.gitattributes so that "git diff --check" will warn about future
cases.
While at it, add *.pm to the set of extensions .gitattributes
knows about, and remove some obsolete entries for files that
we don't have in the tree anymore.
Per followup discussion of commit
5a892c9b1.
Discussion: https://postgr.es/m/E1nfcV1-000kOR-E5@gemulon.postgresql.org
Peter Eisentraut [Wed, 20 Apr 2022 14:11:14 +0000 (16:11 +0200)]
Fix incorrect format placeholders