Fujii Masao [Tue, 18 Feb 2020 04:13:15 +0000 (13:13 +0900)]
Make inherited LOCK TABLE perform access permission checks on parent table only.
Previously, LOCK TABLE command through a parent table checked
the permissions on not only the parent table but also the children
tables inherited from it. This was a bug and inherited queries should
perform access permission checks on the parent table only. This
commit fixes LOCK TABLE so that it does not check the permission
on children tables.
This patch is applied only in the master branch. We decided not to
back-patch because it's not hard to imagine that there are some
applications expecting the old behavior and the change breaks their
security.
Author: Amit Langote
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAHGQGwE+GauyG7POtRfRwwthAGwTjPQYdFHR97+LzA4RHGnJxA@mail.gmail.com
Michael Paquier [Tue, 18 Feb 2020 03:20:55 +0000 (12:20 +0900)]
Remove duplicated words in comments
Author: Daniel Gustafsson
Reviewed-by: Vik Fearing
Discussion: https://postgr.es/m/
EBC3BFEB-664C-4063-81ED-
29F1227DB012@yesql.se
Michael Paquier [Tue, 18 Feb 2020 01:49:44 +0000 (10:49 +0900)]
Fix grammar in monitoring.sgml
This is related to progress reporting for ANALYZE and partitioned
tables.
Author: Amit Langote
Reviewed-by: Daniel Gustafsson, Julien Rouhaud
Discussion: https://postgr.es/m/CA+HiwqGx6C=-bFTX=ryMThyvM7CcSC3b1x8_6zh4Uo41Kvu-zw@mail.gmail.com
Tom Lane [Mon, 17 Feb 2020 23:40:02 +0000 (18:40 -0500)]
Teach pg_dump to dump comments on RLS policy objects.
This was unaccountably omitted in the original RLS patch.
The SQL syntax is basically the same as for comments on triggers,
so crib code from dumpTrigger().
Per report from Marc Munro. Back-patch to all supported branches.
Discussion: https://postgr.es/m/
1581889298.18009.15.camel@bloodnok.com
Peter Eisentraut [Mon, 17 Feb 2020 14:19:58 +0000 (15:19 +0100)]
Optimize update of tables with generated columns
When updating a table row with generated columns, only recompute those
generated columns whose base columns have changed in this update and
keep the rest unchanged. This can result in a significant performance
benefit. The required information was already kept in
RangeTblEntry.extraUpdatedCols; we just have to make use of it.
Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/
b05e781a-fa16-6b52-6738-
761181204567@2ndquadrant.com
Peter Eisentraut [Mon, 17 Feb 2020 14:19:58 +0000 (15:19 +0100)]
Fill in extraUpdatedCols in logical replication
The extraUpdatedCols field of the target RTE records which generated
columns are affected by an update. This is used in a variety of
places, including per-column triggers and foreign data wrappers. When
an update was initiated by a logical replication subscription, this
field was not filled in, so such an update would not affect generated
columns in a way that is consistent with normal updates. To fix,
factor out some code from analyze.c to fill in extraUpdatedCols in the
logical replication worker as well.
Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/
b05e781a-fa16-6b52-6738-
761181204567@2ndquadrant.com
Fujii Masao [Mon, 17 Feb 2020 07:16:08 +0000 (16:16 +0900)]
Add description about GSSOpenServer wait event into document.
This commit also updates wait event enum into alphabetical order.
Previously the enum entry for GSSOpenServer was added out-of-order.
Back-patch to v12 where commit
b0b39f72b9 introduced
GSSOpenServer wait event. In v12, the commit doesn't include
the update of wait event enum, not to break ABI.
Author: Fujii Masao
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/
949931aa-4ed4-d867-a7b5-
de9c02b2292b@oss.nttdata.com
Fujii Masao [Mon, 17 Feb 2020 06:33:32 +0000 (15:33 +0900)]
Add description about LogicalRewriteTruncate wait event into document.
Back-patch to v10 where commit
249cf070e3 introduced
LogicalRewriteTruncate wait event.
Author: Fujii Masao
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/
949931aa-4ed4-d867-a7b5-
de9c02b2292b@oss.nttdata.com
Tom Lane [Sun, 16 Feb 2020 17:20:18 +0000 (12:20 -0500)]
Try again to work around Windows' ERROR_SHARING_VIOLATION in pg_ctl.
Commit
0da33c762 introduced an unfortunate regression in pg_ctl on
Windows: if the log file specified with -l doesn't exist yet, and
pg_ctl is running with Administrator privileges, then the log file
might get created with permissions that prevent the postmaster from
writing on it. (It seems that whether this happens depends on whether
the log file is inside the user's home directory or not, and perhaps
on other phase-of-the-moon conditions, which may explain why we failed
to notice it sooner.)
To fix, just don't create the log file if it doesn't exist yet. The
case where we need to wait obviously only occurs with a pre-existing
log file.
In passing, switch from using fopen() to plain open(), saving a few
cycles.
Per bug #16259 from Jonathan Katz and Heath Lord. Back-patch to v12,
as the faulty commit was.
Alexander Lakhin
Discussion: https://postgr.es/m/16259-
c5ebed32a262a8b1@postgresql.org
Tom Lane [Sat, 15 Feb 2020 20:22:40 +0000 (15:22 -0500)]
Update obsolete comment.
Noted by Justin Pryzby, though I chose to just rip out the stale text,
as it's in no way relevant to this particular function.
Discussion: https://postgr.es/m/
20200212182337.GZ1412@telsasoft.com
Tom Lane [Sat, 15 Feb 2020 20:13:44 +0000 (15:13 -0500)]
Clarify coding in Catalog::AddDefaultValues.
Make it a bit shorter and better-commented; no functional change.
Alvaro Herrera and Tom Lane
Discussion: https://postgr.es/m/
20200212182337.GZ1412@telsasoft.com
Tom Lane [Sat, 15 Feb 2020 19:58:30 +0000 (14:58 -0500)]
Run "make reformat-dat-files".
Mostly to make sure the previous commit didn't break this.
Discussion: https://postgr.es/m/
20200212182337.GZ1412@telsasoft.com
Tom Lane [Sat, 15 Feb 2020 19:57:27 +0000 (14:57 -0500)]
Don't require pg_class.dat to contain correct relnatts values.
Practically everybody who's ever added a column to one of the bootstrap
catalogs has been burnt by the need to update the relnatts field in the
initial pg_class data to match. Now that we use Perl scripts to
generate postgres.bki, we can have the machines take care of that,
by filling the field during genbki.pl.
While at it, use the BKI_DEFAULTS mechanism to eliminate repetitive
specifications of other column values in pg_class.dat, too. They
weren't particularly a maintenance problem, but this way is prettier
(certainly the spotty previous usage of BKI_DEFAULTS wasn't pretty).
No catversion bump needed, since this doesn't actually change the
contents of postgres.bki.
Per gripe from Justin Pryzby, though this is quite different from
his originally proposed solution.
Amit Langote, John Naylor, Tom Lane
Discussion: https://postgr.es/m/
20200212182337.GZ1412@telsasoft.com
Peter Geoghegan [Sat, 15 Feb 2020 02:38:35 +0000 (18:38 -0800)]
Recreate website's formatting for "website" doc builds.
The stylesheets used for the HTML documentation rendered on
postgresql.org have shifted, and no longer matched what was expected by
"make STYLE=website html" builds performed locally. Local doc builds
did not reflect other aspects of the website, including font and
margins.
This patch updates the references to use the current set of stylesheets
that are used by the documentation on postgresql.org. This also wraps
the documentation preview in a HTML container so it can keep the content
within similar margins to those found on the website.
The documentation on building the docs is updated to reflect this
change, and to let the documentation builder know that an external
network connection is required to properly preview documentation built
with "make STYLE=website html" (which was true prior to this patch too,
but not mentioned).
Author: Jonathan Katz
Reported-By: Tom Lane
Discussion: https://postgr.es/m/1375.
1581446233@sss.pgh.pa.us
Tom Lane [Fri, 14 Feb 2020 16:20:07 +0000 (11:20 -0500)]
Remove pg_regress' --load-language option.
We haven't used this option since inventing extensions. As of commit
50fc694e4 it's actually formally equivalent to --load-extension, so
let's just drop it.
Discussion: https://postgr.es/m/6853.
1581627393@sss.pgh.pa.us
Michael Paquier [Fri, 14 Feb 2020 03:38:44 +0000 (12:38 +0900)]
Remove some dead code in contrib/adminpack/
Since its introduction in
fe59e56, the code in charge of validating and
converting a file path includes some extra handling for absolute paths
pointing to an external log_directory, but this has never been used.
Author: Antonin Houska
Reviewed-by: Julien Rouhaud, Michael Paquier
Discussion: https://postgr.es/m/32663.
1581592539@antos
Tom Lane [Thu, 13 Feb 2020 20:02:35 +0000 (15:02 -0500)]
Mark some contrib modules as "trusted".
This allows these modules to be installed into a database without
superuser privileges (assuming that the DBA or sysadmin has installed
the module's files in the expected place). You only need CREATE
privilege on the current database, which by default would be
available to the database owner.
The following modules are marked trusted:
btree_gin
btree_gist
citext
cube
dict_int
earthdistance
fuzzystrmatch
hstore
hstore_plperl
intarray
isn
jsonb_plperl
lo
ltree
pg_trgm
pgcrypto
seg
tablefunc
tcn
tsm_system_rows
tsm_system_time
unaccent
uuid-ossp
In the future we might mark some more modules trusted, but there
seems to be no debate about these, and on the whole it seems wise
to be conservative with use of this feature to start out with.
Discussion: https://postgr.es/m/32315.
1580326876@sss.pgh.pa.us
Jeff Davis [Thu, 13 Feb 2020 17:43:51 +0000 (09:43 -0800)]
Logical Tape Set: lazily allocate read buffer.
The write buffer was already lazily-allocated, so this is more
symmetric. It also means that a freshly-rewound tape (whether for
reading or writing) is not consuming memory for the buffer.
Discussion: https://postgr.es/m/
97c46a59c27f3c38e486ca170fcbc618d97ab049.camel%40j-davis.com
Tom Lane [Thu, 13 Feb 2020 18:37:43 +0000 (13:37 -0500)]
Avoid a performance regression in float overflow/underflow detection.
Commit
6bf0bc842 replaced float.c's CHECKFLOATVAL() macro with static
inline subroutines, but that wasn't too well thought out. In the original
coding, the unlikely condition (isinf(result) or result == 0) was checked
first, and the inf_is_valid or zero_is_valid condition only afterwards.
The inline-subroutine coding caused that to be swapped around, which is
pretty horrid for performance because (a) in common cases the is_valid
condition is twice as expensive to evaluate (e.g., requiring two isinf()
calls not one) and (b) in common cases the is_valid condition is false,
requiring us to perform the unlikely-condition check anyway. Net result
is that one isinf() call becomes two or three, resulting in visible
performance loss as reported by Keisuke Kuroda.
The original fix proposal was to revert the replacement of the macro,
but on second thought, that macro was just a bad idea from the beginning:
if anything it's a net negative for readability of the code. So instead,
let's just open-code all the overflow/underflow tests, being careful to
test the unlikely condition first (and mark it unlikely() to help the
compiler get the point).
Also, rather than having N copies of the actual ereport() calls, collapse
those into out-of-line error subroutines to save some code space. This
does mean that the error file/line numbers won't be very helpful for
figuring out where the issue really is --- but we'd already burned that
bridge by putting the ereports into static inlines.
In HEAD, check_float[48]_val() are gone altogether. In v12, leave them
present in float.h but unused in the core code, just in case some
extension is depending on them.
Emre Hasegeli, with some kibitzing from me and Andres Freund
Discussion: https://postgr.es/m/CANDwggLe1Gc1OrRqvPfGE=kM9K0FSfia0hbeFCEmwabhLz95AA@mail.gmail.com
Peter Geoghegan [Wed, 12 Feb 2020 22:08:34 +0000 (14:08 -0800)]
Doc: Restructure B-Tree support routine docs.
Use a top-level "variablelist", with one item per B-Tree support
function. This structure matches the structure used by various
"Extensibility" sections in other documentation chapters for other index
access methods.
An explicit list makes it much clearer where each item begins and ends.
This wasn't really a problem before now, but an upcoming patch that adds
deduplication to nbtree will need to have its own new B-Tree support
function. Ease the burden of translators by tidying up btree.sgml ahead
of committing the deduplication patch.
Tom Lane [Wed, 12 Feb 2020 19:33:49 +0000 (14:33 -0500)]
Remove long-dead comments.
These should've been dropped by
a8bb8eb58, but evidently were
missed. Not important enough to back-patch.
Tom Lane [Wed, 12 Feb 2020 19:13:13 +0000 (14:13 -0500)]
Doc: fix old oversights in GRANT/REVOKE documentation.
The GRANTED BY clause in GRANT/REVOKE ROLE has been there since 2005
but was never documented. I'm not sure now whether that was just an
oversight or was intentional (given the limited capability of the
option). But seeing that pg_dumpall does emit code that uses this
option, it seems like not documenting it at all is a bad idea.
Also, when we upgraded the syntax to allow CURRENT_USER/SESSION_USER
as the privilege recipient, the role form of GRANT was incorrectly
not modified to show that, and REVOKE's docs weren't touched at all.
Although I'm not that excited about GRANTED BY, the other oversight
seems serious enough to justify a back-patch.
Discussion: https://postgr.es/m/3070.
1581526786@sss.pgh.pa.us
Andres Freund [Wed, 12 Feb 2020 05:14:08 +0000 (21:14 -0800)]
Try to harden insert-conflict-specconflict against autovacuum.
Looks like guaibasaurus had a autovacuum running during the
controller_print_speculative_locks step (just added in
43e08419708). Which does indeed seem quite possible.
Avoid the problem by only looking for the backends participating in
the test.
Michael Paquier [Wed, 12 Feb 2020 04:31:14 +0000 (13:31 +0900)]
Add %x to default PROMPT1 and PROMPT2 in psql
%d can be used to track if the current connection is in a transaction
block or not, and adding it by default to the prompt has the advantage
to not need a modification of .psqlrc, something not possible depending
on the environment.
This discussion has happened across various sources, and there was a
strong consensus in favor of this change.
Author: Vik Fearing
Reviewed-by: Fabien Coelho
Discussion: https://postgr.es/m/
09502c40-cfe1-bb29-10f9-
4b3fa7b2bbb2@2ndquadrant.com
Andres Freund [Wed, 12 Feb 2020 00:32:11 +0000 (16:32 -0800)]
Test additional speculative conflict scenarios.
Previously, the speculative insert tests did not cover the case when a
tuple t is inserted into a table with a unique index on a column but
before it can insert into the index, a concurrent transaction has
inserted a conflicting value into the index and the insertion of tuple t
must be aborted.
The basic permutation is one session successfully inserts into the table
and an associated unique index while a concurrent session successfully
inserts into the table but discovers a conflict before inserting into
the index and must abort the insertion.
Several variants on this include:
- swap which session is successful
- first session insert transaction does not commit, so second session
must wait on a transaction lock
- first session insert does not "complete", so second session must wait
on a speculative insertion lock
Also, refactor the existing TOAST table upsert test to be in the same
spec and reuse the steps.
Author: Melanie Plageman, Ashwin Agrawal, Andres Freund
Reviewed-by: Andres Freund, Taylor Vesely
Discussion: https://postgr.es/m/CAAKRu_ZRmxy_OEryfY3G8Zp01ouhgw59_-_Cm8n7LzRH5BAvng@mail.gmail.com
Fujii Masao [Wed, 12 Feb 2020 00:08:22 +0000 (09:08 +0900)]
Fix bug in pg_basebackup -F plain -R.
Commit
caba97a9d9 changed pg_basebackup -F plain -R so that
it overwrote postgresql.auto.conf in the backup, with new connection
setting. This could cause the existing postgresql.auto.conf settings
in the server to get lost unexpectedly. This is a bug.
This commit fixes the bug by making pg_basebackup -F plain -R
append the connection setting into postgresql.auto.conf in the backup.
Author: Fujii Masao
Reviewed-by: Sergei Kornilov
Discussion: https://postgr.es/m/
250dcf2a-94e7-c05e-824a-
73cfb38a48a4@oss.nttdata.com
Peter Eisentraut [Tue, 11 Feb 2020 22:47:36 +0000 (23:47 +0100)]
Document the pg_upgrade -j/--jobs option as taking an argument
Thomas Munro [Tue, 11 Feb 2020 04:22:37 +0000 (17:22 +1300)]
Use pg_pwrite() in more places.
This removes some lseek() system calls.
Author: Thomas Munro
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/CA%2BhUKGJ%2BoHhnvqjn3%3DHro7xu-YDR8FPr0FL6LF35kHRX%3D_bUzg%40mail.gmail.com
Peter Eisentraut [Mon, 10 Feb 2020 19:47:50 +0000 (20:47 +0100)]
Canonicalize some URLs
Peter Eisentraut [Mon, 10 Feb 2020 18:27:05 +0000 (19:27 +0100)]
psql: Remove one use of HAVE_UNIX_SOCKETS
This use was not protecting any unportable code, it was just omitting
the code because it wouldn't be used. Remove the use to reduce code
complexity a bit.
Discussion: https://www.postgresql.org/message-id/flat/
54bde68c-d134-4eb8-5bd3-
8af33b72a010@2ndquadrant.com
Peter Eisentraut [Mon, 10 Feb 2020 16:12:46 +0000 (17:12 +0100)]
Simplify passing of configure arguments to pg_config
The previous system had configure put the value into the makefiles and
then have the makefiles pass them to the build of pg_config. That was
put in place when pg_config was a shell script. We can simplify that
by having configure put the value into pg_config.h directly. This
also makes the standard build system match how the MSVC build system
already does it.
Discussion: https://www.postgresql.org/message-id/flat/
6e457870-cef5-5f1d-b57c-
fc89cfb8a788%402ndquadrant.com
Jeff Davis [Mon, 10 Feb 2020 18:20:10 +0000 (10:20 -0800)]
Change signature of TupleHashTableHash().
Commit
4eaea3db introduced TupleHashTableHash(), but the signature
didn't match the other exposed functions. Separate it into internal
and external versions. The external version hides the details behind
an API more consistent with the other external functions, and the
internal version is still suitable for simplehash.
Alvaro Herrera [Mon, 10 Feb 2020 15:14:58 +0000 (12:14 -0300)]
createuser: fix parsing of --connection-limit argument
The original coding failed to quote the argument properly.
Reported-by: Daniel Gustafsson
Discussion:
1B8AE66C-85AB-4728-9BB4-
612E8E61C219@yesql.se
Alvaro Herrera [Mon, 10 Feb 2020 14:47:09 +0000 (11:47 -0300)]
Fix priv checks for ALTER <object> DEPENDS ON EXTENSION
Marking an object as dependant on an extension did not have any
privilege check whatsoever; this allowed any user to mark objects as
droppable by anyone able to DROP EXTENSION, which could be used to cause
system-wide havoc. Disallow by checking that the calling user owns the
mentioned object.
(No constraints are placed on the extension.)
Security: CVE-2020-1720
Reported-by: Tom Lane
Discussion: 31605.
1566429043@sss.pgh.pa.us
Michael Paquier [Mon, 10 Feb 2020 06:48:21 +0000 (15:48 +0900)]
Revert "pg_upgrade: Fix quoting of some arguments in pg_ctl command"
This reverts commit
d1c0b61. The patch has some downsides that require
more attention, as discussed with Noah Misch.
Backpatch-through: 9.5
Amit Kapila [Mon, 10 Feb 2020 04:01:18 +0000 (09:31 +0530)]
Fix typos.
Reported-by: Justin Pryzby
Author: Justin Pryzby
Discussion: https://postgr.es/m/
20200206021432.GA24549@telsasoft.com
Amit Kapila [Mon, 10 Feb 2020 03:04:43 +0000 (08:34 +0530)]
doc: Spell checking
Reported-by: Justin Pryzby
Author: Justin Pryzby
Backpatch-through: 9.6
Discussion: https://postgr.es/m/
20200206021432.GA24549@telsasoft.com
Michael Paquier [Mon, 10 Feb 2020 01:48:43 +0000 (10:48 +0900)]
pg_upgrade: Fix quoting of some arguments in pg_ctl command
The previous coding forgot to apply shell quoting to the socket
directory and the data folder, leading to failures when running
pg_upgrade. This refactors the code generating the pg_ctl command
starting clusters to use a more correct shell quoting. Failures are
easier to trigger in 12 and newer versions by using a value of
--socketdir that includes quotes, but it is also possible to cause
failures with quotes included in the default socket directory used by
pg_upgrade or the data folders of the clusters involved in the
upgrade.
As 9.4 is going to be EOL'd with the next minor release, nobody is
likely going to upgrade to it now so this branch is not included in the
set of branches fixed.
Author: Michael Paquier
Reviewed-by: Álvaro Herrera, Noah Misch
Backpatch-through: 9.5
Thomas Munro [Mon, 10 Feb 2020 00:20:00 +0000 (13:20 +1300)]
psql: Fix %w length in PROMPT2 when PROMPT1 contains a newline.
The width of the invisible PROMPT2 must take into account, in order
for user input to be aligned with the first line, that PROMPT1 can
contain newlines.
Author: Maxence Ahlouche
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/CAJeaomVyLSP_Wj%3D0FtYNTuoopWHyFarhUtYKDHs0HHv%2Bb%3DN9sA%40mail.gmail.com
Tom Lane [Sun, 9 Feb 2020 19:19:46 +0000 (14:19 -0500)]
Revert "docs: change "default role" wording to "predefined role""
This reverts commit
0e936a2148472e6c364aee8c3e298dc16dc4240a.
Per discussion, we can't change the section title without some
web-site work, so revert this change temporarily.
Discussion: https://postgr.es/m/
157742545062.1149.
11052653770497832538@wrigleys.postgresql.org
Tom Lane [Sun, 9 Feb 2020 17:02:57 +0000 (12:02 -0500)]
Store the deletion horizon XID for a deleted GIN page on the right page.
Commit
b10714080 moved the GinPageSetDeleteXid() call to a spot where
the "page" variable was pointing to the wrong page, causing the XID
to be inserted on a page that's not being deleted, thus allowing later
GinPageIsRecyclable tests to recycle the deleted page too soon.
It might be a good idea to stop using the single "page" variable for
multiple purposes in this function. But for the moment I just moved
the GinPageSetDeleteXid() call down beside the GinPageSetDeleted()
call, which seems like a more logical place for it anyway.
Back-patch to v11, as the faulty patch was. (Fortunately, the bug
hasn't made it into any release yet.)
Discussion: https://postgr.es/m/21620.
1581098806@sss.pgh.pa.us
Alvaro Herrera [Fri, 7 Feb 2020 21:27:18 +0000 (18:27 -0300)]
Fix failure to create FKs correctly in partitions
On a multi-level partioned table, when adding a partition not directly
connected to the root table, foreign key constraints referencing the
root were not cloned to the new partition, leading to the FK being
possibly inadvertently violated later on.
This was caused by fuzzy thinking in CloneFkReferenced (commit
f56f8f8da6af): it was skipping constraints marked as having parents on
the theory that cloning those would create duplicates; but that's only
correct for the top level of the partitioning hierarchy. For levels
below that one, such constraints must still be considered and only
skipped if later on we see that we'd create duplicates. Apparently, I
(Álvaro) wrote the comments right but the code implemented something
slightly different.
Author: Jehan-Guillaume de Rorthais
Discussion: https://postgr.es/m/
20200206004948.
238352db@firost
Alvaro Herrera [Fri, 7 Feb 2020 20:09:36 +0000 (17:09 -0300)]
Fix TRUNCATE .. CASCADE on partitions
When running TRUNCATE CASCADE on a child of a partitioned table
referenced by another partitioned table, the truncate was not applied to
partitions of the referencing table; this could leave rows violating the
constraint in the referencing partitioned table. Repair by walking the
pg_constraint chain all the way up to the topmost referencing table.
Note: any partitioned tables containing FKs that reference other
partitioned tables should be checked for possible violating rows, if
TRUNCATE has occurred in partitions of the referenced table.
Reported-by: Christophe Courtois
Author: Jehan-Guillaume de Rorthais
Discussion: https://postgr.es/m/
20200204183906.
115f693e@firost
Fujii Masao [Fri, 7 Feb 2020 13:00:21 +0000 (22:00 +0900)]
Fix bug in Tid scan.
Commit
147e3722f7 changed Tid scan so that it calls table_beginscan()
and uses the scan option for seq scan. This change caused two issues.
(1) The change caused Tid scan to take a predicate lock on the entire
relation in serializable transaction even when relation-level
lock is not necessary. This could lead to an unexpected
serialization error.
(2) The change caused Tid scan to increment the number of seq_scan
in pg_stat_*_tables views even though it's not seq scan. This
could confuse the users.
This commit adds the scan option for Tid scan and makes Tid scan
use it, to avoid those issues.
Back-patch to v12, where the bug was introduced.
Author: Tatsuhito Kasahara
Reviewed-by: Kyotaro Horiguchi, Masahiko Sawada, Fujii Masao
Discussion: https://postgr.es/m/CAP0=ZVKy+gTbFmB6X_UW0pP3WaeJ-fkUWHoD-pExS=at3CY76g@mail.gmail.com
Andres Freund [Fri, 7 Feb 2020 06:13:52 +0000 (22:13 -0800)]
jit: Reference expression step functions via llvmjit_types.
The main benefit of doing so is that this allows llvm to ensure that
types match - previously that'd only be detected by a crash within the
called function. There were a number of cases where we passed a
superfluous parameter...
To avoid needing to add all the functions to llvmjit.{c,h}, instead
get them from the llvm module for llvmjit_types.c. Also use that for
the functions from llvmjit_types already in llvmjit.h.
Author: Soumyadeep Chakraborty and Andres Freund
Discussion: https://postgr.es/m/CADwEdooww3wZv-sXSfatzFRwMuwa186LyTwkBfwEW6NjtooBPA@mail.gmail.com
Michael Paquier [Fri, 7 Feb 2020 06:08:17 +0000 (15:08 +0900)]
Bump catalog version for the addition of leader_pid in pg_stat_activity
Oversight in commit
b025f32.
Per private report from Julien Rouhaud.
Jeff Davis [Fri, 7 Feb 2020 03:39:47 +0000 (19:39 -0800)]
Introduce TupleHashTableHash() and LookupTupleHashEntryHash().
Expose two new entry points: one for only calculating the hash value
of a tuple, and another for looking up a hash entry when the hash
value is already known. This will be useful for disk-based Hash
Aggregation to avoid recomputing the hash value for the same tuple
after saving and restoring it from disk.
Discussion: https://postgr.es/m/
37091115219dd522fd9ed67333ee8ed1b7e09443.camel%40j-davis.com
Andres Freund [Fri, 7 Feb 2020 03:45:45 +0000 (19:45 -0800)]
jit: Remove redundancies in expression evaluation code generation.
This merges the code emission for a number of opcodes by handling the
behavioural difference more locally. This reduces code, and also
improves the generated code a bit in some cases, by removing redundant
constants.
Author: Andres Freund
Discussion: https://postgr.es/m/
20191023163849.sosqbfs5yenocez3@alap3.anarazel.de
Andres Freund [Fri, 7 Feb 2020 03:09:06 +0000 (19:09 -0800)]
jit: Reference functions by name in IOCOERCE steps.
Previously we used constant function pointer addresses, which prevents
inlining and other related optimizations.
Author: Andres Freund
Discussion: https://postgr.es/m/
20191023163849.sosqbfs5yenocez3@alap3.anarazel.de
Andres Freund [Fri, 7 Feb 2020 03:06:16 +0000 (19:06 -0800)]
expression eval: Don't redundantly keep track of AggState.
It's already tracked via ExprState->parent, so we don't need to also
include it in ExprEvalStep. When that code originally was written
ExprState->parent didn't exist, but it since has been introduced in
6719b238e8f.
Author: Andres Freund
Discussion: https://postgr.es/m/
20191023163849.sosqbfs5yenocez3@alap3.anarazel.de
Andres Freund [Fri, 7 Feb 2020 03:04:50 +0000 (19:04 -0800)]
expression eval, jit: Minor code cleanups.
This mostly consists of using C99 style for loops, moving variables
into narrower scopes, and a smattering of other minor improvements.
Done separately to make it easier to review patches with actual
functional changes.
Author: Andres Freund
Discussion: https://postgr.es/m/
20191023163849.sosqbfs5yenocez3@alap3.anarazel.de
Michael Paquier [Fri, 7 Feb 2020 03:41:10 +0000 (12:41 +0900)]
Fix typo in proc.c
Author: Julien Rouhaud
Discussion: https://postgr.es/m/
20200206082333.GA95343@nol
Michael Paquier [Fri, 7 Feb 2020 01:51:17 +0000 (10:51 +0900)]
Revert "Prevent running pg_basebackup as root"
This reverts commit
7bae0ad, as this is not ideal with the tar format,
and we may want to explore more options like what is done by tar with
some equivalents of --owner and --group, but for pg_basebackup.
Per complaints from Magnus Hagander and Stephen Frost.
Discussion: https://postgr.es/m/
20200205172259.GW3195@tamriel.snowman.net
Michael Paquier [Thu, 6 Feb 2020 23:10:40 +0000 (08:10 +0900)]
Revert "Add GUC checks for ssl_min_protocol_version and ssl_max_protocol_version"
This reverts commit
41aadee, as the GUC checks could run on older values
with the new values used, and result in incorrect errors if both
parameters are changed at the same time.
Per complaint from Tom Lane.
Discussion: https://postgr.es/m/27574.
1581015893@sss.pgh.pa.us
Backpatch-through: 12
Peter Eisentraut [Thu, 6 Feb 2020 20:21:04 +0000 (21:21 +0100)]
Ensure relcache consistency around generated columns
In certain transient states, it's possible that a table has attributes
with attgenerated set but no default expressions in pg_attrdef yet.
In that case, the old code path would not set
relation->rd_att->constr->has_generated_stored, unless
relation->rd_att->constr was also populated for some other reason.
There was probably no practical impact, but it's better to keep this
consistent.
Reported-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/flat/
20200115181105.ad6ab6dlgyww3lb6%40alap3.anarazel.de
Jeff Davis [Thu, 6 Feb 2020 19:49:56 +0000 (11:49 -0800)]
Refactor hash_agg_entry_size().
Consolidate the calculations for hash table size estimation. This will
help with upcoming Hash Aggregation work that will add additional call
sites.
Jeff Davis [Thu, 6 Feb 2020 18:06:25 +0000 (10:06 -0800)]
Logical Tape Set: use min heap for freelist.
Previously, the freelist of blocks was tracked as an
occasionally-sorted array. A min heap is more resilient to larger
freelists or more frequent changes between reading and writing.
Discussion: https://postgr.es/m/
97c46a59c27f3c38e486ca170fcbc618d97ab049.camel%40j-davis.com
Amit Kapila [Thu, 6 Feb 2020 10:27:02 +0000 (15:57 +0530)]
Fix typo.
Reported-by: Amit Langote
Author: Amit Langote
Backpatch-through: 9.6, where it was introduced
Discussion: https://postgr.es/m/CA+HiwqFNADeukaaGRmTqANbed9Fd81gLi08AWe_F86_942Gspw@mail.gmail.com
Fujii Masao [Thu, 6 Feb 2020 05:43:21 +0000 (14:43 +0900)]
Fix bug in LWLock statistics mechanism.
Previously PostgreSQL built with -DLWLOCK_STATS could report
more than one LWLock statistics entries for the same backend
process and the same LWLock. This is strange and only one
statistics should be output in that case, instead.
The cause of this issue is that the key variable used for
LWLock stats hash table was not fully initialized. The key
consists of two fields and they were initialized. But
the following 4 bytes allocated in the key variable for
the alignment was not initialized. So even if the same key
was specified, hash_search(HASH_ENTER) could not find
the existing entry for that key and created new one.
This commit fixes this issue by initializing the key
variable with zero. As the side effect of this commit,
the volume of LWLock statistics output would be reduced
very much.
Back-patch to v10, where commit
3761fe3c20 introduced the issue.
Author: Fujii Masao
Reviewed-by: Julien Rouhaud, Kyotaro Horiguchi
Discussion: https://postgr.es/m/
26359edb-798a-568f-d93a-
6aafac49752d@oss.nttdata.com
Michael Paquier [Thu, 6 Feb 2020 00:18:06 +0000 (09:18 +0900)]
Add leader_pid to pg_stat_activity
This new field tracks the PID of the group leader used with parallel
query. For parallel workers and the leader, the value is set to the
PID of the group leader. So, for the group leader, the value is the
same as its own PID. Note that this reflects what PGPROC stores in
shared memory, so as leader_pid is NULL if a backend has never been
involved in parallel query. If the backend is using parallel query or
has used it at least once, the value is set until the backend exits.
Author: Julien Rouhaud
Reviewed-by: Sergei Kornilov, Guillaume Lelarge, Michael Paquier, Tomas
Vondra
Discussion: https://postgr.es/m/CAOBaU_Yy5bt0vTPZ2_LUM6cUcGeqmYNoJ8-Rgto+c2+w3defYA@mail.gmail.com
Andrew Gierth [Wed, 5 Feb 2020 19:49:47 +0000 (19:49 +0000)]
Force tuple conversion when the source has missing attributes.
Tuple conversion incorrectly concluded that no conversion was needed
as long as all the attributes lined up. But if the source tuple has a
missing attribute (from addition of a column with default), then the
destination tupdesc might not reflect the same default. The typical
symptom was that the affected columns would be unexpectedly NULL.
Repair by always forcing conversion if the source has missing
attributes, which will be filled in by the deform operation. (In
theory we could optimize for when the destination has the same
default, but that seemed overkill.)
Backpatch to 11 where missing attributes were added.
Per bug #16242.
Vik Fearing (discovery, code, testing) and me (analysis, testcase).
Discussion: https://postgr.es/m/16242-
d1c9fca28445966b@postgresql.org
Alvaro Herrera [Wed, 5 Feb 2020 19:59:29 +0000 (16:59 -0300)]
Make vacuum buffer counters 64 bits wide
Using 32 bit counters means they can now realistically wrap around when
vacuuming extremely large tables. Because they're signed integers,
stats printed by vacuum look very odd when they do.
We'd love to backpatch this, but refrain because the variables are
exported and could cause third-party code to break.
Reviewed-by: Julien Rouhaud, Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/
20200131205926.GA16367@alvherre.pgsql
Alvaro Herrera [Wed, 5 Feb 2020 16:21:43 +0000 (13:21 -0300)]
ALTER SUBSCRIPTION / REFRESH docs: explain copy_data
The docs are ambiguous as to which tables would be copied over when the
copy_data parameter is true in ALTER SUBSCRIPTION ... REFRESH PUBLICATION.
Make it clear that it only applies to tables which are new in the
publication.
Author: David Christensen (reword by Álvaro Herrera)
Discussion: https://postgr.es/m/
95339420-7F09-4F8C-ACC0-
8F1CFAAD9CD7@endpoint.com
Noah Misch [Wed, 5 Feb 2020 16:26:41 +0000 (08:26 -0800)]
When a TAP file has non-zero exit status, retain temporary directories.
PostgresNode already retained base directories in such cases. Stop
using $SIG{__DIE__}, which is redundant with the exit status check, in
lieu of proliferating it to TestLib. Back-patch to 9.6, where commit
88802e068017bee8cea7a5502a712794e761c7b5 introduced retention on
failure.
Reviewed by Daniel Gustafsson.
Discussion: https://postgr.es/m/
20200202170155.GA3264196@rfd.leadboat.com
Thomas Munro [Wed, 5 Feb 2020 04:35:57 +0000 (17:35 +1300)]
Add kqueue(2) support to the WaitEventSet API.
Use kevent(2) to wait for events on the BSD family of operating
systems and macOS. This is similar to the epoll(2) support added
for Linux by commit
98a64d0bd.
Author: Thomas Munro
Reviewed-by: Andres Freund, Marko Tiikkaja, Tom Lane
Tested-by: Mateusz Guzik, Matteo Beccati, Keith Fiske, Heikki Linnakangas, Michael Paquier, Peter Eisentraut, Rui DeSousa, Tom Lane, Mark Wong
Discussion: https://postgr.es/m/CAEepm%3D37oF84-iXDTQ9MrGjENwVGds%2B5zTr38ca73kWR7ez_tA%40mail.gmail.com
Thomas Munro [Tue, 4 Feb 2020 23:21:03 +0000 (12:21 +1300)]
Handle lack of DSM slots in parallel btree build, take 2.
Commit
74618e77 added a new check intended to fix a bug, but put
it in the wrong place so that parallel btree build was always
disabled. Do the check after we've actually tried to create
a DSM segment. Back-patch to 11, like the earlier commit.
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-WzmDABkJzrNnvf%2BOULK-_A_j9gkYg_Dz-H62jzNv4eKQTw%40mail.gmail.com
Tom Lane [Tue, 4 Feb 2020 18:07:13 +0000 (13:07 -0500)]
Fix handling of "Subplans Removed" field in EXPLAIN output.
Commit
499be013d added this field in a rather poorly-thought-through
manner, with the result being that rather than being a field of the
Append or MergeAppend plan node as intended (and as it seems to be,
in text format), it was actually an element of the "Plans" subgroup.
At least in JSON format, that's flat out invalid syntax, because
"Plans" is an array not an object.
While it's not hard to move the generation of the field so that it
appears where it's supposed to, this does result in a visible change
in field order in text format, in cases where a Append or MergeAppend
plan node has any InitPlans attached. That's slightly annoying to
do in stable branches; but the alternative of continuing to emit
broken non-text formats seems worse.
Also, since the set of fields emitted is not supposed to be
data-dependent in non-text formats, make sure that "Subplans Removed"
appears in Append and MergeAppend nodes even when it's zero, in those
formats. (The previous coding made it look like it could appear in
some other node types such as BitmapAnd, but we don't actually support
runtime pruning there, so don't emit it in those cases.)
Per bug #16171 from Mahadevan Ramachandran. Fix by Daniel Gustafsson
and Tom Lane, reviewed by Hamid Akhtar. Back-patch to v11 where this
code came in.
Discussion: https://postgr.es/m/16171-
b72259ab75505fa2@postgresql.org
Michael Paquier [Tue, 4 Feb 2020 04:56:04 +0000 (13:56 +0900)]
Fix fuzzy error handling in pg_basebackup when opening gzFile
First, this code did not bother checking for a failure when calling
dup(). Then, per zlib, gzerror() returns NULL for a NULL input, which
can happen if passing down to gzdopen() an invalid file descriptor or if
there was an allocation failure.
No back-patch is done as this would unlikely be a problem in the field.
Per Coverity.
Reported-by: Tom Lane
Alvaro Herrera [Mon, 3 Feb 2020 21:59:12 +0000 (18:59 -0300)]
Add missing break out seqscan loop in logical replication
When replica identity is FULL (an admittedly unusual case), the loop
that searches for tuples in execReplication.c didn't stop scanning the
table when once a matching tuple was found. Add the missing 'break'.
Note slight behavior change: we now return the first matching tuple
rather than the last one. They are supposed to be indistinguishable
anyway, so this shouldn't matter.
Author: Konstantin Knizhnik
Discussion: https://postgr.es/m/
379743f6-ae91-b866-f7a2-
5624e6d2b0a4@postgrespro.ru
Michael Paquier [Mon, 3 Feb 2020 05:48:42 +0000 (14:48 +0900)]
Add declaration-level assertions for compile-time checks
Those new assertions can be used at file scope, outside of any function
for compilation checks. This commit provides implementations for C and
C++, and fallback implementations.
Author: Peter Smith
Reviewed-by: Andres Freund, Kyotaro Horiguchi, Dagfinn Ilmari Mannsåker,
Michael Paquier
Discussion: https://postgr.es/m/
201DD0641B056142AC8C6645EC1B5F62014B8E8030@SYD1217
Tom Lane [Sun, 2 Feb 2020 18:09:33 +0000 (13:09 -0500)]
Fix assorted error-cleanup bugs in SSL min/max protocol version code.
The error exits added to initialize_SSL() failed to clean up the
partially-built SSL_context, and some of them also leaked the
result of SSLerrmessage(). Make them match other error-handling
cases in that function.
The error exits added to connectOptions2() failed to set conn->status
like every other error exit in that function.
In passing, make the SSL_get_peer_certificate() error exit look more
like all the other calls of SSLerrmessage().
Oversights in commit
ff8ca5fad. Coverity whined about leakage of the
SSLerrmessage() results; I noted the rest in manual code review.
Andrew Gierth [Sat, 1 Feb 2020 21:57:14 +0000 (21:57 +0000)]
Optimizations for integer to decimal output.
Using a lookup table of digit pairs reduces the number of divisions
needed, and calculating the length upfront saves some work; these
ideas are taken from the code previously committed for floats.
David Fetter, reviewed by Kyotaro Horiguchi, Tels, and me.
Discussion: https://postgr.es/m/
20190924052620.GP31596%40fetter.org
Michael Paquier [Sat, 1 Feb 2020 09:30:25 +0000 (18:30 +0900)]
Prevent running pg_basebackup as root
Similarly to pg_upgrade, pg_ctl and initdb, a root user is able to use
--version and --help, but cannot execute the actual operation to avoid
the creation of files with permissions incompatible with the
postmaster.
This is a behavior change, so not back-patching is done.
Author: Ian Barwick
Discussion: https://postgr.es/m/CABvVfJVqOdD2neLkYdygdOHvbWz_5K_iWiqY+psMfA=FeAa3qQ@mail.gmail.com
Thomas Munro [Sat, 1 Feb 2020 01:29:13 +0000 (14:29 +1300)]
Fix memory leak on DSM slot exhaustion.
If we attempt to create a DSM segment when no slots are available,
we should return the memory to the operating system. Previously
we did that if the DSM_CREATE_NULL_IF_MAXSEGMENTS flag was
passed in, but we didn't do it if an error was raised. Repair.
Back-patch to 9.4, where DSM segments arrived.
Author: Thomas Munro
Reviewed-by: Robert Haas
Reported-by: Julian Backes
Discussion: https://postgr.es/m/CA%2BhUKGKAAoEw-R4om0d2YM4eqT1eGEi6%3DQot-3ceDR-SLiWVDw%40mail.gmail.com
Tom Lane [Fri, 31 Jan 2020 22:26:40 +0000 (17:26 -0500)]
Fix not-quite-right string comparison in parse_jsonb_index_flags().
This code would accept "strinX", where X is any 1-byte character,
as meaning "string". Clearly it wasn't meant to do that.
No back-patch, since this doesn't affect correct queries and
there's some tiny chance we'd break somebody's incorrect query
in a minor release.
Report and patch by Dominik Czarnota.
Discussion: https://postgr.es/m/CABEVAa1dU0mDCAfaT8WF2adVXTDsLVJy_izotg6ze_hh-cn8qQ@mail.gmail.com
Tom Lane [Fri, 31 Jan 2020 22:03:55 +0000 (17:03 -0500)]
Fix CheckAttributeType's handling of collations for ranges.
Commit
fc7695891 changed CheckAttributeType to recurse into ranges,
but made it pass down the wrong collation (always InvalidOid, since
ranges as such have no collation). This would result in guaranteed
failure when considering a range type whose subtype is collatable.
Embarrassingly, we lack any regression tests that would expose such
a problem (but fortunately, somebody noticed before we shipped this
bug in any release).
Fix it to pass down the range's subtype collation property instead,
and add some regression test cases to exercise collatable-subtype
ranges a bit more. Back-patch to all supported branches, as the
previous patch was.
Report and patch by Julien Rouhaud, test cases tweaked by me
Discussion: https://postgr.es/m/CAOBaU_aBWqNweiGUFX0guzBKkcfJ8mnnyyGC_KBQmO12Mj5f_A@mail.gmail.com
Tom Lane [Fri, 31 Jan 2020 19:41:49 +0000 (14:41 -0500)]
Fix parallel pg_dump/pg_restore for failure to create worker processes.
If we failed to fork a worker process, or create a communication pipe
for one, WaitForTerminatingWorkers would suffer an assertion failure
if assert-enabled, otherwise crash or go into an infinite loop. This
was a consequence of not accounting for the startup condition where
we've not yet forked all the workers.
The original bug was that ParallelBackupStart would set workerStatus to
WRKR_IDLE before it had successfully forked a worker. I made things
worse in commit
b7b8cc0cf by not understanding the undocumented fact
that the WRKR_TERMINATED state was also meant to represent the case
where a worker hadn't been started yet: I changed enum T_WorkerStatus
so that *all* the worker slots were initially in WRKR_IDLE state. But
this wasn't any more broken in practice, since even one slot in the
wrong state would keep WaitForTerminatingWorkers from terminating.
In v10 and later, introduce an explicit T_WorkerStatus value for
worker-not-started, in hopes of preventing future oversights of the
same ilk. Before that, just document that WRKR_TERMINATED is supposed
to cover that case (partly because it wasn't actively broken, and
partly because the enum is exposed outside parallel.c in those branches,
so there's microscopically more risk involved in changing it).
In all branches, introduce a WORKER_IS_RUNNING status test macro
to hide which T_WorkerStatus values mean that, and be more careful
not to access ParallelSlot fields till we're sure they're valid.
Per report from Vignesh C, though this is my patch not his.
Back-patch to all supported branches.
Discussion: https://postgr.es/m/CALDaNm1Luv-E3sarR+-unz-BjchquHHyfP+YC+2FS2pt_J+wxg@mail.gmail.com
Peter Eisentraut [Fri, 31 Jan 2020 15:26:12 +0000 (16:26 +0100)]
Allow building without default socket directory
We have code paths for Unix socket support and no Unix socket support.
Now add a third variant: Unix socket support but do not use a Unix
socket by default in the client or the server, only if you explicitly
specify one. This will be useful when we enable Unix socket support
on Windows.
To implement this, tweak things so that setting DEFAULT_PGSOCKET_DIR
to "" has the desired effect. This mostly already worked like that;
only a few places needed to be adjusted. Notably, the reference to
DEFAULT_PGSOCKET_DIR in UNIXSOCK_PATH() could be removed because all
callers already resolve an empty socket directory setting with a
default if appropriate.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/
75f72249-8ae6-322a-63df-
4fe03eeccb9f@2ndquadrant.com
Peter Eisentraut [Fri, 31 Jan 2020 10:50:32 +0000 (11:50 +0100)]
Sprinkle some const decorations
This might help clarify the API a bit.
Michael Paquier [Fri, 31 Jan 2020 04:57:56 +0000 (13:57 +0900)]
Fix typo in recently-added TAP test for replication slots
Oversight in commit
b0afdca.
Thomas Munro [Fri, 31 Jan 2020 04:21:47 +0000 (17:21 +1300)]
Report time spent in posix_fallocate() as a wait event.
When allocating DSM segments with posix_fallocate() on Linux (see commit
899bd785), report this activity as a wait event exactly as we would if
we were using file-backed DSM rather than shm_open()-backed DSM.
Author: Thomas Munro
Discussion: https://postgr.es/m/CA%2BhUKGKCSh4GARZrJrQZwqs5SYp0xDMRr9Bvb%2BHQzJKvRgL6ZA%40mail.gmail.com
Thomas Munro [Fri, 31 Jan 2020 03:58:38 +0000 (16:58 +1300)]
Adjust DSM and DSA slot usage constants.
When running a lot of large parallel queries concurrently, or a plan with
a lot of separate Gather nodes, it is possible to run out of DSM slots.
There are better solutions to these problems requiring architectural
redesign work, but for now, let's adjust the constants so that it's more
difficult to hit the limit.
1. Previously, a DSA area would create up to four segments at each size
before doubling the size. After this commit, it will create only two at
each size, so it ramps up faster and therefore needs fewer slots.
2. Previously, the total limit on DSM slots allowed for 2 per connection.
Switch to 5 per connection.
Also remove an obsolete nearby comment.
Author: Thomas Munro
Reviewed-by: Robert Haas, Andres Freund
Discussion: https://postre.es/m/CA%2BhUKGL6H2BpGbiF7Lj6QiTjTGyTLW_vLR%3DSn2tEBeTcYXiMKw%40mail.gmail.com
Tom Lane [Thu, 30 Jan 2020 23:25:55 +0000 (18:25 -0500)]
In jsonb_plpython.c, suppress warning message from gcc 10.
Very recent gcc complains that PLyObject_ToJsonbValue could return
a pointer to a local variable. I think it's wrong; but the coding
is fragile enough, and the savings of one palloc() minimal enough,
that it seems better to just do a palloc() all the time. (My other
idea of tweaking the if-condition doesn't suppress the warning.)
Back-patch to v11 where this code was introduced.
Discussion: https://postgr.es/m/21547.
1580170366@sss.pgh.pa.us
Thomas Munro [Thu, 30 Jan 2020 21:25:34 +0000 (10:25 +1300)]
Handle lack of DSM slots in parallel btree build.
If no DSM slots are available, a ParallelContext can still be
created, but its seg pointer is NULL. Teach parallel btree build
to cope with that by falling back to a regular non-parallel build,
to avoid crashing with a segmentation fault.
Back-patch to 11, where parallel CREATE INDEX landed.
Reported-by: Nicola Contu
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/CA%2BhUKGJgJEBnkuODBVomyK3MWFvDBbMVj%3Dgdt6DnRPU-5sQ6UQ%40mail.gmail.com
Alvaro Herrera [Thu, 30 Jan 2020 16:42:14 +0000 (13:42 -0300)]
Clean up newlines following left parentheses
We used to strategically place newlines after some function call left
parentheses to make pgindent move the argument list a few chars to the
left, so that the whole line would fit under 80 chars. However,
pgindent no longer does that, so the newlines just made the code
vertically longer for no reason. Remove those newlines, and reflow some
of those lines for some extra naturality.
Reviewed-by: Michael Paquier, Tom Lane
Discussion: https://postgr.es/m/
20200129200401.GA6303@alvherre.pgsql
Alvaro Herrera [Thu, 30 Jan 2020 16:32:04 +0000 (13:32 -0300)]
Remove excess parens in ereport() calls
Cosmetic cleanup, not worth backpatching.
Discussion: https://postgr.es/m/
20200129200401.GA6303@alvherre.pgsql
Reviewed-by: Tom Lane, Michael Paquier
Tom Lane [Thu, 30 Jan 2020 15:51:31 +0000 (10:51 -0500)]
Fix test failure with non-MULTIPLICITY Perl interpreters.
Per buildfarm.
Fujii Masao [Thu, 30 Jan 2020 15:42:06 +0000 (00:42 +0900)]
Make inherited TRUNCATE perform access permission checks on parent table only.
Previously, TRUNCATE command through a parent table checked the
permissions on not only the parent table but also the children tables
inherited from it. This was a bug and inherited queries should perform
access permission checks on the parent table only. This commit fixes
that bug.
Back-patch to all supported branches.
Author: Amit Langote
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAHGQGwFHdSvifhJE+-GSNqUHSfbiKxaeQQ7HGcYz6SC2n_oDcg@mail.gmail.com
Michael Paquier [Thu, 30 Jan 2020 02:14:02 +0000 (11:14 +0900)]
Fix slot data persistency when advancing physical replication slots
Advancing a physical replication slot with pg_replication_slot_advance()
did not mark the slot as dirty if any advancing was done, preventing the
follow-up checkpoint to flush the slot data to disk. This caused the
advancing to be lost even on clean restarts. This does not happen for
logical slots as any advancing marked the slot as dirty. Per
discussion, the original feature has been implemented so as in the event
of a crash the slot may move backwards to a past LSN. This property is
kept and more documentation is added about that.
This commit adds some new TAP tests to check the persistency of physical
and logical slots after advancing across clean restarts.
Author: Alexey Kondratov, Michael Paquier
Reviewed-by: Andres Freund, Kyotaro Horiguchi, Craig Ringer
Discussion: https://postgr.es/m/
059cc53a-8b14-653a-a24d-
5f867503b0ee@postgrespro.ru
Backpatch-through: 11
Tom Lane [Thu, 30 Jan 2020 00:13:34 +0000 (19:13 -0500)]
Fix vcregress.pl for new plperl test case.
As of commit
50fc694e4, the plperl tests don't want preinstalled
plperl languages; they now install those themselves. I'd removed the
--load-extension options from the GNUmakefile, but missed teaching
the MSVC build script about that.
Per buildfarm.
Tom Lane [Wed, 29 Jan 2020 23:42:43 +0000 (18:42 -0500)]
Invent "trusted" extensions, and remove the pg_pltemplate catalog.
This patch creates a new extension property, "trusted". An extension
that's marked that way in its control file can be installed by a
non-superuser who has the CREATE privilege on the current database,
even if the extension contains objects that normally would have to be
created by a superuser. The objects within the extension will (by
default) be owned by the bootstrap superuser, but the extension itself
will be owned by the calling user. This allows replicating the old
behavior around trusted procedural languages, without all the
special-case logic in CREATE LANGUAGE. We have, however, chosen to
loosen the rules slightly: formerly, only a database owner could take
advantage of the special case that allowed installation of a trusted
language, but now anyone who has CREATE privilege can do so.
Having done that, we can delete the pg_pltemplate catalog, moving the
knowledge it contained into the extension script files for the various
PLs. This ends up being no change at all for the in-core PLs, but it is
a large step forward for external PLs: they can now have the same ease
of installation as core PLs do. The old "trusted PL" behavior was only
available to PLs that had entries in pg_pltemplate, but now any
extension can be marked trusted if appropriate.
This also removes one of the stumbling blocks for our Python 2 -> 3
migration, since the association of "plpythonu" with Python 2 is no
longer hard-wired into pg_pltemplate's initial contents. Exactly where
we go from here on that front remains to be settled, but one problem
is fixed.
Patch by me, reviewed by Peter Eisentraut, Stephen Frost, and others.
Discussion: https://postgr.es/m/5889.
1566415762@sss.pgh.pa.us
Tom Lane [Wed, 29 Jan 2020 16:06:59 +0000 (11:06 -0500)]
Teach plpgsql's "make clean" to remove generated test files.
Copy the rules that src/test/regress/GNUmakefile uses for this purpose.
Since these files are .gitignore'd, the mistake wasn't obvious unless
you happened to look at "git status --ignored" in an allegedly clean
tree.
Oversight in commit
1858b105b. No need for back-patch since that's
not in the back branches.
Robert Haas [Wed, 29 Jan 2020 15:50:17 +0000 (10:50 -0500)]
Add jsonapi.c to Mkvcbuild.pm's @pgcommonallfiles.
My recent commit
beb4699091e9fab1c5f465056bef35c9ddf7f9fc caused
some buildfarm breakage, as reported by Tom Lane. Try to repair.
This fix is extracted from a larger patch by Andrew Dunstan.
Discussion: http://postgr.es/m/
8440ddc9-8347-ca64-1405-
845d10e054cd@2ndQuadrant.com
Discussion: http://postgr.es/m/14178.
1580312751@sss.pgh.pa.us
Robert Haas [Wed, 29 Jan 2020 15:19:24 +0000 (10:19 -0500)]
Move jsonapi.c and jsonapi.h to src/common.
To make this work, (1) makeJsonLexContextCstringLen now takes the
encoding to be used as an argument; (2) check_stack_depth() is made to
do nothing in frontend code, and (3) elog(ERROR, ...) is changed to
pg_log_fatal + exit in frontend code.
Mark Dilger, reviewed and slightly revised by me.
Discussion: http://postgr.es/m/CA+TgmoYfOXhd27MUDGioVh6QtpD0C1K-f6ObSA10AWiHBAL5bA@mail.gmail.com
Peter Eisentraut [Wed, 29 Jan 2020 14:43:32 +0000 (15:43 +0100)]
Fail if recovery target is not reached
Before, if a recovery target is configured, but the archive ended
before the target was reached, recovery would end and the server would
promote without further notice. That was deemed to be pretty wrong.
With this change, if the recovery target is not reached, it is a fatal
error.
Based-on-patch-by: Leif Gunnar Erlandsen <leif@lako.no>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/
993736dd3f1713ec1f63fc3b653839f5@lako.no
Michael Paquier [Wed, 29 Jan 2020 06:08:19 +0000 (15:08 +0900)]
Fix dispsize for libpq connection parameters channel_binding and gssencmode
channel_binding's longest allowed value is not "7", it is actually "8".
gssencmode also got that wrong.
A similar mistake has been fixed as of
f4051e3.
Backpatch down to v12, where gssencmode has been introduced.
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/
20200128053633.GD1552@paquier.xyz
Backpatch-through: 12
Amit Kapila [Wed, 29 Jan 2020 05:38:50 +0000 (11:08 +0530)]
Add --parallel option to vacuumdb command.
Commit
40d964ec99 allowed vacuum command to leverage multiple CPUs by
invoking parallel workers to process indexes. This commit provides a
'--parallel' option to specify the parallel degree used by vacuum command.
Author: Masahiko Sawada, with few modifications by me
Reviewed-by: Mahendra Singh and Amit Kapila
Discussion: https://postgr.es/m/CAD21AoDTPMgzSkV4E3SFo1CH_x50bf5PqZFQf4jmqjk-C03BWg@mail.gmail.com
Tom Lane [Tue, 28 Jan 2020 22:26:37 +0000 (17:26 -0500)]
Fix dangling pointer in EvalPlanQual machinery.
EvalPlanQualStart() supposed that it could re-use the relsubs_rowmark
and relsubs_done arrays from a prior instantiation. But since they are
allocated in the es_query_cxt of the recheckestate, that's just wrong;
EvalPlanQualEnd() will blow away that storage. Therefore we were using
storage that could have been reallocated to something else, causing all
sorts of havoc.
I think this was modeled on the old code's handling of es_epqTupleSlot,
but since the code was anyway clearing the arrays at re-use, there's
clearly no expectation of importing any outside state. So it's just
a dubious savings of a couple of pallocs, which is negligible compared
to setting up a new planstate tree. Therefore, just allocate the
arrays always. (I moved the allocations slightly for readability.)
In principle this bug could cause a problem whenever EPQ rechecks are
needed in more than one target table of a ModifyTable plan node.
In practice it seems not quite so easy to trigger as that; I couldn't
readily duplicate a crash with a partitioned target table, for instance.
That's probably down to incidental choices about when to free or
reallocate stuff. The added isolation test case does seem to reliably
show an assertion failure, though.
Per report from Oleksii Kliukin. Back-patch to v12 where the bug was
introduced (evidently by commit
3fb307bc4).
Discussion: https://postgr.es/m/
EEF05F66-2871-4786-992B-
5F45C92FEE2E@hintbits.com
Heikki Linnakangas [Tue, 28 Jan 2020 10:55:30 +0000 (12:55 +0200)]
Fix randAccess setting in ReadRecord()
Commit
38a957316d got this backwards.
Author: Kyotaro Horiguchi
Discussion: https://www.postgresql.org/message-id/
20200128.194408.
2260703306774646445.horikyota.ntt@gmail.com
Thomas Munro [Tue, 28 Jan 2020 07:30:40 +0000 (20:30 +1300)]
Fix compile error on HP C.
Per build farm animal anole, after commit
6f38d4dac3.