Peter Eisentraut [Thu, 8 Jul 2021 07:38:52 +0000 (09:38 +0200)]
Improve error messages about mismatching relkind
Most error messages about a relkind that was not supported or
appropriate for the command was of the pattern
"relation \"%s\" is not a table, foreign table, or materialized view"
This style can become verbose and tedious to maintain. Moreover, it's
not very helpful: If I'm trying to create a comment on a TOAST table,
which is not supported, then the information that I could have created
a comment on a materialized view is pointless.
Instead, write the primary error message shorter and saying more
directly that what was attempted is not possible. Then, in the detail
message, explain that the operation is not supported for the relkind
the object was. To simplify that, add a new function
errdetail_relkind_not_supported() that does this.
In passing, make use of RELKIND_HAS_STORAGE() where appropriate,
instead of listing out the relkinds individually.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/flat/
dc35a398-37d0-75ce-07ea-
1dd71d98f8ec@2ndquadrant.com
Tom Lane [Wed, 7 Jul 2021 19:21:25 +0000 (15:21 -0400)]
Fix crash in postgres_fdw for provably-empty remote UPDATE/DELETE.
In
86dc90056, I'd written find_modifytable_subplan with the assumption
that if the immediate child of a ModifyTable is a Result, it must be
a projecting Result with a subplan. However, if the UPDATE or DELETE
has a provably-constant-false WHERE clause, that's not so: we'll
generate a dummy subplan with a childless Result. Add the missing
null-check so we don't crash on such cases.
Per report from Alexander Pyhalov.
Discussion: https://postgr.es/m/
b9a6f53549456b2f3e2fd150dcd79d72@postgrespro.ru
Fujii Masao [Wed, 7 Jul 2021 12:54:47 +0000 (21:54 +0900)]
doc: Fix description about pg_stat_statements.track_planning.
This commit fixes wrong wording like "a fewer kinds"
in the description about track_planning option.
Back-patch to v13 where pg_stat_statements.track_planning was added.
Author: Justin Pryzby
Reviewed-by: Julien Rouhaud, Fujii Masao
Discussion: https://postgr.es/m/
20210418233615.GB7256@telsasoft.com
David Rowley [Wed, 7 Jul 2021 04:29:17 +0000 (16:29 +1200)]
Use a hash table to speed up NOT IN(values)
Similar to
50e17ad28, which allowed hash tables to be used for IN clauses
with a set of constants, here we add the same feature for NOT IN clauses.
NOT IN evaluates the same as: WHERE a <> v1 AND a <> v2 AND a <> v3.
Obviously, if we're using a hash table we must be exactly equivalent to
that and return the same result taking into account that either side of
the condition could contain a NULL. This requires a little bit of
special handling to make work with the hash table version.
When processing NOT IN, the ScalarArrayOpExpr's operator will be the <>
operator. To be able to build and lookup a hash table we must use the
<>'s negator operator. The planner checks if that exists and is hashable
and sets the relevant fields in ScalarArrayOpExpr to instruct the executor
to use hashing.
Author: David Rowley, James Coleman
Reviewed-by: James Coleman, Zhihong Yu
Discussion: https://postgr.es/m/CAApHDvoF1mum_FRk6D621edcB6KSHBi2+GAgWmioj5AhOu2vwQ@mail.gmail.com
Fujii Masao [Wed, 7 Jul 2021 02:13:40 +0000 (11:13 +0900)]
postgres_fdw: Tighten up allowed values for batch_size, fetch_size options.
Previously the values such as '100$%$#$#', '9,223,372,' were accepted and
treated as valid integers for postgres_fdw options batch_size and fetch_size.
Whereas this is not the case with fdw_startup_cost and fdw_tuple_cost options
for which an error is thrown. This was because endptr was not used
while converting strings to integers using strtol.
This commit changes the logic so that it uses parse_int function
instead of strtol as it serves the purpose by returning false in case
if it is unable to convert the string to integer. Note that
this function also rounds off the values such as '100.456' to 100 and
'100.567' or '100.678' to 101.
While on this, use parse_real for fdw_startup_cost and fdw_tuple_cost options.
Since parse_int and parse_real are being used for reloptions and GUCs,
it is more appropriate to use in postgres_fdw rather than using strtol
and strtod directly.
Back-patch to v14.
Author: Bharath Rupireddy
Reviewed-by: Ashutosh Bapat, Tom Lane, Kyotaro Horiguchi, Fujii Masao
Discussion: https://postgr.es/m/CALj2ACVMO6wY5Pc4oe1OCgUOAtdjHuFsBDw8R5uoYR86eWFQDA@mail.gmail.com
Michael Paquier [Wed, 7 Jul 2021 01:55:15 +0000 (10:55 +0900)]
Refactor SASL code with a generic interface for its mechanisms
The code of SCRAM and SASL have been tightly linked together since SCRAM
exists in the core code, making hard to apprehend the addition of new
SASL mechanisms, but these are by design different facilities, with
SCRAM being an option for SASL. This refactors the code related to both
so as the backend and the frontend use a set of callbacks for SASL
mechanisms, documenting while on it what is expected by anybody adding a
new SASL mechanism.
The separation between both layers is neat, using two sets of callbacks
for the frontend and the backend to mark the frontier between both
facilities. The shape of the callbacks is now directly inspired from
the routines used by SCRAM, so the code change is straight-forward, and
the SASL code is moved into its own set of files. These will likely
change depending on how and if new SASL mechanisms get added in the
future.
Author: Jacob Champion
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/
3d2a6f5d50e741117d6baf83eb67ebf1a8a35a11.camel@vmware.com
Tom Lane [Tue, 6 Jul 2021 22:10:11 +0000 (18:10 -0400)]
Allow CustomScan providers to say whether they support projections.
Previously, all CustomScan providers had to support projections,
but there may be cases where this is inconvenient. Add a flag
bit to say if it's supported.
Important item for the release notes: this is non-backwards-compatible
since the default is now to assume that CustomScan providers can't
project, instead of assuming that they can. It's fail-soft, but could
result in visible performance penalties due to adding unnecessary
Result nodes.
Sven Klemm, reviewed by Aleksander Alekseev; some cosmetic fiddling
by me.
Discussion: https://postgr.es/m/CAMCrgp1kyakOz6c8aKhNDJXjhQ1dEjEnp+6KNT3KxPrjNtsrDg@mail.gmail.com
Alvaro Herrera [Tue, 6 Jul 2021 21:48:41 +0000 (17:48 -0400)]
Improve TestLib::system_or_bail error reporting
The original coding was not quoting the complete failing command, and it
wasn't printing the reason for the failure either. Do both.
This is cosmetic only, so no backpatch.
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/
202106301524.eq5pblzstapj@alvherre.pgsql
Tom Lane [Tue, 6 Jul 2021 18:23:09 +0000 (14:23 -0400)]
Reduce the cost of planning deeply-nested views.
Joel Jacobson reported that deep nesting of trivial (flattenable)
views results in O(N^3) growth of planning time for N-deep nesting.
It turns out that a large chunk of this cost comes from copying around
the "subquery" sub-tree of each view's RTE_SUBQUERY RTE. But once we
have successfully flattened the subquery, we don't need that anymore,
because the planner isn't going to do anything else interesting with
that RTE. We already zap the subquery pointer during setrefs.c (cf.
add_rte_to_flat_rtable), but it's useless baggage earlier than that
too. Clearing the pointer as soon as pull_up_simple_subquery is done
with the RTE reduces the cost from O(N^3) to O(N^2); which is still
not great, but it's quite a lot better. Further improvements will
require rethinking of the RTE data structure, which is being considered
in another thread.
Patch by me; thanks to Dean Rasheed for review.
Discussion: https://postgr.es/m/
797aff54-b49b-4914-9ff9-
aa42564a4d7d@www.fastmail.com
Tom Lane [Tue, 6 Jul 2021 16:36:12 +0000 (12:36 -0400)]
Avoid doing catalog lookups in postgres_fdw's conversion_error_callback.
As in
50371df26, this is a bad idea since the callback can't really
know what error is being thrown and thus whether or not it is safe
to attempt catalog accesses. Rather than pushing said accesses into
the mainline code where they'd usually be a waste of cycles, we can
look at the query's rangetable instead.
This change does mean that we'll be printing query aliases (if any
were used) rather than the table or column's true name. But that
doesn't seem like a bad thing: it's certainly a more useful definition
in self-join cases, for instance. In any case, it seems unlikely that
any applications would be depending on this detail, so it seems safe
to change.
Patch by me. Original complaint by Andres Freund; Bharath Rupireddy
noted the connection to conversion_error_callback.
Discussion: https://postgr.es/m/
20210106020229.ne5xnuu6wlondjpe@alap3.anarazel.de
Tom Lane [Tue, 6 Jul 2021 14:34:51 +0000 (10:34 -0400)]
Doc: add info about timestamps with fractional-minute UTC offsets.
Our code has supported fractional-minute UTC offsets for ages, but
there was no mention of the possibility in the main docs, and only
a very indirect reference in Appendix B. Improve that.
Discussion: https://postgr.es/m/
162543102827.697.
5755498651217979813@wrigleys.postgresql.org
Amit Kapila [Tue, 6 Jul 2021 02:16:50 +0000 (07:46 +0530)]
Refactor function parse_subscription_options.
Instead of using multiple parameters in parse_subscription_options
function signature, use the struct SubOpts that encapsulate all the
subscription options and their values. It will be useful for future work
where we need to add other options in the subscription. Also, use bitmaps
to pass the supported and retrieve the specified options much like the way
it is done in the commit
a3dc926009.
Author: Bharath Rupireddy
Reviewed-By: Peter Smith, Amit Kapila, Alvaro Herrera
Discussion: https://postgr.es/m/CALj2ACXtoQczfNsDQWobypVvHbX2DtgEHn8DawS0eGFwuo72kw@mail.gmail.com
David Rowley [Tue, 6 Jul 2021 00:38:50 +0000 (12:38 +1200)]
Fix typo in comment
Author: James Coleman
Discussion: https://postgr.es/m/CAAaqYe8f8ENA0i1PdBtUNWDd2sxHSMgscNYbjhaXMuAdfBrZcg@mail.gmail.com
David Rowley [Tue, 6 Jul 2021 00:24:43 +0000 (12:24 +1200)]
Reduce the number of pallocs when building partition bounds
In each of the create_*_bound() functions for LIST, RANGE and HASH
partitioning, there were a large number of palloc calls which could be
reduced down to a much smaller number.
In each of these functions, an array was built so that we could qsort it
before making the PartitionBoundInfo. For LIST and HASH partitioning, an
array of pointers was allocated then each element was allocated within
that array. Since the number of items of each dimension is known
beforehand, we can just allocate a single chunk of memory for this.
Similarly, with all partition strategies, we're able to reduce the number
of allocations to build the ->datums field. This is an array of Datum
pointers, but there's no need for the Datums that each element points to
to be singly allocated. One big chunk will do. For RANGE partitioning,
the PartitionBoundInfo->kind field can get the same treatment.
We can apply the same optimizations to partition_bounds_copy(). Doing
this might have a small effect on cache performance when searching for the
correct partition during partition pruning or DML on a partitioned table.
However, that's likely to be small and this is mostly about reducing
palloc overhead.
Author: Nitin Jadhav, Justin Pryzby, David Rowley
Reviewed-by: Justin Pryzby, Zhihong Yu
Discussion: https://postgr.es/m/flat/CAMm1aWYFTqEio3bURzZh47jveiHRwgQTiSDvBORczNEz2duZ1Q@mail.gmail.com
Michael Paquier [Mon, 5 Jul 2021 23:10:59 +0000 (08:10 +0900)]
Use WaitLatch() instead of pg_usleep() at the end of backups
This concerns pg_stop_backup() and BASE_BACKUP, when waiting for the
WAL segments required for a backup to be archived. This simplifies a
bit the handling of the wait event used in this code path.
Author: Bharath Rupireddy
Reviewed-by: Michael Paquier, Stephen Frost
Discussion: https://postgr.es/m/CALj2ACU4AdPCq6NLfcA-ZGwX7pPCK5FgEj-CAU0xCKzkASSy_A@mail.gmail.com
Tom Lane [Mon, 5 Jul 2021 20:51:57 +0000 (16:51 -0400)]
Reduce overhead of cache-clobber testing in LookupOpclassInfo().
Commit
03ffc4d6d added logic to bypass all caching behavior in
LookupOpclassInfo when CLOBBER_CACHE_ALWAYS is enabled. It doesn't
look like I stopped to think much about what that would cost, but
recent investigation shows that the cost is enormous: it roughly
doubles the time needed for cache-clobber test runs.
There does seem to be value in this behavior when trying to test
the opclass-cache loading logic itself, but for other purposes the
cost is excessive. Hence, let's back off to doing this only when
debug_invalidate_system_caches_always is at least 3; or in older
branches, when CLOBBER_CACHE_RECURSIVELY is defined.
While here, clean up some other minor issues in LookupOpclassInfo.
Re-order the code so we aren't left with broken cache entries (leading
to later core dumps) in the unlikely case that we suffer OOM while
trying to allocate space for a new entry. (That seems to be my
oversight in
03ffc4d6d.) Also, in >= v13, stop allocating one array
entry too many. That's evidently left over from sloppy reversion in
851b14b0c.
Back-patch to all supported branches, mainly to reduce the runtime
of cache-clobbering buildfarm animals.
Discussion: https://postgr.es/m/
1370856.
1625428625@sss.pgh.pa.us
Tom Lane [Mon, 5 Jul 2021 18:34:47 +0000 (14:34 -0400)]
Rethink blocking annotations in detach-partition-concurrently-[34].
In
741d7f104, I tried to make the reports from canceled steps come out
after the pg_cancel_backend() steps, since that was the most common
ordering before. However, that doesn't ensure that a canceled step
doesn't report even later, as shown in a recent failure on buildfarm
member idiacanthus. Rather than complicating things even more with
additional annotations, let's just force the cancel's effect to be
reported first. It's not *that* unnatural-looking.
Back-patch to v14 where these test cases appeared.
Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=idiacanthus&dt=2021-07-02%2001%3A40%3A04
Dean Rasheed [Mon, 5 Jul 2021 09:16:42 +0000 (10:16 +0100)]
Prevent numeric overflows in parallel numeric aggregates.
Formerly various numeric aggregate functions supported parallel
aggregation by having each worker convert partial aggregate values to
Numeric and use numeric_send() as part of serializing their state.
That's problematic, since the range of Numeric is smaller than that of
NumericVar, so it's possible for it to overflow (on either side of the
decimal point) in cases that would succeed in non-parallel mode.
Fix by serializing NumericVars instead, to avoid the overflow risk and
ensure that parallel and non-parallel modes work the same.
A side benefit is that this improves the efficiency of the
serialization/deserialization code, which can make a noticeable
difference to performance with large numbers of parallel workers.
No back-patch due to risk from changing the binary format of the
aggregate serialization states, as well as lack of prior field
complaints and low probability of such overflows in practice.
Patch by me. Thanks to David Rowley for review and performance
testing, and Ranier Vilela for an additional suggestion.
Discussion: https://postgr.es/m/CAEZATCUmeFWCrq2dNzZpRj5+6LfN85jYiDoqm+ucSXhb9U2TbA@mail.gmail.com
Peter Eisentraut [Mon, 5 Jul 2021 06:26:00 +0000 (08:26 +0200)]
doc: Fix quoting markup
Amit Kapila [Mon, 5 Jul 2021 04:06:11 +0000 (09:36 +0530)]
Doc: Hash Indexes.
A new chapter for Hash Indexes, designed to help users understand how
they work and when to use them.
Backpatch-through 10 where we have made hash indexes durable.
Author: Simon Riggs
Reviewed-By: Justin Pryzby, Amit Kapila
Discussion: https://postgr.es/m/CANbhV-HRjNPYgHo--P1ewBrFJ-GpZPb9_25P7=Wgu7s7hy_sLQ@mail.gmail.com
Michael Paquier [Sun, 4 Jul 2021 11:59:04 +0000 (20:59 +0900)]
doc: Mention requirement to --enable-tap-tests on section for TAP tests
Author: Greg Sabino Mullane
Discussion: https://postgr.es/m/CAKAnmmJYH2FBn_+Vwd2FD5SaKn8hjhAXOCHpZc6n4wXaUaW_SA@mail.gmail.com
Backpatch-through: 9.6
David Rowley [Sun, 4 Jul 2021 10:28:38 +0000 (22:28 +1200)]
Doc: mention that VACUUM can't utilize over 1GB of RAM
Document that setting maintenance_work_mem to values over 1GB has no
effect on VACUUM.
Reported-by: Martín Marqués
Author: Laurenz Albe
Discussion: https://postgr.es/m/CABeG9LsZ2ozUMcqtqWu_-GiFKB17ih3p8wBHXcpfnHqhCnsc7A%40mail.gmail.com
Backpatch-through: 9.6, oldest supported release
David Rowley [Sun, 4 Jul 2021 06:47:31 +0000 (18:47 +1200)]
Cleanup some aggregate code in the executor
Here we alter the code that calls build_pertrans_for_aggref() so that the
function no longer needs to special-case whether it's dealing with an
aggtransfn or an aggcombinefn. This allows us to reuse the
build_aggregate_transfn_expr() function and just get rid of the
build_aggregate_combinefn_expr() completely.
All of the special case code that was in build_pertrans_for_aggref() has
been moved up to the calling functions.
This saves about a dozen lines of code in nodeAgg.c and a few dozen more
in parse_agg.c
Also, rename a few variables in nodeAgg.c to try to make it more clear
that we're working with either a aggtransfn or an aggcombinefn. Some of
the old names would have you believe that we were always working with an
aggtransfn.
Discussion: https://postgr.es/m/CAApHDvptMQ9FmF0D67zC_w88yVnoNVR2+kkOQGUrCmdxWxLULQ@mail.gmail.com
Tom Lane [Sat, 3 Jul 2021 15:21:40 +0000 (11:21 -0400)]
Further restrict the scope of no-exit()-in-libpq test.
Disable this check altogether in --enable-coverage builds,
because newer versions of gcc insert exit() as well as abort()
calls for that. Also disable it on AIX and Solaris, because
those platforms tend to provide facilities such as libldap
as static libraries, which then get included in libpq's shlib.
We can't expect such libraries to honor our coding rules.
(That platform list might need additional tweaking, but I think
this is enough to keep the buildfarm happy.)
Per reports from Jacob Champion and Noah Misch.
Discussion: https://postgr.es/m/
3128896.
1624742969@sss.pgh.pa.us
Bruce Momjian [Sat, 3 Jul 2021 00:42:46 +0000 (20:42 -0400)]
doc: adjust "cities" example to be consistent with other SQL
Reported-by: tom@crystae.net
Discussion: https://postgr.es/m/
162345756191.14472.
9754568432103008703@wrigleys.postgresql.org
Backpatch-through: 9.6
Bruce Momjian [Fri, 2 Jul 2021 22:00:30 +0000 (18:00 -0400)]
docs: clarify new aggressive vacuum mode for multi-xacts
Reported-by: eric.mutta@gmail.com
Discussion: https://postgr.es/m/
162395467510.686.
11947486273299446208@wrigleys.postgresql.org
Backpatch-through: 14
Tom Lane [Fri, 2 Jul 2021 20:04:54 +0000 (16:04 -0400)]
Don't try to print data type names in slot_store_error_callback().
The existing code tried to do syscache lookups in an already-failed
transaction, which is problematic to say the least. After some
consideration of alternatives, the best fix seems to be to just drop
type names from the error message altogether. The table and column
names seem like sufficient localization. If the user is unsure what
types are involved, she can check the local and remote table
definitions.
Having done that, we can also discard the LogicalRepTypMap hash
table, which had no other use. Arguably, LOGICAL_REP_MSG_TYPE
replication messages are now obsolete as well; but we should
probably keep them in case some other use emerges. (The complexity
of removing something from the replication protocol would likely
outweigh any savings anyhow.)
Masahiko Sawada and Bharath Rupireddy, per complaint from Andres
Freund. Back-patch to v10 where this code originated.
Discussion: https://postgr.es/m/
20210106020229.ne5xnuu6wlondjpe@alap3.anarazel.de
Peter Eisentraut [Fri, 2 Jul 2021 09:59:55 +0000 (11:59 +0200)]
Use InvalidBucket instead of -1 where appropriate
Reported-by: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAEudQAp%3DZwKjrP4L%2BCzqV7SmWiaQidPPRqj4tqdjDG4KBx5yrg%40mail.gmail.com
Michael Paquier [Fri, 2 Jul 2021 03:58:34 +0000 (12:58 +0900)]
Use WaitLatch() instead of pg_usleep() at end-of-vacuum truncation
This has the advantage to make a process more responsive when the
postmaster dies, even if the wait time was rather limited as there was
only a 50ms timeout here. Another advantage of this change is for
monitoring, as we gain a new wait event for the end-of-vacuum
truncation.
Author: Bharath Rupireddy
Reviewed-by: Aleksander Alekseev, Thomas Munro, Michael Paquier
Discussion: https://postgr.es/m/CALj2ACU4AdPCq6NLfcA-ZGwX7pPCK5FgEj-CAU0xCKzkASSy_A@mail.gmail.com
Thomas Munro [Fri, 2 Jul 2021 01:07:16 +0000 (13:07 +1200)]
Remove some dead stores.
Remove redundant local variable assignments left behind by commit
2fc7af5e966.
Author: Quan Zongliang <quanzongliang@yeah.net>
Reviewed-by: Jacob Champion <pchampion@vmware.com>
Discussion: https://postgr.es/m/
de141d14-4fd6-3148-99bf-
856b71aa948a%40yeah.net
Michael Paquier [Fri, 2 Jul 2021 00:35:12 +0000 (09:35 +0900)]
Simplify error handing of jsonapi.c for the frontend
This commit removes a dependency to the central logging facilities in
the JSON parsing routines of src/common/, which existed to log errors
when seeing error codes that do not match any existing values in
JsonParseErrorType, which is not something that should never happen.
The routine providing a detailed error message based on the error code
is made backend-only, the existing code being unsafe to use in the
frontend as the error message may finish by being palloc'd or point to a
static string, so there is no way to know if the memory of the message
should be pfree'd or not. The only user of this routine in the frontend
was pg_verifybackup, that is changed to use a more generic error message
on parsing failure.
Note that making this code more resilient to OOM failures if used in
shared libraries would require much more work as a lot of code paths
still rely on palloc() & friends, but we are not sure yet if we need to
go down to that. Still, removing the dependency to logging is a step
toward more portability.
This cleans up the handling of check_stack_depth() while on it, as it
exists only in the backend.
Per discussion with Jacob Champion and Tom Lane.
Discussion: https://postgr.es/m/YNwL7kXwn3Cckbd6@paquier.xyz
Peter Eisentraut [Thu, 1 Jul 2021 20:23:37 +0000 (22:23 +0200)]
doc: Remove inappropriate <acronym> tags
Andrew Dunstan [Thu, 1 Jul 2021 19:38:06 +0000 (15:38 -0400)]
add missing tag from commit
b8c4261e5e
Peter Eisentraut [Thu, 1 Jul 2021 19:27:39 +0000 (21:27 +0200)]
doc: Clean up title case use
Andrew Dunstan [Thu, 1 Jul 2021 18:21:09 +0000 (14:21 -0400)]
Add new make targets world-bin and install-world-bin
These are the same as world and install-world respectively, but without
building or installing the documentation. There are many reasons for
wanting to be able to do this, including speed, lack of documentation
building tools, and wanting to build other formats of the documentation.
Plans for simplifying the buildfarm client code include using these
targets.
Backpatch to all live branches.
Discussion: https://postgr.es/m/
6a421136-d462-b043-a8eb-
e75b2861f3df@dunslane.net
Tom Lane [Thu, 1 Jul 2021 17:33:05 +0000 (13:33 -0400)]
Add --clobber-cache option to initdb, for CCA testing.
Commit
4656e3d66 replaced the "#define CLOBBER_CACHE_ALWAYS"
testing mechanism with a GUC, which has been a great help for
doing cache-clobber testing in more efficient ways; but there
is a gap in the implementation. The only way to do cache-clobber
testing during an initdb run is to use the old method with #define,
because one can't set the GUC from outside. Improve this by
adding a switch to initdb for the purpose.
(Perhaps someday we should let initdb pass through arbitrary
"-c NAME=VALUE" switches. Quoting difficulties dissuaded me
from attempting that right now, though.)
Back-patch to v14 where
4656e3d66 came in.
Discussion: https://postgr.es/m/
1582507.
1624227029@sss.pgh.pa.us
Alvaro Herrera [Thu, 1 Jul 2021 16:56:30 +0000 (12:56 -0400)]
Don't reset relhasindex for partitioned tables on ANALYZE
Commit
0e69f705cc1a introduced code to analyze partitioned table;
however, that code fails to preserve pg_class.relhasindex correctly.
Fix by observing whether any indexes exist rather than accidentally
falling through to assuming none do.
Backpatch to 14.
Author: Alexander Pyhalov <a.pyhalov@postgrespro.ru>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Zhihong Yu <zyu@yugabyte.com>
Discussion: https://postgr.es/m/CALNJ-vS1R3Qoe5t4tbzxrkpBtzRbPq1dDcW4RmA_a+oqweF30w@mail.gmail.com
Tom Lane [Thu, 1 Jul 2021 14:45:12 +0000 (10:45 -0400)]
Improve build-time check that libpq doesn't call exit().
Further fixes for commit
dc227eb82. Per suggestion from
Peter Eisentraut, use a stamp-file to control when the check
is run, avoiding repeated executions during "make all".
Also, remove "-g" switch for nm: it's useless and some versions
of nm consider it to conflict with "-u". (Thanks to Noah Misch
for running down that portability issue.)
Discussion: https://postgr.es/m/
3128896.
1624742969@sss.pgh.pa.us
Andrew Dunstan [Thu, 1 Jul 2021 12:29:10 +0000 (08:29 -0400)]
Fix prove_installcheck to use correct paths when used with PGXS
The prove_installcheck recipe in src/Makefile.global.in was emitting
bogus paths for a couple of elements when used with PGXS. Here we create
a separate recipe for the PGXS case that does it correctly. We also take
the opportunity to make the make the file more readable by breaking up
the prove_installcheck and prove_check recipes across several lines, and
to remove the setting for REGRESS_SHLIB to src/test/recovery/Makefile,
which is the only set of tests that actually need it.
Backpatch to all live branches
Discussion: https://postgr.es/m/
f2401388-936b-f4ef-a07c-
a0bcc49b3300@dunslane.net
Heikki Linnakangas [Thu, 1 Jul 2021 12:32:57 +0000 (15:32 +0300)]
Allow specifying pg_waldump --rmgr option multiple times.
Before, if you specified multiple --rmgr options, only the last one took
effect. It seems more sensible to select all the specified resource
managers.
Reviewed-By: Daniel Gustafsson, Julien Rouhaud
Discussion: https://www.postgresql.org/message-id/
98344bc2-e222-02ad-a75b-
81ffc614c155%40iki.fi
Peter Eisentraut [Thu, 1 Jul 2021 07:17:44 +0000 (09:17 +0200)]
Add tests for UNBOUNDED syntax ambiguity
There is a syntactic ambiguity in the SQL standard. Since UNBOUNDED
is a non-reserved word, it could be the name of a function parameter
and be used as an expression. There is a grammar hack to resolve such
cases as the keyword. Add some tests to record this behavior.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/
b2a09a77-3c8f-7c68-c9b7-
824054f87d98%40enterprisedb.com
David Rowley [Thu, 1 Jul 2021 03:29:06 +0000 (15:29 +1200)]
Improve various places that double the size of a buffer
Several places were performing a tight loop to determine the first power
of 2 number that's > or >= the required memory. Instead of using a loop
for that, we can use pg_nextpower2_32 or pg_nextpower2_64. When we need a
power of 2 number equal to or greater than a given amount, we just pass
the amount to the nextpower2 function. When we need a power of 2 greater
than the amount, we just pass the amount + 1.
Additionally, in tsearch there were a couple of locations that were
performing a while loop when a simple "if" would have done. In both of
these locations only 1 item is being added, so the loop could only have
ever iterated once. Changing the loop into an if statement makes the code
very slightly more optimal as the condition is checked once rather than
twice.
There are quite a few remaining locations that increase the size of the
buffer in the following form:
while (reqsize >= buflen)
{
buflen *= 2;
buf = repalloc(buf, buflen);
}
These are not touched in this commit. repalloc will error out for sizes
larger than MaxAllocSize. Changing these to use pg_nextpower2_32 would
remove the chance of that error being raised. It's unclear from the code
if the sizes could ever become that large, so err on the side of caution.
Discussion: https://postgr.es/m/CAApHDvp=tns7RL4PH0ZR0M+M-YFLquK7218x=0B_zO+DbOma+w@mail.gmail.com
Reviewed-by: Zhihong Yu
Tom Lane [Wed, 30 Jun 2021 14:52:20 +0000 (10:52 -0400)]
Fix portability fallout from commit
dc227eb82.
Give up on trying to mechanically forbid abort() within libpq.
Even though there are no such calls in the source code, we've now
seen three different scenarios where build toolchains silently
insert such calls: gcc does it for profiling, some platforms
implement assert() using it, and icc does so for no visible reason.
Checking for accidental use of exit() seems considerably more
important than checking for abort(), so we'll settle for doing
that for now.
Also, filter out __cxa_atexit() to avoid a false match. It seems
that OpenBSD inserts a call to that despite the fact that libpq
contains no C++ code.
Discussion: https://postgr.es/m/
3128896.
1624742969@sss.pgh.pa.us
Fujii Masao [Wed, 30 Jun 2021 11:57:07 +0000 (20:57 +0900)]
doc: Improve descriptions of tup_returned and tup_fetched in pg_stat_database
Previously the descriptions of tup_returned and tup_fetched columns
in pg_stat_database view were confusing. This commit improves them
so that they represent the following formulas of those columns
more accurately.
* pg_stat_database.tup_returned
= sum(pg_stat_all_tables.seq_tup_read)
+ sum(pg_stat_all_indexes.idx_tup_read)
* pg_stat_database.tup_fetched
= sum(pg_stat_all_tables.idx_tup_fetch)
In these formulas, note that the counters for some system catalogs
like pg_database shared across all databases of a cluster are excluded
from the calculations of sum.
Author: Masahiro Ikeda
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/
9eeeccdb-5dd7-90f9-2807-
a4b5d2b76ca3@oss.nttdata.com
Fujii Masao [Wed, 30 Jun 2021 10:33:18 +0000 (19:33 +0900)]
doc: Add type information for postgres_fdw parameters.
Author: Shinya Kato
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/TYAPR01MB2896DEB25C3B0D57F6139768C40F9@TYAPR01MB2896.jpnprd01.prod.outlook.com
Peter Eisentraut [Wed, 30 Jun 2021 06:29:03 +0000 (08:29 +0200)]
genbki stricter error handling
Instead of just writing warnings for invalid cross-catalog lookups,
count the errors and error out at the end.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/
ca8ee41d-241b-1bf3-71f0-
aaf1add6d3c5%40enterprisedb.com
Amit Kapila [Wed, 30 Jun 2021 05:59:53 +0000 (11:29 +0530)]
Replace magic constants used in pg_stat_get_replication_slot().
A few variables have been using 10 as a magic constant while
PG_STAT_GET_REPLICATION_SLOT_COLS can be used instead.
Author: Masahiko Sawada
Reviewed-By: Amit Kapila
Backpatch-through: 14, where it was introduced
Discussion: https://postgr.es/m/CAD21AoBvqODDfmD17DkEuPCvV2KbruukXQ2Vwrv5Xi-TsAsTJA@mail.gmail.com
Amit Kapila [Wed, 30 Jun 2021 04:07:59 +0000 (09:37 +0530)]
Allow streaming the changes after speculative aborts.
Until now, we didn't allow to stream the changes in logical replication
till we receive speculative confirm or the next DML change record after
speculative inserts. The reason was that we never use to process
speculative aborts but after commit
4daa140a2f it is possible to process
them so we can allow streaming once we receive speculative abort after
speculative insertion.
We decided to backpatch to 14 where the feature for streaming in progress
transactions have been introduced as this is a minor change and makes that
functionality better.
Author: Amit Kapila
Reviewed-By: Dilip Kumar
Backpatch-through: 14
Discussion: https://postgr.es/m/CAA4eK1KdqmTCtrBR6oFfGELrLLbDLDedL6zACcsUOQuTJBj1vw@mail.gmail.com
Amit Kapila [Wed, 30 Jun 2021 03:15:47 +0000 (08:45 +0530)]
Allow enabling two-phase option via replication protocol.
Extend the replication command CREATE_REPLICATION_SLOT to support the
TWO_PHASE option. This will allow decoding commands like PREPARE
TRANSACTION, COMMIT PREPARED and ROLLBACK PREPARED for slots created with
this option. The decoding of the transaction happens at prepare command.
This patch also adds support of two-phase in pg_recvlogical via a new
option --two-phase.
This option will also be used by future patches that allow streaming of
transactions at prepare time for built-in logical replication. With this,
the out-of-core logical replication solutions can enable replication of
two-phase transactions via replication protocol.
Author: Ajin Cherian
Reviewed-By: Jeff Davis, Vignesh C, Amit Kapila
Discussion:
https://postgr.es/m/
02DA5F5E-CECE-4D9C-8B4B-
418077E2C010@postgrespro.ru
https://postgr.es/m/
64b9f783c6e125f18f88fbc0c0234e34e71d8639.camel@j-davis.com
Michael Paquier [Wed, 30 Jun 2021 02:48:53 +0000 (11:48 +0900)]
Fix incorrect PITR message for transaction ROLLBACK PREPARED
Reaching PITR on such a transaction would cause the generation of a LOG
message mentioning a transaction committed, not aborted.
Oversight in
4f1b890.
Author: Simon Riggs
Discussion: https://postgr.es/m/CANbhV-GJ6KijeCgdOrxqMCQ+C8QiK657EMhCy4csjrPcEUFv_Q@mail.gmail.com
Backpatch-through: 9.6
Michael Paquier [Wed, 30 Jun 2021 00:58:59 +0000 (09:58 +0900)]
Optimize pg_checksums --enable where checksum is already set
This commit prevents pg_checksums to do a rewrite of a block if it has
no need to, in the case where the computed checksum matches with what's
already stored in the block read. This is helpful to accelerate
successive runs of the tool when the previous ones got interrupted, for
example.
The number of blocks and files written is tracked and reported by the
tool once finished. Note that the final flush of the data folder
happens even if no blocks are written, as it could be possible that a
previous interrupted run got stopped while doing a flush.
Author: Greg Sabino Mullane
Reviewed-by: Paquier Michael, Julien Rouhaud
Discussion: https://postgr.es/m/CAKAnmmL+k6goxmVzQJB+0bAR0PN1sgo6GDUXJhyhUmVMze1QAw@mail.gmail.com
Alexander Korotkov [Tue, 29 Jun 2021 20:18:09 +0000 (23:18 +0300)]
Fixes for multirange selectivity estimation
* Fix enumeration of the multirange operators in calc_multirangesel() and
calc_multirangesel() switches.
* Add more regression tests for matching to empty ranges/multiranges.
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/
c5269c65-f967-77c5-ff7c-
15e621c47f6a%40gmail.com
Author: Alexander Korotkov
Backpatch-through: 14, where multiranges were introduced
Alvaro Herrera [Tue, 29 Jun 2021 19:01:29 +0000 (15:01 -0400)]
Fix libpq state machine in pipeline mode
The original coding required that PQpipelineSync had been called before
the first call to PQgetResult, and failure to do that would result in an
unexpected NULL result being returned. Fix by setting the right state
when a query is sent, rather than leaving it unchanged and having
PQpipelineSync apply the necessary state change.
A new test case to verify the behavior is added, which relies on the new
PQsendFlushRequest() function added by commit
a7192326c74d.
Backpatch to 14, where pipeline mode was added.
Reported-by: Boris Kolpackov <boris@codesynthesis.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/boris.
20210616110321@codesynthesis.com
Alvaro Herrera [Tue, 29 Jun 2021 18:37:39 +0000 (14:37 -0400)]
Add PQsendFlushRequest to libpq
This new libpq function allows the application to send an 'H' message,
which instructs the server to flush its outgoing buffer.
This hasn't been needed so far because the Sync message already requests
a buffer; and I failed to realize that this was needed in pipeline mode
because PQpipelineSync also causes the buffer to be flushed. However,
sometimes it is useful to request a flush without establishing a
synchronization point.
Backpatch to 14, where pipeline mode was introduced in libpq.
Reported-by: Boris Kolpackov <boris@codesynthesis.com>
Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/
202106252350.t76x73nt643j@alvherre.pgsql
Tom Lane [Tue, 29 Jun 2021 18:34:31 +0000 (14:34 -0400)]
Fix bogus logic for reporting which hash partition conflicts.
Commit
efbfb6424 added logic for reporting exactly which existing
partition conflicts when complaining that a new hash partition's
modulus isn't compatible with the existing ones. However, it
misunderstood the partitioning data structure, and would select
the wrong partition in some cases, or crash outright due to fetching
a bogus table OID in other cases.
Per bug #17076 from Alexander Lakhin. Fix by Amit Langote;
some further work on the code comments by me.
Discussion: https://postgr.es/m/17076-
89a16ae835d329b9@postgresql.org
Tom Lane [Tue, 29 Jun 2021 15:46:17 +0000 (11:46 -0400)]
Add a build-time check that libpq doesn't call exit() or abort().
Directly exiting or aborting seems like poor form for a general-purpose
library. Now that libpq liberally uses bits out of src/common/,
it's very easy to accidentally include code that would do something
unwanted like calling exit(1) after OOM --- see for example
8ec00dc5c.
Hence, add a simple cross-check that no such calls have made it into
libpq.so.
The cross-check depends on nm(1) being available and being able to
work on a shared library, which probably isn't true everywhere.
But we can just make the test silently do nothing if nm fails.
As long as the check is effective on common platforms, that should
be good enough. (By the same logic, I've not worried about providing
an equivalent test in MSVC builds.)
Discussion: https://postgr.es/m/
3128896.
1624742969@sss.pgh.pa.us
Tom Lane [Tue, 29 Jun 2021 15:31:08 +0000 (11:31 -0400)]
Remove libpq's use of abort(3) to handle mutex failure cases.
Doing an abort() seems all right in development builds, but not in
production builds of general-purpose libraries. However, the functions
that were doing this lack any way to report a failure back up to their
callers. It seems like we can just get away with ignoring failures in
production builds, since (a) no such failures have been reported in the
dozen years that the code's been like this, and (b) failure to enforce
mutual exclusion during fe-auth.c operations would likely not cause any
problems anyway in most cases. (The OpenSSL callbacks that use this
macro are obsolete, so even less likely to cause interesting problems.)
Possibly a better answer would be to break compatibility of the
pgthreadlock_t callback API, but in the absence of field problem
reports, it doesn't really seem worth the trouble.
Discussion: https://postgr.es/m/
3131385.
1624746109@sss.pgh.pa.us
Noah Misch [Tue, 29 Jun 2021 07:44:57 +0000 (00:44 -0700)]
Remove literal backslash from Perl \Q ... \E.
The behavior changed sometime after Perl 5.8.9, and "man perlre" says it
"may lead to confusing results." Per buildfarm member gaur. This
repairs commit
a7a7be1f2fa6b9f0f48e69f12256d8f588af729b.
Discussion: https://postgr.es/m/
20210629053627.GA2061079@rfd.leadboat.com
Peter Eisentraut [Tue, 29 Jun 2021 05:57:16 +0000 (07:57 +0200)]
Add index OID macro argument to DECLARE_INDEX
Instead of defining symbols such as AmOidIndexId explicitly, include
them as an argument of DECLARE_INDEX() and have genbki.pl generate the
way as the table OID symbols from the CATALOG() declaration.
Reviewed-by: John Naylor <john.naylor@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/
ccef1e46-a404-25b1-9b4c-
85f2c08e1f28%40enterprisedb.com
Michael Paquier [Tue, 29 Jun 2021 04:57:45 +0000 (13:57 +0900)]
Bump XLOG_PAGE_MAGIC for format changes related to FPW compression
Oversight in
4035cd5, spotted by Tom Lane.
Discussion: https://postgr.es/m/365778.
1624941613@sss.pgh.pa.us
Michael Paquier [Tue, 29 Jun 2021 02:54:11 +0000 (11:54 +0900)]
Fix compilation warning in xloginsert.c
This is reproducible with gcc using at least -O0. The last checks
validating the compression of a block could not be reached with this
variable not set, but let's be clean.
Oversight in
4035cd5, per buildfarm member lapwing.
Michael Paquier [Tue, 29 Jun 2021 02:17:55 +0000 (11:17 +0900)]
Add support for LZ4 with compression of full-page writes in WAL
The logic is implemented so as there can be a choice in the compression
used when building a WAL record, and an extra per-record bit is used to
track down if a block is compressed with PGLZ, LZ4 or nothing.
wal_compression, the existing parameter, is changed to an enum with
support for the following backward-compatible values:
- "off", the default, to not use compression.
- "pglz" or "on", to compress FPWs with PGLZ.
- "lz4", the new mode, to compress FPWs with LZ4.
Benchmarking has showed that LZ4 outclasses easily PGLZ. ZSTD would be
also an interesting choice, but going just with LZ4 for now makes the
patch minimalistic as toast compression is already able to use LZ4, so
there is no need to worry about any build-related needs for this
implementation.
Author: Andrey Borodin, Justin Pryzby
Reviewed-by: Dilip Kumar, Michael Paquier
Discussion: https://postgr.es/m/
3037310D-ECB7-4BF1-AF20-
01C10BB33A33@yandex-team.ru
Noah Misch [Tue, 29 Jun 2021 01:34:56 +0000 (18:34 -0700)]
Skip WAL recycling and preallocation during archive recovery.
The previous commit addressed the chief consequences of a race condition
between InstallXLogFileSegment() and KeepFileRestoredFromArchive(). Fix
three lesser consequences. A spurious durable_rename_excl() LOG message
remained possible. KeepFileRestoredFromArchive() wasted the proceeds of
WAL recycling and preallocation. Finally, XLogFileInitInternal() could
return a descriptor for a file that KeepFileRestoredFromArchive() had
already unlinked. That felt like a recipe for future bugs.
Discussion: https://postgr.es/m/
20210202151416.GB3304930@rfd.leadboat.com
Noah Misch [Tue, 29 Jun 2021 01:34:56 +0000 (18:34 -0700)]
Don't ERROR on PreallocXlogFiles() race condition.
Before a restartpoint finishes PreallocXlogFiles(), a startup process
KeepFileRestoredFromArchive() call can unlink the preallocated segment.
If a CHECKPOINT sql command had elicited the restartpoint experiencing
the race condition, that sql command failed. Moreover, the restartpoint
omitted its log_checkpoints message and some inessential resource
reclamation. Prevent the ERROR by skipping open() of the segment.
Since these consequences are so minor, no back-patch.
Discussion: https://postgr.es/m/
20210202151416.GB3304930@rfd.leadboat.com
Noah Misch [Tue, 29 Jun 2021 01:34:56 +0000 (18:34 -0700)]
Remove XLogFileInit() ability to unlink a pre-existing file.
Only initdb used it. initdb refuses to operate on a non-empty directory
and generally does not cope with pre-existing files of other kinds.
Hence, use the opportunity to simplify.
Discussion: https://postgr.es/m/
20210202151416.GB3304930@rfd.leadboat.com
Noah Misch [Tue, 29 Jun 2021 01:34:55 +0000 (18:34 -0700)]
In XLogFileInit(), fix *use_existent postcondition to suit callers.
Infrequently, the mismatch caused log_checkpoints messages and
TRACE_POSTGRESQL_CHECKPOINT_DONE() to witness an "added" count too high
by one. Since that consequence is so minor, no back-patch.
Discussion: https://postgr.es/m/
20210202151416.GB3304930@rfd.leadboat.com
Noah Misch [Tue, 29 Jun 2021 01:34:55 +0000 (18:34 -0700)]
Remove XLogFileInit() ability to skip ControlFileLock.
Cold paths, initdb and end-of-recovery, used it. Don't optimize them.
Discussion: https://postgr.es/m/
20210202151416.GB3304930@rfd.leadboat.com
Noah Misch [Tue, 29 Jun 2021 01:34:55 +0000 (18:34 -0700)]
Dump COMMENT ON SCHEMA public.
As we do for other attributes of the public schema, omit the COMMENT
command when its payload would match what initdb had installed. For
dumps that do carry this new COMMENT command, non-superusers restoring
them are likely to get an error.
Reviewed by Asif Rehman.
Discussion: https://postgr.es/m/
ab48a34c-60f6-e388-502a-
3e5fe46a2dae@postgresfriends.org
Noah Misch [Tue, 29 Jun 2021 01:34:55 +0000 (18:34 -0700)]
Dump public schema ownership and security labels.
As a side effect, this corrects dumps of public schema ACLs in databases
where the DBA dropped and recreated that schema.
Reviewed by Asif Rehman.
Discussion: https://postgr.es/m/
20201229134924.GA1431748@rfd.leadboat.com
Tom Lane [Mon, 28 Jun 2021 22:05:46 +0000 (18:05 -0400)]
Doc: further updates for RELEASE_CHANGES process notes.
Mention expectations for email notifications of appropriate lists
when a branch is made or retired.
(I've been doing that informally for years, but it's better to have
it written down.)
Peter Geoghegan [Mon, 28 Jun 2021 20:08:46 +0000 (13:08 -0700)]
Improve pgindent release workflow.
Update RELEASE_CHANGES to direct the reader towards completing the steps
outlined in the pgindent README, both as a pre-beta task and as a task
to be performed when starting a new development cycle.
This makes it less likely that somebody will miss updating the
.git-blame-ignore-revs file when running pgindent against the tree as a
routine release change task.
Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAH2-Wz=2PjF4As8dWECArsXxLKganYmQ-s0UeGqHHbHhqDKqeA@mail.gmail.com
Tom Lane [Mon, 28 Jun 2021 18:17:41 +0000 (14:17 -0400)]
Don't use abort(3) in libpq's fe-print.c.
Causing a core dump on out-of-memory seems pretty unfriendly,
and surely is far outside the expected behavior of a general-purpose
library. Just print an error message (as we did already) and return.
These functions unfortunately don't have an error return convention,
but code using them is probably just looking for a quick-n-dirty
print method and wouldn't bother to check anyway.
Although these functions are semi-deprecated, it still seems
appropriate to back-patch this. In passing, also back-patch
b90e6cef1, just to reduce cosmetic differences between the
branches.
Discussion: https://postgr.es/m/
3122443.
1624735363@sss.pgh.pa.us
Tom Lane [Mon, 28 Jun 2021 16:40:37 +0000 (12:40 -0400)]
Don't depend on -fwrapv semantics in pgbench's random() function.
Instead use the common/int.h functions to check for integer overflow
in a more C-standard-compliant fashion. This is motivated by recent
failures on buildfarm member moonjelly, where it appears that
development-tip gcc is optimizing without regard to the -fwrapv
switch. Presumably that's a gcc bug that will be fixed soon, but
we might as well install cleaner coding here rather than wait.
(This does not address the question of whether we'll ever be able
to get rid of using -fwrapv. Testing shows that this spot is the
only place where doing so creates visible regression test failures,
but unfortunately that proves very little.)
Back-patch to v12. The common/int.h functions exist in v11, but
that branch doesn't use them in any client-side code. I judge
that this case isn't interesting enough in the real world to take
even a small risk of issues from being the first such use.
Tom Lane and Fabien Coelho
Discussion: https://postgr.es/m/73927.
1624815543@sss.pgh.pa.us
Peter Geoghegan [Mon, 28 Jun 2021 16:22:24 +0000 (09:22 -0700)]
Add pgindent commit to git-blame-ignore-revs file.
Add entry for recent commit
e1c1c30f.
Andrew Dunstan [Mon, 28 Jun 2021 15:31:16 +0000 (11:31 -0400)]
Stamp HEAD as 15devel.
Let the hacking begin ...
Andrew Dunstan [Mon, 28 Jun 2021 15:05:54 +0000 (11:05 -0400)]
Pre branch pgindent / pgperltidy run
Along the way make a slight adjustment to
src/include/utils/queryjumble.h to avoid an unused typedef.
Peter Eisentraut [Mon, 28 Jun 2021 06:36:44 +0000 (08:36 +0200)]
Message style improvements
Amit Kapila [Mon, 28 Jun 2021 05:26:53 +0000 (10:56 +0530)]
Improve RelationGetIdentityKeyBitmap().
We were using RelationGetIndexList() to update the relation's replica
identity index but instead, we can directly use RelationGetReplicaIndex()
which uses the same functionality. This is a minor code readability
improvement.
Author: Japin Li
Reviewed-By: Takamichi Osumi, Amit Kapila
Discussion: https://postgr.es/m/
4C99A862-69C8-431F-960A-
81B1151F1B89@enterprisedb.com
Amit Kapila [Mon, 28 Jun 2021 03:59:38 +0000 (09:29 +0530)]
Fix race condition in TransactionGroupUpdateXidStatus().
When we cannot immediately acquire XactSLRULock in exclusive mode at
commit time, we add ourselves to a list of processes that need their XIDs
status update. We do this if the clog page where we need to update the
current transaction status is the same as the group leader's clog page,
otherwise, we allow the caller to clear it by itself. Now, when we can't
add ourselves to any group, we were not clearing the current proc if it
has already become a member of some group which was leading to an
assertion failure when the same proc was assigned to another backend after
the current backend exits.
Reported-by: Alexander Lakhin
Bug: 17072
Author: Amit Kapila
Tested-By: Alexander Lakhin
Backpatch-through: 11, where it was introduced
Discussion: https://postgr.es/m/17072-
2f8764857ef2c92a@postgresql.org
Thomas Munro [Mon, 28 Jun 2021 03:17:43 +0000 (15:17 +1200)]
Change recovery_init_sync_method to PGC_SIGHUP.
The setting has no effect except during startup. It's still nice to be
able to change it dynamically, which is expected to be pretty useful to
an admin following crash recovery when restarting the cluster is not so
appealing.
Per discussions following commits
2941138e6 and
61752afb2.
Author: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/
20210529192321.GM2082%40telsasoft.com
Michael Paquier [Mon, 28 Jun 2021 03:11:18 +0000 (12:11 +0900)]
Fix variable initialization with ALTER SUBSCRIPTION DROP PUBLICATION
copy_data is not a supported option with this sub-command of ALTER
SUBSCRIPTION, which would not make a variable related to it initialized
after parsing the option set in DefElems. A refresh could then refer to
it.
Author: Ranier Vilela
Reviewed-by: Peter Smith
Discussion: https://postgr.es/m/CAEudQAp5P8nr=ze2Gv=BMj=DJFZnrvendZCZcC-fos3QiDe2sg@mail.gmail.com
Michael Paquier [Mon, 28 Jun 2021 02:17:05 +0000 (11:17 +0900)]
Add test for CREATE INDEX CONCURRENTLY with not-so-immutable predicate
83158f7 has improved index_set_state_flags() so as it is possible to use
transactional updates when updating pg_index state flags, but there was
not really a test case which stressed directly the possibility it fixed.
This commit adds such a test, using a predicate that looks valid in
appearance but calls a stable function.
Author: Andrey Lepikhov
Discussion: https://postgr.es/m/
9b905019-5297-7372-0ad2-
e1a4bb66a719@postgrespro.ru
Backpatch-through: 9.6
Tom Lane [Sun, 27 Jun 2021 16:45:04 +0000 (12:45 -0400)]
Remove memory leaks in isolationtester.
specscanner.l leaked a kilobyte of memory per token of the spec file.
Apparently somebody thought that the introductory code block would be
executed once; but it's once per yylex() call.
A couple of functions in isolationtester.c leaked small amounts of
memory due to not bothering to free one-time allocations. Might
as well improve these so that valgrind gives this program a clean
bill of health. Also get rid of an ugly static variable.
Coverity complained about one of the one-time leaks, which led me
to try valgrind'ing isolationtester, which led to discovery of the
larger leak.
Peter Eisentraut [Sun, 27 Jun 2021 07:41:16 +0000 (09:41 +0200)]
Error message refactoring
Take some untranslatable things out of the message and replace by
format placeholders, to reduce translatable strings and reduce
translation mistakes.
Tom Lane [Sat, 26 Jun 2021 19:45:16 +0000 (15:45 -0400)]
Doc: update v14 release notes for revert of commit
c0cb87fbb.
Tom Lane [Sat, 26 Jun 2021 18:20:17 +0000 (14:20 -0400)]
Remove undesirable libpq dependency on stringinfo.c.
Commit
c0cb87fbb unwisely introduced a dependency on the StringInfo
machinery in fe-connect.c. We must not use that in libpq, because
it will do a summary exit(1) if it hits OOM, and that is not
appropriate behavior for a general-purpose library. The goal of
allowing arbitrary line lengths in service files doesn't seem like
it's worth a lot of effort, so revert back to the previous method
of using a stack-allocated buffer and failing on buffer overflow.
This isn't an exact revert though. I kept that patch's refactoring
to have a single exit path, as that seems cleaner than having each
error path know what to do to clean up. Also, I made the fixed-size
buffer 1024 bytes not 256, just to push off the need for an expandable
buffer some more.
There is more to do here; in particular the lack of any mechanical
check for this type of mistake now seems pretty hazardous. But this
fix gets us back to the level of robustness we had in v13, anyway.
Discussion: https://postgr.es/m/
daeb22ec6ca8ef61e94d766a9b35fb03cabed38e.camel@vmware.com
Michael Paquier [Sat, 26 Jun 2021 04:52:48 +0000 (13:52 +0900)]
Remove non-existing variable reference in MSVC's Solution.pm
The version string is grabbed from PACKAGE_VERSION in pg_config.h in the
MSVC build since
8f4fb4c6, but an error message referenced a variable
that existed before that. This had no consequences except if one messes
up enough with the version number of the build.
Author: Anton Voloshin
Discussion: https://postgr.es/m/
af79ee1b-9962-b299-98e1-
f90a289e19e6@postgrespro.ru
Backpatch-through: 13
Michael Paquier [Sat, 26 Jun 2021 03:39:54 +0000 (12:39 +0900)]
Remove some useless logs from the TAP tests of pgbench
002_pgbench_no_server was printing some array pointers instead of the
actual contents of those arrays for the expected outputs of stdout and
stderr for a tested command. This does not add any new information that
can help with debugging as the test names allow to track failure
locations, if any.
This commit simply removes those logs as the rest of the printed
information is redundant with command_checks_all().
Per discussion with Andrew Dunstan and Álvaro Herrera.
Discussion: https://postgr.es/m/YNXNFaG7IgkzZanD@paquier.xyz
Backpatch-through: 11
Tom Lane [Fri, 25 Jun 2021 17:59:38 +0000 (13:59 -0400)]
Remove unnecessary failure cases in RemoveRoleFromObjectPolicy().
It's not really necessary for this function to open or lock the
relation associated with the pg_policy entry it's modifying. The
error checks it's making on the rel are if anything counterproductive
(e.g., if we don't want to allow installation of policies on system
catalogs, here is not the place to prevent that). In particular, it
seems just wrong to insist on an ownership check. That has the net
effect of forcing people to use superuser for DROP OWNED BY, which
surely is not an effect we want. Also there is no point in rebuilding
the dependencies of the policy expressions, which aren't being
changed. Lastly, locking the table also seems counterproductive; it's
not helping to prevent race conditions, since we failed to re-read the
pg_policy row after acquiring the lock. That means that concurrent
DDL would likely result in "tuple concurrently updated/deleted"
errors; which is the same behavior this code will produce, with less
overhead.
Per discussion of bug #17062. Back-patch to all supported versions,
as the failure cases this eliminates seem just as undesirable in 9.6
as in HEAD.
Discussion: https://postgr.es/m/
1573181.
1624220108@sss.pgh.pa.us
Tom Lane [Fri, 25 Jun 2021 15:18:15 +0000 (11:18 -0400)]
Doc: remove commit
f560209c6 from v14 release notes.
Now that this has been back-patched, it's no longer a new feature
for v14.
Michael Paquier [Fri, 25 Jun 2021 11:15:24 +0000 (20:15 +0900)]
Cleanup some code related to pgbench log checks in TAP tests
This fixes a couple of problems within the so-said code of this commit
subject:
- Replace the use of open() with slurp_file(), fixing an issue reported
by buildfarm member fairywren whose perl installation keep around CRLF
characters, causing the matching patterns for the logs to fail.
- Remove the eval block, which is not really necessary.
This set of issues has come into light after fixing a different issue
with
c13585fe, and this is wrong since this code has been introduced.
Reported-by: Andrew Dunstan, and buildfarm member fairywren
Author: Michael Paquier
Reviewed-by: Andrew Dunstan
Discussion: https://postgr.es/m/
0f49303e-7784-b3ee-200b-
cbf67be2eb9e@dunslane.net
Backpatch-through: 11
Peter Eisentraut [Fri, 25 Jun 2021 09:40:06 +0000 (11:40 +0200)]
Put option listing back into alphabetical order
Peter Eisentraut [Fri, 25 Jun 2021 07:51:24 +0000 (09:51 +0200)]
Fixes in ALTER SUBSCRIPTION DROP PUBLICATION code
ALTER SUBSCRIPTION DROP PUBLICATION does not actually support
copy_data option, so remove it from tab completion.
Also, reword the error message that is thrown when all the
publications from a subscription are specified to be dropped.
Also, made few doc and cosmetic adjustments.
Author: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Reviewed-by: Japin Li <japinli@hotmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CALDaNm21RwsDzs4xj14ApteAF7auyyomHNnp+NEL-sH8m-jMvQ@mail.gmail.com
Peter Eisentraut [Fri, 25 Jun 2021 06:11:10 +0000 (08:11 +0200)]
doc: Change reloption data type spelling for consistency
Use "floating point" rather than "float4", like everywhere else in
this context.
Author: Shinya11.Kato@nttdata.com
Discussion: https://www.postgresql.org/message-id/flat/TYAPR01MB28965989AF84B57FC351B97BC40F9@TYAPR01MB2896.jpnprd01.prod.outlook.com
Peter Eisentraut [Fri, 25 Jun 2021 05:55:34 +0000 (07:55 +0200)]
Remove redundant variable pageSize in gistinitpage
In gistinitpage, pageSize variable looks redundant, instead just
pass BLCKSZ. This will be consistent with its peers BloomInitPage,
brin_page_init and SpGistInitPage.
Author: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/CALj2ACWj=V1k5591eeZK2sOg2FYuBUp6azFO8tMkBtGfXf8PMQ@mail.gmail.com
Amit Kapila [Fri, 25 Jun 2021 02:52:44 +0000 (08:22 +0530)]
Doc: Fix minor formatting issue in logicaldecoding.sgml.
Author: Guillaume Lelarge
Discussion: https://www.postgresql.org/message-id/CAECtzeXf3_oZoU6mgFCOy5+pDZ5n4XtH0Da4a5n_KacraVWiHQ@mail.gmail.com
Michael Paquier [Fri, 25 Jun 2021 02:29:03 +0000 (11:29 +0900)]
doc: Add acronyms for MITM and SNI
This adds MITM and SNI as acronyms, as the documentation already had
them marked up with <acronym>.
While on it, make sure to spell man-in-the-middle with dashes
consistently, and add acronyms for those new terms where appropriate.
Author: Daniel Gustafsson
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/
CE12DD5C-4BB3-4166-BC9A-
39779568734C@yesql.se
Michael Paquier [Fri, 25 Jun 2021 00:56:44 +0000 (09:56 +0900)]
Add more debugging information with log checks in TAP tests of pgbench
fairywren is not happy with the pattern checks introduced by
c13585f.
I am not sure if this outlines a bug in pgbench or if the regex patterns
used in the tests are too restrictive for this buildfarm member's
environment. This adds more debugging information to show the log
entries that do not match with the expected pattern, to help in finding
out what's happening. That seems like a good addition in the long-term
anyway as that may not be the only issue in this area.
Discussion: https://postgr.es/m/YNUad2HvgW+6eXyo@paquier.xyz
Michael Paquier [Thu, 24 Jun 2021 23:40:16 +0000 (08:40 +0900)]
doc: Move remove_temp_files_after_crash to section for developer options
The main goal of this option is to allow inspecting temporary files for
debugging purposes, so moving the parameter there is natural.
Oversight in
cd91de0.
Reported-by: Justin Pryzby
Author: Euler Taveira
Discussion: https://postgr.es/m/
20210612004347.GP16435@telsasoft.com
Thomas Munro [Thu, 24 Jun 2021 21:55:26 +0000 (09:55 +1200)]
Prepare for forthcoming LLVM 13 API change.
LLVM 13 (due out in September) has changed the semantics of
LLVMOrcAbsoluteSymbols(), so we need to bump some reference counts to
avoid a double-free that causes crashes and bad query results.
A proactive change seems necessary to avoid having a window of time
where our respective latest releases would interact badly. It's
possible that the situation could change before then, though.
Thanks to Fabien Coelho for monitoring bleeding edge LLVM and Andres
Freund for tracking down the change.
Back-patch to 11, where the JIT code arrived.
Discussion: https://postgr.es/m/CA%2BhUKGLEy8mgtN7BNp0ooFAjUedDTJj5dME7NxLU-m91b85siA%40mail.gmail.com