Prefetch data referenced by the WAL, take II.
authorThomas Munro <tmunro@postgresql.org>
Thu, 7 Apr 2022 07:28:40 +0000 (19:28 +1200)
committerThomas Munro <tmunro@postgresql.org>
Thu, 7 Apr 2022 07:42:14 +0000 (19:42 +1200)
commit5dc0418fab281d017a61a5756240467af982bdfd
treecdcfda92621a9a7cd458999ede8f3974ef2a0bc1
parent9553b4115f1879f66935f42fff0b798ef91866d0
Prefetch data referenced by the WAL, take II.

Introduce a new GUC recovery_prefetch.  When enabled, look ahead in the
WAL and try to initiate asynchronous reading of referenced data blocks
that are not yet cached in our buffer pool.  For now, this is done with
posix_fadvise(), which has several caveats.  Since not all OSes have
that system call, "try" is provided so that it can be enabled where
available.  Better mechanisms for asynchronous I/O are possible in later
work.

Set to "try" for now for test coverage.  Default setting to be finalized
before release.

The GUC wal_decode_buffer_size limits the distance we can look ahead in
bytes of decoded data.

The existing GUC maintenance_io_concurrency is used to limit the number
of concurrent I/Os allowed, based on pessimistic heuristics used to
infer that I/Os have begun and completed.  We'll also not look more than
maintenance_io_concurrency * 4 block references ahead.

Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Reviewed-by: Alvaro Herrera <alvherre@2ndquadrant.com> (earlier version)
Reviewed-by: Andres Freund <andres@anarazel.de> (earlier version)
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> (earlier version)
Tested-by: Tomas Vondra <tomas.vondra@2ndquadrant.com> (earlier version)
Tested-by: Jakub Wartak <Jakub.Wartak@tomtom.com> (earlier version)
Tested-by: Dmitry Dolgov <9erthalion6@gmail.com> (earlier version)
Tested-by: Sait Talha Nisanci <Sait.Nisanci@microsoft.com> (earlier version)
Discussion: https://postgr.es/m/CA%2BhUKGJ4VJN8ttxScUFM8dOKX0BrBiboo5uz1cq%3DAovOddfHpA%40mail.gmail.com
27 files changed:
doc/src/sgml/config.sgml
doc/src/sgml/monitoring.sgml
doc/src/sgml/wal.sgml
src/backend/access/transam/Makefile
src/backend/access/transam/xlog.c
src/backend/access/transam/xlogprefetcher.c [new file with mode: 0644]
src/backend/access/transam/xlogreader.c
src/backend/access/transam/xlogrecovery.c
src/backend/access/transam/xlogutils.c
src/backend/catalog/system_views.sql
src/backend/storage/buffer/bufmgr.c
src/backend/storage/freespace/freespace.c
src/backend/storage/ipc/ipci.c
src/backend/storage/smgr/md.c
src/backend/utils/adt/pgstatfuncs.c
src/backend/utils/misc/guc.c
src/backend/utils/misc/postgresql.conf.sample
src/include/access/xlog.h
src/include/access/xlogprefetcher.h [new file with mode: 0644]
src/include/access/xlogreader.h
src/include/access/xlogutils.h
src/include/catalog/catversion.h
src/include/catalog/pg_proc.dat
src/include/utils/guc.h
src/include/utils/guc_tables.h
src/test/regress/expected/rules.out
src/tools/pgindent/typedefs.list