Fix handling of missing files when using pg_rewind with online source

When working with an online source cluster, pg_rewind gets a list of all
the files in the source data directory using a WITH RECURSIVE query,
returning a NULL result for a file's metadata if it gets removed between
the moment it is listed in a directory and the moment its metadata is
obtained with pg_stat_file() (say a recycled WAL segment).  The query
result was processed in such a way that for each tuple we checked only
that the first file's metadata was NULL.  This could have two
consequences, both resulting in a failure of the rewind:
- If the first tuple referred to a removed file, all files from the
source would be ignored.
- Any file actually missing would not be considered as such.

While on it, rework slightly the code so as no values are saved if we
know that a file is going to be skipped.

Issue introduced by b36805f, so backpatch down to 9.5.

Author: Justin Pryzby, Michael Paquier
Reviewed-by: Daniel Gustafsson, Masahiko Sawada
Discussion: https://postgr.es/m/20200713061010.GC23581@telsasoft.com
Backpatch-through: 9.5

diff --git a/src/bin/pg_rewind/libpq_fetch.c b/src/bin/pg_rewind/libpq_fetch.c
index 1dbbceab0bd48d6e78174955fd4d75cf2773a723..c44648f82318c8a0156977a3c06d222a3b644855 100644 (file)
--- a/src/bin/pg_rewind/libpq_fetch.c
+++ b/src/bin/pg_rewind/libpq_fetch.c
@@ -214,13 +214,13 @@ libpqProcessFileList(void)
    /* Read result to local variables */
    for (i = 0; i < PQntuples(res); i++)
    {
-       char       *path = PQgetvalue(res, i, 0);
-       int64       filesize = atol(PQgetvalue(res, i, 1));
-       bool        isdir = (strcmp(PQgetvalue(res, i, 2), "t") == 0);
-       char       *link_target = PQgetvalue(res, i, 3);
+       char       *path;
+       int64       filesize;
+       bool        isdir;
+       char       *link_target;
        file_type_t type;
 
-       if (PQgetisnull(res, 0, 1))
+       if (PQgetisnull(res, i, 1))
        {
            /*
             * The file was removed from the server while the query was
@@ -229,6 +229,11 @@ libpqProcessFileList(void)
            continue;
        }
 
+       path = PQgetvalue(res, i, 0);
+       filesize = atol(PQgetvalue(res, i, 1));
+       isdir = (strcmp(PQgetvalue(res, i, 2), "t") == 0);
+       link_target = PQgetvalue(res, i, 3);
+
        if (link_target[0])
            type = FILE_TYPE_SYMLINK;
        else if (isdir)