Fix memory in Incremental Sort rescans

The Incremental Sort had a couple issues, resulting in ing memory
during rescans, possibly triggering OOM. The code had a couple of
related flaws:

1. During rescans, the sort states were reset but then also set to NULL
   (despite the comment saying otherwise). ExecIncrementalSort then
   sees NULL and initializes a new sort state, ing the memory used
   by the old one.

2. Initializing the sort state also automatically rebuilt the info about
   presorted keys, ing the already initialized info. presorted_keys
   was also unnecessarily reset to NULL.

 by James Coleman, based on es by Laurenz Albe and Tom Lane.
Back to 13, where Incremental Sort was introduced.

Author: James Coleman, Laurenz Albe, Tom Lane
Reported-by: Laurenz Albe, Zu-Ming Jiang
Back-through: 13
Discussion: https://postgr.es/m/b2bd02dff61af15e3526293e2771f874cf2a3be7.camel%40cybertec.at
Discussion: https://postgr.es/m/db03c582-086d-e7cd-d4a1-3bc722f81765%40inf.ethz.ch

diff --git a/src/backend/executor/nodeIncrementalSort.c b/src/backend/executor/nodeIncrementalSort.c
index 34257ce34be9d693aa5a6312c399d250bfb550fc..7683e3341cde9abbafa18cde1cba1f44a20b3903 100644 (file)
--- a/src/backend/executor/nodeIncrementalSort.c
+++ b/src/backend/executor/nodeIncrementalSort.c
@@ -1140,7 +1140,6 @@ ExecReScanIncrementalSort(IncrementalSortState *node)
    node->outerNodeDone = false;
    node->n_fullsort_remaining = 0;
    node->bound_Done = 0;
-   node->presorted_keys = NULL;
 
    node->execution_status = INCSORT_LOADFULLSORT;
 
@@ -1153,15 +1152,9 @@ ExecReScanIncrementalSort(IncrementalSortState *node)
     * cause a .
     */
    if (node->fullsort_state != NULL)
-   {
        tuplesort_reset(node->fullsort_state);
-       node->fullsort_state = NULL;
-   }
    if (node->prefixsort_state != NULL)
-   {
        tuplesort_reset(node->prefixsort_state);
-       node->prefixsort_state = NULL;
-   }
 
    /*
     * If chgParam of subnode is not null, then the plan will be re-scanned by