Comparing changes

This commit introduces the following new table AM APIs for parallel table vacuuming: - parallel_vacuum_compute_workers - parallel_vacuum_estimate - parallel_vacuum_initialize - parallel_vacuum_initialize_worker - parallel_vacuum_collect_dead_items While parallel_vacuum_compute_workers is required, other new callbacks are optional. There is no code using these new APIs for now. Upcoming parallel vacuum es utilize these APIs. Reviewed-by: Amit Kapila <[email protected]> Reviewed-by: Hayato Kuroda <[email protected]> Reviewed-by: Peter Smith <[email protected]> Reviewed-by: Tomas Vondra <[email protected]> Reviewed-by: Dilip Kumar <[email protected]> Reviewed-by: Melanie Plageman <[email protected]> Discussion: https://postgr.es/m/CAD21AoAEfCNv-GgaDheDJ+s-p_Lv1H24AiJeNoPGCmZNSwL1YA@mail.gmail.com

…ad items. Previously, parallel vacuum was available only for index vacuuming and index cleanup, ParallelVacuumState was initialized only when the table has at least two indexes that are eligible for parallel index vacuuming and cleanup. This commit extends vacuumparallel.c to support parallel table vacuuming. parallel_vacuum_init() now initializes ParallelVacuumState to perform parallel heap scan to collect dead items, or paralel index vacuuming/cleanup, or both. During the initialization, it asks the table AM for the number of parallel workers required for parallel table vacuuming. If >0, it enables parallel table vacuuming and calls further table AM APIs such as parallel_vacuum_estimate. For parallel table vacuuming, this commit introduces parallel_vacuum_collect_dead_items_begin() function, which can be used to collect dead items in the table (for example, the first pass over heap table in lazy vacuum for heap tables). Heap table AM disables the parallel heap vacuuming for now, but an upcoming uses it. Reviewed-by: Amit Kapila <[email protected]> Reviewed-by: Hayato Kuroda <[email protected]> Reviewed-by: Peter Smith <[email protected]> Reviewed-by: Tomas Vondra <[email protected]> Reviewed-by: Dilip Kumar <[email protected]> Reviewed-by: Melanie Plageman <[email protected]> Discussion: https://postgr.es/m/CAD21AoAEfCNv-GgaDheDJ+s-p_Lv1H24AiJeNoPGCmZNSwL1YA@mail.gmail.com

This is a pure refactoring for upcoming parallel heap scan, which requires storing relation statistics and relation data such as extant oldest XID/MXID collected during lazy heap scan to a shared memory area. Reviewed-by: Amit Kapila <[email protected]> Reviewed-by: Hayato Kuroda <[email protected]> Reviewed-by: Peter Smith <[email protected]> Reviewed-by: Tomas Vondra <[email protected]> Reviewed-by: Dilip Kumar <[email protected]> Reviewed-by: Melanie Plageman <[email protected]> Discussion: https://postgr.es/m/CAD21AoAEfCNv-GgaDheDJ+s-p_Lv1H24AiJeNoPGCmZNSwL1YA@mail.gmail.com

This feature allows the vacuum to leverage multiple CPUs in order to collect dead items (i.e. the first pass over heap table) with parallel workers. The parallel degree for parallel heap vacuuming is determined based on the number of blocks to vacuum unless PARALLEL option of VACUUM command is specified, and further limited by max_parallel_maintenance_workers. For the parallel heap scan to collect dead items, we utilize a parallel block table scan, controlled by ParallelBlockTableScanDesc, in conjunction with the read stream. The workers' parallel scan descriptions are stored in the DSM space, enabling different parallel workers to resume the heap scan (phase 1) after a cycle of heap vacuuming and index vacuuming (phase 2 and 3) from their previous state. However, due to the potential presence of pinned buffers loaded by the read stream's look-ahead mechanism, we cannot abruptly stop phase 1 even when the space of dead_items TIDs exceeds the limit. Therefore, once the space of dead_items TIDs exceeds the limit, we begin processing pages without attempting to retrieve additional blocks by look-ahead mechanism until the read stream is exhausted, even if the the memory limit is surpassed. While this approach may increase the memory usage, it typically doesn't pose a significant problem, as processing a few 10s-100s buffers doesn't substantially increase the size of dead_items TIDs. When the parallel heap scan to collect dead items is enabled, we disable eager scanning. This is because parallel vacuum is available only in the VACUUM command and would not occur frequently, which doesn't align with the purpose of eager scanning. Reviewed-by: Amit Kapila <[email protected]> Reviewed-by: Hayato Kuroda <[email protected]> Reviewed-by: Peter Smith <[email protected]> Reviewed-by: Tomas Vondra <[email protected]> Reviewed-by: Dilip Kumar <[email protected]> Reviewed-by: Melanie Plageman <[email protected]> Reviewed-by: Andres Freund <[email protected]> Discussion: https://postgr.es/m/CAD21AoAEfCNv-GgaDheDJ+s-p_Lv1H24AiJeNoPGCmZNSwL1YA@mail.gmail.com

This branch was automatically generated by a robot using es from an email thread registered at: https://commitfest.postgresql.org//5326 The branch will be overwritten each time a new version is posted to the thread, and also periodically to check for bitrot caused by changes on the master branch. (es): https://www.postgresql.org/message-id/CAD21AoCdxc6jLfk5fc1a5-2DgxFikrjFPa6-A5b8pn27i4yKRg@mail.gmail.com Author(s): Masahiko Sawada

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comparing changes

Open a pull request

Commits on May 18, 2025

This comparison is taking too long to generate.