Re: [PATCH 3/4] mm: compaction: Update pageblock skip when first migration candidate is not at the start

[Vlastimil Babka <vbabka@suse.cz>]

On 5/29/23 12:33, Mel Gorman wrote:
> On Thu, May 25, 2023 at 03:37:43PM +0200, Vlastimil Babka wrote:
>> On 5/15/23 13:33, Mel Gorman wrote:
>> > isolate_migratepages_block should mark a pageblock as skip if scanning
>> > started on an aligned pageblock boundary but it only updates the skip
>> > flag if the first migration candidate is also aligned. Tracing during
>> > a compaction stress load (mmtests: workload-usemem-stress-numa-compact)
>> > that many pageblocks are not marked skip causing excessive scanning of
>> > blocks that had been recently checked. Update pageblock skip based on
>> > "valid_page" which is set if scanning started on a pageblock boundary.
>> > 
>> > Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
>> 
>> I wonder if this has an unintended side-effect that if we resume
>> isolate_migratepages_block() of a partially compacted pageblock to finish
>> it, test_and_set_skip() will now tell us to abort, because we already set
>> the skip bit in the previous call. This would include the
>> cc->finish_pageblock rescan cases.
>> 
>> So unless I miss something that already prevents that, I agree we should not
>> tie setting the skip bit to pageblock_aligned(pfn), but maybe if we are not
>> pageblock aligned, we should ignore the already-set skip bit, as it was most
>> likely being set by us in the previous iteration and should not prevent us
>> from finishing the pageblock?
>> 
> 
> Hmm, I think you're right. While it should not hit the original bug,
> migration candidates are missed until the next compaction scan which
> could be tricky to detect. Something like this as a separate patch?
> Build tested only but the intent is for an unaligned start to set the skip
> bet if already unset but otherwise complete the scan. Like earlier fixes,
> this might overscan some pageblocks in a given context but we are probably
> hitting the limits on how compaction can run efficiently in the current
> scheme without causing other side-effects :(

Yeah that should work! I think it should be even folded to 3/4 but if you
want separate, fine too.

> diff --git a/mm/compaction.c b/mm/compaction.c
> index 91af6a8b7a98..761a2dd7d78a 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -792,6 +792,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
>  	bool skip_on_failure = false;
>  	unsigned long next_skip_pfn = 0;
>  	bool skip_updated = false;
> +	bool start_aligned;
>  	int ret = 0;
>  
>  	cc->migrate_pfn = low_pfn;
> @@ -824,6 +825,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
>  	}
>  
>  	/* Time to isolate some pages for migration */
> +	start_aligned = pageblock_aligned(start_pfn);
>  	for (; low_pfn < end_pfn; low_pfn++) {
>  
>  		if (skip_on_failure && low_pfn >= next_skip_pfn) {
> @@ -1069,10 +1071,15 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
>  
>  			lruvec_memcg_debug(lruvec, page_folio(page));
>  
> -			/* Try get exclusive access under lock */
> +			/* Try get exclusive access under lock. Isolation is
> +			 * only aborted if the start was pageblock aligned
> +			 * as this may be a partial resumed scan that set
> +			 * the bit on a recent scan but the scan must reach
> +			 * the end of the pageblock.
> +			 */
>  			if (!skip_updated && valid_page) {
>  				skip_updated = true;
> -				if (test_and_set_skip(cc, valid_page))
> +				if (test_and_set_skip(cc, valid_page) && start_aligned)
>  					goto isolate_abort;
>  			}
>