From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D72CC433DB for ; Fri, 5 Mar 2021 16:06:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BCCDF65079 for ; Fri, 5 Mar 2021 16:06:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BCCDF65079 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3AB986B006C; Fri, 5 Mar 2021 11:06:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 382B66B0070; Fri, 5 Mar 2021 11:06:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 272886B0072; Fri, 5 Mar 2021 11:06:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0238.hostedemail.com [216.40.44.238]) by kanga.kvack.org (Postfix) with ESMTP id 0ABE86B006C for ; Fri, 5 Mar 2021 11:06:23 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id BDF8718025BED for ; Fri, 5 Mar 2021 16:06:22 +0000 (UTC) X-FDA: 77886297804.02.7C77319 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf22.hostedemail.com (Postfix) with ESMTP id 78874C0007ED for ; Fri, 5 Mar 2021 16:06:18 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1614960378; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=KaVkVhRkdIPeBEfcZK2BZahxFa1648i+OZhY5tS4IkY=; b=kinwnXlFoqC2i/vX+HThGlOYJeXAMp+xYZkoErq4bFuwLPcfQJeQWN8No6wddf499EEUVC lm7WURMddPIFC0AeSCn8+mczXsdt+HZ3FPIb1hOGVrDtnIEvsn+DZtLna8RS9gTKlqxWNq hme3madqe5qteeuNCXfCkH+u0o2aAyM= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id EFDEBACCF; Fri, 5 Mar 2021 16:06:17 +0000 (UTC) Date: Fri, 5 Mar 2021 17:06:17 +0100 From: Michal Hocko To: Minchan Kim Cc: Andrew Morton , linux-mm , LKML , joaodias@google.com, surenb@google.com, cgoldswo@codeaurora.org, willy@infradead.org, david@redhat.com, vbabka@suse.cz, linux-fsdevel@vger.kernel.org Subject: Re: [PATCH 1/2] mm: disable LRU pagevec during the migration temporarily Message-ID: References: <20210302210949.2440120-1-minchan@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 78874C0007ED X-Stat-Signature: 4ketsmhrtwmgmd7n5z6zpx7na9qospnp Received-SPF: none (suse.com>: No applicable sender policy available) receiver=imf22; identity=mailfrom; envelope-from=""; helo=mx2.suse.de; client-ip=195.135.220.15 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614960378-743283 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 03-03-21 12:23:22, Minchan Kim wrote: > On Wed, Mar 03, 2021 at 01:49:36PM +0100, Michal Hocko wrote: > > On Tue 02-03-21 13:09:48, Minchan Kim wrote: > > > LRU pagevec holds refcount of pages until the pagevec are drained. > > > It could prevent migration since the refcount of the page is greater > > > than the expection in migration logic. To mitigate the issue, > > > callers of migrate_pages drains LRU pagevec via migrate_prep or > > > lru_add_drain_all before migrate_pages call. > > > > > > However, it's not enough because pages coming into pagevec after the > > > draining call still could stay at the pagevec so it could keep > > > preventing page migration. Since some callers of migrate_pages have > > > retrial logic with LRU draining, the page would migrate at next trail > > > but it is still fragile in that it doesn't close the fundamental race > > > between upcoming LRU pages into pagvec and migration so the migration > > > failure could cause contiguous memory allocation failure in the end. > > > > > > To close the race, this patch disables lru caches(i.e, pagevec) > > > during ongoing migration until migrate is done. > > > > > > Since it's really hard to reproduce, I measured how many times > > > migrate_pages retried with force mode below debug code. > > > > > > int migrate_pages(struct list_head *from, new_page_t get_new_page, > > > .. > > > .. > > > > > > if (rc && reason == MR_CONTIG_RANGE && pass > 2) { > > > printk(KERN_ERR, "pfn 0x%lx reason %d\n", page_to_pfn(page), rc); > > > dump_page(page, "fail to migrate"); > > > } > > > > > > The test was repeating android apps launching with cma allocation > > > in background every five seconds. Total cma allocation count was > > > about 500 during the testing. With this patch, the dump_page count > > > was reduced from 400 to 30. > > > > Have you seen any improvement on the CMA allocation success rate? > > Unfortunately, the cma alloc failure rate with reasonable margin > of error is really hard to reproduce under real workload. > That's why I measured the soft metric instead of direct cma fail > under real workload(I don't want to make some adhoc artificial > benchmark and keep tunes system knobs until it could show > extremly exaggerated result to convice patch effect). > > Please say if you belive this work is pointless unless there is > stable data under reproducible scenario. I am happy to drop it. Well, I am not saying that this is pointless. In the end the resulting change is relatively small and it provides a useful functionality for other users (e.g. hotplug). That should be a sufficient justification. I was asking about CMA allocation success rate because that is a much more reasonable metric than how many times something has retried because retries can help to increase success rate and the patch doesn't really remove those. If you want to use number of retries as a metric then the average allocation latency would be more meaningful. -- Michal Hocko SUSE Labs