From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA08EC4360C for ; Wed, 25 Sep 2019 07:08:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7DDE721D7F for ; Wed, 25 Sep 2019 07:08:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7DDE721D7F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2848C6B0283; Wed, 25 Sep 2019 03:08:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 234A46B0284; Wed, 25 Sep 2019 03:08:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14AD36B0285; Wed, 25 Sep 2019 03:08:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0149.hostedemail.com [216.40.44.149]) by kanga.kvack.org (Postfix) with ESMTP id E1A916B0283 for ; Wed, 25 Sep 2019 03:08:22 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 776F487CC for ; Wed, 25 Sep 2019 07:08:22 +0000 (UTC) X-FDA: 75972564444.21.head04_2befd649adb25 X-HE-Tag: head04_2befd649adb25 X-Filterd-Recvd-Size: 4486 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Wed, 25 Sep 2019 07:08:21 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id EF702AF5A; Wed, 25 Sep 2019 07:08:19 +0000 (UTC) Date: Wed, 25 Sep 2019 09:08:17 +0200 From: Michal Hocko To: David Rientjes Cc: Andrea Arcangeli , Linus Torvalds , Andrew Morton , Mel Gorman , Vlastimil Babka , "Kirill A. Shutemov" , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [patch for-5.3 0/4] revert immediate fallback to remote hugepages Message-ID: <20190925070817.GH23050@dhcp22.suse.cz> References: <20190904205522.GA9871@redhat.com> <20190909193020.GD2063@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190909193020.GD2063@dhcp22.suse.cz> User-Agent: Mutt/1.10.1 (2018-07-13) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let me revive this thread as there was no follow up. On Mon 09-09-19 21:30:20, Michal Hocko wrote: [...] > I believe it would be the best to start by explaining why we do not see > the same problem with order-0 requests. We do not enter the slow path > and thus the memory reclaim if there is any other node to pass through > watermakr as well right? So essentially we are relying on kswapd to keep > nodes balanced so that allocation request can be satisfied from a local > node. We do have kcompactd to do background compaction. Why do we want > to rely on the direct compaction instead? What is the fundamental > difference? I am especially interested about this part. The more I think about this the more I am convinced that the underlying problem really is in the pre mature fallback in the fast path. Does the almost-patch below helps your workload? It effectively reduces the fast path for higher order allocations to the local/requested node. The justification is that watermark check might be too strict for those requests as it is primary order-0 oriented. Low watermark target simply has no meaning for the higher order requests AFAIU. The min-low gap is giving kswapd a chance to balance and be more local node friendly while we do not have anything like that in compaction. diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ff5484fdbdf9..09036cf55fca 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4685,7 +4685,7 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid, { struct page *page; unsigned int alloc_flags = ALLOC_WMARK_LOW; - gfp_t alloc_mask; /* The gfp_t that was actually used for allocation */ + gfp_t fastpath_mask, alloc_mask; /* The gfp_t that was actually used for allocation */ struct alloc_context ac = { }; /* @@ -4698,7 +4698,7 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid, } gfp_mask &= gfp_allowed_mask; - alloc_mask = gfp_mask; + fastpath_mask = alloc_mask = gfp_mask; if (!prepare_alloc_pages(gfp_mask, order, preferred_nid, nodemask, &ac, &alloc_mask, &alloc_flags)) return NULL; @@ -4710,8 +4710,17 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid, */ alloc_flags |= alloc_flags_nofragment(ac.preferred_zoneref->zone, gfp_mask); - /* First allocation attempt */ - page = get_page_from_freelist(alloc_mask, order, alloc_flags, &ac); + /* + * First allocation attempt. If we have a high order allocation then do not fall + * back to a remote node just based on the watermark check on the requested node + * because compaction might easily free up a requested order and then it would be + * better to simply go to the slow path. + * TODO: kcompactd should help here but nobody has woken it up unless we hit the + * slow path so we might need some tuning there as well. + */ + if (order && (gfp_mask & __GFP_DIRECT_RECLAIM)) + fastpath_mask |= __GFP_THISNODE; + page = get_page_from_freelist(fastpath_mask, order, alloc_flags, &ac); if (likely(page)) goto out; -- Michal Hocko SUSE Labs