From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5DC0C433F5 for ; Wed, 23 Feb 2022 19:49:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2C4CD8D0058; Wed, 23 Feb 2022 14:49:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 24DA48D0053; Wed, 23 Feb 2022 14:49:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0EE5A8D0058; Wed, 23 Feb 2022 14:49:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0088.hostedemail.com [216.40.44.88]) by kanga.kvack.org (Postfix) with ESMTP id D64498D0053 for ; Wed, 23 Feb 2022 14:49:50 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6E680998C9 for ; Wed, 23 Feb 2022 19:49:50 +0000 (UTC) X-FDA: 79175084940.18.028B5B4 Received: from mail-yw1-f179.google.com (mail-yw1-f179.google.com [209.85.128.179]) by imf16.hostedemail.com (Postfix) with ESMTP id E331B180007 for ; Wed, 23 Feb 2022 19:49:49 +0000 (UTC) Received: by mail-yw1-f179.google.com with SMTP id 00721157ae682-2d66f95f1d1so1664127b3.0 for ; Wed, 23 Feb 2022 11:49:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=8ffcbCLr7F7iwruaVWvkUbmkoeQwsuHvf36Ao1DCjcI=; b=Bz3B8ils3xqTiZi9e5L8Tytb83Wo70tAi9jv43XOJFrgPIyvDGldOE8axKPNwkj0jR lQPN6WWb9yoi6Elkz8HCbOQAMFUO4y7VcHJoYPCfi2JDXy+FbmctPO1cfSKM5Ejx1WoE xD753kmJqqh0gu5pmwKwxo1GUaipn4lnkTTvF/iWwwB6xvkVNZNTytTrVYYi/xV7MN55 V2sIGDy8uPz3dLu9mSS7vIVizeG5OMgsYL8Kw101YkfGZXmD3kcn4rIHjz+EWZb0OCzz sSeEzdIUEuwri5ys+PXjzX7VdZDHScEHws6g+DIAOa4UJx5ZMcoW6BV4ZTr8yTMIeQ6l 3Axg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8ffcbCLr7F7iwruaVWvkUbmkoeQwsuHvf36Ao1DCjcI=; b=cKoH795sLs4A7rSM34Ri5MZH8JAb29eKDvYzOWYjQ2F4gmpKjLTxEpdJPGdHFOgCkk uo6wnlOBCEDrXBRAL+v4Hh5IXNluYp74qE+JcnhH8GrVmD6x8qiyFX3uc9YikXoVs4EA D3FbEzxXBbjfG20Tx51YbiT+3bnpoI+VeMOxKm77ZzC9S+eKKDGNWcXgLjaEWHw9gLqv kRdSvG3Thwt+gW4otKEPe7kaMd7Y/PgG1roCmU6VWQXWZPQqJqK/ebaJff93xhlJssQB F6g27+HksJ7bxURcL63P9rXpksTVdiiuST2JBaAsHvVEqKRzVgMu1O9x1O9uFe3tzwab Xkog== X-Gm-Message-State: AOAM532JQ4MrvfJ80Ljk0gw+NCzKbz/+4pslmyH7VCzbssyIjKw4kUNC 5FKUCMeYpjInr7Hu2W7cLVkyX6EzucaUD80vqfI8Vg== X-Google-Smtp-Source: ABdhPJypDXqdPY+XlYgbvLbgmXAbIXhkwMu4dCulfhH56KD58PB0Yg3Vt3GRhjA9WUWYMlziTnl6Oew+89EhJcgizRw= X-Received: by 2002:a81:9858:0:b0:2d6:ecb0:71b7 with SMTP id p85-20020a819858000000b002d6ecb071b7mr1122772ywg.507.1645645788714; Wed, 23 Feb 2022 11:49:48 -0800 (PST) MIME-Version: 1.0 References: <20220223194018.1296629-1-surenb@google.com> In-Reply-To: From: Suren Baghdasaryan Date: Wed, 23 Feb 2022 11:49:37 -0800 Message-ID: Subject: Re: [PATCH v2 1/1] mm: count time in drain_all_pages during direct reclaim as memory pressure To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, mhocko@suse.com, pmladek@suse.com, peterz@infradead.org, guro@fb.com, shakeelb@google.com, minchan@kernel.org, timmurray@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com Content-Type: text/plain; charset="UTF-8" Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Bz3B8ils; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of surenb@google.com designates 209.85.128.179 as permitted sender) smtp.mailfrom=surenb@google.com X-Rspam-User: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: E331B180007 X-Stat-Signature: r7h4m1kjxjfnf9bp9tfirndwe6t3pmtc X-HE-Tag: 1645645789-86958 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Feb 23, 2022 at 11:43 AM Suren Baghdasaryan wrote: > > On Wed, Feb 23, 2022 at 11:40 AM Suren Baghdasaryan wrote: > > > > When page allocation in direct reclaim path fails, the system will > > make one attempt to shrink per-cpu page lists and free pages from > > high alloc reserves. Draining per-cpu pages into buddy allocator can > > be a very slow operation because it's done using workqueues and the > > task in direct reclaim waits for all of them to finish before > > proceeding. Currently this time is not accounted as psi memory stall. > > > > While testing mobile devices under extreme memory pressure, when > > allocations are failing during direct reclaim, we notices that psi > > events which would be expected in such conditions were not triggered. > > After profiling these cases it was determined that the reason for > > missing psi events was that a big chunk of time spent in direct > > reclaim is not accounted as memory stall, therefore psi would not > > reach the levels at which an event is generated. Further investigation > > revealed that the bulk of that unaccounted time was spent inside > > drain_all_pages call. > > > > A typical captured case when drain_all_pages path gets activated: > > > > __alloc_pages_slowpath took 44.644.613ns > > __perform_reclaim took 751.668ns (1.7%) > > drain_all_pages took 43.887.167ns (98.3%) > > > > PSI in this case records the time spent in __perform_reclaim but > > ignores drain_all_pages, IOW it misses 98.3% of the time spent in > > __alloc_pages_slowpath. > > > > Annotate __alloc_pages_direct_reclaim in its entirety so that delays > > from handling page allocation failure in the direct reclaim path are > > accounted as memory stall. > > > > Reported-by: Tim Murray > > Signed-off-by: Suren Baghdasaryan > > Acked-by: Johannes Weiner > > --- > > changes in v2: > > - Added captured sample case to show the delay numbers, per Michal Hocko > > - Moved annotation from __perform_reclaim into __alloc_pages_direct_reclaim, > > per Minchan Kim > > > > mm/page_alloc.c | 11 ++++++----- > > 1 file changed, 6 insertions(+), 5 deletions(-) > > > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > > index 3589febc6d31..2e9fbf28938f 100644 > > --- a/mm/page_alloc.c > > +++ b/mm/page_alloc.c > > @@ -4595,13 +4595,12 @@ __perform_reclaim(gfp_t gfp_mask, unsigned int order, > > const struct alloc_context *ac) > > { > > unsigned int noreclaim_flag; > > - unsigned long pflags, progress; > > + unsigned long progress; > > > > cond_resched(); > > > > /* We now go into synchronous reclaim */ > > cpuset_memory_pressure_bump(); > > - psi_memstall_enter(&pflags); > > fs_reclaim_acquire(gfp_mask); > > noreclaim_flag = memalloc_noreclaim_save(); > > > > @@ -4610,7 +4609,6 @@ __perform_reclaim(gfp_t gfp_mask, unsigned int order, > > > > memalloc_noreclaim_restore(noreclaim_flag); > > fs_reclaim_release(gfp_mask); > > - psi_memstall_leave(&pflags); > > > > cond_resched(); > > > > @@ -4624,11 +4622,13 @@ __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order, > > unsigned long *did_some_progress) > > { > > struct page *page = NULL; > > + unsigned long pflags; > > bool drained = false; > > > > + psi_memstall_enter(&pflags); > > *did_some_progress = __perform_reclaim(gfp_mask, order, ac); > > if (unlikely(!(*did_some_progress))) > > - return NULL; > > + goto out; > > > > retry: > > page = get_page_from_freelist(gfp_mask, order, alloc_flags, ac); > > @@ -4644,7 +4644,8 @@ __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order, > > drained = true; > > goto retry; > > } > > - > > + psi_memstall_leave(&pflags); > > Oh, psi_memstall_leave should have been *after* the "out" label. Will > fix and repost. Fixed in v3: https://lore.kernel.org/all/20220223194812.1299646-1-surenb@google.com/ > > > +out: > > return page; > > } > > > > -- > > 2.35.1.473.g83b2b277ed-goog > >