From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.3 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 037A3C43461 for ; Mon, 14 Sep 2020 10:07:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7CA682078D for ; Mon, 14 Sep 2020 10:07:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LQEpOLVM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7CA682078D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9C64E6B0037; Mon, 14 Sep 2020 06:07:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9762D6B005A; Mon, 14 Sep 2020 06:07:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8656C6B0062; Mon, 14 Sep 2020 06:07:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0235.hostedemail.com [216.40.44.235]) by kanga.kvack.org (Postfix) with ESMTP id 722EF6B0037 for ; Mon, 14 Sep 2020 06:07:50 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 2F76DBF04 for ; Mon, 14 Sep 2020 10:07:50 +0000 (UTC) X-FDA: 77261240700.12.mass57_5f085f027107 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id 00678180078A0 for ; Mon, 14 Sep 2020 10:07:49 +0000 (UTC) X-HE-Tag: mass57_5f085f027107 X-Filterd-Recvd-Size: 6121 Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Mon, 14 Sep 2020 10:07:49 +0000 (UTC) Received: by mail-wm1-f51.google.com with SMTP id q9so10033867wmj.2 for ; Mon, 14 Sep 2020 03:07:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=uVsG0Kkb3udXixOXzcX/0nWy6iMIi/f0mWhUtwVsfPg=; b=LQEpOLVMWVXWQajzwR4iS7CsWBaOQXjEAZrW/0AbAvZo/cIGqyeTJ6om5D+vZbISyK 0QTG3tbRdRvfu5g6id4Ze2iJ37oCVKkaGaX+DLTCKkb4j5uRVSZEFDwh0K5gBXCzPywg 6AG0xezXTQ1Z30tZVWI0/AsYdyEStjM191omA81i99oQbB1XWpaDQfY4R1pWfzHI3K66 X2vlQl20cKyloNby1I5kcdH7EFzDh9TjpcNa30/c7/nHE78C+Op94vjmfDDqQsUIro2c 90d16uQPY8bJVW/ZY8Xf3Rx2B+cHo48nuoxm9lo4ZbeYFMKgb8UsJkbJzUDUJy+uyRlI P0Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=uVsG0Kkb3udXixOXzcX/0nWy6iMIi/f0mWhUtwVsfPg=; b=cxng+oAuQWi7yx37Dg70LoQBVwoT3hxGnzfLkh26VnHSTQNfHA3Yf0RV+v5XAaZUDy xnwvpIGVzUEfQCnAFahSOZKTo9VoX0L+5tcMdyjH6zn5zs6Jw6IUPlO2b5tGYxf3PvlW gM1fsAM2VXJgBMFk+9VxJjtI1wVWAd9Kq+A41LbJkRhFO3rtIn/Q6Uw8vTBWWFob4jCO IZcy3hGBbcpFzlr4Mwke5NFT9lZuOoB/ZFYiWFHgREN+yRPK13h9gW7ID9sfDER68xwL luaKDehtb5DQ3ucqWSD2yjnD9ljNqnRPbnQ9ELoGuLM0eMwOiLE4rjgciVmcf9aRAXQc 8Afg== X-Gm-Message-State: AOAM531PlIlGrnm5gvTsDQyN+3FdJe28JmDKKVM/i+6q0XdKR9bKCRXA Guh6EvLPCqKN9w9nvdfAXd1hx3TcqAZGbw== X-Google-Smtp-Source: ABdhPJy7yaAiQnoYnvIRyEyNMlmXPfg0k2YcwNad5xpdcHp3LRUdvhMwnY1tCif6jgR1LLtk+5L9Ug== X-Received: by 2002:a1c:770c:: with SMTP id t12mr15252565wmi.121.1600078067958; Mon, 14 Sep 2020 03:07:47 -0700 (PDT) Received: from localhost.localdomain (188.147.112.12.nat.umts.dynamic.t-mobile.pl. [188.147.112.12]) by smtp.gmail.com with ESMTPSA id t4sm20485749wrr.26.2020.09.14.03.07.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Sep 2020 03:07:47 -0700 (PDT) From: mateusznosek0@gmail.com To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Mateusz Nosek , akpm@linux-foundation.org Subject: [RFC PATCH] mm/page_alloc.c: micro-optimization reduce oom critical section size Date: Mon, 14 Sep 2020 12:06:54 +0200 Message-Id: <20200914100654.21746-1-mateusznosek0@gmail.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-Rspamd-Queue-Id: 00678180078A0 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mateusz Nosek Most operations from '__alloc_pages_may_oom' do not require oom_mutex hol= d. Exception is 'out_of_memory'. The patch refactors '__alloc_pages_may_oom' to reduce critical section size and improve overall system performance. Signed-off-by: Mateusz Nosek --- mm/page_alloc.c | 45 ++++++++++++++++++++++++--------------------- 1 file changed, 24 insertions(+), 21 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b9bd75cacf02..b07f950a5825 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3935,18 +3935,7 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int= order, .order =3D order, }; struct page *page; - - *did_some_progress =3D 0; - - /* - * Acquire the oom lock. If that fails, somebody else is - * making progress for us. - */ - if (!mutex_trylock(&oom_lock)) { - *did_some_progress =3D 1; - schedule_timeout_uninterruptible(1); - return NULL; - } + bool success; =20 /* * Go through the zonelist yet one more time, keep very high watermark @@ -3959,14 +3948,17 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned in= t order, ~__GFP_DIRECT_RECLAIM, order, ALLOC_WMARK_HIGH|ALLOC_CPUSET, ac); if (page) - goto out; + return page; + + /* Check if somebody else is making progress for us. */ + *did_some_progress =3D mutex_is_locked(&oom_lock); =20 /* Coredumps can quickly deplete all memory reserves */ if (current->flags & PF_DUMPCORE) - goto out; + return NULL; /* The OOM killer will not help higher order allocs */ if (order > PAGE_ALLOC_COSTLY_ORDER) - goto out; + return NULL; /* * We have already exhausted all our reclaim opportunities without any * success so it is time to admit defeat. We will skip the OOM killer @@ -3976,12 +3968,12 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned in= t order, * The OOM killer may not free memory on a specific node. */ if (gfp_mask & (__GFP_RETRY_MAYFAIL | __GFP_THISNODE)) - goto out; + return NULL; /* The OOM killer does not needlessly kill tasks for lowmem */ if (ac->highest_zoneidx < ZONE_NORMAL) - goto out; + return NULL; if (pm_suspended_storage()) - goto out; + return NULL; /* * XXX: GFP_NOFS allocations should rather fail than rely on * other request to make a forward progress. @@ -3992,8 +3984,20 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int= order, * failures more gracefully we should just bail out here. */ =20 + /* + * Acquire the oom lock. If that fails, somebody else is + * making progress for us. + */ + if (!mutex_trylock(&oom_lock)) { + *did_some_progress =3D 1; + schedule_timeout_uninterruptible(1); + return NULL; + } + success =3D out_of_memory(&oc); + mutex_unlock(&oom_lock); + /* Exhausted what can be done so it's blame time */ - if (out_of_memory(&oc) || WARN_ON_ONCE(gfp_mask & __GFP_NOFAIL)) { + if (success || WARN_ON_ONCE(gfp_mask & __GFP_NOFAIL)) { *did_some_progress =3D 1; =20 /* @@ -4004,8 +4008,7 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int = order, page =3D __alloc_pages_cpuset_fallback(gfp_mask, order, ALLOC_NO_WATERMARKS, ac); } -out: - mutex_unlock(&oom_lock); + return page; } =20 --=20 2.20.1