From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A39BC55178 for ; Sat, 24 Oct 2020 17:53:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B03612225F for ; Sat, 24 Oct 2020 17:53:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B03612225F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=surriel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C10136B005D; Sat, 24 Oct 2020 13:53:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BC1666B0062; Sat, 24 Oct 2020 13:53:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AD7E46B0068; Sat, 24 Oct 2020 13:53:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id 7D4A76B005D for ; Sat, 24 Oct 2020 13:53:14 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 127E8362D for ; Sat, 24 Oct 2020 17:53:14 +0000 (UTC) X-FDA: 77407565508.27.brush68_2b0591927264 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id E935C3D679 for ; Sat, 24 Oct 2020 17:53:13 +0000 (UTC) X-HE-Tag: brush68_2b0591927264 X-Filterd-Recvd-Size: 4280 Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Sat, 24 Oct 2020 17:53:13 +0000 (UTC) Received: from imladris.surriel.com ([96.67.55.152]) by shelob.surriel.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1kWNix-0001VD-Js; Sat, 24 Oct 2020 13:53:03 -0400 Message-ID: Subject: Re: [PATCH v4] mm,thp,shmem: limit shmem THP alloc gfp_mask From: Rik van Riel To: Matthew Wilcox Cc: Hugh Dickins , Yu Xu , Andrew Morton , Mel Gorman , Andrea Arcangeli , linux-kernel@vger.kernel.org, kernel-team@fb.com, linux-mm@kvack.org, Vlastimil Babka , Michal Hocko Date: Sat, 24 Oct 2020 13:53:03 -0400 In-Reply-To: <20201024020922.GH20115@casper.infradead.org> References: <20201023204804.3f8d19c1@imladris.surriel.com> <20201024020922.GH20115@casper.infradead.org> Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="=-EsNs16WKP7UDH8iBf2MT" User-Agent: Evolution 3.34.4 (3.34.4-1.fc31) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --=-EsNs16WKP7UDH8iBf2MT Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, 2020-10-24 at 03:09 +0100, Matthew Wilcox wrote: > On Fri, Oct 23, 2020 at 08:48:04PM -0400, Rik van Riel wrote: > > The allocation flags of anonymous transparent huge pages can be > > controlled > > through the files in /sys/kernel/mm/transparent_hugepage/defrag, > > which can > > help the system from getting bogged down in the page reclaim and > > compaction > > code when many THPs are getting allocated simultaneously. > >=20 > > However, the gfp_mask for shmem THP allocations were not limited by > > those > > configuration settings, and some workloads ended up with all CPUs > > stuck > > on the LRU lock in the page reclaim code, trying to allocate dozens > > of > > THPs simultaneously. > >=20 > > This patch applies the same configurated limitation of THPs to > > shmem > > hugepage allocations, to prevent that from happening. > >=20 > > This way a THP defrag setting of "never" or "defer+madvise" will > > result > > in quick allocation failures without direct reclaim when no 2MB > > free > > pages are available. > >=20 > > With this patch applied, THP allocations for tmpfs will be a little > > more aggressive than today for files mmapped with MADV_HUGEPAGE, > > and a little less aggressive for files that are not mmapped or > > mapped without that flag. >=20 > How about this code path though? >=20 > shmem_get_pages() [ in i915 ] > shmem_read_mapping_page_gfp(__GFP_NORETRY | __GFP_NOWARN) > shmem_getpage_gfp() > shmem_alloc_and_acct_page() > shmem_alloc_hugepage() >=20 > I feel like the NORETRY from i915 should override whatever is set > in sysfs for anon THPs. What do others think? It looks like currently the only way to get a THP allocation with __GFP_DIRECT_RECLAIM and without __GFP_NORETRY (which does nothing without __GFP_DIRECT_RECLAIM) is to explicitly do an madvise MADV_HUGEPAGE on a VMA. I am not convinced the i915 driver should override a userspace madvise. --=20 All Rights Reversed. --=-EsNs16WKP7UDH8iBf2MT Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iQEzBAABCAAdFiEEKR73pCCtJ5Xj3yADznnekoTE3oMFAl+Uaf8ACgkQznnekoTE 3oNrlgf+IvnPSk5SIIzWtcHHyoBUttjdDn/ziiCG76djIOfJPxDT1MjfNQIfpfyX tB4txq8aA/m+oj6GMwO59Jlc9+R7Fyj8C21io14RQXJRCamjkRqWqOl5g82AHrq4 Pi8D7hYOV5bHFk1HoODZaj5JGHYi6bq8NS4tVftu5UEXR+eVrwfdwn/0Hgt+EEjP 8zIXZp59TZon1abCGaCangNHFbcn5krBDyQl1sAazwSxdxnnpb2o33+8cwOIugYI u442yl7EvM14AWN0DtYCdbk26SNPThNxTuT1MszAIqouUFTsyepruNcmP/+2mo43 M7K0Ce1tT3gjj7Lrh7ZfOuaR8sRxbw== =x0QG -----END PGP SIGNATURE----- --=-EsNs16WKP7UDH8iBf2MT--