From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.8 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1, USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E93EC4361B for ; Mon, 14 Dec 2020 21:16:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 85CAC224B0 for ; Mon, 14 Dec 2020 21:16:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 85CAC224B0 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9EEC46B0036; Mon, 14 Dec 2020 16:16:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 978F86B005D; Mon, 14 Dec 2020 16:16:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 818566B0068; Mon, 14 Dec 2020 16:16:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0248.hostedemail.com [216.40.44.248]) by kanga.kvack.org (Postfix) with ESMTP id 666F36B0036 for ; Mon, 14 Dec 2020 16:16:54 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 1CBC4246E for ; Mon, 14 Dec 2020 21:16:54 +0000 (UTC) X-FDA: 77593147548.17.cream26_0a039b22741e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id 01DE2180D018B for ; Mon, 14 Dec 2020 21:16:53 +0000 (UTC) X-HE-Tag: cream26_0a039b22741e X-Filterd-Recvd-Size: 6541 Received: from mail-ot1-f67.google.com (mail-ot1-f67.google.com [209.85.210.67]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Mon, 14 Dec 2020 21:16:53 +0000 (UTC) Received: by mail-ot1-f67.google.com with SMTP id j12so17229164ota.7 for ; Mon, 14 Dec 2020 13:16:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=6f8m1ssHwXG9JNfNkbWg5A0BGB7gO6er1DG8qgB1GbU=; b=lQtFHlz+McBNdaTM9p3vvq/RNx3aW3GUOVEPN8x9M3UPEHpq4L2jEg48d1ycfeCGJv WQXIZOV0/JdW2XUwDinnlyZUjL6HFk21AC2WlUWcwXA1Yq8CdcnbfPkr34iPmAL1rcPe KIOtpFe8UcpiaN1oKrRxsA+MRHKS2J9cGhxbsoKpHQCyJskEh9JKXyYUJG77U7GYfn5F Zti+9gbrfQZ7TyIyEU2y0H5NwCds/b9QmEet+7hzUwaQSI2fCUFywiOFGggTXdNnFNpg 69CbsJlhFWHzpYvqAtMRXhTWYkhBkbqGn/zNg3t9ELS0MlKIa3QypXfIGSTHxh/nBjKu 2+nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version; bh=6f8m1ssHwXG9JNfNkbWg5A0BGB7gO6er1DG8qgB1GbU=; b=hhYf+7tubTOV90ZxuUZhCCGXKhuEut/L1BUbJupsrct7Mw/GA4x8oeIx+nbDcMIYhm ooSxvXF3fi1ehPeX7KmYEyIB1jKe6BxAxHhMO7tNmXB4OjPkCnOzk3INAEAephUO2NBq sLl+NE/XDJ3/hx04udooAhUtgmtKVQspPwOopF1DYwvQSp4YinL7oLxcggNHKnxyILg9 am79lK6p3EoFxclm3Zbg5S6QKOkOGurOb7Gdp2RJ8A72OHh8sriY9hnPDYT34ZWphQNu E5UjB7dqjXRJHaclQPKUcM2sF+noCusr7OGy6gFvTBY+xiNC+/9SkkjwrlB//53D1C3D eS1Q== X-Gm-Message-State: AOAM531fRHL+zr2DWRYv+53QPCdcvLdrXLJRexzOu39QUhV504feqG3u oq2Q78hph94gwrYqjGIHXt0pAA== X-Google-Smtp-Source: ABdhPJyLBlt1Orysb4nvb3lC1nALltH2pQLJJ86aKWyzG5xYPWANVcsxbZnzG87CdsOKa8BPNOjfUQ== X-Received: by 2002:a05:6830:1482:: with SMTP id s2mr21216866otq.296.1607980612567; Mon, 14 Dec 2020 13:16:52 -0800 (PST) Received: from eggly.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id f67sm4603670otb.60.2020.12.14.13.16.50 (version=TLS1 cipher=ECDHE-ECDSA-AES128-SHA bits=128/128); Mon, 14 Dec 2020 13:16:51 -0800 (PST) Date: Mon, 14 Dec 2020 13:16:39 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Andrew Morton cc: Rik van Riel , hughd@google.com, xuyu@linux.alibaba.com, mgorman@suse.de, aarcange@redhat.com, willy@infradead.org, linux-kernel@vger.kernel.org, kernel-team@fb.com, linux-mm@kvack.org, vbabka@suse.cz, mhocko@suse.com Subject: Re: [PATCH v6 0/3] mm,thp,shm: limit shmem THP alloc gfp_mask In-Reply-To: <20201124194925.623931-1-riel@surriel.com> Message-ID: References: <20201124194925.623931-1-riel@surriel.com> User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, 24 Nov 2020, Rik van Riel wrote: > The allocation flags of anonymous transparent huge pages can be controlled > through the files in /sys/kernel/mm/transparent_hugepage/defrag, which can > help the system from getting bogged down in the page reclaim and compaction > code when many THPs are getting allocated simultaneously. > > However, the gfp_mask for shmem THP allocations were not limited by those > configuration settings, and some workloads ended up with all CPUs stuck > on the LRU lock in the page reclaim code, trying to allocate dozens of > THPs simultaneously. > > This patch applies the same configurated limitation of THPs to shmem > hugepage allocations, to prevent that from happening. > > This way a THP defrag setting of "never" or "defer+madvise" will result > in quick allocation failures without direct reclaim when no 2MB free > pages are available. > > With this patch applied, THP allocations for tmpfs will be a little > more aggressive than today for files mmapped with MADV_HUGEPAGE, > and a little less aggressive for files that are not mmapped or > mapped without that flag. > > v6: make khugepaged actually obey tmpfs mount flags > v5: reduce gfp mask further if needed, to accomodate i915 (Matthew Wilcox) > v4: rename alloc_hugepage_direct_gfpmask to vma_thp_gfp_mask (Matthew Wilcox) > v3: fix NULL vma issue spotted by Hugh Dickins & tested > v2: move gfp calculation to shmem_getpage_gfp as suggested by Yu Xu Andrew, please don't rush mmthpshmem-limit-shmem-thp-alloc-gfp_mask.patch mmthpshm-limit-gfp-mask-to-no-more-than-specified.patch mmthpshmem-make-khugepaged-obey-tmpfs-mount-flags.patch to Linus in your first wave of mmotm->5.11 sendings. Or, alternatively, go ahead and send them to Linus, but be aware that I'm fairly likely to want adjustments later. Sorry for limping along so far behind, but I still have more re-reading of the threads to do, and I'm still investigating why tmpfs huge=always becomes so ineffective in my testing with these changes, even if I ramp up from default defrag=madvise to defrag=always: 5.10 mmotm thp_file_alloc 4641788 216027 thp_file_fallback 275339 8895647 I've been looking into it off and on for weeks (gfp_mask wrangling is not my favourite task! so tend to find higher priorities to divert me); hoped to arrive at a conclusion before merge window, but still have nothing constructive to say yet, hence my silence so far. Above's "a little less aggressive" appears understatement at present. I respect what Rik is trying to achieve here, and I may end up concluding that there's nothing better to be done than what he has. My kind of hugepage-thrashing-in-low-memory may be so remote from normal usage, and could be skirting the latency horrors we all want to avoid: but I haven't reached that conclusion yet - the disparity in effectiveness still deserves more investigation. (There's also a specific issue with the gfp_mask limiting: I have not yet reviewed the allowing and denying in detail, but it looks like it does not respect the caller's GFP_ZONEMASK - the gfp in shmem_getpage_gfp() and shmem_read_mapping_page_gfp() is there to satisfy the gma500, which wanted to use shmem but could only manage DMA32. I doubt it wants THPS, but shmem_enabled=force forces them.) Thanks, Hugh