From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A03CFC30658 for ; Fri, 5 Jul 2024 09:05:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 396B56B00A2; Fri, 5 Jul 2024 05:05:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 346526B00A3; Fri, 5 Jul 2024 05:05:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2675F6B00A4; Fri, 5 Jul 2024 05:05:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 01FDB6B00A2 for ; Fri, 5 Jul 2024 05:05:42 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0A83FC180C for ; Fri, 5 Jul 2024 09:05:39 +0000 (UTC) X-FDA: 82305115998.14.BE61FEF Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf08.hostedemail.com (Postfix) with ESMTP id 4E10E160020 for ; Fri, 5 Jul 2024 09:05:37 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf08.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720170316; a=rsa-sha256; cv=none; b=RfDkI/mvROscuJyxkGjJOYEfd0ruJXiI7N62R8uh72Qaw8uD5/RFp4rSpppM+BXZbqzi3t g/U8EeCeQF7XTZVgzJD+2dzztA45aWStufnngAPgzYyl91tIM5xhCccqGlrLiCxunY2HCy uK37Tk9Di9+swUWBLZ8Ezt18aVlWt48= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf08.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720170316; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mVpgsA7hwKPqycvAonZIXOWEO5svXiC6dTFlhbieT8Q=; b=KH2C2azkki1TnsMU2VyY2uhMp3RNfArq5DAePBuiVTuLR7jjY0BV1U5vMzDqEqfHKwtKyY Fw71DZaqRyqkvib8qSLYSHi8XacWpQIcHPhBzPW5ZzBM3hgFFUhyjBTWYKiYYV7GyjI15p japlSO10QxIIQvPr6c3wXNfO+rNskjI= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B9179367; Fri, 5 Jul 2024 02:06:01 -0700 (PDT) Received: from [10.57.74.223] (unknown [10.57.74.223]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 23B9E3F762; Fri, 5 Jul 2024 02:05:33 -0700 (PDT) Message-ID: <8df9636e-1816-46d2-9f5b-e3029848655d@arm.com> Date: Fri, 5 Jul 2024 10:05:32 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v5 4/6] mm: shmem: add mTHP support for anonymous shmem Content-Language: en-GB To: Baolin Wang , Bang Li , akpm@linux-foundation.org, hughd@google.com Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, ying.huang@intel.com, 21cnbao@gmail.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <65796c1e72e51e15f3410195b5c2d5b6c160d411.1718090413.git.baolin.wang@linux.alibaba.com> <65c37315-2741-481f-b433-cec35ef1af35@arm.com> <475332ea-a80b-421c-855e-a663d1d5bfc7@linux.alibaba.com> <076550c4-0e8a-4344-9f8a-31ae9e1051b5@linux.alibaba.com> <96625631-ef7d-44a2-ad5f-f7beb64f0ed0@arm.com> <545e23ab-e40a-4f13-8167-c9aa85a34b19@linux.alibaba.com> From: Ryan Roberts In-Reply-To: <545e23ab-e40a-4f13-8167-c9aa85a34b19@linux.alibaba.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 4E10E160020 X-Stat-Signature: rubgbhs5irm4zmswc85zpqdodybts88h X-Rspam-User: X-HE-Tag: 1720170337-529723 X-HE-Meta: U2FsdGVkX18nJwo7dfbEdpGxw8jnf9tOtvtK5JH6geJl3rcuD8aEL6VYAOSqdQJZ5U6ZRTG0r3A/Ez8TmwPdIyfN9siLO3cRQA96N4xk+jIrvLJsaLA11z+p0dq6vH4pzXv49A3xgqBxJvNI4tRKL8+wWx68nItmZl1B7M7QU9g8V5w+VFmXAaN1GHT7/NcLcOXQWNpbJr7CVC7j+BOThIyWl/qqiV/qWz/5m31iKo1YHiZPYH44gmXkagJxzgZd8TU/rf4iqezSUBwqJ/YXXEEpRMrDygyspd7qN1tI1Wlsv+B9N+3BAm4DWPNB3I5YpyJo1zIjBOEz48oAIusVC+APeXGhdR+c5+w1mUkH/XB5dXwnT1pXfG2yUFpuhajN55lWSaEbv+cnbtWCNo2WIgu/LA1rS2U7wRQ/IVmfjd36ikqk2qkI9W6E6KwOkOPvwb1ux6fvi+n+ts1Itv3BvNfFBbGMTEem0YQwI/p/1evjSrCKwgVzz2riWIkDedSidiwGUOS9/wLNmlxh3m67Z2I4nYOeSNUu6f98NZlcb60vsBkxE+Zpl2vDeu+hILVw+ISYl5DJ9+aWsCVUtblF8f4D0nQmw+3XvQ7yZurUSwlHpJNeF6YfVlTnm3WTlSN4CBztfRdg/utgyGWaa+On4Q9WX1TZfVRMKxdfEdgcGdBfcAINAIlSfyeocj8bPpbejGG6SwlabmbX+ghDDVfaXitPF3juE2s6VPMsQdWz8AQd3kDoO1JsYqPOt55ewmhpO1hfFeMPbWqUcoxYj7SC0G3E8wnmA51Lg6TCnuqXRjAvO+qLUHkNQC+pQYiGVPps01v/nb/u/6atdQXrv/9JQkx6/25xR5/7KNm6YmIt2/o6MWnf36gw1TqdIR+PhHYtDm1Fk5wSFSnN2XfjOS3KcL69JQTIpbO55s+o+ceEHTrimMGvmxbH2wM+MqlLx+j+DNPjaTaD9n+tFgk/Cfl IYrwscXO 6HUxeyg6FYLG9D36cJNOM4rjRMp0lbY5xLC/6SuDJAJ2tMAseb5Z9Txq4Ucbo1xGz5FguE23ghBxVrlB5aFQwBF++ZnzwX5NzO8sBHPWW4DbSi934OTapIc5mHoFNPJMrsEhkLgaqvhMZqMgJ1PvnEmYDazhfoSWgEQa780hCNk6G1zL89PAqx8B9qqdZ2o4DLOSdDrpXhoaoSXi0eERB7XLSc8Gmo5zxsveVf3yTcypc4xRVJohrarYwBg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 05/07/2024 09:57, Baolin Wang wrote: > > > On 2024/7/5 16:42, Ryan Roberts wrote: >> On 05/07/2024 04:01, Baolin Wang wrote: >>> >>> >>> On 2024/7/4 22:46, Bang Li wrote: >>>> Hi Bao lin, >>>> >>>> On 2024/7/4 19:15, Baolin Wang wrote: >>>>> >>>>>>> + >>>>>>> +    /* >>>>>>> +     * Only allow inherit orders if the top-level value is 'force', which >>>>>>> +     * means non-PMD sized THP can not override 'huge' mount option now. >>>>>>> +     */ >>>>>>> +    if (shmem_huge == SHMEM_HUGE_FORCE) >>>>>>> +        return READ_ONCE(huge_shmem_orders_inherit); >>>>>> >>>>>> I vaguely recall that we originally discussed that trying to set 'force' >>>>>> on the >>>>>> top level control while any per-size controls were set to 'inherit' would >>>>>> be an >>>>>> error, and trying to set 'force' on any per-size control except the PMD-size >>>>>> would be an error? >>>>> >>>>> Right. >>>>> >>>>>> I don't really understand this logic. Shouldn't we just be looking at the >>>>>> per-size control settings (or the top-level control as a proxy for every >>>>>> per-size control that has 'inherit' set)? >>>>> >>>>> ‘force’ will apply the huge orders for anon shmem and tmpfs, so now we only >>>>> allow pmd-mapped THP to be forced. We should not look at per-size control >>>>> settings for tmpfs now (mTHP for tmpfs will be discussed in future). >>>>> >>>>>> >>>>>> Then for tmpfs, which doesn't support non-PMD-sizes yet, we just always >>>>>> use the >>>>>> PMD-size control for decisions. >>>>>> >>>>>> I'm also really struggling with the concept of shmem_is_huge() existing along >>>>>> side shmem_allowable_huge_orders(). Surely this needs to all be refactored >>>>>> into >>>>>> shmem_allowable_huge_orders()? >>>>> >>>>> I understood. But now they serve different purposes: shmem_is_huge() will be >>>>> used to check the huge orders for the top level, for *tmpfs* and anon shmem; >>>>> whereas shmem_allowable_huge_orders() will only be used to check the per-size >>>>> huge orders for anon shmem (excluding tmpfs now). However, as I plan to add >>>>> mTHP support for tmpfs, I think we can perform some cleanups. >>>> >>>> Please count me in, I'd be happy to contribute to the cleanup and enhancement >>>> process if I can. >>> >>> Good. If you have time, I think you can look at the shmem khugepaged issue from >>> the previous discussion [1], which I don't have time to look at now. >>> >>> " >>> (3) khugepaged >>> >>> khugepaged needs to handle larger folios properly as well. Until fixed, >>> using smaller THP sizes as fallback might prohibit collapsing a >>> PMD-sized THP later. But really, khugepaged needs to be fixed to handle >>> that. >>> " >> >> khugepaged can already collapse "folios of any order less then PMD-size" to >> PMD-size, for anon memory. Infact I modified the selftest to be able to test >> that in commit 9f0704eae8a4 ("selftests/mm/khugepaged: enlighten for multi-size >> THP"). I'd be surprised if khugepaged can't alreay handle the same for shmem? > > I did not test this, but from the comment in hpage_collapse_scan_file(), seems > that compacting small mTHP into a single PMD-mapped THP is not supported yet. > > /* >          * TODO: khugepaged should compact smaller compound pages >          * into a PMD sized page >          */ >         if (folio_test_large(folio)) { >             result = folio_order(folio) == HPAGE_PMD_ORDER && >                     folio->index == start >                     /* Maybe PMD-mapped */ >                     ? SCAN_PTE_MAPPED_HUGEPAGE >                     : SCAN_PAGE_COMPOUND; >             /* >              * For SCAN_PTE_MAPPED_HUGEPAGE, further processing >              * by the caller won't touch the page cache, and so >              * it's safe to skip LRU and refcount checks before >              * returning. >              */ >             break; >         } OK, so the functionality is missing just for shmem/file-backed collapse. Got it. Sorry for the noise. > >> Although the test will definitely want to be extended to test it. > > Right.