From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5B77C433EF for ; Thu, 30 Jun 2022 16:24:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 237968E0005; Thu, 30 Jun 2022 12:24:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C12A8E0001; Thu, 30 Jun 2022 12:24:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0615F8E0005; Thu, 30 Jun 2022 12:24:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E80478E0001 for ; Thu, 30 Jun 2022 12:24:02 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B309A2124C for ; Thu, 30 Jun 2022 16:24:02 +0000 (UTC) X-FDA: 79635423924.30.8C901E8 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf25.hostedemail.com (Postfix) with ESMTP id 0C8B4A002D for ; Thu, 30 Jun 2022 16:24:01 +0000 (UTC) Received: by mail-pf1-f175.google.com with SMTP id 136so13493929pfy.10 for ; Thu, 30 Jun 2022 09:24:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=UQ0ABdxfR6D43G67MI06zZIhVmsQkiYQlJlGftbg3co=; b=U34n6EPZFSdi0BxyjYW9mpzFC4uLov7wQVqlBzu4KgmT9eQTL1rff0zfX9lTeGFG6x oAlz37vRUkmn91Zv1WGucnJQH0McZEV4cs1nm2Ji5UcXzWtixkiPwquhMbXwOjbY4KID g/U02e1hG1qzapf6uMxCMR+iLfmBhnlc96EiMONXF7wsJybq5FfZ6DFeC+8uPyLd2B/W nfedQ4nuV5Ft5fas0ry+rGPn5vbcjWNwjatkIoubdkClM6Mm71HXuzlO4IYki9z5lqP2 CkjIkqZvO/lcSUJqm/uY6NspI/2FaW9jwowoIjxaBXl+q7NZTJnwq8G9Jhy4m1FjzK4K 1Q3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=UQ0ABdxfR6D43G67MI06zZIhVmsQkiYQlJlGftbg3co=; b=K11ZAxToUaq1erEEF1pCB2udmcMtLkNomMfOawrjgrMqObATJ9fs2HdVmhLRZN85HR e9be+ADAKhk4Yu1q6lW4YSA/pZB+0WaE9866dUsACGVkI222xqsRT+geBfupqdvjB906 s/wN/jri2CckrSymNz+Yb0PEKuwAZNLFcQYjmui2V/nRRaeQa8SlDoSu8ti8SKGCTJtj vTKEsFMqZVAneJfTABYXMsH+3OAFwPdcmAhNRkm0qXdnbjchWSFNd6Hy4azZPPANJvt7 dzAVUDcrjaSNOS2Gj+Sg7ylZf1Rd+nCCxChUlhDXlTmjraSvo9LRFALXvGVVh9KHxbMT ScOw== X-Gm-Message-State: AJIora8y1NvJu21l9vmt2lNb0GMganm5ZrUSpf01apoQvDEcLfg3w1qA NJfUiEP2G0cwizAiDOO/gIjRQq8oJgK0V50pagHc2g== X-Google-Smtp-Source: AGRyM1vyljlV3h/CFSJYkNpPc3JzXZ4DEIbWhvcjFs6Qm0gF5Tz+g16pAYIAcWs2EeS1saAxpg3NA+2q2/KUAbSN9II= X-Received: by 2002:a63:5449:0:b0:40d:c8d4:ed6a with SMTP id e9-20020a635449000000b0040dc8d4ed6amr8372360pgm.9.1656606240746; Thu, 30 Jun 2022 09:24:00 -0700 (PDT) MIME-Version: 1.0 References: <20220624173656.2033256-1-jthoughton@google.com> <20220624173656.2033256-5-jthoughton@google.com> In-Reply-To: From: James Houghton Date: Thu, 30 Jun 2022 09:23:49 -0700 Message-ID: Subject: Re: [RFC PATCH 04/26] hugetlb: make huge_pte_lockptr take an explicit shift argument. To: Muchun Song Cc: Mike Kravetz , Peter Xu , David Hildenbrand , David Rientjes , Axel Rasmussen , Mina Almasry , Jue Wang , Manish Mishra , "Dr . David Alan Gilbert" , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656606242; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UQ0ABdxfR6D43G67MI06zZIhVmsQkiYQlJlGftbg3co=; b=jvUrpjd8kMqT91qYMfN5Xjm2FLWZ5XwsnLFGUOw1gzkD44QROk+tGQbCwb1gcJtEipvtug H1uUdbnCpV96rH+EYadi1cKt+L7Y0+FCgG6IQwJQnFzfApnGkW7VSYf5TaJxYkgnCcSiqs kYlnWZBTdi6//tFimt2vkeAzDuQ4NvU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656606242; a=rsa-sha256; cv=none; b=YpX/mimHp/2JRqOJvaJqv0KE4GUomwh6MhHLFjZTfAEv9NRf+gafkLg8coEwtXFBAitxBv IoH50xHkVcaumGMB7XU1d5HV3UZhuIbjrwPWa80aMG4xamklgvEGaGn7WhU7EMvMt3071z WXuKYMV08XRBRLkyKDvOhxMYIsj8Wsc= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=U34n6EPZ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of jthoughton@google.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=jthoughton@google.com Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=U34n6EPZ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of jthoughton@google.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=jthoughton@google.com X-Rspamd-Server: rspam03 X-Rspam-User: X-Stat-Signature: 6ibbuxdj6ghfr1rzhobbbosr6x5hwyeb X-Rspamd-Queue-Id: 0C8B4A002D X-HE-Tag: 1656606241-206286 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Jun 30, 2022 at 2:35 AM Muchun Song wrote: > > On Wed, Jun 29, 2022 at 03:24:45PM -0700, Mike Kravetz wrote: > > On 06/29/22 14:39, James Houghton wrote: > > > On Wed, Jun 29, 2022 at 2:04 PM Mike Kravetz wrote: > > > > > > > > On 06/29/22 14:09, Muchun Song wrote: > > > > > On Mon, Jun 27, 2022 at 01:51:53PM -0700, Mike Kravetz wrote: > > > > > > On 06/24/22 17:36, James Houghton wrote: > > > > > > > This is needed to handle PTL locking with high-granularity mapping. We > > > > > > > won't always be using the PMD-level PTL even if we're using the 2M > > > > > > > hugepage hstate. It's possible that we're dealing with 4K PTEs, in which > > > > > > > case, we need to lock the PTL for the 4K PTE. > > > > > > > > > > > > I'm not really sure why this would be required. > > > > > > Why not use the PMD level lock for 4K PTEs? Seems that would scale better > > > > > > with less contention than using the more coarse mm lock. > > > > > > > > > > > > > > > > Your words make me thing of another question unrelated to this patch. > > > > > We __know__ that arm64 supports continues PTE HugeTLB. huge_pte_lockptr() > > > > > did not consider this case, in this case, those HugeTLB pages are contended > > > > > with mm lock. Seems we should optimize this case. Something like: > > > > > > > > > > diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h > > > > > index 0d790fa3f297..68a1e071bfc0 100644 > > > > > --- a/include/linux/hugetlb.h > > > > > +++ b/include/linux/hugetlb.h > > > > > @@ -893,7 +893,7 @@ static inline gfp_t htlb_modify_alloc_mask(struct hstate *h, gfp_t gfp_mask) > > > > > static inline spinlock_t *huge_pte_lockptr(struct hstate *h, > > > > > struct mm_struct *mm, pte_t *pte) > > > > > { > > > > > - if (huge_page_size(h) == PMD_SIZE) > > > > > + if (huge_page_size(h) <= PMD_SIZE) > > > > > return pmd_lockptr(mm, (pmd_t *) pte); > > > > > VM_BUG_ON(huge_page_size(h) == PAGE_SIZE); > > > > > return &mm->page_table_lock; > > > > > > > > > > I did not check if elsewhere needs to be changed as well. Just a primary > > > > > thought. > > > > > > I'm not sure if this works. If hugetlb_pte_size(hpte) is PAGE_SIZE, > > > then `hpte.ptep` will be a pte_t, not a pmd_t -- I assume that breaks > > > things. So I think, when doing a HugeTLB PT walk down to PAGE_SIZE, we > > > need to separately keep track of the location of the PMD so that we > > > can use it to get the PMD lock. > > > > I assume Muchun was talking about changing this in current code (before > > your changes) where huge_page_size(h) can not be PAGE_SIZE. > > > > Yes, that's what I meant. Right -- but I think my point still stands. If `huge_page_size(h)` is CONT_PTE_SIZE, then the `pte_t *` passed to `huge_pte_lockptr` will *actually* point to a `pte_t` and not a `pmd_t` (I'm pretty sure the distinction is important). So it seems like we need to separately keep track of the real pmd_t that is being used in the CONT_PTE_SIZE case (and therefore, when considering HGM, the PAGE_SIZE case). However, we *can* make this optimization for CONT_PMD_SIZE (maybe this is what you originally meant, Muchun?), so instead of `huge_page_size(h) == PMD_SIZE`, we could do `huge_page_size(h) >= PMD_SIZE && huge_page_size(h) < PUD_SIZE`. > > Thanks.