From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6021BC433FE for ; Thu, 10 Dec 2020 15:43:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B80F123770 for ; Thu, 10 Dec 2020 15:43:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B80F123770 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0E44B6B006E; Thu, 10 Dec 2020 10:43:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0477A6B0070; Thu, 10 Dec 2020 10:43:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E50506B0071; Thu, 10 Dec 2020 10:43:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0103.hostedemail.com [216.40.44.103]) by kanga.kvack.org (Postfix) with ESMTP id CBB596B006E for ; Thu, 10 Dec 2020 10:43:45 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 877D8180AD822 for ; Thu, 10 Dec 2020 15:43:45 +0000 (UTC) X-FDA: 77577792810.23.fork94_5413a00273f9 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 6524737604 for ; Thu, 10 Dec 2020 15:43:45 +0000 (UTC) X-HE-Tag: fork94_5413a00273f9 X-Filterd-Recvd-Size: 5855 Received: from aserp2130.oracle.com (aserp2130.oracle.com [141.146.126.79]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Thu, 10 Dec 2020 15:43:44 +0000 (UTC) Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 0BAFZOwP169583; Thu, 10 Dec 2020 15:43:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=kVIKnPtJaLqjBpJs3IPG0Ef+Ab1GmoQambLDfBkb+QM=; b=OWojDtxhMF4ArgnCZq83u0+Fcl/BmmwTJvrCen4fgko1CC/5y0smRSG+ktrW2cSdhA5v FMaFOtSh0dVyWvWvnmE/ICCoCHXfBudU6JY2BPuss4TVkNyPzCoLPvqdnZ2ZOc+W1vaL QZvjIlDhbpu2u4HLTWX9d0/DQb/Su0ZzhzEhMSSlNQqXlQf1rNlO++5IL4xe+sgVLBS5 SwmXLB1bQCXFIhSwKFs7i4xc3Q3oQ8A9XBZ5HaTWfZucN2IHv4jWfoyp/slqJdxM80dJ Mf+un8k3PA4NkYkBh144sZjlLvCGVtrZDH90mSgPDDmfjqzwaNB+mttLvl+LdlIu1eTx EQ== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2130.oracle.com with ESMTP id 357yqc64w5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 10 Dec 2020 15:43:33 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 0BAFacEV142085; Thu, 10 Dec 2020 15:43:33 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserp3020.oracle.com with ESMTP id 358m41u3mh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 10 Dec 2020 15:43:33 +0000 Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 0BAFhQLL016251; Thu, 10 Dec 2020 15:43:26 GMT Received: from [10.175.193.63] (/10.175.193.63) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 10 Dec 2020 07:43:26 -0800 Subject: Re: [PATCH RFC 6/9] mm/gup: Grab head page refcount once for group of subpages To: Matthew Wilcox Cc: linux-mm@kvack.org, linux-nvdimm@lists.01.org, Muchun Song , Mike Kravetz , Andrew Morton , Jason Gunthorpe References: <20201208172901.17384-1-joao.m.martins@oracle.com> <20201208172901.17384-8-joao.m.martins@oracle.com> <20201208194905.GQ5487@ziepe.ca> <20201209151505.GV5487@ziepe.ca> <20201209162438.GW5487@ziepe.ca> <20201209181406.GQ7338@casper.infradead.org> From: Joao Martins Message-ID: <996c74e8-fa86-bb91-6e95-b4b7efac7c05@oracle.com> Date: Thu, 10 Dec 2020 15:43:22 +0000 MIME-Version: 1.0 In-Reply-To: <20201209181406.GQ7338@casper.infradead.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9830 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 phishscore=0 suspectscore=1 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012100099 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9830 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 mlxlogscore=999 clxscore=1015 malwarescore=0 bulkscore=0 phishscore=0 adultscore=0 spamscore=0 priorityscore=1501 mlxscore=0 lowpriorityscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012100099 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 12/9/20 6:14 PM, Matthew Wilcox wrote: > On Wed, Dec 09, 2020 at 12:24:38PM -0400, Jason Gunthorpe wrote: >> On Wed, Dec 09, 2020 at 04:02:05PM +0000, Joao Martins wrote: >> >>> Today (without the series) struct pages are not represented the way they >>> are expressed in the page tables, which is what I am hoping to fix in this >>> series thus initializing these as compound pages of a given order. But me >>> introducing PGMAP_COMPOUND was to conservatively keep both old (non-compound) >>> and new (compound pages) co-exist. >> >> Oooh, that I didn't know.. That is kind of horrible to have a PMD >> pointing at an order 0 page only in this one special case. > > Uh, yes. I'm surprised it hasn't caused more problems. > There was 1 or 2 problems in the KVM MMU related to zone device pages. See commit e851265a816f ("KVM: x86/mmu: Use huge pages for DAX-backed files") which eventually lead to commit db5432165e9b5 ("KVM: x86/mmu: Walk host page tables to find THP mappings") to be less amenable to metadata changes. >> Still, I think it would be easier to teach record_subpages() that a >> PMD doesn't necessarily point to a high order page, eg do something >> like I suggested for the SGL where it extracts the page order and >> iterates over the contiguous range of pfns. > > But we also see good performance improvements from doing all reference > counts on the head page instead of spread throughout the pages, so we > really want compound pages. Going further than just refcounts and borrowing your (or someone else?) idea, perhaps also a FOLL_HEAD gup flag that would let us only work with head pages (or folios). Which would consequently let us pin/grab bigger swathes of memory e.g. 1G (in 2M head pages) or 512G (in 1G head pages) with just 1 page for storing the struct pages[*]. Albeit I suspect the numbers would have to justify it. Joao [*] One page happens to be what's used for RDMA/umem and vdpa as callers of pin_user_pages*()