From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CFD5C433EF for ; Tue, 2 Nov 2021 18:38:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 258C66112D for ; Tue, 2 Nov 2021 18:38:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 258C66112D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id B0D35940008; Tue, 2 Nov 2021 14:38:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ABDA1940007; Tue, 2 Nov 2021 14:38:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9ACD1940008; Tue, 2 Nov 2021 14:38:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0134.hostedemail.com [216.40.44.134]) by kanga.kvack.org (Postfix) with ESMTP id 8EBAD940007 for ; Tue, 2 Nov 2021 14:38:37 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 54629181AC9CB for ; Tue, 2 Nov 2021 18:38:37 +0000 (UTC) X-FDA: 78764851116.01.DF05CC2 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by imf08.hostedemail.com (Postfix) with ESMTP id 8392130000BC for ; Tue, 2 Nov 2021 18:38:26 +0000 (UTC) Received: by mail-pf1-f178.google.com with SMTP id y4so5967065pfa.5 for ; Tue, 02 Nov 2021 11:38:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=wtwMNok/oS+uqbAcXvNbRmtV8aoGFRlPvBnKAS2OABs=; b=dkUsvt1FXSekVZLjszihzsGgRAeAnAB1GtlL1olNIkdMTc4Vzi+c12TbSA89d04EW5 gK9VzjcQECAHmGa50uE2H/MGxFR3hVZSMH8sMfk2Is/mMBolgXd4IIaocWJjwi7UPXaG D3L2CSxsIcC749fCI0qafHRFXV3Gj+UNrXyNJ40Uhb3JfVADYVrrv/wSxEhLH49B/1O6 xgkuA5r39aUvYXzviXoh9fZ9NBg4tqkuqeNBOmyJICyt2o2dO8qJ7VfQpZfg2bjGmFEX 1qGA2X4LMRtJPesld/eEpsJH2K1UA8G+nJ8BH/mvroNC23RdSOhFuh+LiZ1gY5fxE4iD gNQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wtwMNok/oS+uqbAcXvNbRmtV8aoGFRlPvBnKAS2OABs=; b=27yR6tVGdx8aPtm0C83aAiHy/RckPY+7IGtdO1fOIdQd1Z7l8xrqjz3fZh+BIsu6Nq aJJx1Sb2aFj1RcHYdD+EKNRrPS9s6szGeyB8JqG9Ws3Wd08FuVZc9JwzYdxlMrpHt+Cv AaYeRyBRfhY5kj4gKyTsnLc2SZp4L+OKBH+g6nfV/s1WfxSnc5AgNDQmSfpsu2GMRdLc prdvcWcVnuBQHi+f/lDy8DDGCZZ1+NHMTWa9izJGCzD/s+Z1vOPsSXuKpY2oOfI6fRRe uSLaDulsvCx8GZmO4tu0la3e4dlq9YKZnMFTaM2DqanPpI03eoainjqnX7o03H+Z6Shk hiVA== X-Gm-Message-State: AOAM533e0eOTokqP/yCVbIsQpE+Uyw80pAt0bc6NJpbDo/i8ms7Wvsqk auYnAot9V1UFjMZE41jWrBRYhsbRBuwyxeNXB/jFNA== X-Google-Smtp-Source: ABdhPJyVRrrsb4RUC9XUP9L6QjFp/rQ6P5+Rlce6y59oRmwUDtlYFYFCdn9M2NfT/WwqSdkvL1KAQXuQxGtpnLSrITI= X-Received: by 2002:a63:3fcd:: with SMTP id m196mr28982342pga.417.1635878315614; Tue, 02 Nov 2021 11:38:35 -0700 (PDT) MIME-Version: 1.0 References: <20211028205854.830200-1-almasrymina@google.com> <2fede4d2-9d82-eac9-002b-9a7246b2c3f8@redhat.com> <9fd0a86f-c012-4bb7-78eb-7413346448e0@redhat.com> In-Reply-To: <9fd0a86f-c012-4bb7-78eb-7413346448e0@redhat.com> From: Mina Almasry Date: Tue, 2 Nov 2021 11:38:23 -0700 Message-ID: Subject: Re: [PATCH v1] mm: Add /proc/$PID/pageflags To: David Hildenbrand Cc: Nathan Lewis , Yu Zhao , "Paul E . McKenney" , Jonathan Corbet , Andrew Morton , Peter Xu , Ivan Teterevkov , Florian Schmidt , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 8392130000BC X-Stat-Signature: n4wxym5rf3oiytd7i87emxwafxux8zqb Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=dkUsvt1F; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of almasrymina@google.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=almasrymina@google.com X-HE-Tag: 1635878306-331038 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Nov 2, 2021 at 4:42 AM David Hildenbrand wrote: > > >> Bit 58-60 are still free, no? Bit 57 was recently added for uffd-wp > >> purposes I think. > >> > >> #define PM_SOFT_DIRTY BIT_ULL(55) > >> #define PM_MMAP_EXCLUSIVE BIT_ULL(56) > >> #define PM_UFFD_WP BIT_ULL(57) > >> #define PM_FILE BIT_ULL(61) > >> #define PM_SWAP BIT_ULL(62) > >> #define PM_PRESENT BIT_ULL(63) > >> > >> PM_MMAP_EXCLUSIVE and PM_FILE already go into the direction of "what is > >> mapped" IMHO. So just a thought if something in there (PM_HUGE? PM_THP?) > >> ... could make sense. > >> > > > > Thanks! I _think_ that would work for us, I'll look into confirming. > > To be honest I still wonder if eventually different folks will find > > uses for other page flags and eventually we'll run out of pagemaps > > bits, but I'll yield to whatever you think is best here. > > Using one of the remaining 3 bits should be fine. In the worst case, > we'll need pagemap_ext at some point that provides more bits per PFN, if > we ever run out of bits. > That sounds great to me. Thank you Both Matthew and David for patiently explaining the concerns with /proc/self/pageflags to me and suggesting alternatives that could work :-) > But as mentioned by Matthew, extending mincore() could also work: not > only indicating if the page is resident, but also in which "form" it is > resident. > I need to learn more about mincore() to be honest, from casually reading some docs I didn't get a full understanding on if/why that would work better. I'll do some investigating and upload V2 either with /proc/self/pagemaps or mincore() and why I chose such and we can go from there. > We could separate the cases "cont PTE huge page" vs. "PMD huge page". > So to be completely honest (and I need to confirm), we are using this on x86 and we essentially care that the virt address is mapped by 2MB, so mapped by PMD. I think (but need to confirm) that's what the pageflags HUGE bit refers to as well as does PageHuge() and TransPageHuge(). After confirming I'll upload V2 with the precise info we need (I think it's going to be "PMD huge page" as David says). > I recall that the information (THP / !THP) might be valuable for users: > there was a discussion to let user space decide where to place THP. > (IIRC madvise() extension to have something like MADV_COLLAPSE_THP / > MADV_DISSOLVE_THP) > > -- > Thanks, > > David / dhildenb >