From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB160C433EF for ; Mon, 15 Nov 2021 22:50:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4A41863217 for ; Mon, 15 Nov 2021 22:50:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4A41863217 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id BEBE26B007B; Mon, 15 Nov 2021 17:50:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B72D66B007D; Mon, 15 Nov 2021 17:50:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A13B66B007E; Mon, 15 Nov 2021 17:50:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0153.hostedemail.com [216.40.44.153]) by kanga.kvack.org (Postfix) with ESMTP id 8D79A6B007B for ; Mon, 15 Nov 2021 17:50:39 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 4CCFF18155812 for ; Mon, 15 Nov 2021 22:50:39 +0000 (UTC) X-FDA: 78812660556.17.BCBA290 Received: from mail-il1-f170.google.com (mail-il1-f170.google.com [209.85.166.170]) by imf01.hostedemail.com (Postfix) with ESMTP id A1C9A5093064 for ; Mon, 15 Nov 2021 22:50:22 +0000 (UTC) Received: by mail-il1-f170.google.com with SMTP id m11so18377592ilh.5 for ; Mon, 15 Nov 2021 14:50:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=MbJjt/ncW6eOrgGWMNW+Eh6/4jlzS3HIFhQ82o7o1ZE=; b=NS9lOPe9i5akbD6ebXCrGb+sx/9H3cRucOxDp7O8Ex7janJgSxxGfTF1PuUC87V/Ai 4ATPuckrf4HOO4HhPolfuUO0xfapDfmydxr5ROYfJGTXvz70aZqo1qv8TBP4TAy+hwY+ 01AbCzh3jA/wwIyvVmdOve1kR34KIPnV02+3zmA8s3pUIcUAx8pjY5ks2NjOlIw2f8Fo DmFHhwJmC02NwO+qta37QCnY5eyIJbXsd29hsdciEdg9pWduG7t3NSXhnOPhGu8Neflx +rhpCbOE6MuJVe9aUjK5QRTsdGvIF0pj9Zg0Kzsc3WSwA5RgNlkr20JJ8Lqcfa1XpPS0 8ulg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=MbJjt/ncW6eOrgGWMNW+Eh6/4jlzS3HIFhQ82o7o1ZE=; b=7THC8QxiVB0D1CM0zt58aOgtvGYED6lS+JJHk3E86OMVwN+7ccRpIxDIB08Xity1TJ cmm9MSfHnZKl6h3HsGfruWlzvQgtP7VnfA0HPOOxw8kQhKNvdFYLpWiDlm8s6ITLvSBG dJxCGKO1yoQ09MzJK/cR1ivQOfp4b4ozpfOaZBFeeJ6sVzNMWVPCTcZdMVz4PMI6bspM mnXV1qZD1K6YYBilhlqhymnNdQz6waU4m6rkFvHso6HUTIFnd8ohINawNS94uKEFH+NM Qrb0lAZRPtBMfrJpfWpzeSC/Kv09dIvcfujlfW71hzuBnaTI0gSSSZVko3BO0PqjQMwf Fzbg== X-Gm-Message-State: AOAM530jtHx6AbDCs+PsixscS2YEOC6/HWAjGCW6gZs5FGmb9ouxmiwO Am777Z9x84q/4aXEq5KVYRIANKgHTvJ+ZEErytM04OXM6uptLA== X-Google-Smtp-Source: ABdhPJwFUJw+E4PVvoQj8Sluj9XmoKPe/cZLTm+RfdBC45c2ftLWfz10J0gBd0rqheQ6j55AB2/Z8l0cLhcRf5AFg1Y= X-Received: by 2002:a92:6b0b:: with SMTP id g11mr1622656ilc.146.1637016638060; Mon, 15 Nov 2021 14:50:38 -0800 (PST) MIME-Version: 1.0 References: <20211107235754.1395488-1-almasrymina@google.com> In-Reply-To: From: Mina Almasry Date: Mon, 15 Nov 2021 14:50:26 -0800 Message-ID: Subject: Re: [PATCH v4] mm: Add PM_HUGE_THP_MAPPING to /proc/pid/pagemap To: Peter Xu Cc: David Hildenbrand , Matthew Wilcox , "Paul E . McKenney" , Yu Zhao , Jonathan Corbet , Andrew Morton , Ivan Teterevkov , Florian Schmidt , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A1C9A5093064 X-Stat-Signature: moxb1gbrn1g5ieaiqtdy535od5cu975n Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=NS9lOPe9; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of almasrymina@google.com designates 209.85.166.170 as permitted sender) smtp.mailfrom=almasrymina@google.com X-HE-Tag: 1637016622-713668 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Nov 11, 2021 at 11:41 PM Peter Xu wrote: > > On Wed, Nov 10, 2021 at 09:42:25AM -0800, Mina Almasry wrote: > > Sorry, yes I should update the commit message with this info. The > > issues with smaps are: > > 1. Performance: I've pinged our network service folks to obtain a > > rough perf comparison but I haven't been able to get one. I can try to > > get a performance measurement myself but Peter seems to be also seeing > > this. > > No I was not seeing any real issues in my environment, but I remembered people > complaining about it because smaps needs to walk the whole memory of the > process, then if one program is only interested in some small portion of the > whole memory, it'll be slow because smaps will still need to walk all the > memory anyway. > > > 2. smaps output is human readable and a bit convoluted for userspace to parse. > > IMHO this is not a major issue. AFAIK lots of programs will still try to parse > human readable output like smaps to get some solid numbers. It's just that > it'll be indeed an perf issue if it's only a part of the memory that is of > interest. > > Could we consider exporting a new smaps interface that: > > 1. allows to specify a range of memory, and, > 2. expose information as "struct mem_size_stats" in binary format > (we may want to replace "unsigned long" with "u64", then also have some > versioning or having a "size" field for the struct, though; seems doable) > > I'm wondering whether this could be helpful in even more scenarios. > Sorry could you elaborate more? How do we allow users to specify the range? Are you envisioning a new added syscall? Or is there some way for users to pass the range to the existing /proc/pid/smaps that I'm missing? On Thu, Nov 11, 2021 at 11:43 PM Peter Xu wrote: > > On Wed, Nov 10, 2021 at 09:50:13AM -0800, Mina Almasry wrote: > > On Tue, Nov 9, 2021 at 11:03 PM Peter Xu wrote: > > > > > > The ending "_MAPPING" seems redundant to me, how about just call it "PM_THP" or > > > "PM_HUGE" (as THP also means HUGE already)? > > > > > > > So I want to make it clear that the flag is set only when the page is > > PMD mappend and is a THP (not hugetlbfs or some other PMD device > > mapping). PM_THP would imply the flag is set only if the underlying > > page is THP without regard to whether it's actually PMD mapped or not. > > I see, that's fine. > > However as I mentioned I still think HUGE and THP dup with each other. > Meanwhile, "MAPPING" does not sound like a boolean status on whether it's thp > mapped.. > > If you still prefer this approach, how about PM_THP_MAPPED? PM_THP_MAPPED sounds good to me. TBH I think I still prefer this approach because it's a very simple 2 line patch which addresses the concrete use case I have well. I'm not too familiar with the smaps code to be honest but I think adding a range-based smaps API will be a sizeable patch to add a syscall, handle a stable interface, and handle cases where the memory range doesn't match a VMA boundary. I'm not sure the performance benefit would justify this patch and I'm not sure the extra info from smaps would be widely useful. However if you insist and folks believe this is the better approach I can prototype a range-based smaps and test its performance to see if it works for us as well, just let me know what kind of API you're envisioning. > > -- > Peter Xu >