From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D06CEECAAA1 for ; Sun, 23 Oct 2022 20:16:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 671B0940008; Sun, 23 Oct 2022 16:16:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 62250940007; Sun, 23 Oct 2022 16:16:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E921940008; Sun, 23 Oct 2022 16:16:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3D7B3940007 for ; Sun, 23 Oct 2022 16:16:44 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0ABF2160656 for ; Sun, 23 Oct 2022 20:16:44 +0000 (UTC) X-FDA: 80053322328.04.A4D6674 Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com [209.85.167.170]) by imf02.hostedemail.com (Postfix) with ESMTP id A2FE980004 for ; Sun, 23 Oct 2022 20:16:43 +0000 (UTC) Received: by mail-oi1-f170.google.com with SMTP id u15so9031402oie.2 for ; Sun, 23 Oct 2022 13:16:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=L1yJrBm1zn8SsyErR4f1/ZvN1KDaJleCkqUq/AOdhu8=; b=jLw2qhpgAORoIlqfrBx49W3wtNa7rjk5WfKxVvNUpbMRZlZ4pSh97O9QGQSNk3Yjdh yIgh1ZN1JJQ3HOgVqxzHyEwgNLUSZEQO2VUbAifhdNbmGZ+it5TS39F/oA08gBsNQypL Te8pj3B8Wm1Tlf5nAWIp2zDE2Qj1gv/6PStyxeltT/Wqb/hGgikHt+TNM5oHLH41kyU7 hyoUJLAUbQZNA9Qd4rWCdEjjGGpTYHmbaROGXidatd/KBAefoJkAxKTe4CVf20WSH/hQ KvhVvv6P9h424Bhfcqkx8HrO4VNWMtMi0/9c8qWqsA/qmo9hu0FBVcrp2GUSDjD2AZk+ zFrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=L1yJrBm1zn8SsyErR4f1/ZvN1KDaJleCkqUq/AOdhu8=; b=kfxg9I52LPXR4uOvC5/136o8SwHi/5GdyS/9K3ac2Myk4gDaHhRVpA/hT8LGvkhVXJ oz3RCQxewcSJtHhCm5Y1FSo9auhCs/uQ9rLSKxKdtta++pOeUXpgsIAfOL9WobgbUASs 92fSofglwERo5Qprsay3nogckrcP4Nc9DQi2torzzPqvSzkHJQ1lZFifDftX5lmhWzw7 VzZR8X2c3+1qtyju3Se2pmky1l17bapDBt2ZJ/Y+YbPvvRA5LGFZLJeG6edIxkHvz9BO 6XyK1wqKjzmJC2yS/R+qthL6dmMIh4ATgRLUaQSZLGkie4yc9yC/JIGGXDP/J3Gd7zX9 BLHQ== X-Gm-Message-State: ACrzQf31oEaoPTmCgoOiplLi0x7s+fYW6B4li7cYj6soGQA6Bp9i6auA 00erEisLVfSCLuSl952Q+WOG+A== X-Google-Smtp-Source: AMsMyM7ai7/LrlShKt0cDcHy/wofRnIoUy4OQ+EdVaPjQHqH5uKzvbOZHVjZ4EHzAumeshE2t5L6vw== X-Received: by 2002:a05:6808:10c6:b0:355:3ff8:b88e with SMTP id s6-20020a05680810c600b003553ff8b88emr17313996ois.209.1666556202691; Sun, 23 Oct 2022 13:16:42 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id m4-20020a4ae3c4000000b00475d676d2d4sm5973709oov.16.2022.10.23.13.16.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Oct 2022 13:16:42 -0700 (PDT) Date: Sun, 23 Oct 2022 13:16:33 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: =?ISO-2022-JP?Q?=1B$B2+=5B=3F=1B=28J?= cc: Hugh Dickins , Mike Kravetz , Muchun Song , Andi Kleen , Andrew Morton , linux-mm@kvack.org, linux-kernel Subject: Re: [PATCH] mm: hugetlb: support get/set_policy for hugetlb_vm_ops In-Reply-To: Message-ID: References: <20221012081526.73067-1-huangjie.albert@bytedance.com> <5f7ef6ee-6241-9912-f434-962be53272c@google.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="-1463760895-794960718-1666556202=:8441" ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=jLw2qhpg; spf=pass (imf02.hostedemail.com: domain of hughd@google.com designates 209.85.167.170 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666556203; a=rsa-sha256; cv=none; b=NZBLgNJGjTYR21K1Te7zBn/x4Tu4aNBcwyADYV7oo0+ToK/pFpYRywWUQXRSYnfneKtXc+ r4qAfma6DhR0yn4tBN4yqTv15PrRJfPYI/M3o8xV7/kTiFuGSOPyJF7J7bNoY7XQ9mfNX7 +LL5rd3wo9eYAbhugJyrr6yc2mBEMVk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666556203; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=L1yJrBm1zn8SsyErR4f1/ZvN1KDaJleCkqUq/AOdhu8=; b=ku85MHPgIM5i5+rmAmHacd0Q/Q+lqMSfD7FAL7jWjZocHJLx8MI/qn842SFLPcdj13jo1d DBrGBQn/JJVQLzjG9CEqUjSNhmDJ486j7VHq3V9jWDg/gJszb9RJqQCVhHxBUkD4yuLKju 0nMgz3aeV/7z+hv/gpRhJCuZn5Kx/Qs= Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=jLw2qhpg; spf=pass (imf02.hostedemail.com: domain of hughd@google.com designates 209.85.167.170 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Rspamd-Server: rspam10 X-Stat-Signature: f7pxbycuen6et9hs9rfxst9u6419mkmm X-Rspamd-Queue-Id: A2FE980004 X-HE-Tag: 1666556203-922501 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---1463760895-794960718-1666556202=:8441 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE On Wed, 19 Oct 2022, =E9=BB=84=E6=9D=B0 wrote: > Hugh Dickins =E4=BA=8E2022=E5=B9=B410=E6=9C=8813=E6=97= =A5=E5=91=A8=E5=9B=9B 03:45=E5=86=99=E9=81=93=EF=BC=9A > > On Wed, 12 Oct 2022, Albert Huang wrote: > > > From: "huangjie.albert" > > > > > > implement these two functions so that we can set the mempolicy to > > > the inode of the hugetlb file. This ensures that the mempolicy of > > > all processes sharing this huge page file is consistent. > > > > > > In some scenarios where huge pages are shared: > > > if we need to limit the memory usage of vm within node0, so I set qem= u's > > > mempilciy bind to node0, but if there is a process (such as virtiofsd= ) > > > shared memory with the vm, in this case. If the page fault is trigger= ed > > > by virtiofsd, the allocated memory may go to node1 which depends on > > > virtiofsd. > > > > > > Signed-off-by: huangjie.albert > > > > Aha! Congratulations for noticing, after all this time. hugetlbfs > > contains various little pieces of code that pretend to be supporting > > shared NUMA mempolicy, but in fact there was nothing connecting it up. > > > > It will be for Mike to decide, but personally I oppose adding > > shared NUMA mempolicy support to hugetlbfs, after eighteen years. > > > > The thing is, it will change the behaviour of NUMA on hugetlbfs: > > in ways that would have been sensible way back then, yes; but surely > > those who have invested in NUMA and hugetlbfs have developed other > > ways of administering it successfully, without shared NUMA mempolicy. > > > > At the least, I would expect some tests to break (I could easily be > > wrong), and there's a chance that some app or tool would break too. >=20 > Hi : Hugh >=20 > Can you share some issues here? Sorry, I don't think I can: precisely because it's been such a relief to know that hugetlbfs is not in the shared NUMA mempolicy game, I've given no thought to what issues it might have if it joined the game. Not much memory is wasted on the unused fields in hugetlbfs_inode_info, just a few bytes per inode, that aspect doesn't concern me much. Reference counting of shared mempolicy has certainly been a recurrent problem in the past (see mpol_needs_cond_ref() etc): stable nowadays I believe; but whether supporting hugetlbfs would cause new problems to surface there, I don't know; but whatever, those would just be bugs to be fixed. /proc/pid/numa_maps does not represent shared NUMA mempolicies correctly: not for tmpfs, and would not for hugetlbfs. I did have old patches to fix that, but not patches that I'm ever likely to have time to resurrect and present and push. My main difficulties in tmpfs were with how to deal correctly and consistently with non-hugepage-aligned mempolicies when hugepages are in use. In the case of hugetlbfs, it would be simpler, since you're always dealing in hugepages of a known size: I recommend being as strict as possible, demanding correctly aligned mempolicy or else EINVAL. (That may already be enforced, I've not looked.) But my main concern in extending shared NUMA mempolicy to hugetlbfs is exactly what I already said earlier: The thing is, it will change the behaviour of NUMA on hugetlbfs: in ways that would have been sensible way back then, yes; but surely those who have invested in NUMA and hugetlbfs have developed other ways of administering it successfully, without shared NUMA mempolicy. At the least, I would expect some tests to break (I could easily be wrong), and there's a chance that some app or tool would break too. It's a risk, and a body of complication, that I would keep away from myself. The shared mempolicy rbtree: makes sense, but no madvise() since has implemented such a tree, to attach its advice to ranges of the shared object rather than to vma. Hugh ---1463760895-794960718-1666556202=:8441--