From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E186C001DF for ; Fri, 28 Jul 2023 22:14:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 78D9F6B0078; Fri, 28 Jul 2023 18:14:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 73E116B007B; Fri, 28 Jul 2023 18:14:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 605696B007D; Fri, 28 Jul 2023 18:14:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 52F916B0078 for ; Fri, 28 Jul 2023 18:14:33 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 1EA7C1A03E7 for ; Fri, 28 Jul 2023 22:14:33 +0000 (UTC) X-FDA: 81062425626.09.EE89D63 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf26.hostedemail.com (Postfix) with ESMTP id 11BFE140015 for ; Fri, 28 Jul 2023 22:14:30 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=ZtiNC4ej; spf=pass (imf26.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.210.181 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690582471; a=rsa-sha256; cv=none; b=EEwZcQlY1P25yz/GDrCKBJBFf/+gfM/7RCY7zQZDVe1KLsalpwabQ2FZObh/eXFTGiFfCm +rOmEBUEj/hsaaps8o6NNcjXeC/OBzR09WGZIx960507LGi1YqjCi03aFwyp3FCWYBPaJy g3kmPw3tfO7tX10Y8EqvX4M2Y7QPCYY= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=ZtiNC4ej; spf=pass (imf26.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.210.181 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690582471; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j23ImFvrdRCadrPSwwknJnJ5y4yBBTyt7l0nVVVM2Yc=; b=SLnSy4uYG0woZ+4rqCMmufcycR94Dovq3SYuSOu6Y5moa92wAyFDC5On1pnFdIHfsEUynm GFdVLQHDbmcdxh+1c7quJ6939nrzUf8fjgAw+348P9tfgMXwCwiKhYb9eHrThPFGzRUlRT VX9vDxw+a3w1ivSz0zt0j4NH0SSPjAM= Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-686ed1d2594so2495014b3a.2 for ; Fri, 28 Jul 2023 15:14:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1690582469; x=1691187269; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=j23ImFvrdRCadrPSwwknJnJ5y4yBBTyt7l0nVVVM2Yc=; b=ZtiNC4ej7OZeywn8wHEAqgDTQCL96l3JSg/+ffzCXaHNuNTx//E5tRxv+/5LyUCmzy wJU7VJ1iIxDjSGiYmpFU6nsZJLAX9pZspnX9lclWTUB+5LrMVyrOHTTcH7n9g0ri1s6/ jxQOuM+oNZ2mdhzAwawrFQ9KDOH6pNdZsBMXiJbfKH2lCh7pATaIgYWj96/7at6d67Va mRmHHE1OyMNNCbopzrhaqGthSM5xveatfXj9L6soKQqTrUS+/DH4oq1w4/EnW2qpPV1M Jdy66HRsCu5/ykHeyesJo9rLKHxuC2ZI1slvAXgyW67g1fPexWbnVuxbXMH7GbD4QMwG mmUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690582469; x=1691187269; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=j23ImFvrdRCadrPSwwknJnJ5y4yBBTyt7l0nVVVM2Yc=; b=WQs0Olx4MXKP3LH4T+EjMjCrpyeZoPE6Omoc4IxoD+wSqtu67Ks9M7Xup3Oai0oBVh Im3Gi0Zqj35R5xHBX4GN1q1xTcMbEEa+otJlc8pj0HQWzvhFXrBm7zzIC03+nu57FBoF TcebsJp4AISR/9/hgE03G8s5WETCZbkxVaDJg3gDDXhzn8ycvsjAAH6nXEJn/9GGKR3e rDfkR5cOhRZaNofnnLCVnZO0jzMq1GCqCJmvmAarut5hHZmf5Yy+WOA2daEh9s19wpiT f6j4j6Aj2XM1adAU9RON11739j0weAEoLAYAlmMYKhpWY2g2Km7XBXn0RlBSzgU6ko6v xoQA== X-Gm-Message-State: ABy/qLbJMu0q6ExWtWOJDDxUMCydCA2wWc2tOm5IMKtd4s+rNVriwWDp 2pb6v5IAJ5Oav2n2rwgnZRaWoQ== X-Google-Smtp-Source: APBJJlF0fafCFeexldp3nrIcTl/TKK4y9sr2ROkSLVlVpnM5bPmorNZnR4SuXn5KRsO7VBS9InuRKQ== X-Received: by 2002:a17:902:988e:b0:1b2:1a79:147d with SMTP id s14-20020a170902988e00b001b21a79147dmr3058566plp.2.1690582469499; Fri, 28 Jul 2023 15:14:29 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-25-194.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.25.194]) by smtp.gmail.com with ESMTPSA id jl21-20020a170903135500b001b531e8a000sm4056368plb.157.2023.07.28.15.14.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Jul 2023 15:14:28 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1qPVj8-001vv5-5d; Fri, 28 Jul 2023 19:14:26 -0300 Date: Fri, 28 Jul 2023 19:14:26 -0300 From: Jason Gunthorpe To: David Hildenbrand Cc: Peter Xu , Linus Torvalds , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Andrew Morton , liubo , Matthew Wilcox , Hugh Dickins , John Hubbard Subject: Re: [PATCH v1 0/4] smaps / mm/gup: fix gup_can_follow_protnone fallout Message-ID: References: <20230727212845.135673-1-david@redhat.com> <69a5f457-63b6-2d4f-e5c0-4b3de1e6c9f1@redhat.com> <22262495-c92c-20fa-dddf-eee4ce635b12@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <22262495-c92c-20fa-dddf-eee4ce635b12@redhat.com> X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 11BFE140015 X-Stat-Signature: 4ou9aktfydrc74kz8q534rb1f4xbdrzw X-Rspam-User: X-HE-Tag: 1690582470-970863 X-HE-Meta: U2FsdGVkX180iZ3Rx1dnoBiHHHbzSDR8AZLHHlx4dLlQuOvaQ60Kcp1fHvEEWP2To20vw+xjUMTEZFAvmi+DsJ/H6I+JmSZGPqYj1PcP15xFV5yvTkl84lrDh+Hg0zjs7KyS8x/so8nDtEOGYHgLk9d/w+N0D4Soy0FDZkspDGf+Ui9krg7M7dm6qHcwPxfIpBqLboj+p7r867Hfjv1BKiCOYzLjc/21J+vAcg9aoMuS/sA+GSdVlsMoZ/cJVGMh/2WKMWMQQX38CSmJdR97zCwyqpClvKIBzXMh1ec49nufv0noRvRIR13flgAMGSksIwLVlvWlIgZHqJcnDLB1foMty6WpWinp9dXzorcLFVDBuf4CDTN+qH0biCZMqB/P+sF4lz/yvobRy3n8r2/ZbEEp9riQCLNpRnRNzUjpdjSS46k87/8G2CUp0wSc2BqRlMYczf57v1hOC/fRmrdjiGaea2tLKp4agYsU0Yz81os29s4vhpYiocJiA9Z1F3TMyydjXJxBqBptZU5CNBgutr0dOpFBb7uHiezG7Sm7+q7dl8QwhmTD+vlev5TbXX5G+OsOxOBoILjj1p71KJ0JXEj+BT35piXiwh4qEIDrAPgNcrMEoAWrw81qrzqALhsKy0qynZ0ogLb632p7hi02p+qSTK1WTb59BQJrcJ6n2aWXpcPvIdtHNBaAo8KlUH4+a/OdHEkbBsRN8dwyx3Lx6hZZEseNtpxXVJzMqlz87BX5H8rdsVOX34y1/SMP1fz4OsA3ak4OwS+dvkGo9i8OxKVXFcefzx1sE2UEZ8mO/x2noftWPJzpH+VkCe+kMdi30/ApH016HlqpRvMvOhs3S3L2DzOvgJu2vrLQZVYvDQQ7nzY1C4ekJbHM03HCruPN2GgUMhB/r05CYm9uOoMrDFdBqyQzSN3+GSPPHjs5l/liJxUt1x64ink1O17jibrJuJJj9TX6PWsz7gw0F0V yn7mEmyT PRT43P6Y7mLETw/btVexsI6JHL9sJWXGvcunCZO+Bn+TlPqJFnHc+ZOHcmvEYGv3EVmgeRvMI776ZNt+bxSRIhw3P2EWp8EcLHCt61ECn7LmRo4tuufECrdGRhHLbG8nCaqEHcFZMYuqP+Cmh/cZKlRvH3vHsgsqSHPVMkaFXDNH/ztTzStelKtfTOHr51J8wGu9HA2tN0nMk+SyClAz1ilOGLpdzg7rpm9ENeq2+EKek7Oc36IE3QkEDeLoUCKHTCDFgQYQFAndFNgVs08Fxv7zCfAMlab+i1MUuPxKrLJQdETsFIa+LYgzBim5B0okVypiC50wPsLpmwctILgcpmwSqxnBYCH08iwGmqZ/m1kxXJdd1/QqBFM3b+w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Jul 28, 2023 at 11:31:49PM +0200, David Hildenbrand wrote: > * vfio triggers FOLL_PIN|FOLL_LONGTERM from a random QEMU thread. > Where should we migrate that page to? Would it actually be counter- > productive to migrate it to the NUMA node of the setup thread? The > longterm pin will turn the page unmovable, yes, but where to migrate > it to? For VFIO & KVM you actively don't get any kind of numa balancing or awareness. In this case qemu should probably strive to put the memory on the numa node of the majorty of CPUs early on because it doesn't get another shot at it. In other cases it depends quite alot. Eg DPDK might want its VFIO buffers to NUMA'd to the node that is close to the device, not the CPU. Or vice versa. There is alot of micro sensitivity here at high data rates. I think people today manually tune this by deliberately allocating the memory to specific numas and then GUP should just leave it alone. FWIW, I'm reading this thread and I have no idea what the special semantic is KVM needs from GUP, so I'm all for better documentation on the GUP flag :) Jason