From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CDBBC54FB9 for ; Thu, 16 Nov 2023 11:11:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2625C440009; Thu, 16 Nov 2023 06:11:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 211B56B0458; Thu, 16 Nov 2023 06:11:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 101A4440009; Thu, 16 Nov 2023 06:11:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 01CA46B0456 for ; Thu, 16 Nov 2023 06:11:56 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id D1DC4A0BF6 for ; Thu, 16 Nov 2023 11:11:56 +0000 (UTC) X-FDA: 81463552632.19.0C90F5C Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by imf04.hostedemail.com (Postfix) with ESMTP id E34404001D for ; Thu, 16 Nov 2023 11:11:53 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf04.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700133115; a=rsa-sha256; cv=none; b=mwcaD8sp/vKrEjZmEjvl+qdghXsLbVXev8Cz3Ue7I3ja9agcLf2BpTnPyWPXcjtVP8EaCg Y1FnUV339d6/YEtuSjiVJsKw5KwrpPDKM5l2uBq1ReZfUvglC+HqA9uYQbrED0NJQNy2gk zqrYJyy+oZ37lRRLO6/STsbNK8T8cqc= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf04.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700133115; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qa4zNvD3R63oZSQAhdocCl4kKoiV4mt+N880Q8UM01c=; b=sq7l9tnaWpF9ea7rj3e+iFxwLlexCWJ4zwIkE6NeKyOC5ma29DHVksKatINupkHg7j+Aig uo5DpJH/xOT1FxkIHnf1ZP+zXhM4GjrttQFeQu74kZ44YFi4KHyR4wub2bUfJ59twhaegS GQtUcQu5k4sblggdXfCMoOqARnHjFvE= Received: from dggpemm500005.china.huawei.com (unknown [172.30.72.57]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4SWHKZ5v5XzMmsp; Thu, 16 Nov 2023 19:06:42 +0800 (CST) Received: from [10.69.30.204] (10.69.30.204) by dggpemm500005.china.huawei.com (7.185.36.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Thu, 16 Nov 2023 19:11:19 +0800 Subject: Re: [PATCH RFC 3/8] memory-provider: dmabuf devmem memory provider To: Mina Almasry CC: Jason Gunthorpe , Jakub Kicinski , , , , , Willem de Bruijn , Kaiyuan Zhang , Jesper Dangaard Brouer , Ilias Apalodimas , Eric Dumazet , =?UTF-8?Q?Christian_K=c3=b6nig?= , Matthew Wilcox , Linux-MM References: <20231113130041.58124-1-linyunsheng@huawei.com> <20231113130041.58124-4-linyunsheng@huawei.com> <20231113180554.1d1c6b1a@kernel.org> <0c39bd57-5d67-3255-9da2-3f3194ee5a66@huawei.com> From: Yunsheng Lin Message-ID: <28b65d3b-4df4-ce8b-00b6-abe565c0ab70@huawei.com> Date: Thu, 16 Nov 2023 19:11:19 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 8bit X-Originating-IP: [10.69.30.204] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemm500005.china.huawei.com (7.185.36.74) X-CFilter-Loop: Reflected X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: E34404001D X-Stat-Signature: 6q37q3mr1feo4814ybcaku4qjf1zybc7 X-Rspam-User: X-HE-Tag: 1700133113-510822 X-HE-Meta: U2FsdGVkX1+4pFdVainpLgY2MySo0K17sBvpzMMjwufLtN8W9dkQi6rGdAw8AhOEk7KuLjTHQ35BSEbREyMXmvB6dZX+lCp00syvRq8EQwOTA2jc1gsTTbGAoPWZ4eWOoLkB/V2FvhXpDjnFMSWTggsEVJ72FBp9ajOYief9BwsyIvJ+G7QC4yLyZqB2CEmYgdAWLX1VOAp/XRnlZfec8oY6GufltY2ulyNb4mb89CBgZRmbJVhFucyMCetwOVXHekXmVgOl5n6i+xmiZQGwwRzH88qaguBjHaK/uzw5wCVc4OnNmZhKA7mBHtYaZUVLWqx/gQTqeBqmKdg0PNyQlcIeldjmmgGq95/+89UaWj6MCDngWVgYhvPBoE5PkvSoyzbtkMtNnaxzWG7HdbQCqUYsLiy8E3sC7ijWjDc6TSVtui44hyCS3MXuHIna355eiJLNCYJ0xDdALnW35VAAtcaYmkJstwynp0O4+gQbapMT1Hjvvrfj+eBa7Y9c8ichmmfQDN2/BalvFiUmd1tq7ZMzId5x08qIrNHKQHOPGg6rMX36vlRUITTp0fUVSppDp0uZVr+BwKxa1DrrxFHrj1iueqvxTkoJwvHedVIcy1mwbFghgys3nPFLUoWPic7u18a1MG/iQ1OfRTY1btG4B6flsOGd0jGXpBGnHy0oxN5lRfxI9XIz5oUjXhcY0Cas1t5MGx3v5EGZu9idvOibpL3WV8+CMeQpIcrU4It1Etjukzb5nccoBSxCmUZ7nQpz8rdbOtbYpasQJzCGL+51dITdZQeI9LaQrpEsV8sTszo7Fculz7U3iH3fQ0SCc7mGETyICVpUZ7FBH3AGMW3jz+6CHV7QpV/nCTmRnTNwp/G3z8qyBtlyZs2BWjhlcn/Lo54eWrcA/J77gINZTElddyFrUtUQC+j2EquD/+Ke1qCkEPR2IglqhsbevBHgmXE5Y4Cl5E7o+bs03KUIPWW CyMnWqRz +e8lYwoUCTVLc6jz+PhbeynY3JWs7LVV5x2EuxiGaJVH8ssT1/FIccOhCEb5Jbii4JUfUW9+xX1OYdqE85mk+kVQlWuOsV8xRbrquscYsPzGdqhlLHLLvAbBbK/tv7xVdb69qUD/4QD+cdQr4VZY40sJ0MYsXfEpAGu/MYtf7m0mljKb0nnFGQqHRTLcTM0iUGeBLtAxO3DCipGOP+tVicZgi8lwIJmxB97RIYPISzmtLdHywQYCm6fXZ6A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2023/11/16 1:44, Mina Almasry wrote: > On Wed, Nov 15, 2023 at 1:21 AM Yunsheng Lin wrote: >> >> On 2023/11/14 21:16, Jason Gunthorpe wrote: >>> On Tue, Nov 14, 2023 at 04:21:26AM -0800, Mina Almasry wrote: >>> >>>> Actually because you put the 'strtuct page for devmem' in >>>> skb->bv_frag, the net stack will grab the 'struct page' for devmem >>>> using skb_frag_page() then call things like page_address(), kmap, >>>> get_page, put_page, etc, etc, etc. >>> >>> Yikes, please no. If net has its own struct page look alike it has to >>> stay entirely inside net. A non-mm owned struct page should not be >>> passed into mm calls. It is just way too hacky to be seriously >>> considered :( >> >> Yes, that is something this patchset is trying to do, defining its own >> struct page look alike for page pool to support devmem. >> >> struct page for devmem will not be called into the mm subsystem, so most >> of the mm calls is avoided by calling into the devmem memory provider' >> ops instead of calling mm calls. >> >> As far as I see for now, only page_ref_count(), page_is_pfmemalloc() and >> PageTail() is called for devmem page, which should be easy to ensure that >> those call for devmem page is consistent with the struct page owned by mm. > > I'm not sure this is true. These 3 calls are just the calls you're > aware of. In your proposal you're casting mirror pages into page* and > releasing them into the net stack. You need to scrub the entire net > stack for mm calls, i.e. all driver code and all skb_frag_page() call > sites. Of the top of my head, the driver is probably calling > page_address() and illegal_highdma() is calling PageHighMem(). TCP > zerocopy receive is calling vm_insert_pages(). For net stack part, I believe your patch below is handling to aovid those mm calls? I don't include it in this patchset as I thought it is obvious that whatever the proposal is, we always need those checking. Maybe we should have included it to avoid this kind of confusion. https://lore.kernel.org/all/20231106024413.2801438-10-almasrymina@google.com/ For driver part, I was thinking if the driver supports devmem, it should check that if it can call page_address() related call on a specific 'stuct page', or maybe we should introduce a new helper to make it obvious?