From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61223CDB474 for ; Tue, 17 Oct 2023 12:17:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 941A88D0021; Tue, 17 Oct 2023 08:17:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C9658D000C; Tue, 17 Oct 2023 08:17:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 769D58D0021; Tue, 17 Oct 2023 08:17:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 63F208D000C for ; Tue, 17 Oct 2023 08:17:51 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 33D41140C85 for ; Tue, 17 Oct 2023 12:17:51 +0000 (UTC) X-FDA: 81354854742.10.E618E0F Received: from mail-lj1-f174.google.com (mail-lj1-f174.google.com [209.85.208.174]) by imf03.hostedemail.com (Postfix) with ESMTP id 26FF22002A for ; Tue, 17 Oct 2023 12:17:48 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b=Hg94OWog; dmarc=pass (policy=none) header.from=linaro.org; spf=pass (imf03.hostedemail.com: domain of ilias.apalodimas@linaro.org designates 209.85.208.174 as permitted sender) smtp.mailfrom=ilias.apalodimas@linaro.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697545069; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0NE2es9FYFS0mSlu5ve+Yss2hukel82LAPmeDG92pME=; b=vb4S6UjI/b3jtBpPwzTaPJQYA+juBCOJPr6A10idElscaBjVpoD0Yqt3od2BkZIuPT67tu hfAEFZWRsijmN+SRUPZUMQZkHQV1LIan7xZwK/+iS3uDXXnKtDxvteoBBj5dbmXosnANtY 3rfLce2O0X/9j5vwxSEZyTlvEHWCyXs= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b=Hg94OWog; dmarc=pass (policy=none) header.from=linaro.org; spf=pass (imf03.hostedemail.com: domain of ilias.apalodimas@linaro.org designates 209.85.208.174 as permitted sender) smtp.mailfrom=ilias.apalodimas@linaro.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697545069; a=rsa-sha256; cv=none; b=QKpg0EyugUqboGrboUC9Bti8UxjoxmF9PrN3hJxBHkYjUagDhkRFqCfTxRxbL393ksuIw9 t6S0LSC8rE4ko/GAILAAAHfFnidNXsJepk64a/HgxBtSDMm8nbJ4mrVDsmDa+CD/LJ5g7o O7RzpWFVM+a/CXCD5A8PY1f9CEYU4oE= Received: by mail-lj1-f174.google.com with SMTP id 38308e7fff4ca-2c515527310so38516931fa.2 for ; Tue, 17 Oct 2023 05:17:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1697545067; x=1698149867; darn=kvack.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=0NE2es9FYFS0mSlu5ve+Yss2hukel82LAPmeDG92pME=; b=Hg94OWogoIxoEoTmD8cJ/XQvDpra6afKKz18+MUHIAWEavawWhGFOfTlGoilEZXWpD IQXeaR/lHr/yZp6luB67nD4IWtpf6xbmlg1Cnkc1b4YGvI3AMVOyMjLFQov8blYyLUIA kWYigjI0e5nPBi3VtagqRr0O+b86wHemn+hzabBX8Aj+OqXieibbs76+hC5seKeTPrg5 PnENMj79WMKoT3bw00Xz5xrqxttU4qJjSpOZ/klu9PrFT31p1fKHytgVG6Kf/nR34BQS cky8Ug7QAfmS37wHRvhUtLtRW64W9fTN5vo2wf3E1wBzd2vle7Lvi3f8x4Z1TNnC2YG5 TvHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697545067; x=1698149867; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0NE2es9FYFS0mSlu5ve+Yss2hukel82LAPmeDG92pME=; b=sI2VH/bO52vAf01fZk+dRUj28XU5lu/O/3PEZnYSz/n47/9syKardPw04mFn7uhEFP hUtAyO7ktjzCyBffRn/III66q3Z7VlGmKd+uV+pALMjDOXClVhi54LmHZv8mupNBS5PP x06xAFf3f3RggEJSWjk8u/YT91uSFaZi1H6HnjK8w0/hI1gFj0HKq14BYfoW0fLAbgfd PM4IwRCgMsMiEYBmUYH0XeSE+CVsutU0ABxKg3pc5DzVviSHscIULLcj2xCy5mBytTq/ eexWwoVWso8OVcPHpNrwLceLLzbUYYm/JJyl0S9JyRN4FJZfxULPGd9ABBOfHRYHer7z DOcg== X-Gm-Message-State: AOJu0YzjMryg3xhwibrqieXLV3slKGnNtsyOzPiA8IwALGi9daWmGrQo wlnmHRtiVEqMtSv5BiwozM91tPsQyXqFy7YhiNNgEQ== X-Google-Smtp-Source: AGHT+IHkBhMWQoFFD2034ar1d+95+wi0b8n/21RlF6F0EEWfujXdsGMEiOi84J1LtX9CXVmccciqfJIbbLbaczLUkXc= X-Received: by 2002:a05:6512:558:b0:500:7685:83d with SMTP id h24-20020a056512055800b005007685083dmr1675449lfl.48.1697545067186; Tue, 17 Oct 2023 05:17:47 -0700 (PDT) MIME-Version: 1.0 References: <20231013064827.61135-1-linyunsheng@huawei.com> <20231013064827.61135-2-linyunsheng@huawei.com> In-Reply-To: <20231013064827.61135-2-linyunsheng@huawei.com> From: Ilias Apalodimas Date: Tue, 17 Oct 2023 15:17:11 +0300 Message-ID: Subject: Re: [PATCH net-next v11 1/6] page_pool: fragment API support for 32-bit arch with 64-bit DMA To: Yunsheng Lin Cc: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Alexander Lobakin , Lorenzo Bianconi , Alexander Duyck , Liang Chen , Guillaume Tucker , Matthew Wilcox , Linux-MM , Jesper Dangaard Brouer , Eric Dumazet Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Stat-Signature: o5bk9sgem4fy5gkihsyfjr3zrtbi56fi X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 26FF22002A X-HE-Tag: 1697545068-372556 X-HE-Meta: U2FsdGVkX1+48IGuFjsrKhIVppBRaWJa3oajJ+v78UN4LQTMEh6I35aD04o1eVk840qvbzOIIiNMnjmMR1gVjhcd5MEkuVyKGgn4Kz6z47kS9Jn7sJfyAEydi+vUsJQZskqWy02GE4hTXC63DkgzVn7chOx34oLlaThzlRbC6wjUy0YdrnP+SdM5oXer5c4ncTe3XfCAA+1GvTTwzD7m678B1sUCOFnT9acr8qZpLfJ4OHNpDMvtWP7/ZyoTfz7XzBmX4EaEIvkIavj0zoCHqMrHO4sRtF3GuPQ4sCoojhp2lOQ7HpgjqUNTsYRoZlqwCteewAXPMiDCQjRsc8bd1OoAkSIUvJcwMYqXQXKIxrRa9yKFTcI/GF/FUt85kr6N/pMegq6EhBhrfVGaTT5vyn00wShLNydniYrb4SgzIJYDcb2LibxPYwGExIMlIBNNaiYlD4t86RjKBJS2KeL5Z4+z6oFvfmJeWoANj6PHGQQ54MZTMnjiOLrKv/qzggQJ3Wvrkq/h4b5HOQ+K0yMCGawUVU8+FfBRTEy19/3H6WCQXvlmROphcfBUHM5YIQR16sfvqlwqqUqWce4HloLKDqwa9VUWV2A44RPZg0sQTWdyo542c6OY4Hlt7s/w0mtjGY22vZvcRDPLKmK015pdqq+FXRv9dGOXH3goSUa2dvsMo5T+F+Rkjlhj0OcuF0anwAwJNJHoaWYN2mKKBTXXhI95xr5BLFXEhp8Yk3nYr6QI402emEm+fQ2pC+PFedHFcIgteomc59wV2Za/PmHQZ4nCSL0JXz2U625qHlYG6EUobKiOk+RITq2MY4CoxtFxoDdKpIFSGGQ9K/o1EY04H7oPWLh9KpfT80udBNoAjZfRUSHRV65zfMrDo/T7ybH3fvzEJ/kZurzIoB7Hf0PRep1QV6ja2DGQDKAAXDM/MSWneeJnc9cVWe+c1KPYyABM7/bxoQk9xCWvi+KewV+ NMu+ui0P 5zHMewIq/izp6cHpORwt8go9nv1zg+oqbNvvHKkLvHCYOUMpsA+j328iUnNg9Z/4U7vB7l/qk9iFFPq31dtjp5GZck3ZMxDk6ZuRvfyVi1hyKpWJVuQrII8RkdPJBYgj5CJaZMzg20npXs6ZWu11w6w7tI7k6AjzGR9Yze50AxFkFnrw6P5hPfNpVAwE/2C/loPgQPRBgn3oG3ZJ9WDg8EtPGn2Le9YxIjuGUPInUuzZ5AaYp4Ta2u27/0127THFViGCiqz9+4Be0pZp3VQtJ3z5CyABhDwj0MzMvZvw/EFRKMbnYakSle+ubfliM4Ryws/JxYttzs5y08eNbs12p94664zGaihTQA4XbwG5yRSIUMZoNwlCQCQHfZi39tJ6SxTDL77pwE28yOPzm+ZnPSFB7MLlzjr3GGfQGPVSE2Hi5IxAbR2BTxzIBq/OtWhmbEYbCK49gCLUwhwRD6TFHsnKPMgfBakL4QexN1uTOilSFW0EhD+Vk05dxMwk1wU716ggAVAH2g+clL/xAs3djl2/smDOEiKvlAkMLaOQ4eqK1jfNrNZlIMEQY+Cc3ZKwVDCMf64q1lMJhbnyDNdWnV4tHiWM4x5TyL1T/W5ofl9c1wVrM/diLfLJ8rUib58bHtlfihIhOcFTupOxQl5Pd5ulUfCBepELex3GpO+Ve57sjkGGs5AwPPtcJIRwonZu6+M3deEz0bNBZSDU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Yunsheng re-sending, HTML was somehow enabled and the ML dropped the public part. Apologies for the late reply (and the noise) On Fri, 13 Oct 2023 at 09:47, Yunsheng Lin wrote: > > Currently page_pool_alloc_frag() is not supported in 32-bit > arch with 64-bit DMA because of the overlap issue between > pp_frag_count and dma_addr_upper in 'struct page' for those > arches, which seems to be quite common, see [1], which means > driver may need to handle it when using fragment API. > > It is assumed that the combination of the above arch with an > address space >16TB does not exist, as all those arches have > 64b equivalent, it seems logical to use the 64b version for a > system with a large address space. It is also assumed that dma > address is page aligned when we are dma mapping a page aligned > buffer, see [2]. > > That means we're storing 12 bits of 0 at the lower end for a > dma address, we can reuse those bits for the above arches to > support 32b+12b, which is 16TB of memory. > > If we make a wrong assumption, a warning is emitted so that > user can report to us. > > 1. https://lore.kernel.org/all/20211117075652.58299-1-linyunsheng@huawei.com/ > 2. https://lore.kernel.org/all/20230818145145.4b357c89@kernel.org/ > > Tested-by: Alexander Lobakin > Signed-off-by: Jakub Kicinski > Signed-off-by: Yunsheng Lin > CC: Lorenzo Bianconi > CC: Alexander Duyck > CC: Liang Chen > CC: Alexander Lobakin > CC: Guillaume Tucker > CC: Matthew Wilcox > CC: Linux-MM > --- > include/linux/mm_types.h | 13 +------------ > include/net/page_pool/helpers.h | 20 ++++++++++++++------ > net/core/page_pool.c | 14 +++++++++----- > 3 files changed, 24 insertions(+), 23 deletions(-) > > diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h > index 36c5b43999e6..74b49c4c7a52 100644 > --- a/include/linux/mm_types.h > +++ b/include/linux/mm_types.h > @@ -125,18 +125,7 @@ struct page { > struct page_pool *pp; > unsigned long _pp_mapping_pad; > unsigned long dma_addr; > - union { > - /** > - * dma_addr_upper: might require a 64-bit > - * value on 32-bit architectures. > - */ > - unsigned long dma_addr_upper; > - /** > - * For frag page support, not supported in > - * 32-bit architectures with 64-bit DMA. > - */ > - atomic_long_t pp_frag_count; > - }; > + atomic_long_t pp_frag_count; > }; > struct { /* Tail pages of compound page */ > unsigned long compound_head; /* Bit zero is set */ > diff --git a/include/net/page_pool/helpers.h b/include/net/page_pool/helpers.h > index 8e7751464ff5..8f64adf86f5b 100644 > --- a/include/net/page_pool/helpers.h > +++ b/include/net/page_pool/helpers.h > @@ -197,7 +197,7 @@ static inline void page_pool_recycle_direct(struct page_pool *pool, > page_pool_put_full_page(pool, page, true); > } > > -#define PAGE_POOL_DMA_USE_PP_FRAG_COUNT \ > +#define PAGE_POOL_32BIT_ARCH_WITH_64BIT_DMA \ > (sizeof(dma_addr_t) > sizeof(unsigned long)) > > /** > @@ -211,17 +211,25 @@ static inline dma_addr_t page_pool_get_dma_addr(struct page *page) > { > dma_addr_t ret = page->dma_addr; > > - if (PAGE_POOL_DMA_USE_PP_FRAG_COUNT) > - ret |= (dma_addr_t)page->dma_addr_upper << 16 << 16; > + if (PAGE_POOL_32BIT_ARCH_WITH_64BIT_DMA) > + ret <<= PAGE_SHIFT; > > return ret; > } > > -static inline void page_pool_set_dma_addr(struct page *page, dma_addr_t addr) > +static inline bool page_pool_set_dma_addr(struct page *page, dma_addr_t addr) > { > + if (PAGE_POOL_32BIT_ARCH_WITH_64BIT_DMA) { > + page->dma_addr = addr >> PAGE_SHIFT; > + > + /* We assume page alignment to shave off bottom bits, > + * if this "compression" doesn't work we need to drop. > + */ > + return addr != (dma_addr_t)page->dma_addr << PAGE_SHIFT; > + } > + > page->dma_addr = addr; > - if (PAGE_POOL_DMA_USE_PP_FRAG_COUNT) > - page->dma_addr_upper = upper_32_bits(addr); > + return false; > } > > static inline bool page_pool_put(struct page_pool *pool) > diff --git a/net/core/page_pool.c b/net/core/page_pool.c > index 77cb75e63aca..8a9868ea5067 100644 > --- a/net/core/page_pool.c > +++ b/net/core/page_pool.c > @@ -211,10 +211,6 @@ static int page_pool_init(struct page_pool *pool, > */ > } > > - if (PAGE_POOL_DMA_USE_PP_FRAG_COUNT && > - pool->p.flags & PP_FLAG_PAGE_FRAG) > - return -EINVAL; > - > #ifdef CONFIG_PAGE_POOL_STATS > pool->recycle_stats = alloc_percpu(struct page_pool_recycle_stats); > if (!pool->recycle_stats) > @@ -359,12 +355,20 @@ static bool page_pool_dma_map(struct page_pool *pool, struct page *page) > if (dma_mapping_error(pool->p.dev, dma)) > return false; > > - page_pool_set_dma_addr(page, dma); > + if (page_pool_set_dma_addr(page, dma)) > + goto unmap_failed; > > if (pool->p.flags & PP_FLAG_DMA_SYNC_DEV) > page_pool_dma_sync_for_device(pool, page, pool->p.max_len); > > return true; > + > +unmap_failed: > + WARN_ON_ONCE("unexpected DMA address, please report to netdev@"); > + dma_unmap_page_attrs(pool->p.dev, dma, > + PAGE_SIZE << pool->p.order, pool->p.dma_dir, > + DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_WEAK_ORDERING); > + return false; > } > > static void page_pool_set_pp_info(struct page_pool *pool, > -- > 2.33.0 > That looks fine wrt what we discussed with Jakub, Acked-by: Ilias Apalodimas