From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4EDAC3DA6E for ; Mon, 8 Jan 2024 16:14:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34DC26B007D; Mon, 8 Jan 2024 11:14:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2FD7E6B007E; Mon, 8 Jan 2024 11:14:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1C4CB6B0080; Mon, 8 Jan 2024 11:14:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0AC196B007D for ; Mon, 8 Jan 2024 11:14:16 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id CBD3AC080D for ; Mon, 8 Jan 2024 16:14:15 +0000 (UTC) X-FDA: 81656640870.29.F2C9163 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf17.hostedemail.com (Postfix) with ESMTP id DE59B40023 for ; Mon, 8 Jan 2024 16:14:13 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=YTRoID3+; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf17.hostedemail.com: domain of alexander.duyck@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=alexander.duyck@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704730453; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=unjvTIPzqvEYfu9k31l2UbfWfPgY7ha+9toKsyi5Vr8=; b=Whmwab/rDedgtdis334Byq2fmDGUqhgw5pqHZu9F6MYc3HUtYWvaf8y/ZWPBud1+mCxTrB Xr2aQoomSrHMNtY+rcXtoSwSWKE0H0HuEQKX6Le8NXTYOcwIf+Ps3nI3GcKpb9DhDgqdqE CwyT9JY7leup9FfHrUqPCApO3z7mww4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=YTRoID3+; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf17.hostedemail.com: domain of alexander.duyck@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=alexander.duyck@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704730453; a=rsa-sha256; cv=none; b=Ns+/5ky3aE4tpaPGxRvT1xxJDQIUk3/6qUt+zzHTALXPx0i/oBriAZ61E1UjwVu2WzqHVy YmmwAGKDvFh05FiLmiAe4arQmIjM4RR8vW0C/8tlFXhxDGZJ8Scg/CP0UVk8OI4CvDw9yU 30FXVFTwgwrUilNNUJKCUBVqdW6o/MY= Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-28b400f08a4so1764354a91.1 for ; Mon, 08 Jan 2024 08:14:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1704730452; x=1705335252; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=unjvTIPzqvEYfu9k31l2UbfWfPgY7ha+9toKsyi5Vr8=; b=YTRoID3+AfDfTpqtzCD98GeReKIBy9PRQPdQeMdnkZW56/UpB0PXnqLiK5GaOLxri5 LDh2GUOZzCAVH3SRiojx/81nwKeEGAX00KKkfIElAx7oto+bspjcMHwJ3+dITi8MBuno 3PHySPYI10HmYNRW2jQUfZQdW3y1yyEqRxA+OYOFf4lrxNhHEUNy81uMuUnJqI8KqqHs 0IakCnq++994TOZDkHOcReJ1IsEUwF+tvBPPix4HcTYRIlMl8BHPCMVpAxBePMkoTyvn L1jUKbopdaT5wlNOTxGfZ4AUjrJbbJK2xBVSqSih/TzG8O0377b2+1NmQKzPm4Fuxwsr XS+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704730452; x=1705335252; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=unjvTIPzqvEYfu9k31l2UbfWfPgY7ha+9toKsyi5Vr8=; b=n/15gP5oWT6Z5fBL0aRHL3W749KCypQ9z57b2U/jftD3rXv9deyxktg4PBc8HTUaqd 44qe4nFlv/XHLJ87TUswW9ofLXtD48qdwqWgFyFUWyJkkSDScAOTLFmC3Zrj0f3firUu mrMtL/2lTz5MerlKL7Wo/4hs1Tw+f0XtcDdvQdHO68c7yxwLUHh/vgpuMfLEgBmlDM11 KtuRtaQ3Ngp9K+5ZsYnHypkOlHxQIyfomNwQK9Eaa3Z0YsQCXLnLMJinvxZfetWDYIi+ YkqC5Z2zH0ykPnKrf3T0Nee3bBKM7waaAUscMhHLHxwi3MH879wfElX5tJOomhIbbooM hR/Q== X-Gm-Message-State: AOJu0YxK0li4nZ+CFjRDLajzNV8/3LsxNetKRnWIwogfxi6muLJoKXwY AVGFWLJa0lgIa9DO/q5h268UgSPepzjxISXzRI4= X-Google-Smtp-Source: AGHT+IGGbd9uEm1HwzeQk0zSJgcIrych+A4ioq0XePO8wLwmpeqipVfchKXfctSdfiFT9SV1RSAuYB49yeFctZANvgM= X-Received: by 2002:a17:90a:604e:b0:28d:19d3:8c58 with SMTP id h14-20020a17090a604e00b0028d19d38c58mr1670693pjm.73.1704730452552; Mon, 08 Jan 2024 08:14:12 -0800 (PST) MIME-Version: 1.0 References: <20240103095650.25769-1-linyunsheng@huawei.com> <20240103095650.25769-3-linyunsheng@huawei.com> <1d40427d-78e3-ef40-a63f-206c0697bda2@huawei.com> In-Reply-To: <1d40427d-78e3-ef40-a63f-206c0697bda2@huawei.com> From: Alexander Duyck Date: Mon, 8 Jan 2024 08:13:35 -0800 Message-ID: Subject: Re: [PATCH net-next 2/6] page_frag: unify gfp bits for order 3 page allocation To: Yunsheng Lin Cc: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, "Michael S. Tsirkin" , Jason Wang , Andrew Morton , Eric Dumazet , kvm@vger.kernel.org, virtualization@lists.linux.dev, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Stat-Signature: ssa7dcup8e6aghow59kcxjndgxmerdmc X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: DE59B40023 X-HE-Tag: 1704730453-491768 X-HE-Meta: U2FsdGVkX1+gr4hwfdJgJ7+nsCQ3B6dwfJneqOd3FBNWZFzRcY3Rtdv4T3DCJfUpG1cvaSvWfpge3IgcSWTXoZgO1jC10fAmD31cdVXrwta8XheypYFYx5Q5+2uB0Y6yjhlOBassWBlWUIS41SFs+4hezCbr0QnMAmY2zGzKzp2QzJiMRsY5JGsU+H8wSXq9P6FSf06CR8kx8Jea+ithr5Kfk82NdDEDdqlX0Q6on5BincgrNtYLDBl3AbR15CarhACnVnHNdTyz2Z+AJ0DFPy2ifkmSTN/8WGRgO5yakvWAX8f6MA1dPv7j+umzFl27/SRBJbKd8ZNVRBaCV5EBPMcmpsL++Czmh2jySCnLOOUrYynEasbBFQv4rqy8xjOdmH7xKHaFGMrFFVmydAIvDIUHATltJWkk+5vCuscFUaQDlY3dGlI8TzAF0uHEyeicKMB9Im9Ia3Qu+sEofcA8Ka5o4tJZAyfAzyt0nEdtd3G5awy+KyaLRc8WdisXNOA75Eohq8y6xbFfPvX5qGxhZ43XByQ+fIf7y23vJatryuSe/4VGoLbm482mik5bZ8GqMFI5vxTrnyjKBfw08lfwMTcp/syHD1qkrNN75M98RP9yLVs0n+Ggyn1f0RoWsSDpL4MLssWzjIFUYWgZMX4y6oqG3lU4OM0PNHx8LjzcXSp+gjf/ZVcHt7PYi7s1VbG316QrJpC6jiNwMwl0Zsjk1lOO1DyKpSZmFoQ2uZ9ckwW7BMi6+/7YI2f3cXqyTxwygSghMTmv23PnFaCnq5wsKVqCYToaOrcE1hd5XLvrtJbETWcybZ5UCoWO34o3T/8WYMCu3TbZ3rN1bRMb/G8E+4hiI/Q6y+ZHIhjE57P6exVQPu/m6WQikmuuPHEMrRU8ShifL9GyRZaoIOta9r0U6OsGNkJnLyOY4S1FN1d5NIhvVh+nX5QkqBqFLhPjEOukY+8AI1IokNcPJHbS0JE mAr5nMmV KS01N9M6IAoD1Ix6eiQXyI3bvRZU1yJK+wVsND1bUZU/QU0sUMUgegv5xz4f9yzgjXPf/QM/mb2sfme8W/BTsg3qS0WwpjT2edS7E+/Arek0EAEjhgHA2sR55Um7G8IoLak9wzublqdBplRI69a88+/yNXIP3rTLXQNVa2JxmebCzgX017NX/KKLDsYT9C76hZ5K8jYAHN3ODjTgq6Isu5NJdtC+y5WCS31jmfVkjqiv44qFbUkKVgLSJFDkjBSjJUn7jrQj1Vdfa35extUZ9aBYTI9OhFMBGHPf9OlGM/bazHDGQ9x5pq8M3vJ0Kqa6AXxquHbGBi6j9scYJskPuWXrjPYMxvDCiOu+I10QCLxWVoLDmPOQOsq1BeUBpBA7CqpleLVtrcdMLBf0vmpDwk4Mw/QRzKbbozuuE6F8nwhwhH+7zHbAMhIjQgGHuCEcnGdT28YMfEqcaLGnfN1d8h45WE5eJJ5x2VHg3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Jan 8, 2024 at 12:25=E2=80=AFAM Yunsheng Lin wrote: > > On 2024/1/5 23:35, Alexander H Duyck wrote: > > On Wed, 2024-01-03 at 17:56 +0800, Yunsheng Lin wrote: > >> Currently there seems to be three page frag implementions > >> which all try to allocate order 3 page, if that fails, it > >> then fail back to allocate order 0 page, and each of them > >> all allow order 3 page allocation to fail under certain > >> condition by using specific gfp bits. > >> > >> The gfp bits for order 3 page allocation are different > >> between different implementation, __GFP_NOMEMALLOC is > >> or'd to forbid access to emergency reserves memory for > >> __page_frag_cache_refill(), but it is not or'd in other > >> implementions, __GFP_DIRECT_RECLAIM is masked off to avoid > >> direct reclaim in skb_page_frag_refill(), but it is not > >> masked off in __page_frag_cache_refill(). > >> > >> This patch unifies the gfp bits used between different > >> implementions by or'ing __GFP_NOMEMALLOC and masking off > >> __GFP_DIRECT_RECLAIM for order 3 page allocation to avoid > >> possible pressure for mm. > >> > >> Signed-off-by: Yunsheng Lin > >> CC: Alexander Duyck > >> --- > >> drivers/vhost/net.c | 2 +- > >> mm/page_alloc.c | 4 ++-- > >> net/core/sock.c | 2 +- > >> 3 files changed, 4 insertions(+), 4 deletions(-) > >> > >> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c > >> index f2ed7167c848..e574e21cc0ca 100644 > >> --- a/drivers/vhost/net.c > >> +++ b/drivers/vhost/net.c > >> @@ -670,7 +670,7 @@ static bool vhost_net_page_frag_refill(struct vhos= t_net *net, unsigned int sz, > >> /* Avoid direct reclaim but allow kswapd to wake */ > >> pfrag->page =3D alloc_pages((gfp & ~__GFP_DIRECT_RECLAIM)= | > >> __GFP_COMP | __GFP_NOWARN | > >> - __GFP_NORETRY, > >> + __GFP_NORETRY | __GFP_NOMEMALLO= C, > >> SKB_FRAG_PAGE_ORDER); > >> if (likely(pfrag->page)) { > >> pfrag->size =3D PAGE_SIZE << SKB_FRAG_PAGE_ORDER; > >> diff --git a/mm/page_alloc.c b/mm/page_alloc.c > >> index 9a16305cf985..1f0b36dd81b5 100644 > >> --- a/mm/page_alloc.c > >> +++ b/mm/page_alloc.c > >> @@ -4693,8 +4693,8 @@ static struct page *__page_frag_cache_refill(str= uct page_frag_cache *nc, > >> gfp_t gfp =3D gfp_mask; > >> > >> #if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE) > >> - gfp_mask |=3D __GFP_COMP | __GFP_NOWARN | __GFP_NORETRY | > >> - __GFP_NOMEMALLOC; > >> + gfp_mask =3D (gfp_mask & ~__GFP_DIRECT_RECLAIM) | __GFP_COMP | > >> + __GFP_NOWARN | __GFP_NORETRY | __GFP_NOMEMALLOC; > >> page =3D alloc_pages_node(NUMA_NO_NODE, gfp_mask, > >> PAGE_FRAG_CACHE_MAX_ORDER); > >> nc->size =3D page ? PAGE_FRAG_CACHE_MAX_SIZE : PAGE_SIZE; > >> diff --git a/net/core/sock.c b/net/core/sock.c > >> index 446e945f736b..d643332c3ee5 100644 > >> --- a/net/core/sock.c > >> +++ b/net/core/sock.c > >> @@ -2900,7 +2900,7 @@ bool skb_page_frag_refill(unsigned int sz, struc= t page_frag *pfrag, gfp_t gfp) > >> /* Avoid direct reclaim but allow kswapd to wake */ > >> pfrag->page =3D alloc_pages((gfp & ~__GFP_DIRECT_RECLAIM)= | > >> __GFP_COMP | __GFP_NOWARN | > >> - __GFP_NORETRY, > >> + __GFP_NORETRY | __GFP_NOMEMALLO= C, > >> SKB_FRAG_PAGE_ORDER); > >> if (likely(pfrag->page)) { > >> pfrag->size =3D PAGE_SIZE << SKB_FRAG_PAGE_ORDER; > > > > Looks fine to me. > > > > One thing you may want to consider would be to place this all in an > > inline function that could just consolidate all the code. > > Do you think it is possible to further unify the implementations of the > 'struct page_frag_cache' and 'struct page_frag', so adding a inline > function for above is unnecessary? Actually the skb_page_frag_refill seems to function more similarly to how the Intel drivers do in terms of handling fragments. It is basically slicing off pieces until either it runs out of them and allocates a new one, or if the page reference count is one without pre-allocating the references. However, with that said many of the core bits are the same so it might be possible to look at unifiying at least pieces of this. For example the page_frag has the same first 3 members as the page_frag_cache so it might be possible to look at refactoring things further to unify more of the frag_refill logic.