From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD1D4C2BD09 for ; Mon, 24 Jun 2024 18:26:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4066D6B0394; Mon, 24 Jun 2024 14:26:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3B6D26B0395; Mon, 24 Jun 2024 14:26:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 22FCF6B0399; Mon, 24 Jun 2024 14:26:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id F32776B0394 for ; Mon, 24 Jun 2024 14:26:37 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 9622EA11F7 for ; Mon, 24 Jun 2024 18:26:37 +0000 (UTC) X-FDA: 82266612834.14.922F749 Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) by imf27.hostedemail.com (Postfix) with ESMTP id 9060140016 for ; Mon, 24 Jun 2024 18:26:35 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WUXleK19; spf=pass (imf27.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.167.42 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719253577; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8QY3hLjglDBcIn3zXNfyok4jWrdOzSelcx/I3VKtoZU=; b=fylPhHrLWWCFCXbWX6OR0AaOLv/+yxOeusaidY6AgtCbDeZH01aRJM+rPi/OSvzjFP25/D aWuNWpdyJ0D8p1xo5yNWEh9eX4hOU8P1/EkH0Zf3lTVIOFCwAelYmTLjUacWaEGSvEs2Kf qfiquNMTQs+1QCvv0LJt50QXHzjEoPs= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WUXleK19; spf=pass (imf27.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.167.42 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719253577; a=rsa-sha256; cv=none; b=VKyTl83g27MX+Jryc3Uh4Tw03Njyz6xZdBZRcsr/eBAWen+sMIAwnlv34np8BSxoaN1cYP XWv2cLf7AIRmdPiO6wBotxI+6TcflbyMXPjV494nN2T6IxnJniMj8zp+rnYvB8u9HGSAPv tU+aQmFJEylwHOBQ2ZdWWm2OgFQUIsQ= Received: by mail-lf1-f42.google.com with SMTP id 2adb3069b0e04-52cdf4bc083so3255930e87.2 for ; Mon, 24 Jun 2024 11:26:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719253594; x=1719858394; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=8QY3hLjglDBcIn3zXNfyok4jWrdOzSelcx/I3VKtoZU=; b=WUXleK19F8cVj8uYznDT3ryE6PoevI9YMJV0TlwDRFoAnfJYYxUcceVCIGuhjKu7ov 4igXgHs1z7V6ecny2GnmVADFsXne9ZLVjmztmLs/aOs/o0Hk1UIxxuxYE4mbq0JPOOKA RQnmE7YQ+wS1D1+C4JNi51oxRFD3ATh5DsoPZywEdKyFOS0EuD2fMTKphXd/PJWAYY0c XTABHj3FYk64mOVzwJ2YW87/bQFwdwiHqf1wmOtIPHCYuWR2whB1FieCHPX40hnLVkkC S7XQ2wCjVtARxBYQLzPhne2GoOddDQb3JbVQRCLh1rd2ZOM0hwc72LzkfdH3aB89BJuw F1zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719253594; x=1719858394; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8QY3hLjglDBcIn3zXNfyok4jWrdOzSelcx/I3VKtoZU=; b=LnfxYg5C86Ki9/BBkJGqVrxpdJtZvL5YzlvEo2y8GEq/7F/BzyJ1UC/Tgc71xEmIVt Ps4ctimRG8XWco87K70QXM5Jc4TSIiIuoBY2PTfBlnBqX4Ns/nSUlkg2a7HcBb1WtTlG UyfQi4qeViJETRJpoIYBiakRnOz9uGstpMETB5zLgxeomyA9U3yLWZLtG76zF80ixP+8 iCdyqUb+TiGAVnstagQRC7pEgAYIwU9nI/G/CZnLogn1GjoqF5E7hy4/ynGHG2PjcMjT qjK+Ehjzo/ZxBqxT9/8RyLC3UdnT+ywt5KhCs8hBOJfrTJWZJsA0PV24GyrPtQe1VCmU We7g== X-Forwarded-Encrypted: i=1; AJvYcCW23UgrZZ29eujqg1DS2DSBgKEltbGqQ36k0BEp1n0PkyjkMxS2WVWz2Mh3MkTDN8/p2qlsGTC9hUryJhhojUB6hzM= X-Gm-Message-State: AOJu0YxEvoVwRQoae7xkdCz3szw5HmXrIkKjjnkYqZbvivRxYdylMYZ8 pJEyZIebc3Pz0BHs8Mf5ALv/NbfHb8rB919ErLI9rJtNdH7sV65i4vhAixn6My2wgw== X-Google-Smtp-Source: AGHT+IE4h3ga/z+Y+Sd3WyTIsGf+yCxLn2kq84TioTsOo+/QrFFHZbpbPJkF+2UXnFPTlTeyZyLVMA== X-Received: by 2002:a05:6512:6c3:b0:52c:83c2:9670 with SMTP id 2adb3069b0e04-52ce0646e8bmr4740430e87.69.1719253593248; Mon, 24 Jun 2024 11:26:33 -0700 (PDT) Received: from ?IPV6:2001:16a2:df66:b900:46d:aa3:6645:bcd8? ([2001:16a2:df66:b900:46d:aa3:6645:bcd8]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4248179d1fdsm149929305e9.7.2024.06.24.11.26.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 24 Jun 2024 11:26:32 -0700 (PDT) Message-ID: <167b2f11-a013-440f-b196-5d8c0ea5d9b3@gmail.com> Date: Mon, 24 Jun 2024 21:26:29 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [linux-next:master] [mm] 0fa2857d23: WARNING:at_mm/page_alloc.c:#__alloc_pages_noprof To: Yosry Ahmed , Hugh Dickins , Yury Norov , Rasmus Villemoes Cc: kernel test robot , oe-lkp@lists.linux.dev, lkp@intel.com, Linux Memory Management List , Andrew Morton , Chengming Zhou , Nhat Pham , David Hildenbrand , "Huang, Ying" , Johannes Weiner , Matthew Wilcox , Shakeel Butt , Andi Kleen , linux-kernel@vger.kernel.org, "Vlastimil Babka (SUSE)" , Yury Norov , Rasmus Villemoes References: <202406241651.963e3e78-oliver.sang@intel.com> <12fb19d1-3e57-4880-be59-0e83cdc4b7f1@gmail.com> <61d19ec8-2ba7-e156-7bb7-f746dae8e120@google.com> <5b3e732c-d23d-41ef-ae5c-947fa3e866ab@gmail.com> Content-Language: en-US From: Usama Arif In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9060140016 X-Stat-Signature: 3uf9xx3p5tr3sox7izjez4jtc6a4iq1r X-Rspam-User: X-HE-Tag: 1719253595-735549 X-HE-Meta: U2FsdGVkX19OrTaf9aSXxIdwXoXsWneF5KlsY9g4W84iAo5KfylWX7nT2FLgtI3JkiZfQmjkpW0ECGqn5INop4mB8jLtN9q3PbP0fU64ZDmfplrqQ254Eunq0O17P+g1yX1B5D49xvGkuZLtCZXb6hcmajafwF2StYsF2Ja1AJpwemqbwAQrjg3oZkdBIHe+jrwrmrd05n1vsmFWYanK5PJugZ6tFyYNzn6ysfL52YNZrblHJfqybCp1lHaIqWs6XJrOMkE8td9lx/pY72HBxwqaUB20m/6udAhMpPrzGYxlsjrFPupqFMpTaXZ2js+emKG9vxsCI/gs4ICYpEFGfrb9+RitR8gULRc2LFrRQI+LXMFhUUsouIDgLi67i5v4E4EKfFi+jwPRPNrEdh8Fp3hPTIeMqSo8WOSWwfwYmvsHtcH4I03hgh3VFqYJzNcqJ+3TBI+qPMvRvTdfBQqPjB3f5tIIauXoVlz4GQha8A1iYNkIQUYfIyA6u46/nutAWFQ0fxKUWIAVMlpaP6ZlAcf+VhZTJDmz34jBbnHXxdgF3ElUYNdjOpSmUt9q35ST7FthEUhGFZpO523vYnT4+YevMWyIKRVmS5M4X6zaH9kWinXRvOX5LedU6lLStLaGGeettxLvNSjzk3nijcUhANIm0jwIOqXqRdRCzmlq466G/FG/tg8HUd1xzDnFoaqaBw1yuRy9gnm3DhOpwWHG9bgui4aC6DesW8+ncSsRBVQWgCbtbE1yXpk4zV7iKWqoJGBEWR75mllhQ3FnrinZUKZx6T8hdnIcokjOFJE+wQ86RBrwpZERSfVbMND6ksjDfScF1UlpXzpcjeznxfzK1cgpGwmd34sgFU9yFgcE1ag11O4KGfQ0mGRjWxbmEXVy11Bc7DOxMx/bX7qLc5aTavv8z9VoDaxLddK86GT9GcuxpaPAR4eoJeNAH78TXmbNbeVFOxU11EtZBSDVeCP wcqt8YmQ uQSxnZWDajhOvfWoevF/8lNlTvNowhgRkiwDEsZu2LHo6m1oOGFrCWrTW/DHsApMtImPpu7IasvHgp8IqIqdJm/XJB4xbIXSuKT1hNQECvcxYsG2q6ra1rviME/MTEDvwq3zaizCOaWaYwDHb31rL0NFRsQzZ96iC00PAJJZ3IowfbelQ7aWh/Uvz2qDkIFddGHRWQDdUdbrV7OOpcPfcJqUfcly60yjpZJjIO0bUCQhjHm5I1129Er5ET6MMD6EJgyx4XB/a8Cn7//8uY1yZwyhUG01uRR55zgGk/+2X5bBkgoAGa1E01y/Vy0rRsAaTWEbWGPwSYNMhKNOgZP6xikrrI4t9+xS0N4h08+OPhax451ib728xLdaAkWFJDDps6o5sff22aJFjIxIYGvZOzv4Sf3ed3t391arOIUsjJXU3n3gRx24FxqhpvfR47GQI7q7fL8UNMizT5MZIRYpnqEbdCOtfpMccV4Lz8p7Qnzkptlk3YzCSpy0iBlkU6aqaiZQF/YUridjAgj8/WyJz/o2w2mYHSNjaia97QYVGxcjWLJaH6iZ+7N7fygVYnVeR242Q X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 24/06/2024 20:31, Yosry Ahmed wrote: > On Mon, Jun 24, 2024 at 10:26 AM Usama Arif wrote: >> >> On 24/06/2024 19:56, Yosry Ahmed wrote: >>> [..] >>>>>> - p->zeromap = bitmap_zalloc(maxpages, GFP_KERNEL); >>>>>> + p->zeromap = kvzalloc(DIV_ROUND_UP(maxpages, 8), GFP_KERNEL); >>>>> No, 8 is not right for 32-bit kernels. I think you want >>>>> p->zeromap = kvzalloc(BITS_TO_LONGS(maxpages), GFP_KERNEL); >>>>> but please check it carefully, I'm easily confused by such conversions. >>>>> >>>>> Hugh >>>> Ah yes, didnt take into account 32-bit kernel. I think its supposed to be >>>> >>>> p->zeromap = kvzalloc(BITS_TO_LONGS(maxpages) * sizeof(unsigned long), >>>> GFP_KERNEL); >>> You can do something similar to bitmap_zalloc() and use: >>> >>> kvmalloc_array(BITS_TO_LONGS(nbits), sizeof(unsigned long), GFP_KERNEL >>> | __GFP_ZERO) >>> >>> I don't see a kvzalloc_array() variant to use directly, but it should >>> be trivial to add it. I can see other users of kvmalloc_array() that >>> pass in __GFP_ZERO (e.g. fs/ntfs3/bitmap.c). >>> >>> , or you could take it a step further and add bitmap_kvzalloc(), >>> assuming the maintainers are open to that. >> Thanks! bitmap_kvzalloc makes most sense to me. It doesnt make sense >> that bitmap should only be limited to MAX_PAGE_ORDER size. I can add >> this patch below at the start of the series and use it in the patch for >> zeropage swap optimization. >> >> >> bitmap: add support for virtually contiguous bitmap >> >> The current bitmap_zalloc API limits the allocation to MAX_PAGE_ORDER, >> which prevents larger order bitmap allocations. Introduce >> bitmap_kvzalloc that will allow larger allocations of bitmap. >> kvmalloc_array still attempts to allocate physically contiguous memory, >> but upon failure, falls back to non-contiguous (vmalloc) allocation. >> >> Suggested-by: Yosry Ahmed >> Signed-off-by: Usama Arif >> > LGTM with a small fix below. > >> diff --git a/include/linux/bitmap.h b/include/linux/bitmap.h >> index 8c4768c44a01..881c2ff2e834 100644 >> --- a/include/linux/bitmap.h >> +++ b/include/linux/bitmap.h >> @@ -131,9 +131,11 @@ struct device; >> */ >> unsigned long *bitmap_alloc(unsigned int nbits, gfp_t flags); >> unsigned long *bitmap_zalloc(unsigned int nbits, gfp_t flags); >> +unsigned long *bitmap_kvzalloc(unsigned int nbits, gfp_t flags); >> unsigned long *bitmap_alloc_node(unsigned int nbits, gfp_t flags, int >> node); >> unsigned long *bitmap_zalloc_node(unsigned int nbits, gfp_t flags, int >> node); >> void bitmap_free(const unsigned long *bitmap); >> +void bitmap_kvfree(const unsigned long *bitmap); >> >> DEFINE_FREE(bitmap, unsigned long *, if (_T) bitmap_free(_T)) >> >> diff --git a/lib/bitmap.c b/lib/bitmap.c >> index b97692854966..eabbfb85fb45 100644 >> --- a/lib/bitmap.c >> +++ b/lib/bitmap.c >> @@ -727,6 +727,13 @@ unsigned long *bitmap_zalloc(unsigned int nbits, >> gfp_t flags) >> } >> EXPORT_SYMBOL(bitmap_zalloc); >> >> +unsigned long *bitmap_kvzalloc(unsigned int nbits, gfp_t flags) >> +{ >> + return kvmalloc_array(BITS_TO_LONGS(nbits), sizeof(unsigned long), >> + flags | __GFP_ZERO); >> +} >> +EXPORT_SYMBOL(bitmap_zalloc); > EXPORT_SYMBOL(bitmap_kvzalloc)* Actually, does it make more sense to change the behaviour of the current APIs like below instead of above patch? Or is there an expectation that the current bitmap API is supposed to work only on physically contiguous bits? I believe in the kernel if the allocation/free starts with 'k' its physically contiguous and with "kv" its physically contiguous if possible, otherwise virtually contiguous. The bitmap functions dont have either, so we could change the current implementation. I believe it would not impact the current users of the functions as the first attempt is physically contiguous which is how it works currently, and only upon failure it would be virtual and it would increase the use of current bitmap API to greater than MAX_PAGE_ORDER size allocations. Yury Norov and Rasmus Villemoes, any views on this? Thanks diff --git a/include/linux/slab.h b/include/linux/slab.h index 7247e217e21b..ad771dc81afa 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -804,6 +804,7 @@ kvmalloc_array_node_noprof(size_t n, size_t size, gfp_t flags, int node)  #define kvcalloc_node_noprof(_n,_s,_f,_node) kvmalloc_array_node_noprof(_n,_s,(_f)|__GFP_ZERO,_node)  #define kvcalloc_noprof(...) kvcalloc_node_noprof(__VA_ARGS__, NUMA_NO_NODE) +#define kvmalloc_array_node(...) alloc_hooks(kvmalloc_array_node_noprof(__VA_ARGS__))  #define kvmalloc_array(...) alloc_hooks(kvmalloc_array_noprof(__VA_ARGS__))  #define kvcalloc_node(...) alloc_hooks(kvcalloc_node_noprof(__VA_ARGS__))  #define kvcalloc(...) alloc_hooks(kvcalloc_noprof(__VA_ARGS__)) diff --git a/lib/bitmap.c b/lib/bitmap.c index b97692854966..272164dcbef1 100644 --- a/lib/bitmap.c +++ b/lib/bitmap.c @@ -716,7 +716,7 @@ void bitmap_fold(unsigned long *dst, const unsigned long *orig,  unsigned long *bitmap_alloc(unsigned int nbits, gfp_t flags)  { -       return kmalloc_array(BITS_TO_LONGS(nbits), sizeof(unsigned long), +       return kvmalloc_array(BITS_TO_LONGS(nbits), sizeof(unsigned long),                              flags);  }  EXPORT_SYMBOL(bitmap_alloc); @@ -729,7 +729,7 @@ EXPORT_SYMBOL(bitmap_zalloc);  unsigned long *bitmap_alloc_node(unsigned int nbits, gfp_t flags, int node)  { -       return kmalloc_array_node(BITS_TO_LONGS(nbits), sizeof(unsigned long), +       return kvmalloc_array_node(BITS_TO_LONGS(nbits), sizeof(unsigned long),                                   flags, node);  }  EXPORT_SYMBOL(bitmap_alloc_node); @@ -742,7 +742,7 @@ EXPORT_SYMBOL(bitmap_zalloc_node);  void bitmap_free(const unsigned long *bitmap)  { -       kfree(bitmap); +       kvfree(bitmap);  }  EXPORT_SYMBOL(bitmap_free);