From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A43CBC4332F for ; Sun, 16 Oct 2022 09:10:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 747148E0002; Sun, 16 Oct 2022 05:10:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6F57C8E0001; Sun, 16 Oct 2022 05:10:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5BCB38E0002; Sun, 16 Oct 2022 05:10:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 475CC8E0001 for ; Sun, 16 Oct 2022 05:10:24 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 0ECB91C6001 for ; Sun, 16 Oct 2022 09:10:24 +0000 (UTC) X-FDA: 80026241568.03.1FB6A78 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf16.hostedemail.com (Postfix) with ESMTP id A5F0A18001E for ; Sun, 16 Oct 2022 09:10:23 +0000 (UTC) Received: by mail-pj1-f53.google.com with SMTP id a6-20020a17090abe0600b0020d7c0c6650so11757670pjs.0 for ; Sun, 16 Oct 2022 02:10:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=CLyMeXzmotkiR8j5Gpy9vXymc5jaUcrTQlSZ8Ryl3Ug=; b=pLCXPAQfZB5t+OnsYhA9LPn/MAn97Opqa6IgzFWza6gcOAf+tRph5dnuSppcco5wOk Hae/j+WykDaCwqtjbeS/Ow5AZkb8VsdyrHumnecPUuSHNIeVRLlM5M97VtUAzhnsskxm ONsaGDY4nikPal5zxmyyI4odJjG0NfG3gfxe3ocFEi76KEdxl3qdn+/mZHGzUp2cbHgM gLIIiGaMNELbR9lEquePgyFG6ifXhBK5iFX9j1OwnO6Rupn8owJiFnRWk20aZgHTa7KE 0Quj/bOSxT6dLZR09/2V0aEYPKoQMpK1ZpmEibTcAxcC3fmAudZ2TnfrJrG7sW2hyWyw PoTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=CLyMeXzmotkiR8j5Gpy9vXymc5jaUcrTQlSZ8Ryl3Ug=; b=q7HpvNoYSgjWysJr3jQLZcNP0bRLjcW8cbWt58JcfV9pDdymxjkv7BlOPoYZ/ifAAb Mn3/V37fmN//IpXHT5mvkr6OA4Q0rz1azgtRteGWeyT4fde9p1SQegzagoVN4QcXuidr gq9M+6YiKaTeo7khgwkTRRb/B/W4KoEB0GG2KEEdqG5ewR4kFCpkli+QGwb8vnfsTZ92 FXnjnIfBpocC4IZEyxUA87uzpwoWprrVgcmatlq3MnAeWD7MOiJWgH/hADZrChaEJKHs yLXTlgZvIADkCdkTfa3ZXaKcdvl9+UI55w0Jt8JlfK9vnI64DYeo0cmgATTac0n+qbd4 xorw== X-Gm-Message-State: ACrzQf1BiHoErHkMC547kDoHmx30k7ftq3NI6qJrWw4IWauLLfWGZx04 4IM8PhuIX9ccmorBbrem0KI= X-Google-Smtp-Source: AMsMyM6jnh2QfXY198Icbz7Y0ot32GA/pYALs5HjjqoyMyj80qqOyEp5f6swFeo2IIhIjJkckxqAFA== X-Received: by 2002:a17:90a:a401:b0:20a:da56:b073 with SMTP id y1-20020a17090aa40100b0020ada56b073mr7437677pjp.1.1665911422528; Sun, 16 Oct 2022 02:10:22 -0700 (PDT) Received: from hyeyoo ([114.29.91.56]) by smtp.gmail.com with ESMTPSA id p65-20020a625b44000000b0053e4296e1d3sm4709558pfb.198.2022.10.16.02.10.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 16 Oct 2022 02:10:21 -0700 (PDT) Date: Sun, 16 Oct 2022 18:10:15 +0900 From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Vlastimil Babka Cc: Guenter Roeck , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v4 10/17] mm/slab: kmalloc: pass requests larger than order-1 page to page allocator Message-ID: References: <20220817101826.236819-1-42.hyeyoo@gmail.com> <20220817101826.236819-11-42.hyeyoo@gmail.com> <20221014205818.GA1428667@roeck-us.net> <7ad9081b-9082-2cbb-5732-f87366dca801@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7ad9081b-9082-2cbb-5732-f87366dca801@suse.cz> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665911423; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CLyMeXzmotkiR8j5Gpy9vXymc5jaUcrTQlSZ8Ryl3Ug=; b=6NrYcFIGRWBdqFL9Zpp8vIMkgLSdVUfjd7ni0X8MpkKbtkf8c3NncFeOT8X/c5Lu+Kk1Sc qHV5XpU2A2TCGIixh7h+ytZdvWXLBSvC2ZqZ32h3l+sZ7jxgfpm1VJidSYCFg14quLxM/g 54kMuICNsWzQyqK/fVh9a4lGg0OYPCs= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=pLCXPAQf; spf=pass (imf16.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665911423; a=rsa-sha256; cv=none; b=tGaxd1dWQ4/GH/xJ96LAuEgkmbmcj+V7wmFbKgkT45x4bw2582VvnHtU5Ro1jPDnte9W3G zwJLsD5CteCXZP51d5f/my91/pH3F9HyOhg28xVBo2fWcKcH13aZNUNSXTbWHOJQGjoXGV jAXqK8kqpnp59rt72fYRuXOE7cHr3gY= Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=pLCXPAQf; spf=pass (imf16.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: ubf3zaqaj7fpnxh7e3jasoqb675ofz79 X-Rspamd-Queue-Id: A5F0A18001E X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1665911423-62901 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, Oct 15, 2022 at 09:39:08PM +0200, Vlastimil Babka wrote: > On 10/15/22 01:48, Hyeonggon Yoo wrote: > > On Fri, Oct 14, 2022 at 01:58:18PM -0700, Guenter Roeck wrote: > >> Hi, > >> > >> On Wed, Aug 17, 2022 at 07:18:19PM +0900, Hyeonggon Yoo wrote: > >> > There is not much benefit for serving large objects in kmalloc(). > >> > Let's pass large requests to page allocator like SLUB for better > >> > maintenance of common code. > >> > > >> > Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> > >> > Reviewed-by: Vlastimil Babka > >> > --- > >> > >> This patch results in a WARNING backtrace in all mips and sparc64 > >> emulations. > >> > >> ------------[ cut here ]------------ > >> WARNING: CPU: 0 PID: 0 at mm/slab_common.c:729 kmalloc_slab+0xc0/0xdc > >> Modules linked in: > >> CPU: 0 PID: 0 Comm: swapper Not tainted 6.0.0-11990-g9c9155a3509a #1 > >> Stack : ffffffff 801b2a18 80dd0000 00000004 00000000 00000000 81023cd4 00000000 > >> 81040000 811a9930 81040000 8104a628 81101833 00000001 81023c78 00000000 > >> 00000000 00000000 80f5d858 81023b98 00000001 00000023 00000000 ffffffff > >> 00000000 00000064 00000002 81040000 81040000 00000001 80f5d858 000002d9 > >> 00000000 00000000 80000000 80002000 00000000 00000000 00000000 00000000 > >> ... > >> Call Trace: > >> [<8010a2bc>] show_stack+0x38/0x118 > >> [<80cf5f7c>] dump_stack_lvl+0xac/0x104 > >> [<80130d7c>] __warn+0xe0/0x224 > >> [<80cdba5c>] warn_slowpath_fmt+0x64/0xb8 > >> [<8028c058>] kmalloc_slab+0xc0/0xdc > >> > >> irq event stamp: 0 > >> hardirqs last enabled at (0): [<00000000>] 0x0 > >> hardirqs last disabled at (0): [<00000000>] 0x0 > >> softirqs last enabled at (0): [<00000000>] 0x0 > >> softirqs last disabled at (0): [<00000000>] 0x0 > >> ---[ end trace 0000000000000000 ]--- > >> > >> Guenter > > > > Hi. > > > > Thank you so much for this report! > > > > Hmm so SLAB tries to find kmalloc cache for freelist index array using > > kmalloc_slab() directly, and it becomes problematic when size of the > > array is larger than PAGE_SIZE * 2. > > Hmm interesting, did you find out how exactly that can happen in practice, > or what's special about mips and sparc64 here? IIUC if page size is large, number of objects per slab is quite large and so the possiblity of failing to use objfreelist slab is higher, and then it tries to use off slab. > Because normally > calculate_slab_order() will only go up to slab_max_order, which AFAICS can > only go up to SLAB_MAX_ORDER_HI, thus 1, unless there's a boot command line > override. AFAICS with mips default configuration and without setting slab_max_order, It seems SLAB actually does not use too big freelist index array. But it hits the warning because of tricky logic. For example if the condition is true on > if (freelist_cache->size > cachep->size / 2) > continue; or on (before kmalloc is up, in case of kmem_cache) > freelist_cache = kmalloc_slab(freelist_size, 0u); > if (!freelist_cache) > continue; it increases gfporder over and over until 'num' becomes larger than SLAB_MAX_OBJS. (regardless of slab_max_order). I think adding below would be more robust. diff --git a/mm/slab.c b/mm/slab.c index d1f6e2c64c2e..1321aca1887c 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1679,7 +1679,7 @@ static size_t calculate_slab_order(struct kmem_cache *cachep, } else { freelist_cache = kmalloc_slab(freelist_size, 0u); if (!freelist_cache) - continue; + break; freelist_cache_size = freelist_cache->size; /* @@ -1692,7 +1692,7 @@ static size_t calculate_slab_order(struct kmem_cache *cachep, /* check if off slab has enough benefit */ if (freelist_cache_size > cachep->size / 2) - continue; + break; } /* Found something acceptable - save it away */ > And if we have two pages for objects, surely even with small objects they > can't be smaller than freelist_idx_t, so if the number of objects fits into > two pages (order 1), then the freelist array should also fit in two pages? That's right but on certain condition it seem to go larger than slab_max_order. (from code inspection) > > Thanks, > Vlastimil > > > Will send a fix soon. > > -- Thanks, Hyeonggon