From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FAA8C2D0F3 for ; Wed, 1 Apr 2020 18:16:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EE82F2063A for ; Wed, 1 Apr 2020 18:16:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="K+d7Rorm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE82F2063A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7F6598E0007; Wed, 1 Apr 2020 14:16:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 77F698E0006; Wed, 1 Apr 2020 14:16:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 66EFA8E0007; Wed, 1 Apr 2020 14:16:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0019.hostedemail.com [216.40.44.19]) by kanga.kvack.org (Postfix) with ESMTP id 506588E0006 for ; Wed, 1 Apr 2020 14:16:39 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 161235008 for ; Wed, 1 Apr 2020 18:16:39 +0000 (UTC) X-FDA: 76660091718.13.drain72_915464e4d310c X-HE-Tag: drain72_915464e4d310c X-Filterd-Recvd-Size: 6001 Received: from mail-lf1-f67.google.com (mail-lf1-f67.google.com [209.85.167.67]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Wed, 1 Apr 2020 18:16:38 +0000 (UTC) Received: by mail-lf1-f67.google.com with SMTP id n20so473794lfl.10 for ; Wed, 01 Apr 2020 11:16:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:date:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=ohKhIvr0zaGkZNZ8iCKmPfOv0u8WbM7PXyOQtVWyFt8=; b=K+d7Rorms1Cni3yLww4dnDdrJOs14OO60FrW3zNhaJMt4znxIyzrsMPueYF6XwN29x uw01a0igiJEkp4Pqos6HPFTtFqeJc7Mt/ioE/Hx+si9FW41oZ4MsMclcGXAMd+he1DyU 4Z6N4MWZoI64SqGI+BgU3YQ+XJxOc6wbqJQJ7X2bRJFJD4aMWPugN6dB1GfFVMWXV59V FDGHyJVKb4Nhe8nvWOOZfNb+3wylIfpCjNTWAMnuwuCnxBQPuyij8XzEqEOWaqnbQqOD X1azb/qNUAmS8rQAjCdXEYpBtO0M2Wtn1Lvpz0SnCUWA+8gZ85Xg/gdI0vu2RfFOti/s CyUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=ohKhIvr0zaGkZNZ8iCKmPfOv0u8WbM7PXyOQtVWyFt8=; b=IGwOoSDp4XHgiDvBL8NW4iD5w3spx+lXL9fqropdiLPOCA6p0WEHqQIlsBNxJEUpgW 9y7RBHd5lP4SCpMa60M1y0ny6TJ+MACyxh3t1tdlotgOm2AlQpI0vYHGsCTvq6EyTqry RVsh9qA1dcvFUoJ17BHcmVzQkrUYNO5Tls1vcLUJGpMPwV6cBw8JuLq1wTowEwcxwexc km4jYvmcZcMGWDPcdTyfTG9gx5eO2VGYjXqENHG7UOToNdIUltsSNxpwSfpWeAFpX2FK sl392XekXZi+fm+cCw39nuxNRPbr4Wv/OCu1PUZ83QhWfOafmpvFqi0h5GUgHIsg6Jxu 8u5A== X-Gm-Message-State: AGi0PuZDhAVixXLnenS/LHhpsUmFSLBlrVVJzEtTl7ne41yqS5+HKT57 3vz0cP6wcaKe9u+BJybSy28= X-Google-Smtp-Source: APiQypLUT/Urx2WPTfjLpIiARF0DyHJ3Fb08TbT+Qv/xlBHs3Zk7A9+cT+CQ8O/7hciGtwRKiKAabA== X-Received: by 2002:a19:4843:: with SMTP id v64mr15013886lfa.171.1585764996901; Wed, 01 Apr 2020 11:16:36 -0700 (PDT) Received: from pc636 (h5ef52e31.seluork.dyn.perspektivbredband.net. [94.245.46.49]) by smtp.gmail.com with ESMTPSA id b28sm1854143ljp.90.2020.04.01.11.16.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Apr 2020 11:16:35 -0700 (PDT) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Wed, 1 Apr 2020 20:16:01 +0200 To: "Paul E. McKenney" Cc: Uladzislau Rezki , Joel Fernandes , linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, willy@infradead.org, peterz@infradead.org, neilb@suse.com, vbabka@suse.cz, mgorman@suse.de, Andrew Morton , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Steven Rostedt Subject: Re: [PATCH RFC] rcu/tree: Use GFP_MEMALLOC for alloc memory to free memory pattern Message-ID: <20200401181601.GA4042@pc636> References: <20200331131628.153118-1-joel@joelfernandes.org> <20200331140433.GA26498@pc636> <20200331150911.GC236678@google.com> <20200331160119.GA27614@pc636> <20200331183000.GD236678@google.com> <20200401122550.GA32593@pc636> <20200401134745.GV19865@paulmck-ThinkPad-P72> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200401134745.GV19865@paulmck-ThinkPad-P72> User-Agent: Mutt/1.10.1 (2018-07-13) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > > > > > > Right. Per discussion with Paul, we discussed that it is better if we > > > pre-allocate N number of array blocks per-CPU and use it for the cache. > > > Default for N being 1 and tunable with a boot parameter. I agree with this. > > > > > As discussed before, we can make use of memory pool API for such > > purpose. But i am not sure if it should be one pool per CPU or > > one pool per NR_CPUS, that would contain NR_CPUS * N pre-allocated > > blocks. > > There are advantages and disadvantages either way. The advantage of the > per-CPU pool is that you don't have to worry about something like lock > contention causing even more pain during an OOM event. One potential > problem wtih the per-CPU pool can happen when callbacks are offloaded, > in which case the CPUs needing the memory might never be getting it, > because in the offloaded case (RCU_NOCB_CPU=y) the CPU posting callbacks > might never be invoking them. > > But from what I know now, systems built with CONFIG_RCU_NOCB_CPU=y > either don't have heavy callback loads (HPC systems) or are carefully > configured (real-time systems). Plus large systems would probably end > up needing something pretty close to a slab allocator to keep from dying > from lock contention, and it is hard to justify that level of complexity > at this point. > > Or is there some way to mark a specific slab allocator instance as being > able to keep some amount of memory no matter what the OOM conditions are? > If not, the current per-CPU pre-allocated cache is a better choice in the > near term. > As for mempool API: mempool_alloc() just tries to make regular allocation taking into account passed gfp_t bitmask. If it fails due to memory pressure, it uses reserved preallocated pool that consists of number of desirable elements(preallocated when a pool is created). mempoll_free() returns an element to to pool, if it detects that current reserved elements are lower then minimum allowed elements, it will add an element to reserved pool, i.e. refill it. Otherwise just call kfree() or whatever we define as "element-freeing function." > > If not, the current per-CPU pre-allocated cache is a better choice in the > near term. > OK. I see your point. Thank you for your comments and view :) -- Vlad Rezki