From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 834B7ECE58C for ; Tue, 8 Oct 2019 23:09:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 36F3520873 for ; Tue, 8 Oct 2019 23:09:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 36F3520873 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8A2EF8E0005; Tue, 8 Oct 2019 19:09:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8534F8E0003; Tue, 8 Oct 2019 19:09:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 768F68E0005; Tue, 8 Oct 2019 19:09:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0128.hostedemail.com [216.40.44.128]) by kanga.kvack.org (Postfix) with ESMTP id 558698E0003 for ; Tue, 8 Oct 2019 19:09:45 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id D85AA5008 for ; Tue, 8 Oct 2019 23:09:44 +0000 (UTC) X-FDA: 76022161488.25.leaf71_1ac7a9b99bd16 X-HE-Tag: leaf71_1ac7a9b99bd16 X-Filterd-Recvd-Size: 4346 Received: from out30-57.freemail.mail.aliyun.com (out30-57.freemail.mail.aliyun.com [115.124.30.57]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Tue, 8 Oct 2019 23:09:43 +0000 (UTC) X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01451;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=12;SR=0;TI=SMTPD_---0TeUt30J_1570576174; Received: from US-143344MP.local(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0TeUt30J_1570576174) by smtp.aliyun-inc.com(127.0.0.1); Wed, 09 Oct 2019 07:09:40 +0800 Subject: Re: [PATCH] mm: thp: move deferred split queue to memcg's nodeinfo To: "Kirill A. Shutemov" , Michal Hocko Cc: Vlastimil Babka , kirill.shutemov@linux.intel.com, ktkhai@virtuozzo.com, hannes@cmpxchg.org, hughd@google.com, shakeelb@google.com, rientjes@google.com, akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <1569968203-64647-1-git-send-email-yang.shi@linux.alibaba.com> <20191002084304.GI15624@dhcp22.suse.cz> <30421920-4fdb-767a-6ef2-60187932c414@suse.cz> <20191007143030.GN2381@dhcp22.suse.cz> <20191008144437.fr374cxtpnrnnjsv@box> From: Yang Shi Message-ID: Date: Tue, 8 Oct 2019 16:09:29 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <20191008144437.fr374cxtpnrnnjsv@box> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 10/8/19 7:44 AM, Kirill A. Shutemov wrote: > On Mon, Oct 07, 2019 at 04:30:30PM +0200, Michal Hocko wrote: >> On Mon 07-10-19 16:19:59, Vlastimil Babka wrote: >>> On 10/2/19 10:43 AM, Michal Hocko wrote: >>>> On Wed 02-10-19 06:16:43, Yang Shi wrote: >>>>> The commit 87eaceb3faa59b9b4d940ec9554ce251325d83fe ("mm: thp: make >>>>> deferred split shrinker memcg aware") makes deferred split queue per >>>>> memcg to resolve memcg pre-mature OOM problem. But, all nodes end up >>>>> sharing the same queue instead of one queue per-node before the commit. >>>>> It is not a big deal for memcg limit reclaim, but it may cause global >>>>> kswapd shrink THPs from a different node. >>>>> >>>>> And, 0-day testing reported -19.6% regression of stress-ng's madvise >>>>> test [1]. I didn't see that much regression on my test box (24 threads, >>>>> 48GB memory, 2 nodes), with the same test (stress-ng --timeout 1 >>>>> --metrics-brief --sequential 72 --class vm --exclude spawn,exec), I saw >>>>> average -3% (run the same test 10 times then calculate the average since >>>>> the test itself may have most 15% variation according to my test) >>>>> regression sometimes (not every time, sometimes I didn't see regression >>>>> at all). >>>>> >>>>> This might be caused by deferred split queue lock contention. With some >>>>> configuration (i.e. just one root memcg) the lock contention my be worse >>>>> than before (given 2 nodes, two locks are reduced to one lock). >>>>> >>>>> So, moving deferred split queue to memcg's nodeinfo to make it NUMA >>>>> aware again. >>>>> >>>>> With this change stress-ng's madvise test shows average 4% improvement >>>>> sometimes and I didn't see degradation anymore. >>>> My concern about this getting more and more complex >>>> (http://lkml.kernel.org/r/20191002084014.GH15624@dhcp22.suse.cz) holds >>>> here even more. Can we step back and reconsider the whole thing please? >>> What about freeing immediately after split via workqueue and also have a >>> synchronous version called before going oom? Maybe there would be also >>> other things that would benefit from this scheme instead of traditional >>> reclaim and shrinkers? >> That is exactly what we have discussed some time ago. > Yes, I've posted the patch: > > http://lkml.kernel.org/r/20190827125911.boya23eowxhqmopa@box > > But I still not sure that the approach is right. I expect it to trigger > performance regressions. For system with pleanty of free memory, we will > just pay split cost for nothing in many cases. This is exactly what I'm concerned about as well. >