From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 718F5C433DF for ; Thu, 11 Jun 2020 06:07:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3B7272075F for ; Thu, 11 Jun 2020 06:07:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B7272075F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 97B0A8D0079; Thu, 11 Jun 2020 02:07:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 92A868D004C; Thu, 11 Jun 2020 02:07:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 840378D0079; Thu, 11 Jun 2020 02:07:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0118.hostedemail.com [216.40.44.118]) by kanga.kvack.org (Postfix) with ESMTP id 6D3598D004C for ; Thu, 11 Jun 2020 02:07:00 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1DCDC180AEF50 for ; Thu, 11 Jun 2020 06:07:00 +0000 (UTC) X-FDA: 76915897800.27.bread17_511148d26dd1 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id EB65513177 for ; Thu, 11 Jun 2020 06:06:59 +0000 (UTC) X-HE-Tag: bread17_511148d26dd1 X-Filterd-Recvd-Size: 6592 Received: from out30-42.freemail.mail.aliyun.com (out30-42.freemail.mail.aliyun.com [115.124.30.42]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Thu, 11 Jun 2020 06:06:58 +0000 (UTC) X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R191e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01419;MF=alex.shi@linux.alibaba.com;NM=1;PH=DS;RN=16;SR=0;TI=SMTPD_---0U.Eg.NV_1591855608; Received: from IT-FVFX43SYHV2H.local(mailfrom:alex.shi@linux.alibaba.com fp:SMTPD_---0U.Eg.NV_1591855608) by smtp.aliyun-inc.com(127.0.0.1); Thu, 11 Jun 2020 14:06:48 +0800 Subject: Re: [PATCH v11 00/16] per memcg lru lock To: Hugh Dickins Cc: akpm@linux-foundation.org, mgorman@techsingularity.net, tj@kernel.org, khlebnikov@yandex-team.ru, daniel.m.jordan@oracle.com, yang.shi@linux.alibaba.com, willy@infradead.org, hannes@cmpxchg.org, lkp@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, shakeelb@google.com, iamjoonsoo.kim@lge.com, richard.weiyang@gmail.com References: <1590663658-184131-1-git-send-email-alex.shi@linux.alibaba.com> <31943f08-a8e8-be38-24fb-ab9d25fd96ff@linux.alibaba.com> From: Alex Shi Message-ID: <730c595b-f4bf-b16a-562e-de25b9b7eb97@linux.alibaba.com> Date: Thu, 11 Jun 2020 14:06:48 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 X-Rspamd-Queue-Id: EB65513177 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000355, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: =E5=9C=A8 2020/6/10 =E4=B8=8A=E5=8D=8811:22, Hugh Dickins =E5=86=99=E9=81= =93: > On Mon, 8 Jun 2020, Alex Shi wrote: >> =E5=9C=A8 2020/6/8 =E4=B8=8B=E5=8D=8812:15, Hugh Dickins =E5=86=99=E9=81= =93: >>>> 24 files changed, 487 insertions(+), 312 deletions(-) >>> Hi Alex, >>> >>> I didn't get to try v10 at all, waited until Johannes's preparatory >>> memcg swap cleanup was in mmotm; but I have spent a while thrashing >>> this v11, and can happily report that it is much better than v9 etc: >>> I believe this memcg lru_lock work will soon be ready for v5.9. >>> >>> I've not yet found any flaw at the swapping end, but fixes are needed >>> for isolate_migratepages_block() and mem_cgroup_move_account(): I've >>> got a series of 4 fix patches to send you (I guess two to fold into >>> existing patches of yours, and two to keep as separate from me). >>> >>> I haven't yet written the patch descriptions, will return to that >>> tomorrow. I expect you will be preparing a v12 rebased on v5.8-rc1 >>> or v5.8-rc2, and will be able to include these fixes in that. >> >> I am very glad to get your help on this feature!=20 >> >> and looking forward for your fixes tomorrow. :) >> >> Thanks a lot! >> Alex >=20 > Sorry, Alex, the news is not so good today. >=20 > You'll have noticed I sent nothing yesterday. That's because I got > stuck on my second patch: could not quite convince myself that it > was safe. Hi Hugh, Thanks a lot for your help and effort! I very appreciate for this. >=20 > I keep hinting at these patches, and I can't complete their writeups > until I'm convinced; but to give you a better idea of what they do: >=20 > 1. Fixes isolate_fail and isolate_abort in isolate_migratepages_block()= . I guess I know this after mm-compaction-avoid-vm_bug_onpageslab-in-page_m= apcount.patch was removed. > 2. Fixes unsafe use of trylock_page() in __isolate_lru_page_prepare(). > 3. Reverts 07/16 inversion of lock ordering in split_huge_page_to_list(= ). > 4. Adds lruvec lock protection in mem_cgroup_move_account(). Sorry for can't follow you for above issues. Anyway, I will send out new = patchset with the first issue fixed. and then let's discussion base on it. >=20 > In the second, I was using rcu_read_lock() instead of trylock_page() > (like in my own patchset), but could not quite be sure of the case when > PageSwapCache gets set at the wrong moment. Gave up for the night, and > in the morning abandoned that, instead just shifting the call to > __isolate_lru_page_prepare() after the get_page_unless_zero(), > where that trylock_page() becomes safe (no danger of stomping on page > flags while page is being freed or newly allocated to another owner). Sorry, I don't know the problem of trylock_page here? Could you like to describe it as a race? >=20 > I thought that a very safe change, but best to do some test runs with > it in before finalizing. And was then unpleasantly surprised to hit a > VM_BUG_ON_PAGE(lruvec_memcg(lruvec) !=3D page->mem_cgroup) from > lock_page_lruvec_irqsave < relock_page_lruvec < pagevec_lru_move_fn < > pagevec_move_tail < lru_add_drain_cpu after 6 hours on one machine. > Then similar but < rotate_reclaimable_page after 8 hours on another. >=20 > Only seen once before: that's what drove me to add patch 4 (with 3 to > revert the locking before it): somehow, when adding the lruvec locking > there, I just took it for granted that your patchset would have the > appropriate locking (or TestClearPageLRU magic) at the other end. >=20 > But apparently not. And I'm beginning to think that TestClearPageLRU > was just to distract the audience from the lack of proper locking. >=20 > I have certainly not concluded that yet, but I'm having to think about > an area of the code which I'd imagined you had under control (and I'm > puzzled why my testing has found it so very hard to hit). If we're > lucky, I'll find that pagevec_move_tail is a special case, and > nothing much else needs changing; but I doubt that will be so. >=20 > There's one other unexplained and unfixed bug I've seen several times > while exercising mem_cgroup_move_account(): refcount_warn_saturate() > from where __mem_cgroup_clear_mc() calls mem_cgroup_id_get_many(). > I'll be glad if that goes away when the lruvec locking is fixed, > but don't understand the connection. And it's quite possible that > this refcounting bug has nothing to do with your changes: I have > not succeeded in reproducing it on 5.7 nor on 5.7-rc7-mm1, > but I didn't really try long enough to be sure. >=20 > (I should also warn, that I'm surprised by the amount of change > 11/16 makes to mm/mlock.c: I've not been exercising mlock at all.) yes, that is a bit complex. I have tried the mlock cases in selftest with your swap&build case. They are all fine with 300 times run. >=20 > Taking a break for the evening, > Hugh >=20