From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3806C433E0 for ; Wed, 3 Jun 2020 08:21:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 690A120678 for ; Wed, 3 Jun 2020 08:21:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 690A120678 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linutronix.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B0C5780007; Wed, 3 Jun 2020 04:21:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ABCC08E0006; Wed, 3 Jun 2020 04:21:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D2AE80007; Wed, 3 Jun 2020 04:21:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0137.hostedemail.com [216.40.44.137]) by kanga.kvack.org (Postfix) with ESMTP id 833888E0006 for ; Wed, 3 Jun 2020 04:21:52 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 3A808181AC9BF for ; Wed, 3 Jun 2020 08:21:52 +0000 (UTC) X-FDA: 76887207264.11.beef25_4a722518e4d47 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 15B7D180F8B82 for ; Wed, 3 Jun 2020 08:21:52 +0000 (UTC) X-HE-Tag: beef25_4a722518e4d47 X-Filterd-Recvd-Size: 3381 Received: from Galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Wed, 3 Jun 2020 08:21:51 +0000 (UTC) Received: from bigeasy by Galois.linutronix.de with local (Exim 4.80) (envelope-from ) id 1jgOeg-0004fB-Dr; Wed, 03 Jun 2020 10:21:46 +0200 Date: Wed, 3 Jun 2020 10:21:45 +0200 From: Sebastian Andrzej Siewior To: Hillf Danton Cc: linux-mm , LKML , Konstantin Khlebnikov , "Ahmed S. Darwish" , Peter Zijlstra , Thomas Gleixner Subject: Re: [RFC PATCH] mm: swap: remove lru drain waiters Message-ID: <20200603082145.jfgmzozx42rjxegx@linutronix.de> References: <20200601143734.9572-1-hdanton@sina.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20200601143734.9572-1-hdanton@sina.com> X-Rspamd-Queue-Id: 15B7D180F8B82 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2020-06-01 22:37:34 [+0800], Hillf Danton wrote: > > After updating the lru drain sequence, new comers avoid waiting for > the current drainer, because he is flushing works on each online CPU, > by trying to lock the mutex; the drainer OTOH tries to do works for > those who fail to acquire the lock by checking the lru drain sequence > after releasing lock. > > See eef1a429f234 ("mm/swap.c: piggyback lru_add_drain_all() calls") > for reasons why we can skip waiting for the lock. > > The memory barriers around the sequence and the lock come together > to remove waiters without their drain works bandoned. > > Cc: Sebastian Andrzej Siewior > Cc: Konstantin Khlebnikov > Signed-off-by: Hillf Danton > --- > This is inspired by one of the works from Sebastian. Not me, it was Ahmed. > --- a/mm/swap.c > +++ b/mm/swap.c > @@ -714,10 +714,11 @@ static void lru_add_drain_per_cpu(struct > */ > void lru_add_drain_all(void) > { > - static seqcount_t seqcount = SEQCNT_ZERO(seqcount); > + static unsigned int lru_drain_seq; > static DEFINE_MUTEX(lock); > static struct cpumask has_work; > - int cpu, seq; > + int cpu; > + unsigned int seq; > > /* > * Make sure nobody triggers this path before mm_percpu_wq is fully > @@ -726,18 +727,16 @@ void lru_add_drain_all(void) > if (WARN_ON(!mm_percpu_wq)) > return; > > - seq = raw_read_seqcount_latch(&seqcount); > + lru_drain_seq++; > + smp_mb(); > > - mutex_lock(&lock); > +more_work: > > - /* > - * Piggyback on drain started and finished while we waited for lock: > - * all pages pended at the time of our enter were drained from vectors. > - */ > - if (__read_seqcount_retry(&seqcount, seq)) > - goto done; > + if (!mutex_trylock(&lock)) > + return; > > - raw_write_seqcount_latch(&seqcount); > + smp_mb(); > + seq = lru_drain_seq; > > cpumask_clear(&has_work); > > @@ -759,8 +758,11 @@ void lru_add_drain_all(void) > for_each_cpu(cpu, &has_work) > flush_work(&per_cpu(lru_add_drain_work, cpu)); > > -done: > mutex_unlock(&lock); > + > + smp_mb(); > + if (seq != lru_drain_seq) > + goto more_work; > } > #else > void lru_add_drain_all(void) > -- Sebastian