Re: [PATCH] mm/swap.c: workaround for_each_cpu() bug on UP kernel.

[Michal Hocko <mhocko@kernel.org>]

On Tue 12-02-19 13:06:20, Andrew Morton wrote:
> On Tue, 12 Feb 2019 12:29:54 +0100 Michal Hocko <mhocko@kernel.org> wrote:
> 
> > On Tue 12-02-19 12:21:17, Michal Hocko wrote:
> > > On Tue 12-02-19 19:25:46, Tetsuo Handa wrote:
> > > > On 2019/02/12 19:11, Michal Hocko wrote:
> > > > > This patch is ugly as hell! I do agree that for_each_cpu not working on
> > > > > CONFIG_SMP=n sucks but why do we even care about lru_add_drain_all when
> > > > > there is a single cpu? Why don't we simply do
> > > > > 
> > > > > diff --git a/mm/swap.c b/mm/swap.c
> > > > > index aa483719922e..952f24b09070 100644
> > > > > --- a/mm/swap.c
> > > > > +++ b/mm/swap.c
> > > > > @@ -660,6 +660,7 @@ static void lru_add_drain_per_cpu(struct work_struct *dummy)
> > > > >  
> > > > >  static DEFINE_PER_CPU(struct work_struct, lru_add_drain_work);
> > > > >  
> > > > > +#ifdef CONFIG_SMP
> > > > >  /*
> > > > >   * Doesn't need any cpu hotplug locking because we do rely on per-cpu
> > > > >   * kworkers being shut down before our page_alloc_cpu_dead callback is
> > > > > @@ -702,6 +703,10 @@ void lru_add_drain_all(void)
> > > > >  
> > > > >  	mutex_unlock(&lock);
> > > > >  }
> > > > > +#else
> > > > > +#define lru_add_drain_all() lru_add_drain()
> > > > > +
> > > > > +#endif
> > > > 
> > > > If there is no need to evaluate the "if" conditions, I'm fine with this shortcut.
> > > 
> > > lru_add_drain does drain only pagevecs which have pages and so we do not
> > > really have to duplicate the check. There is also no need to defer the
> > > execution to the workqueue for a local cpu. So we are left with only the
> > > lock to prevent parallel execution but the preemption disabling acts the
> > > same purpose on UP so the approach should be equivalent from the
> > > correctness point of view.
> > 
> > The patch with the full changelog follows:
> > 
> > 
> > >From db104f132bd6e1c02ecbe65e62c12caa7e4e2e2a Mon Sep 17 00:00:00 2001
> > From: Michal Hocko <mhocko@suse.com>
> > Date: Tue, 12 Feb 2019 12:25:28 +0100
> > Subject: [PATCH] mm: handle lru_add_drain_all for UP properly
> > 
> > Since for_each_cpu(cpu, mask) added by commit 2d3854a37e8b767a ("cpumask:
> > introduce new API, without changing anything") did not evaluate the mask
> > argument if NR_CPUS == 1 due to CONFIG_SMP=n, lru_add_drain_all() is
> > hitting WARN_ON() at __flush_work() added by commit 4d43d395fed12463
> > ("workqueue: Try to catch flush_work() without INIT_WORK().")
> > by unconditionally calling flush_work() [1].
> > 
> > Workaround this issue by using CONFIG_SMP=n specific lru_add_drain_all
> > implementation. There is no real need to defer the implementation to the
> > workqueue as the draining is going to happen on the local cpu. So alias
> > lru_add_drain_all to lru_add_drain which does all the necessary work.
> > 
> > [1] https://lkml.kernel.org/r/18a30387-6aa5-6123-e67c-57579ecc3f38@roeck-us.net
> >
> > ...
> >
> > --- a/mm/swap.c
> > +++ b/mm/swap.c
> > @@ -660,6 +660,7 @@ static void lru_add_drain_per_cpu(struct work_struct *dummy)
> >  
> >  static DEFINE_PER_CPU(struct work_struct, lru_add_drain_work);
> >  
> > +#ifdef CONFIG_SMP
> >  /*
> >   * Doesn't need any cpu hotplug locking because we do rely on per-cpu
> >   * kworkers being shut down before our page_alloc_cpu_dead callback is
> > @@ -702,6 +703,10 @@ void lru_add_drain_all(void)
> >  
> >  	mutex_unlock(&lock);
> >  }
> > +#else
> > +#define lru_add_drain_all() lru_add_drain()
> > +
> > +#endif
> >  
> >  /**
> >   * release_pages - batched put_page()
> 
> How can this even link?  Lots of compilation units call
> lru_add_drain_all() but the implementation just got removed.

Yeah, my bad. Should have compile tested...


From a13b4420f064abc9fe86dbb33f2fe3b508c9fac7 Mon Sep 17 00:00:00 2001
From: Michal Hocko <mhocko@suse.com>
Date: Tue, 12 Feb 2019 12:25:28 +0100
Subject: [PATCH] mm: handle lru_add_drain_all for UP properly

Since for_each_cpu(cpu, mask) added by commit 2d3854a37e8b767a ("cpumask:
introduce new API, without changing anything") did not evaluate the mask
argument if NR_CPUS == 1 due to CONFIG_SMP=n, lru_add_drain_all() is
hitting WARN_ON() at __flush_work() added by commit 4d43d395fed12463
("workqueue: Try to catch flush_work() without INIT_WORK().")
by unconditionally calling flush_work() [1].

Workaround this issue by using CONFIG_SMP=n specific lru_add_drain_all
implementation. There is no real need to defer the implementation to the
workqueue as the draining is going to happen on the local cpu. So alias
lru_add_drain_all to lru_add_drain which does all the necessary work.

[1] https://lkml.kernel.org/r/18a30387-6aa5-6123-e67c-57579ecc3f38@roeck-us.net
Reported-by: Guenter Roeck <linux@roeck-us.net>
Debugged-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Signed-off-by: Michal Hocko <mhocko@suse.com>
---
 mm/swap.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/mm/swap.c b/mm/swap.c
index 4929bc1be60e..12711434a1b9 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -660,6 +660,7 @@ static void lru_add_drain_per_cpu(struct work_struct *dummy)
 
 static DEFINE_PER_CPU(struct work_struct, lru_add_drain_work);
 
+#ifdef CONFIG_SMP
 /*
  * Doesn't need any cpu hotplug locking because we do rely on per-cpu
  * kworkers being shut down before our page_alloc_cpu_dead callback is
@@ -702,6 +703,12 @@ void lru_add_drain_all(void)
 
 	mutex_unlock(&lock);
 }
+#else
+void lru_add_drain_all(void)
+{
+	lru_add_drain();
+}
+#endif
 
 /**
  * release_pages - batched put_page()
-- 
2.20.1

-- 
Michal Hocko
SUSE Labs