From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25CA5C433F5 for ; Tue, 26 Apr 2022 06:30:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5F0786B0075; Tue, 26 Apr 2022 02:30:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 59F426B0078; Tue, 26 Apr 2022 02:30:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4677C6B007B; Tue, 26 Apr 2022 02:30:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id 353EF6B0075 for ; Tue, 26 Apr 2022 02:30:21 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id D8E8C607F7 for ; Tue, 26 Apr 2022 06:30:20 +0000 (UTC) X-FDA: 79398055800.17.BF74BDE Received: from mail-yb1-f172.google.com (mail-yb1-f172.google.com [209.85.219.172]) by imf12.hostedemail.com (Postfix) with ESMTP id 6DB844003E for ; Tue, 26 Apr 2022 06:30:12 +0000 (UTC) Received: by mail-yb1-f172.google.com with SMTP id s30so3169651ybi.8 for ; Mon, 25 Apr 2022 23:30:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Xk1fRrXwMY49fkEklzghzXkAbwf3cL6niuJY5FykJaY=; b=XKpip720WjbGzHfP5YW9gSB8hFdf4hNkRtU+Adgc2GptypWuimrKfm4d/jTk1AJN2S wvD9Q6Sle0BhWy7kUbqvV8S9J14hP2H7Xf0KqXsQ5ogF+LBDqfcEYPCdNRdqYphLhYst 22mhzSuTfN0lo3t2tx9A3sGF+U1uEyp9clr9Xx3jpUbJiTz+16DdaivMegtOh6JHgpou X1AELsmWoTmVvYVEFo3ay4KIKBVJs94KXqylK86UsUPhaTfWV1lgeshVI4fyRf3mURus SOGA0Snj1/dFaXKuZs3n2+NlRmHfLxPyes/tTiY6eQBGZM5hOAjIUScfKhZrHCgCDqiq 8mHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Xk1fRrXwMY49fkEklzghzXkAbwf3cL6niuJY5FykJaY=; b=OWB+H8CbcPqNfApniP1cFbqWmbDhbVCd2LaNgYfuVEtchfFXYTr/scSZ4SBV9dzGGX mW+3zezBf8MaSipZA2Qahhp3wpxcBOhNPwz9w6Tsd0WZY+hNXrXJo4U8fSetmCW9dGOg dmGT3sAROigGHu9dxpN5V4u5e8PALFvas7rm9IcyORdwXXLgFIn12g+WdHFQ/tUnqp73 iULLXU3BhL2RsSmhAPjKocBvt5sSJ4qJKU35Fn34c0TEt27MoubWJFO6+58xg/LBS4gb 9iGqX7blFVv3FZAbdi6v9B/upwo7u5rZcxsWvmuaaoN+0hB4EiiWICJ50xmJ8947WKfU R0kw== X-Gm-Message-State: AOAM533Qxi7P7TlhN80fxocJLp1LURuHuoMs/4BBUsjd/hYTC5j5ZyRB uFC7qE4kKQSPLqw1kJxwnNopZ07EU7jdzNKAPcvOkQ== X-Google-Smtp-Source: ABdhPJxb4B+LcT0jivCZF2yrHw8bSTLtmXMFcK6VBuul7ICiYr9UHhwQS8Dhk2Q4vrNahhA+41kQRQ5ZuNsamNIR9Ec= X-Received: by 2002:a05:6902:1242:b0:644:c30c:cfcc with SMTP id t2-20020a056902124200b00644c30ccfccmr18900379ybu.509.1650954614961; Mon, 25 Apr 2022 23:30:14 -0700 (PDT) MIME-Version: 1.0 References: <20220420095906.27349-1-mgorman@techsingularity.net> In-Reply-To: From: Suren Baghdasaryan Date: Mon, 25 Apr 2022 23:30:03 -0700 Message-ID: Subject: Re: [RFC PATCH 0/6] Drain remote per-cpu directly To: Mel Gorman Cc: Nicolas Saenz Julienne , Marcelo Tosatti , Vlastimil Babka , Michal Hocko , LKML , Linux-MM Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 6DB844003E X-Stat-Signature: 394iqr7wnjhr79c9fs8jit86dtgoajwm Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=XKpip720; spf=pass (imf12.hostedemail.com: domain of surenb@google.com designates 209.85.219.172 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com X-HE-Tag: 1650954612-477617 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Apr 25, 2022 at 7:49 PM Suren Baghdasaryan wrote: > > On Wed, Apr 20, 2022 at 2:59 AM Mel Gorman wrote: > > > > This series has the same intent as Nicolas' series "mm/page_alloc: Remote > > per-cpu lists drain support" -- avoid interference of a high priority > > task due to a workqueue item draining per-cpu page lists. While many > > workloads can tolerate a brief interruption, it may be cause a real-time > > task runnning on a NOHZ_FULL CPU to miss a deadline and at minimum, > > the draining in non-deterministic. > > > > Currently an IRQ-safe local_lock protects the page allocator per-cpu lists. > > The local_lock on its own prevents migration and the IRQ disabling protects > > from corruption due to an interrupt arriving while a page allocation is > > in progress. The locking is inherently unsafe for remote access unless > > the CPU is hot-removed. > > > > This series adjusts the locking. A spin-lock is added to struct > > per_cpu_pages to protect the list contents while local_lock_irq continues > > to prevent migration and IRQ reentry. This allows a remote CPU to safely > > drain a remote per-cpu list. > > > > This series is a partial series. Follow-on work would allow the > > local_irq_save to be converted to a local_irq to avoid IRQs being > > disabled/enabled in most cases. However, there are enough corner cases > > that it deserves a series on its own separated by one kernel release and > > the priority right now is to avoid interference of high priority tasks. > > > > Patch 1 is a cosmetic patch to clarify when page->lru is storing buddy pages > > and when it is storing per-cpu pages. > > > > Patch 2 shrinks per_cpu_pages to make room for a spin lock. Strictly speaking > > this is not necessary but it avoids per_cpu_pages consuming another > > cache line. > > > > Patch 3 is a preparation patch to avoid code duplication. > > > > Patch 4 is a simple micro-optimisation that improves code flow necessary for > > a later patch to avoid code duplication. > > > > Patch 5 uses a spin_lock to protect the per_cpu_pages contents while still > > relying on local_lock to prevent migration, stabilise the pcp > > lookup and prevent IRQ reentrancy. > > > > Patch 6 remote drains per-cpu pages directly instead of using a workqueue. > > This quite possibly solves the issue I was trying to fix in > https://lore.kernel.org/all/20220225012819.1807147-1-surenb@google.com. > I will give it a try and see how it looks. My test shows sizable improvement for the worst case drain_all_pages duration. Before the change I caught cases when a drain_local_pages_wq in the workqueue was delayed by 100+ms (not even counting drain_local_pages_wq execution time itself). With this patchset the worst time I was able to record for drain_all_pages duration was 17ms. > Thanks! > > > > > include/linux/mm_types.h | 5 + > > include/linux/mmzone.h | 12 +- > > mm/page_alloc.c | 333 ++++++++++++++++++++++++--------------- > > 3 files changed, 222 insertions(+), 128 deletions(-) > > > > -- > > 2.34.1 > > > >