From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA07CC433EF for ; Wed, 25 May 2022 17:59:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42DC98D0005; Wed, 25 May 2022 13:59:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E1B48D0002; Wed, 25 May 2022 13:59:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 280948D0005; Wed, 25 May 2022 13:59:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1ACEE8D0002 for ; Wed, 25 May 2022 13:59:04 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id E7815813F8 for ; Wed, 25 May 2022 17:59:03 +0000 (UTC) X-FDA: 79505026566.04.1400B64 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf24.hostedemail.com (Postfix) with ESMTP id 9A1D3180028 for ; Wed, 25 May 2022 17:58:50 +0000 (UTC) Received: by mail-pl1-f171.google.com with SMTP id d22so19200778plr.9 for ; Wed, 25 May 2022 10:59:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Uvr0+sCkmLiSZWxmyZhwpZSJ3wvLpeFEGZEQ2lIkwmU=; b=XD1lcPxkw7S3zRs/8+49lv+DnWFzi3wkan3miLa/UwRLkcxFOFbKudBlfDPS+szJpG jsjAFDoW65A3Wu5UA5VJ/wlK/YdI1WtMbsTDD9zCUBcGmFsEgQ/7lt3V7ZL6U6sI9/9h Q5FEWSWFrkp5sBN12PGg8uFn/mInKIYhhDUe0nBuKmOODcuSDwlB7GMbCwi5HyBcMzMY YoarM1iLKttCCD+EUv/XSSqRzR8ChlVcxmBuwfghwYC9S8CB4YBQr8bONoPEBftpFaW0 uZ7hrjY9BiBLbJQQ20Z+FGz3J+M3WyeiZ35N1qFv67YaFmRvMnuTdVoCWEzoTUFZ2EBb jDHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Uvr0+sCkmLiSZWxmyZhwpZSJ3wvLpeFEGZEQ2lIkwmU=; b=EpTYoaURZLJKv6AQVBHVZcOQziDnmL1KWEGZxfOR+G0+VAx8+/AZYYaOGcAHybMNMk Bt8KrYX0OnWTAXvnLqEdE7yR56wjL7ka/RZzNihMs3N3FYmFGmky3RK/J0skGzumz3+b gGogoihPcdnhivK/BFkHnrRj8tATb03X/RrbZ0nhs7dWdgA6XRRV01Y4Yy3AokHC7MDs VIceI3cRpXSA5Hz9saLNtpylwi2fzu6FfllcKJoZcVRkmS8mab95cduV2BtbeJiwt600 y3quMUEjzTvX6gnChfmWHiNZSTyianUg1GxSjIUcdK4o4Lc7EeDOSJXzM6rrlzkRB5Ym SqKg== X-Gm-Message-State: AOAM531rVs9DZ2yMBOhih8PgyoDIC7WrjDzj5GMfoNxyxHZiWEEpD4M0 fZcVQr93he2RPpuQH9h9evLF6Idz39sMwLmh59w= X-Google-Smtp-Source: ABdhPJzg4gA/odLsF09cCQCnefvzzP1TP5E0bMmyiupwif1s5EvI8nPFJN1tYe1PqN3SDIrj4uXcR3yhM1M+yxK4aMo= X-Received: by 2002:a17:903:32cc:b0:161:9539:fd49 with SMTP id i12-20020a17090332cc00b001619539fd49mr34510785plr.117.1653501542609; Wed, 25 May 2022 10:59:02 -0700 (PDT) MIME-Version: 1.0 References: <20220504214437.2850685-1-zokeefe@google.com> <20220504214437.2850685-5-zokeefe@google.com> In-Reply-To: From: Yang Shi Date: Wed, 25 May 2022 10:58:50 -0700 Message-ID: Subject: Re: [PATCH v5 04/13] mm/khugepaged: make hugepage allocation context-specific To: "Zach O'Keefe" Cc: David Rientjes , Alex Shi , David Hildenbrand , Matthew Wilcox , Michal Hocko , Pasha Tatashin , Peter Xu , SeongJae Park , Song Liu , Vlastimil Babka , Zi Yan , Linux MM , Andrea Arcangeli , Andrew Morton , Arnd Bergmann , Axel Rasmussen , Chris Kennelly , Chris Zankel , Helge Deller , Hugh Dickins , Ivan Kokshaysky , "James E.J. Bottomley" , Jens Axboe , "Kirill A. Shutemov" , Matt Turner , Max Filippov , Miaohe Lin , Minchan Kim , Patrick Xia , Pavel Begunkov , Thomas Bogendoerfer Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9A1D3180028 X-Stat-Signature: 85inycrmp789jm7djsyu99exkgk1it3t X-Rspam-User: Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=XD1lcPxk; spf=pass (imf24.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1653501530-386735 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 17, 2022 at 3:35 PM Zach O'Keefe wrote: > > On Tue, May 17, 2022 at 10:51 AM Yang Shi wrote: > > > > On Fri, May 13, 2022 at 4:56 PM Zach O'Keefe wrote: > > > > > > On Fri, May 13, 2022 at 4:17 PM Yang Shi wrote: > > > > > > > > On Fri, May 13, 2022 at 4:05 PM Zach O'Keefe wrote: > > > > > > > > > > Thanks for the review, David! > > > > > > > > > > On Thu, May 12, 2022 at 1:02 PM David Rientjes wrote: > > > > > > > > > > > > On Wed, 4 May 2022, Zach O'Keefe wrote: > > > > > > > > > > > > > diff --git a/mm/khugepaged.c b/mm/khugepaged.c > > > > > > > index c94bc43dff3e..6095fcb3f07c 100644 > > > > > > > --- a/mm/khugepaged.c > > > > > > > +++ b/mm/khugepaged.c > > > > > > > @@ -92,6 +92,10 @@ struct collapse_control { > > > > > > > > > > > > > > /* Last target selected in khugepaged_find_target_node() */ > > > > > > > int last_target_node; > > > > > > > + > > > > > > > + struct page *hpage; > > > > > > > + int (*alloc_charge_hpage)(struct mm_struct *mm, > > > > > > > + struct collapse_control *cc); > > > > > > > }; > > > > > > > > > > > > > > /** > > > > > > > > > > > > Embedding this function pointer into collapse_contol seems like it would > > > > > > need some pretty strong rationale. Not to say that it should be a > > > > > > non-starter, but I think the changelog needs to clearly indicate why this > > > > > > is better/cleaner than embedding the needed info for a single allocation > > > > > > and charge function to use. If the callbacks would truly be so different > > > > > > that unifying them would be more complex, I think this makes sense. > > > > > > > > > > Mostly, this boils down to khugepaged having different a allocation > > > > > pattern for NUMA/UMA ; the former scans the pages first to determine > > > > > the right node, the latter preallocates before scanning. khugepaged > > > > > has the luxury on UMA systems of just holding onto a hugepage > > > > > indefinitely for the next collapse target. > > > > > > > > > > For MADV_COLLAPSE, we never preallocate, and so its pattern doesn't > > > > > depend on NUMA or UMA configs. Trying to avoid "if (khugepaged) ... > > > > > else" casing, defining this as a context-defined operation seemed > > > > > appropriate. > > > > > > > > > > Collapsing both alloc and charging together was mostly a code > > > > > cleanliness decision resulting from not wanting to embed a ->gfp() > > > > > hook (gfp flags are used both by allocation and memcg charging). > > > > > Alternatively, a .gfp member could exist - it would just need to be > > > > > refreshed periodically in the khugepaged codepath. > > > > > > > > > > That all said - let me take another crack at seeing if I can make this > > > > > work without the need for a function pointer here. > > > > > > > > I had a patch that removed UMA allocation, please refer to > > > > https://lore.kernel.org/linux-mm/20210817202146.3218-1-shy828301@gmail.com/#t > > > > > > > > It was not made upstream due to some requests for further cleanup, but > > > > unfortunately I haven't got time to look into it yet. > > > > > > > > If this page were merged, would that make your life easier? > > > > > > Hey Yang, > > > > > > First, sorry for missing that patch in the first place. I actually > > > have some patches queued up that do a similar cleanup of > > > khugepaged_prealloc_page() that was mentioned, but decided to not > > > include them here. > > > > > > Second, removing the NUMA/UMA story does make this patch easier, I > > > think (esp since the sched change was dropped for now). This is > > > something I wanted while writing this series, but without the larger > > > context referenced in your patch (most users don't build NUMA=n even > > > on single node systems, and the pcp hugepage lists optimization) > > > couldn't justify myself. > > > > Thanks, it would be better to add this patch into your series as a > > prerequisite so that you could make the MADV_COLLAPSE easier. > > > > I don't think I will be able to find time to rework the patch and > > solve all the review comments at any time soon. If you'd like to take > > it, please do it. > > > > Sounds good, Yang. I'll take a crack at it for v6. Hi Zach, Fortunately I got some time, if you haven't spent too much effort in this, I could rework the patch then resubmit. Just let me know. > > Thanks for taking the time, > Zach > > > > > > > Best, > > > Zach