From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CDCDC433EF for ; Thu, 28 Oct 2021 14:13:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0A4FB61073 for ; Thu, 28 Oct 2021 14:13:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0A4FB61073 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 7B4BF6B0071; Thu, 28 Oct 2021 10:13:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 76427940007; Thu, 28 Oct 2021 10:13:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 603FF6B0073; Thu, 28 Oct 2021 10:13:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0055.hostedemail.com [216.40.44.55]) by kanga.kvack.org (Postfix) with ESMTP id 3936B6B0071 for ; Thu, 28 Oct 2021 10:13:29 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A105639F47 for ; Thu, 28 Oct 2021 14:13:28 +0000 (UTC) X-FDA: 78746038896.13.7B0A2D9 Received: from mail-lj1-f175.google.com (mail-lj1-f175.google.com [209.85.208.175]) by imf25.hostedemail.com (Postfix) with ESMTP id 7205CB000198 for ; Thu, 28 Oct 2021 14:13:21 +0000 (UTC) Received: by mail-lj1-f175.google.com with SMTP id q16so11008218ljg.3 for ; Thu, 28 Oct 2021 07:13:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov-name.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=0BjhU3Cna3E9JGjpF83DSSS2jpqfHtqZfRiRgzrxSow=; b=q83ENiFi1iveIgq9JU30MqfVB1wFeBSCxjFEhJuJnLODkJUVFEds1PRtLnNvXx3Pma kQPTHN4fcpLadb3f5wwccV3/OGkes1Am66acJ2zXZi4ZcwF+lJPK65oVcES7VZafA1lQ FwQnxwbknQ80biJTv3NwWHN4XhCmfnx03HOIyaytinm8awbhrRFepdNplINjxnsmsSD6 roksuGKqt3Man1rd2WnGvnERZ77wEyb0aeR4Verd9H+FFlWian/3Kbl+GqH7WBtGE6uS fPZY7mXI66GOjda2KkJXjsp7wAE7GI8xn970SfZioAKL/C/niWhwQEwX/0t9hfrY2KsH 3jdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=0BjhU3Cna3E9JGjpF83DSSS2jpqfHtqZfRiRgzrxSow=; b=cgYbbFE0iy6PIv937b2eMgx+6iHnzLsWWUctphEzBZBe5I66EQPXFzBnJ7FiNwoF7f H1LKZfpfdxKKZWLnUW//Sq1Jk+WsmTzThrQi9+PCggBDiaHD7H8iTsa38S7F7FlFGuhF ek2jartx9q6EEHbVVNyH3nuhUPdWiT/YRviSL7AKwjBRRF/OQF7afsGwXLSuCS5SqPb+ 8PbB59AzOSFGIvwlnI62vzzX1G/pTrgE2jO7gDZCg+qP21nnUXiEw92wsPvsTzFsRynl 8IlvyLAggp4MykW9FpT/kpDLyGtpx9GcQ+UYkykQTEYsJk3F2UFrubcBI3z3XdSgdcAm 5HDg== X-Gm-Message-State: AOAM533GK5XoMJF1Xd4Ut7Ko0G8EgTZUw8sayIurEnQkthPZEUTDJTMg G6N5pIHFulhcdtBFJ25U4u1c5Q== X-Google-Smtp-Source: ABdhPJx0QPD2OkQ/OeUxtR0G8HrJ89WuX3U4SkQ01JmPr4huCqw6ESGbo+fWyGPnZ1VN7//goRTWyw== X-Received: by 2002:a05:651c:1595:: with SMTP id h21mr5048775ljq.418.1635430406537; Thu, 28 Oct 2021 07:13:26 -0700 (PDT) Received: from box.localdomain ([86.57.175.117]) by smtp.gmail.com with ESMTPSA id f40sm326257lfv.156.2021.10.28.07.13.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Oct 2021 07:13:25 -0700 (PDT) Received: by box.localdomain (Postfix, from userid 1000) id 500A110338B; Thu, 28 Oct 2021 17:13:33 +0300 (+03) Date: Thu, 28 Oct 2021 17:13:33 +0300 From: "Kirill A. Shutemov" To: Ning Zhang Cc: linux-mm@kvack.org, Andrew Morton , Johannes Weiner , Michal Hocko , Vladimir Davydov , Yu Zhao Subject: Re: [RFC 0/6] Reclaim zero subpages of thp to avoid memory bloat Message-ID: <20211028141333.kgcjgsnrrjuq4hjx@box.shutemov.name> References: <1635422215-99394-1-git-send-email-ningzhang@linux.alibaba.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1635422215-99394-1-git-send-email-ningzhang@linux.alibaba.com> X-Stat-Signature: u98dd57cr5af1wpizndutdn1bk58h6ij Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=shutemov-name.20210112.gappssmtp.com header.s=20210112 header.b=q83ENiFi; spf=none (imf25.hostedemail.com: domain of kirill@shutemov.name has no SPF policy when checking 209.85.208.175) smtp.mailfrom=kirill@shutemov.name; dmarc=none X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7205CB000198 X-HE-Tag: 1635430401-965004 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Oct 28, 2021 at 07:56:49PM +0800, Ning Zhang wrote: > As we know, thp may lead to memory bloat which may cause OOM. > Through testing with some apps, we found that the reason of > memory bloat is a huge page may contain some zero subpages > (may accessed or not). And we found that most zero subpages > are centralized in a few huge pages. > > Following is a text_classification_rnn case for tensorflow: > > zero_subpages huge_pages waste > [ 0, 1) 186 0.00% > [ 1, 2) 23 0.01% > [ 2, 4) 36 0.02% > [ 4, 8) 67 0.08% > [ 8, 16) 80 0.23% > [ 16, 32) 109 0.61% > [ 32, 64) 44 0.49% > [ 64, 128) 12 0.30% > [ 128, 256) 28 1.54% > [ 256, 513) 159 18.03% > > In the case, there are 187 huge pages (25% of the total huge pages) > which contain more then 128 zero subpages. And these huge pages > lead to 19.57% waste of the total rss. It means we can reclaim > 19.57% memory by splitting the 187 huge pages and reclaiming the > zero subpages. > > This patchset introduce a new mechanism to split the huge page > which has zero subpages and reclaim these zero subpages. > > We add the anonymous huge page to a list to reduce the cost of > finding the huge page. When the memory reclaim is triggering, > the list will be walked and the huge page contains enough zero > subpages may be reclaimed. Meanwhile, replace the zero subpages > by ZERO_PAGE(0). Does it actually help your workload? I mean this will only be triggered via vmscan that was going to split pages and free anyway. You prioritize splitting THP and freeing zero subpages over reclaiming other pages. It may or may not be right thing to do, depending on workload. Maybe it makes more sense to check for all-zero pages just after split_huge_page_to_list() in vmscan and free such pages immediately rather then add all this complexity? > Yu Zhao has done some similar work when the huge page is swap out > or migrated to accelerate[1]. While we do this in the normal memory > shrink path for the swapoff scene to avoid OOM. > > In the future, we will do the proactive reclaim to reclaim the "cold" > huge page proactively. This is for keeping the performance of thp as > for as possible. In addition to that, some users want the memory usage > using thp is equal to the usage using 4K. Proactive reclaim can be harmful if your max_ptes_none allows to recreate THP back. -- Kirill A. Shutemov