From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B790C77B73 for ; Mon, 22 May 2023 05:57:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 88AD5900002; Mon, 22 May 2023 01:57:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 83AC76B0075; Mon, 22 May 2023 01:57:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 75165900002; Mon, 22 May 2023 01:57:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 68C776B0074 for ; Mon, 22 May 2023 01:57:16 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 30A6D1A0AFC for ; Mon, 22 May 2023 05:57:16 +0000 (UTC) X-FDA: 80816833272.04.3C0A30B Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf19.hostedemail.com (Postfix) with ESMTP id 2F92E1A0013 for ; Mon, 22 May 2023 05:57:12 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="I/221L8g"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf19.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684735034; a=rsa-sha256; cv=none; b=rptTzei3WcUk1UjT8nS5kZon3qpaNOxW3vZkRhtq/OMxDndczht5dmoZnnV3XlZFmbMWJj 6BhRUdMfMaeTsj+9QHE8U3yEfvrzHCrjuiAIIVdfUnZCDEHuXOaZTyyQN3HR5z9kc7u/br RF0984U2lHow0LPpiHIW/HbA6DEp7rc= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="I/221L8g"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf19.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684735034; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SfmvEfRz1S8h5FlxQK1YhvOFzmULZvD3kJS9sAB5Go4=; b=5+hUFb00ES/Ht/98q1PP9Q71hf4oX4Kw8u3/CSt1cjkMF83S7yT01072vOznYFLbz739MP bYGNxX+Hk2qMoDenJogNv/D3mbXGWm4mZs+pNu3y09QzPA8wEyhC5Y+VyGs4GIb9g2iTno +gthx9MjwuYVsvjliXvjKx5EWMvJP+w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1684735033; x=1716271033; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=tOMcNfhocZCO/GNSExZGCChK9wg74/hawJud+TAllGI=; b=I/221L8g7XhtozIRFRzXe4J+uWc6HkJuQCCEMJgaVrYg6pROR9FRxLPa iCi8O0Jx3X01WJNyWVPLqNvZey8LY8GZ3AlxBVUkkY39j04UljhfEp27V /ddkWruvRyixa2pShOLxjl6i2aMaTnAAnVRcuL66lLUUCMOExByecAIBc +FXzLpgGtJCSWkrdxKQVzKkH0s1ptD5sUUiSsqe6qoxoiQ3IJ7ryfRJTb 1/1jMNaHKLZ7iB+5FtG3YA8DybyzIjeYADNHP4uhgT/aA/NXmJRb4QcvO ZR7OWCFtudNoQcdt1KJgx2lFJ+09WhrQlskDg5dUCGsZxbYI2UQZe9y0m A==; X-IronPort-AV: E=McAfee;i="6600,9927,10717"; a="332439333" X-IronPort-AV: E=Sophos;i="6.00,183,1681196400"; d="scan'208";a="332439333" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2023 22:57:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10717"; a="736286283" X-IronPort-AV: E=Sophos;i="6.00,183,1681196400"; d="scan'208";a="736286283" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2023 22:57:08 -0700 From: "Huang, Ying" To: David Hildenbrand Cc: Khalid Aziz , akpm@linux-foundation.org, willy@infradead.org, steven.sistare@oracle.com, mgorman@techsingularity.net, khalid@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3] mm, compaction: Skip all non-migratable pages during scan References: <20230517161555.84776-1-khalid.aziz@oracle.com> <87sfbubg3j.fsf@yhuang6-desk2.ccr.corp.intel.com> <5f43a17c-94a0-4bff-b451-e5014de36ed9@redhat.com> Date: Mon, 22 May 2023 13:55:51 +0800 In-Reply-To: <5f43a17c-94a0-4bff-b451-e5014de36ed9@redhat.com> (David Hildenbrand's message of "Fri, 19 May 2023 11:51:00 +0200") Message-ID: <875y8k3o60.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 2F92E1A0013 X-Stat-Signature: z5buyb7m1ycfypqixsn5dqw78turnudc X-HE-Tag: 1684735032-374815 X-HE-Meta: U2FsdGVkX19rXKy5/PfNkCavkLO4wqn9QBEnWhjv0Go9upgFTJiR7SpT9y+dKw870TasJ9Sk9ixD+LYGEYYzGnNRdd+92LwvZIlqV6TZakgZscJq1EKsscBe43/mUhyavTNLq0PrptjKQqbryxR7ch275N0R2TZ0xgjfvYIPe5Of06wAwnXxbqC0c7E+Q9noYOhWMAvZFlK+/HkX6+tzYJaQD2Pz7puXmCErk0qcl1PNMKEl2NYJqNis0P0wIiDJVI964/8dKyjsUYG1ZTAl7FaBF9CEOPcdvdXAVoOrEXJ0R9kxU4Ut3ttYajqI4X2U/A4zvcqBEvgy41K6qZ0XhLEINvmmOgyA+tY0XPOAySOAq8+8aDPKO9pqKkLtXKoWbC775cuo5BeU9LFveedMIrkTCaUEv7imcPRNQbgzX8Gb/s22ssExPW1qjV1GWZ/7pqCupliEoNUYYb+aDqWXJruAOZtBqZb2/LgfGv9qUKL/e5SnCQrMofuuW9DVxCsv37zUMjO6o9ek8BwFG03iKemXGDSAe6O+ExgKqb1igIB5066rxu3LytJE4YuLy/6XX5gOPmo5AGM4Gkc45kCfoimuuLTpeQ3wrO9d3U+P8jii/F5NC8F0kY+DnLWW9LECrHXg0ySkv08S7ZR2wLtsHc/Reqpalx2ZnMzxsVK7FJ7tRicspXGO6r3kXKEQ1rDaUuF2jbMxMSMqyQFwU9kaYxOhjs2JOw+eA5cwwx1hvccDc7Gq+J7fNnpRq3DxPa5WzojkjJaamyyJMqrIlwJbAYiiiBAuNoiQ4jxe+EjH56N2rKLSGUppTRGe9H4Dir1Va4K8iuicSVgppVChQhUmuecFBtn/kdFghS6ATXvkjm/bNskQDGlPZbaa9N/HgRjEtTAYCcQvHtOdJwrhIL1QrA+DEQOKCRND39LY9fWe8mP1cD89GejmeESRJniik4St4B+ZQkTBR/4+zK/Ggic kzqo+bFx r7Hg6H4m0+8LcJRVYl+i6f8fIN0CP4fuRwtDkV2uEP1CaLYG/Ksn3NjOHiZMIkrFx7DB4DIj5V+GiAxZvMNGza2vlOZnmdOi45Gajq9eXS7+Y38LyzS+0p+JNa02AUiWUblZJe62EPBTwUsCEF2IXAXG0eyMp8eLnEkzg0VulsetV/Aw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: David Hildenbrand writes: > On 18.05.23 03:09, Huang, Ying wrote: >> David Hildenbrand writes: >> >>> On 17.05.23 18:15, Khalid Aziz wrote: >>>> Pages pinned in memory through extra refcounts can not be migrated. >>>> Currently as isolate_migratepages_block() scans pages for >>>> compaction, it skips any pinned anonymous pages. All non-migratable >>>> pages should be skipped and not just the anonymous pinned pages. >>>> This patch adds a check for extra refcounts on a page to determine >>>> if the page can be migrated. This was seen as a real issue on a >>>> customer workload where a large number of pages were pinned by vfio >>>> on the host and any attempts to allocate hugepages resulted in >>>> significant amount of cpu time spent in either direct compaction or >>>> in kcompactd scanning vfio pinned pages over and over again that can >>>> not be migrated. >>> >>> How will this change affect alloc_contig_range(), such as used for CMA >>> allocations or virtio-mem? alloc_contig_range() ends up calling >>> isolate_migratepages_range() -> isolate_migratepages_block(). >> IIUC, cc->alloc_contig can be used to distinguish contiguous >> allocation >> and compaction. And, from the original commit which introduced >> anonymous pages skipping (commit 119d6d59dcc0 ("mm, compaction: avoid >> isolating pinned pages ")) and this patch, large number of migration >> failure during compaction causes real issue too. So, I suggest to use >> cc->alloc_contig here. > > Agreed. I further wonder if we want to special-case the !alloc_contig > case also for MIGRATE_CMA and ZONE_MOVABLE, where we cannot have > longterm page pinnings (e.g., vfio pinned pages). This makes sense. The skipping is more accurate in this way. Best Regards, Huang, Ying