From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34E71C6FA83 for ; Wed, 28 Sep 2022 01:42:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF1A48E010E; Tue, 27 Sep 2022 21:42:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B7AAA8E00C1; Tue, 27 Sep 2022 21:42:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F3668E010E; Tue, 27 Sep 2022 21:42:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 88E038E00C1 for ; Tue, 27 Sep 2022 21:42:07 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 5D9E91C617B for ; Wed, 28 Sep 2022 01:42:07 +0000 (UTC) X-FDA: 79959793494.25.B3A7A7B Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf21.hostedemail.com (Postfix) with ESMTP id 6B22D1C0005 for ; Wed, 28 Sep 2022 01:42:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1664329326; x=1695865326; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=VdnIAay/pNa3J+uoxCjvYQR/wUDLr3pNkvZzDguZ7Fk=; b=ak4MftBpAsoRfRcH8R3Jmirh3aEpF20hMsIOKoAa9AB4vBqcOXGElDyP KD7KjBcnz4AGUnuhXq+a9prVsMJRglusbwK92WXkS7bpFWdIOTDuD+eHe vrnFiDzxIoyaGYYw12VdWzSwSMSJU3o1GDzzHxRc0A9IGac/Tun11hpss wDw54WgUovCTQ41XDd5Od13utgKlJ2hM7x6BFJs6+43amLOVwCnpJ2MhK BnCtGk+5M+UrJDOVUYtapCBnY+i3oWiumQcIbK7xqyD/ABdlnwRHPpcm9 zDuEbbeh7ezpBRBdYUBK58MgShvu1NoZi+J7X6qdJA69PudmZIjaDp0jk Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10483"; a="284608765" X-IronPort-AV: E=Sophos;i="5.93,350,1654585200"; d="scan'208";a="284608765" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Sep 2022 18:42:04 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10483"; a="654939615" X-IronPort-AV: E=Sophos;i="5.93,350,1654585200"; d="scan'208";a="654939615" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Sep 2022 18:42:02 -0700 From: "Huang, Ying" To: Alistair Popple Cc: Yang Shi , John Hubbard , , , Andrew Morton , Zi Yan , Baolin Wang , Oscar Salvador , "Matthew Wilcox" Subject: Re: [RFC 2/6] mm/migrate_pages: split unmap_and_move() to _unmap() and _move() In-Reply-To: <87pmfgjnpj.fsf@nvdebian.thelocal> (Alistair Popple's message of "Wed, 28 Sep 2022 10:59:08 +1000") References: <20220921060616.73086-1-ying.huang@intel.com> <20220921060616.73086-3-ying.huang@intel.com> <87o7v2lbn4.fsf@nvdebian.thelocal> <87fsgdllmb.fsf@nvdebian.thelocal> <87ill937qe.fsf@yhuang6-desk2.ccr.corp.intel.com> <46807002-c42c-1232-0938-5b48050171ee@nvidia.com> <87pmfgjnpj.fsf@nvdebian.thelocal> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) Date: Wed, 28 Sep 2022 09:41:28 +0800 Message-ID: <87czbg2s3b.fsf@yhuang6-desk2.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ascii ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664329326; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zxRbku8aoXRZ99m3ZcQYGlFUV+74X2ZE7Sye8QJj6fo=; b=ww+2XxPySeGexMOyjyqW/SAwM8kzSaPDIWchfZb0T0KyuHrht+EGWQ8tr+TBtI/Is2iR05 T47gFuW2sPKWhM4WyhAOAjVtbD9H9U5L7TiEFu7xOhdC1wRM1UkRPNroCY/jX2VDqjXTML TVTDUkFuS1Uo3Uxk1Y0zBwJ5qaGK3pY= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=ak4MftBp; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf21.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664329326; a=rsa-sha256; cv=none; b=Eo7B3q9jCOEohUmxzcKPTcE+GIc9IizETYmnfnZe5AbT7xdcj9GrmPXz2hZuE4SqoZCEp6 7OAJAMG758QfWStlL9NltU3u4vmA7BpLKOHCaSjnJUIcusX7+rD9dfmC56FLYlMU1hUoJB an/oak1FE6k3KdCN27dl+ZExF+kBZF4= Authentication-Results: imf21.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=ak4MftBp; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf21.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com X-Rspamd-Server: rspam02 X-Stat-Signature: nphzr7c6j75toohu35p7n7wrd33unke4 X-Rspamd-Queue-Id: 6B22D1C0005 X-Rspam-User: X-HE-Tag: 1664329326-893774 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Alistair Popple writes: > Yang Shi writes: > >> On Tue, Sep 27, 2022 at 1:35 PM John Hubbard wrote: >>> >>> On 9/26/22 18:51, Huang, Ying wrote: >>> >>> But there might be other cases which may incur deadlock, for example, >>> >>> filesystem writeback IIUC. Some filesystems may lock a bunch of pages >>> >>> then write them back in a batch. The same pages may be on the >>> >>> migration list and they are also dirty and seen by writeback. I'm not >>> >>> sure whether I miss something that could prevent such a deadlock from >>> >>> happening. >>> >> >>> >> I'm not overly familiar with that area but I would assume any filesystem >>> >> code doing this would already have to deal with deadlock potential. >>> > >>> > Thank you very much for pointing this out. I think the deadlock is a >>> > real issue. Anyway, we shouldn't forbid other places in kernel to lock >>> > 2 pages at the same time. >>> > >>> >>> I also agree that we cannot make any rules such as "do not lock > 1 page >>> at the same time, elsewhere in the kernel", because it is already >>> happening, for example in page-writeback.c, which locks PAGEVEC_SIZE >>> (15) pages per batch [1]. > > That's not really the case though. The inner loop of write_cache_page() > only ever locks one page at a time, either directly via the > unlock_page() on L2338 (those goto's are amazing) or indirectly via > (*writepage)() on L2359. > > So there's no deadlock potential there because unlocking any previously > locked page(s) doesn't depend on obtaining the lock for another page. > Unless I've missed something? Yes. This is my understanding too after checking ext4_writepage(). Best Regards, Huang, Ying >>> The only deadlock prevention convention that I see is the convention of >>> locking the pages in order of ascending address. That only helps if >>> everything does it that way, and migrate code definitely does not. >>> However...I thought that up until now, at least, the migrate code relied >>> on trylock (which can fail, and so migration can fail, too), to avoid >>> deadlock. Is that changing somehow, I didn't see it? >> >> The trylock is used by async mode which does try to avoid blocking. >> But sync mode does use lock. The current implementation of migration >> does migrate one page at a time, so it is not a problem. >> >>> >>> >>> [1] https://elixir.bootlin.com/linux/latest/source/mm/page-writeback.c#L2296 >>> >>> thanks, >>> >>> -- >>> John Hubbard >>> NVIDIA >>> >>> > The simplest solution is to batch page migration only if mode == >>> > MIGRATE_ASYNC. Then we may consider to fall back to non-batch mode if >>> > mode != MIGRATE_ASYNC and trylock page fails. >>> > >>> >>>