From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7793C25B0E for ; Wed, 17 Aug 2022 02:45:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BF778D0002; Tue, 16 Aug 2022 22:45:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 66E6B8D0001; Tue, 16 Aug 2022 22:45:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50F7A8D0002; Tue, 16 Aug 2022 22:45:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3EBBF8D0001 for ; Tue, 16 Aug 2022 22:45:42 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 126DCA01C8 for ; Wed, 17 Aug 2022 02:45:42 +0000 (UTC) X-FDA: 79807544124.06.15B21C9 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 9F1901A0060 for ; Wed, 17 Aug 2022 02:45:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1660704341; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=sFFVgLR/Ut+q9B0hkN4LXK9LXiRNRRxKcE0ZSUfZXQc=; b=igBR8u9IhUYcHwo5l1rjuJfd2rcU5ceZRs3Qp1ncSYfPTyixtv34Ch0k2xoMln+3l+R28N D31pbiAHOaXoKgMS3UVUbxJ42LFMZALnE2ySckpRILTsKugI8yMHPTxFrEEVQzt8jaVMeU WjvsMmQ/UqDcpZKSSQWEMcbOeqz+Vqk= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-384-BEkYJTyXNmqGu8Cv6d5qhA-1; Tue, 16 Aug 2022 22:45:40 -0400 X-MC-Unique: BEkYJTyXNmqGu8Cv6d5qhA-1 Received: by mail-qk1-f197.google.com with SMTP id bm11-20020a05620a198b00b006bb2388ef0cso6501182qkb.5 for ; Tue, 16 Aug 2022 19:45:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc; bh=sFFVgLR/Ut+q9B0hkN4LXK9LXiRNRRxKcE0ZSUfZXQc=; b=P6VkK4ASzv7EJlgVOb3gtiqwV3oSfqhQClpvfr7YShCgl7txVoUABho7BDtg9EUJvU BYhHohaxGLOq3qFHcWVC+yKitBwiqqILOsCrW5SHmQ7SV9go05nIUQQPIta/+Qu3Y3bC 1n1Ry6Er0QsBKPdWRiduYn9anb0cegT3Fg9nfDL6umeaFp1FLKr4LII9FGgp601NpXs9 +l5N+TlL8JSAvShvbiScDyNXCYg4i8mUB5QUbeTdVPRDgiM29DvoIujS0iFC3wwdj834 zVsrERLyOAs4CZ9eMzZEZ0B5s0Pdpbc2bGu9LxQk21ryUr/UtOmgEdF9ikOJ10BGUQxK qX8Q== X-Gm-Message-State: ACgBeo1dEpAf+jFeZ/zrY+LuDjcHsy/sCJfQb6zwXb4O8P9ZNFTQIJO7 4OiUZEcQgKoazX68a0ebckTSBocQjWLsj4umJc8xJ+aE8aod3lUaUozPjdaYr/UEfMjPvYDTxZ7 sS8/SxdvDGW0= X-Received: by 2002:a37:64c3:0:b0:6ba:f404:8ff2 with SMTP id y186-20020a3764c3000000b006baf4048ff2mr12451577qkb.397.1660704339577; Tue, 16 Aug 2022 19:45:39 -0700 (PDT) X-Google-Smtp-Source: AA6agR5ynDdWHuXl7m7xK2zUJirotuz+hRQv269GrLiXRjE8J+k1PvWsa2cxvxjorZ2Ww5yhRCP3kw== X-Received: by 2002:a37:64c3:0:b0:6ba:f404:8ff2 with SMTP id y186-20020a3764c3000000b006baf4048ff2mr12451556qkb.397.1660704339333; Tue, 16 Aug 2022 19:45:39 -0700 (PDT) Received: from xz-m1.local (bras-base-aurron9127w-grc-35-70-27-3-10.dsl.bell.ca. [70.27.3.10]) by smtp.gmail.com with ESMTPSA id w17-20020a05620a0e9100b006bb568016easm4148710qkm.116.2022.08.16.19.45.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Aug 2022 19:45:38 -0700 (PDT) Date: Tue, 16 Aug 2022 22:45:37 -0400 From: Peter Xu To: Alistair Popple Cc: huang ying , linux-mm@kvack.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, "Sierra Guiza, Alejandro (Alex)" , Felix Kuehling , Jason Gunthorpe , John Hubbard , David Hildenbrand , Ralph Campbell , Matthew Wilcox , Karol Herbst , Lyude Paul , Ben Skeggs , Logan Gunthorpe , paulus@ozlabs.org, linuxppc-dev@lists.ozlabs.org, Huang Ying , stable@vger.kernel.org Subject: Re: [PATCH v2 1/2] mm/migrate_device.c: Copy pte dirty bit to page Message-ID: References: <6e77914685ede036c419fa65b6adc27f25a6c3e9.1660635033.git-series.apopple@nvidia.com> <871qtfvdlw.fsf@nvdebian.thelocal> MIME-Version: 1.0 In-Reply-To: <871qtfvdlw.fsf@nvdebian.thelocal> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=igBR8u9I; spf=pass (imf19.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660704341; a=rsa-sha256; cv=none; b=hxP9q196sfvymsDknadJXvraR0M7zvRjImtT9MplaxXs1OS4LR3LJpJ58nRtpxnt+iGKq+ c5PcTY6xgFDIKye+JOlCt8269L7H9tuD+c1s8BnNfek2BC/5sr7vUwfElgnHnFQDQOgbZg VRb9bZSPArFHWrZN6vbC9HclL0e6gt4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660704341; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sFFVgLR/Ut+q9B0hkN4LXK9LXiRNRRxKcE0ZSUfZXQc=; b=3haQ/DUTd0YUjgIRwwAxB3t3q3uuRW7muK2h8PsAi6BFuGDl4EBtXCtD401gcUlN7l00bW xiwNtT3ImzASr4VHBHbLW+iDgZfDlOu1a7DynydBxSvD/g1jWdwYGQvcp8yh34So1SPMV+ NOWuiqrw2mxm8zNJgPh9Z/6w5mUh5fI= Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=igBR8u9I; spf=pass (imf19.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: 7dwn8wqhkkh78bfhhydtfgurp3oexsqw X-Rspamd-Server: rspam09 X-Rspam-User: X-Rspamd-Queue-Id: 9F1901A0060 X-HE-Tag: 1660704341-378763 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Aug 17, 2022 at 11:49:03AM +1000, Alistair Popple wrote: > > Peter Xu writes: > > > On Tue, Aug 16, 2022 at 04:10:29PM +0800, huang ying wrote: > >> > @@ -193,11 +194,10 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, > >> > bool anon_exclusive; > >> > pte_t swp_pte; > >> > > >> > + flush_cache_page(vma, addr, pte_pfn(*ptep)); > >> > + pte = ptep_clear_flush(vma, addr, ptep); > >> > >> Although I think it's possible to batch the TLB flushing just before > >> unlocking PTL. The current code looks correct. > > > > If we're with unconditionally ptep_clear_flush(), does it mean we should > > probably drop the "unmapped" and the last flush_tlb_range() already since > > they'll be redundant? > > This patch does that, unless I missed something? Yes it does. Somehow I didn't read into the real v2 patch, sorry! > > > If that'll need to be dropped, it looks indeed better to still keep the > > batch to me but just move it earlier (before unlock iiuc then it'll be > > safe), then we can keep using ptep_get_and_clear() afaiu but keep "pte" > > updated. > > I think we would also need to check should_defer_flush(). Looking at > try_to_unmap_one() there is this comment: > > if (should_defer_flush(mm, flags) && !anon_exclusive) { > /* > * We clear the PTE but do not flush so potentially > * a remote CPU could still be writing to the folio. > * If the entry was previously clean then the > * architecture must guarantee that a clear->dirty > * transition on a cached TLB entry is written through > * and traps if the PTE is unmapped. > */ > > And as I understand it we'd need the same guarantee here. Given > try_to_migrate_one() doesn't do batched TLB flushes either I'd rather > keep the code as consistent as possible between > migrate_vma_collect_pmd() and try_to_migrate_one(). I could look at > introducing TLB flushing for both in some future patch series. should_defer_flush() is TTU-specific code? IIUC the caller sets TTU_BATCH_FLUSH showing that tlb can be omitted since the caller will be responsible for doing it. In migrate_vma_collect_pmd() iiuc we don't need that hint because it'll be flushed within the same function but just only after the loop of modifying the ptes. Also it'll be with the pgtable lock held. Indeed try_to_migrate_one() doesn't do batching either, but IMHO it's just harder to do due to using the vma walker (e.g., the lock is released in not_found() implicitly so iiuc it's hard to do tlb flush batching safely in the loop of page_vma_mapped_walk). Also that's less a concern since the loop will only operate upon >1 ptes only if it's a thp page mapped in ptes. OTOH migrate_vma_collect_pmd() operates on all ptes on a pmd always. No strong opinion anyway, it's just a bit of a pity because fundamentally this patch is removing the batching tlb flush. I also don't know whether there'll be observe-able perf degrade for migrate_vma_collect_pmd(), especially on large machines. Thanks, -- Peter Xu