From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3114C32772 for ; Wed, 17 Aug 2022 19:27:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D418C6B0073; Wed, 17 Aug 2022 15:27:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF12D6B0074; Wed, 17 Aug 2022 15:27:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B920A8D0001; Wed, 17 Aug 2022 15:27:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A7FBE6B0073 for ; Wed, 17 Aug 2022 15:27:39 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 80696A051B for ; Wed, 17 Aug 2022 19:27:39 +0000 (UTC) X-FDA: 79810068996.05.92EC7AC Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf06.hostedemail.com (Postfix) with ESMTP id ED77B1801A0 for ; Wed, 17 Aug 2022 19:27:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1660764458; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dmDD/m9rVXOHRexADGf5nUOz3ccETHTRAhO5DzNVEqY=; b=TEzVsHQ/L8n9jq1OtTOjqfUP+ZLcJe3dwgP9C4oyUyari4p2Vut2FZJ/Jx7SXchLEF746u 92oeADUogsO+p2MJWfesYKooAFZWegOTRd0ijKwJIwljCGoHia1cEGh5lnniQytY/hTgzr pCntWNp/y8opSUVOchH5RpCkEpd+/94= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-42-af3LidO_OFi1IMT2kNm7Ww-1; Wed, 17 Aug 2022 15:27:35 -0400 X-MC-Unique: af3LidO_OFi1IMT2kNm7Ww-1 Received: by mail-qv1-f70.google.com with SMTP id o16-20020a0cecd0000000b0049656c32564so2303687qvq.19 for ; Wed, 17 Aug 2022 12:27:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc; bh=dmDD/m9rVXOHRexADGf5nUOz3ccETHTRAhO5DzNVEqY=; b=20d1qORYpjrXU1FQfNHQJ9Srq2aqRUKtmfMexXRrE2ExQ8L9hWq/Y24f70V/iKzHoK LO3vPUl+CHyIcno/BeFKHDurKqJAl8dGi+D0k5BAObWW26PTNTGypGmGOagcjL6ZIMJj VX1c3iZQoX6LBki7uL1RauQ8pronHtEPDB5ZBvlYg5H7f+yNbabHosoWDjAjUFnp7Voe T0RmYhoz6LWe2bYqMQISENV3GV2OhFdLzthQDT8uTZtCVi/j7jQMWfdGuB5Q00riay6t RQlQ65X6zwbnLRofhsJLO9MRu4NvzXZuI7pm1yEvac1kRSmNU0pchcIshAYZD/2f4PHA QugA== X-Gm-Message-State: ACgBeo1SvdPfO5SQYqvCWSPEOLtH1tao1mZIN0sN1J8RkZ6CVGwbPgeK J3XCbhNBNCzzklG16U4XvbAMuvHUKXEcP/Bu7NyetuZD6949JLiQPahY7BAA1uGAO3KcLY1S4W8 viveE38Y3M7A= X-Received: by 2002:a05:6214:b6c:b0:476:8321:db81 with SMTP id ey12-20020a0562140b6c00b004768321db81mr23492941qvb.100.1660764455107; Wed, 17 Aug 2022 12:27:35 -0700 (PDT) X-Google-Smtp-Source: AA6agR6vqYZw2nSP0al2QOlYxSI+caDpn668z2wTq0sUdA1GVvZZhov4a4lSRj3rYF0hmqY4tBw2Ng== X-Received: by 2002:a05:6214:b6c:b0:476:8321:db81 with SMTP id ey12-20020a0562140b6c00b004768321db81mr23492907qvb.100.1660764454794; Wed, 17 Aug 2022 12:27:34 -0700 (PDT) Received: from xz-m1.local (bras-base-aurron9127w-grc-35-70-27-3-10.dsl.bell.ca. [70.27.3.10]) by smtp.gmail.com with ESMTPSA id bj38-20020a05620a192600b006bb5cdd4031sm5016365qkb.40.2022.08.17.12.27.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Aug 2022 12:27:34 -0700 (PDT) Date: Wed, 17 Aug 2022 15:27:32 -0400 From: Peter Xu To: Nadav Amit Cc: "Huang, Ying" , Alistair Popple , huang ying , Linux MM , Andrew Morton , LKML , "Sierra Guiza, Alejandro (Alex)" , Felix Kuehling , Jason Gunthorpe , John Hubbard , David Hildenbrand , Ralph Campbell , Matthew Wilcox , Karol Herbst , Lyude Paul , Ben Skeggs , Logan Gunthorpe , paulus@ozlabs.org, linuxppc-dev@lists.ozlabs.org, stable@vger.kernel.org Subject: Re: [PATCH v2 1/2] mm/migrate_device.c: Copy pte dirty bit to page Message-ID: References: <6e77914685ede036c419fa65b6adc27f25a6c3e9.1660635033.git-series.apopple@nvidia.com> <871qtfvdlw.fsf@nvdebian.thelocal> <87o7wjtn2g.fsf@nvdebian.thelocal> <87tu6bbaq7.fsf@yhuang6-desk2.ccr.corp.intel.com> <1D2FB37E-831B-445E-ADDC-C1D3FF0425C1@gmail.com> MIME-Version: 1.0 In-Reply-To: <1D2FB37E-831B-445E-ADDC-C1D3FF0425C1@gmail.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660764459; a=rsa-sha256; cv=none; b=nrSk+rLeg9b0D219yydm4lSGXOSYxYbotH1m0Ob9VTNJzV0n7rkwroaJOY8e5yiFU4xnZr cRZ2JpbWG5I4gdXeE9JLtQ6+7Xj2OrLWyFH4v9o+jS4te4yi/YZ29kq2d+YnP3WLUgcPGs XxV6KbAVSLoklZ73VkQ+yBdwhCmFHQo= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="TEzVsHQ/"; spf=pass (imf06.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660764459; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dmDD/m9rVXOHRexADGf5nUOz3ccETHTRAhO5DzNVEqY=; b=zkR5uK8hp0OOdhEjXio7KWDntq0MvDG8ZmNAfZEA6OGQ1yOILie8ppPvqRViyEt2wiG63u 7lHhQvVypIOkf+9463QCQabIW7FTTSrecTcP6Te0/T5CFPdB6opVfO8R/FQ5XoKl5JAZeq CWAaxHKGMIp1YrclWtiJv0o43kfQ2nY= Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="TEzVsHQ/"; spf=pass (imf06.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspam-User: X-Stat-Signature: hsjezgthitmgpcxedtyim1nogmi6k54r X-Rspamd-Queue-Id: ED77B1801A0 X-Rspamd-Server: rspam06 X-HE-Tag: 1660764458-444138 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Aug 17, 2022 at 02:41:19AM -0700, Nadav Amit wrote: > 4. Having multiple TLB flushing infrastructures makes all of these > discussions very complicated and unmaintainable. I need to convince myself > in every occasion (including this one) whether calls to > flush_tlb_batched_pending() and tlb_flush_pending() are needed or not. > > What I would like to have [3] is a single infrastructure that gets a > “ticket” (generation when the batching started), the old PTE and the new PTE > and checks whether a TLB flush is needed based on the arch behavior and the > current TLB generation. If needed, it would update the “ticket” to the new > generation. Andy wanted a ring for pending TLB flushes, but I think it is an > overkill with more overhead and complexity than needed. > > But the current situation in which every TLB flush is a basis for long > discussions and prone to bugs is impossible. > > I hope it helps. Let me know if you want me to revive the patch-set or other > feedback. > > [1] https://lore.kernel.org/all/20220711034615.482895-5-21cnbao@gmail.com/ > [2] https://lore.kernel.org/all/20220718120212.3180-13-namit@vmware.com/ > [3] https://lore.kernel.org/all/20210131001132.3368247-16-namit@vmware.com/ I need more reading on tlb code and also [3] which looks useful to me. It's definitely sad to make tlb flushing so complicated. It'll be great if things can be sorted out someday. In this specific case, the only way to do safe tlb batching in my mind is: pte_offset_map_lock(); arch_enter_lazy_mmu_mode(); // If any pending tlb, do it now if (mm_tlb_flush_pending()) flush_tlb_range(vma, start, end); else flush_tlb_batched_pending(); loop { ... pte = ptep_get_and_clear(); ... if (pte_present()) unmapped++; ... } if (unmapped) flush_tlb_range(walk->vma, start, end); arch_leave_lazy_mmu_mode(); pte_unmap_unlock(); I may miss something, but even if not it already doesn't look pretty. Thanks, -- Peter Xu