From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D012CC4360C for ; Tue, 8 Oct 2019 17:14:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0CDC0206C2 for ; Tue, 8 Oct 2019 17:14:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="ReRkzdzi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0CDC0206C2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B32468E0007; Tue, 8 Oct 2019 13:14:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AE2E98E0003; Tue, 8 Oct 2019 13:14:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F8948E0007; Tue, 8 Oct 2019 13:14:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0047.hostedemail.com [216.40.44.47]) by kanga.kvack.org (Postfix) with ESMTP id 7D1168E0003 for ; Tue, 8 Oct 2019 13:14:49 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 0866D68BF for ; Tue, 8 Oct 2019 17:14:49 +0000 (UTC) X-FDA: 76021267098.07.beef15_8141de18ce440 X-HE-Tag: beef15_8141de18ce440 X-Filterd-Recvd-Size: 7173 Received: from mail-lj1-f195.google.com (mail-lj1-f195.google.com [209.85.208.195]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Tue, 8 Oct 2019 17:14:48 +0000 (UTC) Received: by mail-lj1-f195.google.com with SMTP id y3so18421974ljj.6 for ; Tue, 08 Oct 2019 10:14:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=XJM/CmZg6Z5jzXw4aAwQQ7njeCb11d/hVxoFBeKQhrY=; b=ReRkzdzi+v9g+NGO+NDFI7PGCcrUVdY64ueoH7j4mlWoK/n86aa4CYeKK3a6u/8tBy fYGyPyxGFTm6AWff8+32JPNdkEYWn+z7CJfoNmJULOWfvrjOKYwkyJl1Wkb1GuERxeAl BvLkCTZtIFczxB0eRfSwKT7Q8WRPg5p92N3TA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=XJM/CmZg6Z5jzXw4aAwQQ7njeCb11d/hVxoFBeKQhrY=; b=Mej28DHTgX/7S+yS9zou2AkyvQmp4S6eKYV0YQlC6/OHrfa8CabYJIynMJ7Xc9P/VA A3+UxcdC7DMnjvZIYjFnXBMZLIj32940J3Lybn1uwPGJjskJH6ghj4l1JJGhW7yjz69/ QfblVyXu3BHtBnCUs5NoFGgT6UrE6nQQ37c+FQOqcJoalit7L+7P9Y669s94gtjAT9pV DNC4l6KWgNskwGnIJ9p0o/63c6kwwWRiSQnCKmMwzASfpWh+jvJUBjBExeaX7zxG2+kG MHflReA90XLjVRYD9VpGuPR+8p8Xtl7Ei80qgadwgfsjZ7B2qQYcPXeSl2t5XOlFiff8 G/6A== X-Gm-Message-State: APjAAAV31zYKKLi+aVPxNlS2Tsog33t3vVKIYcylfUzVLLURb1JSBsng A7rtTFIVtiRJlZPfiMt8Z8OQCVKOJnI= X-Google-Smtp-Source: APXvYqxYStcLstx+fFi3cswrECKTNuUumoJNmC/pptpTQ0yWrL7sVPuWDTv3hwrL/z2+lzHgFiqC1Q== X-Received: by 2002:a2e:9a88:: with SMTP id p8mr23176410lji.86.1570554886105; Tue, 08 Oct 2019 10:14:46 -0700 (PDT) Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com. [209.85.167.46]) by smtp.gmail.com with ESMTPSA id 77sm4105480ljj.84.2019.10.08.10.14.45 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 08 Oct 2019 10:14:45 -0700 (PDT) Received: by mail-lf1-f46.google.com with SMTP id r2so12538836lfn.8 for ; Tue, 08 Oct 2019 10:14:45 -0700 (PDT) X-Received: by 2002:a19:7d55:: with SMTP id y82mr20982648lfc.106.1570554422608; Tue, 08 Oct 2019 10:07:02 -0700 (PDT) MIME-Version: 1.0 References: <20191008091508.2682-1-thomas_os@shipmail.org> <20191008091508.2682-6-thomas_os@shipmail.org> In-Reply-To: <20191008091508.2682-6-thomas_os@shipmail.org> From: Linus Torvalds Date: Tue, 8 Oct 2019 10:06:46 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v4 5/9] mm: Add write-protect and clean utilities for address space ranges To: =?UTF-8?Q?Thomas_Hellstr=C3=B6m_=28VMware=29?= Cc: Linux Kernel Mailing List , Linux-MM , Thomas Hellstrom , Andrew Morton , Matthew Wilcox , Will Deacon , Peter Zijlstra , Rik van Riel , Minchan Kim , Michal Hocko , Huang Ying , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , "Kirill A . Shutemov" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Oct 8, 2019 at 2:15 AM Thomas Hellstr=C3=B6m (VMware) wrote: > > Add two utilities to 1) write-protect and 2) clean all ptes pointing into > a range of an address space. The code looks much simpler and easier to understand now, I think, but that also makes some thing more obvious.. > +static int clean_record_pte(pte_t *pte, unsigned long addr, > + unsigned long end, struct mm_walk *walk) > +{ > + struct wp_walk *wpwalk =3D walk->private; > + struct clean_walk *cwalk =3D to_clean_walk(wpwalk); > + pte_t ptent =3D *pte; > + > + if (pte_dirty(ptent)) { > + pgoff_t pgoff =3D ((addr - walk->vma->vm_start) >> PAGE_S= HIFT) + > + walk->vma->vm_pgoff - cwalk->bitmap_pgoff; > + pte_t old_pte =3D ptep_modify_prot_start(walk->vma, addr,= pte); > + > + ptent =3D pte_mkclean(old_pte); > + ptep_modify_prot_commit(walk->vma, addr, pte, old_pte, pt= ent); > + > + wpwalk->total++; > + wpwalk->tlbflush_start =3D min(wpwalk->tlbflush_start, ad= dr); > + wpwalk->tlbflush_end =3D max(wpwalk->tlbflush_end, > + addr + PAGE_SIZE); > + > + __set_bit(pgoff, cwalk->bitmap); The above looks fundamentally racy. You clean the pte in memory, but it remains dirty and writable in the TLB, and you only flush it much later. So now writes can continue to happen to that page, without it necessarily being marked dirty again in the page tables, because all the CPU TLB caches say "it's already dirty". This may be ok - you've moved the diry bit into that bitmap, and you will flush the TLB before then taking action on the bitmap. So you haven't really _lost_ any dirty bits, but it still may be worth a comment. You do have comments about the _other_ issues (ie the whole "If a caller needs to make sure all dirty ptes are picked up and none additional are added..") but you don't actually have comments about the TLB race basically potentially now causing "missing" dirty stuff. And this may actually be a big problem on some architectures. Not x86, and maybe nothing else, but I have this dim memory of some architectures being unhappy due to virtual caches, and writeback when the page table entry says it's read-only and clean. Our normal "clean pages" is very anal about this, and does flush_cache_page(vma, address, pte_pfn(*pte)); entry =3D ptep_clear_flush(vma, address, pte); entry =3D pte_wrprotect(entry); entry =3D pte_mkclean(entry); set_pte_at(vma->vm_mm, address, pte, entry); when it cleans a page, and I note both the cache flush and the "clear_flush()" (which invalidates the pte and does a synchronous TLB flush) and we have magic semantics for that at least on s390 because there are some low-level architecture details wrt flushing TLB entries and modifying them. End result: I think the code is probably ok in practice because you don't mind the slightly fuzzy dirty bit state, and it's _probably_ ok on all architectures that use drm for the TLB invalidation side. But I think it bears a bit of thinking about. Linus