From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8859C07E9C for ; Wed, 14 Jul 2021 16:02:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 976B861370 for ; Wed, 14 Jul 2021 16:02:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 976B861370 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B50016B0075; Wed, 14 Jul 2021 12:02:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AFF4B6B0078; Wed, 14 Jul 2021 12:02:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 978A36B0081; Wed, 14 Jul 2021 12:02:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0031.hostedemail.com [216.40.44.31]) by kanga.kvack.org (Postfix) with ESMTP id 6D7956B0075 for ; Wed, 14 Jul 2021 12:02:06 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 4F418185DCA71 for ; Wed, 14 Jul 2021 16:02:05 +0000 (UTC) X-FDA: 78361659810.28.86EFC70 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf12.hostedemail.com (Postfix) with ESMTP id EEA281000F83 for ; Wed, 14 Jul 2021 16:02:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626278524; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=iDdwf1XmUeNAele69YuflW7I8JQTNLYYUbHmho8CJjk=; b=DKrs4HEqOo975A+FhE5W4ahJ9PHRlgEUOkoVh54op3k3wg4B5RayVVM4xEDfdE2QPbnrhZ 5ujDZ4HnBpeneXBaOGJONOJBNbCS+WEYOXpDhIy/wmughrXGI695YJlZkYOdSlK1vi7P9b vKN0U230QchlyvQaLIEBoRmVaxWSn3Q= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-232-q_DL4fiEM7K2xLxbFkV_Xg-1; Wed, 14 Jul 2021 12:02:03 -0400 X-MC-Unique: q_DL4fiEM7K2xLxbFkV_Xg-1 Received: by mail-qk1-f197.google.com with SMTP id f203-20020a379cd40000b02903b861bec838so1557276qke.7 for ; Wed, 14 Jul 2021 09:02:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=iDdwf1XmUeNAele69YuflW7I8JQTNLYYUbHmho8CJjk=; b=kBmIkxBitG/PkvnhuwZu0PuYiT4JnOSG0Va68l2MSjtoMUp0Dfxh66z+lMXrozc23N 03LiO871KkveTCr1N45TIOIjLnEGnDFsO324XYOxm1l5JGqYtkr1WcDsGHp+h40IYYzI +2bkdyQORMRriVPWLaNXVyxebUyYFaAvqkPNWX6FWQRbsS1onzChWwehQ3F8OGmk+Kpv 11i5D1o97MC9kvk41RxD1aJQwNrjPVXD4+Je2bUnFFrMXBEq0rki9f8Ac5qd1RzEdY9/ cxxXb1KuqshHX2wjsdIVAoJ/2OvTVvuzWZPRWUsSC6iZaBUPpFaTbWgg8XZbX//3rvOg E7gQ== X-Gm-Message-State: AOAM533YURqUUk54VWL4jkp1v7RuQIz5kRrf0HB6w4ygauE6O5Su1zZq 5L5kGAxlPdNYBPMKm0JrcRB7UrvPHXv4iehLDONWCcC2fQQXc69NakKVwoenvhCAs26OW9s0EsT ajORIEL9d85M= X-Received: by 2002:a05:622a:d1:: with SMTP id p17mr9872836qtw.141.1626278519914; Wed, 14 Jul 2021 09:01:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxmBx7ehtHLFv1+6C/w89oUyoBrHdc2FU7lSB7HgqiXueL1nKb1EkEmByVFXTV9sw1ZSVVFYA== X-Received: by 2002:a05:622a:d1:: with SMTP id p17mr9872799qtw.141.1626278519654; Wed, 14 Jul 2021 09:01:59 -0700 (PDT) Received: from t490s (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id y4sm1227470qkc.27.2021.07.14.09.01.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Jul 2021 09:01:58 -0700 (PDT) Date: Wed, 14 Jul 2021 12:01:57 -0400 From: Peter Xu To: Tiberiu Georgescu Cc: akpm@linux-foundation.org, catalin.marinas@arm.com, peterz@infradead.org, chinwen.chang@mediatek.com, linmiaohe@huawei.com, jannh@google.com, apopple@nvidia.com, christian.brauner@ubuntu.com, ebiederm@xmission.com, adobriyan@gmail.com, songmuchun@bytedance.com, axboe@kernel.dk, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, ivan.teterevkov@nutanix.com, florian.schmidt@nutanix.com, carl.waldspurger@nutanix.com, Hugh Dickins , Andrea Arcangeli Subject: Re: [RFC PATCH 0/1] pagemap: report swap location for shared pages Message-ID: References: <20210714152426.216217-1-tiberiu.georgescu@nutanix.com> MIME-Version: 1.0 In-Reply-To: <20210714152426.216217-1-tiberiu.georgescu@nutanix.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: EEA281000F83 X-Stat-Signature: cr316duyh35d3q3y3fqpo7q8p4a4thg3 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DKrs4HEq; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf12.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com X-HE-Tag: 1626278524-682108 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jul 14, 2021 at 03:24:25PM +0000, Tiberiu Georgescu wrote: > When a page allocated using the MAP_SHARED flag is swapped out, its pagemap > entry is cleared. In many cases, there is no difference between swapped-out > shared pages and newly allocated, non-dirty pages in the pagemap interface. > > Example pagemap-test code (Tested on Kernel Version 5.14-rc1): > > #define NPAGES (256) > /* map 1MiB shared memory */ > size_t pagesize = getpagesize(); > char *p = mmap(NULL, pagesize * NPAGES, PROT_READ | PROT_WRITE, > MAP_ANONYMOUS | MAP_SHARED, -1, 0); > /* Dirty new pages. */ > for (i = 0; i < PAGES; i++) > p[i * pagesize] = i; > > Run the above program in a small cgroup, which allows swapping: > > /* Initialise cgroup & run a program */ > $ echo 512K > foo/memory.limit_in_bytes > $ echo 60 > foo/memory.swappiness > $ cgexec -g memory:foo ./pagemap-test > > Check the pagemap report. This is an example of the current expected output: > > $ dd if=/proc/$PID/pagemap ibs=8 skip=$(($VADDR / $PAGESIZE)) count=$COUNT | hexdump -C > 00000000 00 00 00 00 00 00 80 00 00 00 00 00 00 00 80 00 |................| > * > 00000710 e1 6b 06 00 00 00 80 a1 9e eb 06 00 00 00 80 a1 |.k..............| > 00000720 6b ee 06 00 00 00 80 a1 a5 a4 05 00 00 00 80 a1 |k...............| > 00000730 5c bf 06 00 00 00 80 a1 90 b6 06 00 00 00 80 a1 |\...............| > > The first pagemap entries are reported as zeroes, indicating the pages have > never been allocated while they have actually been swapped out. It is > possible for bit 55 (PTE is Soft-Dirty) to be set on all pages of the > shared VMA, indicating some access to the page, but nothing else (frame > location, presence in swap or otherwise). > > This patch addresses the behaviour and modifies pte_to_pagemap_entry() to > make use of the XArray associated with the virtual memory area struct > passed as an argument. The XArray contains the location of virtual pages in > the page cache, swap cache or on disk. If they are on either of the caches, > then the original implementation still works. If not, then the missing > information will be retrieved from the XArray. > > The root cause of the missing functionality is that the PTE for the page > itself is cleared when a swap out occurs on a shared page. Please take a > look at the proposed patch. I would appreciate it if you could verify a > couple of points: > > 1. Why do swappable and non-syncable shared pages have their PTEs cleared > when they are swapped out ? Why does the behaviour differ so much > between MAP_SHARED and MAP_PRIVATE pages? What are the origins of the > approach? My understanding is linux mm treat this differently for file-backed memories, MAP_SHARED is one of this kind. For these memories, ptes can be dropped at any time because it can be reloaded from page cache when faulted again. Anonymous private memories cannot do that, so anonymous private memories keep all things within ptes, including swap entry. > > 2. PM_SOFT_DIRTY and PM_UFFD_WP are two flags that seem to get lost once > the shared page is swapped out. Is there any other way to retrieve > their value in the proposed patch, other than ensuring these flags are > set, when necessary, in the PTE? uffd-wp has no problem on dropping them because uffd-wp does not yet support shmem. Shmem support is posted upstream but still during review: https://lore.kernel.org/lkml/20210527201927.29586-1-peterx@redhat.com/ After that work they'll persist, then we won't have an issue using uffd-wp with shmem swapping; the pagemap part is done in patch 25 of 27: https://lore.kernel.org/lkml/20210527202340.32306-1-peterx@redhat.com/ However I agree soft-dirty seems to be still broken with it. (Cc Hugh and Andrea too) Thanks, -- Peter Xu