From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EA08C43331 for ; Thu, 2 Apr 2020 15:57:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4AE8220757 for ; Thu, 2 Apr 2020 15:57:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4AE8220757 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D99378E0008; Thu, 2 Apr 2020 11:57:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D4A678E0007; Thu, 2 Apr 2020 11:57:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C87BA8E0008; Thu, 2 Apr 2020 11:57:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0191.hostedemail.com [216.40.44.191]) by kanga.kvack.org (Postfix) with ESMTP id B0D3B8E0007 for ; Thu, 2 Apr 2020 11:57:55 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 733C28248047 for ; Thu, 2 Apr 2020 15:57:55 +0000 (UTC) X-FDA: 76663370910.08.apple49_a62f2dca0b36 X-HE-Tag: apple49_a62f2dca0b36 X-Filterd-Recvd-Size: 4547 Received: from out30-42.freemail.mail.aliyun.com (out30-42.freemail.mail.aliyun.com [115.124.30.42]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Thu, 2 Apr 2020 15:57:50 +0000 (UTC) X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01358;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=12;SR=0;TI=SMTPD_---0TuR6onf_1585842736; Received: from US-143344MP.local(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0TuR6onf_1585842736) by smtp.aliyun-inc.com(127.0.0.1); Thu, 02 Apr 2020 23:52:19 +0800 Subject: Re: [PATCH -V2] /proc/PID/smaps: Add PMD migration entry parsing To: Michal Hocko , "Huang, Ying" Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Zi Yan , Andrea Arcangeli , "Kirill A . Shutemov" , Vlastimil Babka , Alexey Dobriyan , Konstantin Khlebnikov , =?UTF-8?B?Su+/vXLvv71tZSBHbGlzc2U=?= References: <20200402020031.1611223-1-ying.huang@intel.com> <20200402064437.GC22681@dhcp22.suse.cz> <87zhbufjyc.fsf@yhuang-dev.intel.com> <20200402074411.GH22681@dhcp22.suse.cz> From: Yang Shi Message-ID: <30321c02-6dc4-889f-142c-532b47a67f66@linux.alibaba.com> Date: Thu, 2 Apr 2020 08:52:12 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <20200402074411.GH22681@dhcp22.suse.cz> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 4/2/20 12:44 AM, Michal Hocko wrote: > On Thu 02-04-20 15:03:23, Huang, Ying wrote: >> Michal Hocko writes: >> >>> On Thu 02-04-20 10:00:31, Huang, Ying wrote: >>>> From: Huang Ying >>>> >>>> Now, when read /proc/PID/smaps, the PMD migration entry in page table is simply >>>> ignored. To improve the accuracy of /proc/PID/smaps, its parsing and processing >>>> is added. >>>> >>>> Before the patch, for a fully populated 400 MB anonymous VMA, sometimes some THP >>>> pages under migration may be lost as follows. >>> Interesting. How did you reproduce this? >>> [...] >> I run the pmbench in background to eat memory, then run >> `/usr/bin/migratepages` and `cat /proc/PID/smaps` every second. The >> issue can be reproduced within 60 seconds. > Please add that information to the changelog. I was probably too > optimistic about the migration duration because I found it highly > unlikely to be visible. I was clearly wrong here. I believe that depends on the page is shared by how many processes. If it is not shared then it should just take dozens micro seconds in my test FYI. > >>>> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c >>>> index 8d382d4ec067..9c72f9ce2dd8 100644 >>>> --- a/fs/proc/task_mmu.c >>>> +++ b/fs/proc/task_mmu.c >>>> @@ -546,10 +546,19 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr, >>>> struct mem_size_stats *mss = walk->private; >>>> struct vm_area_struct *vma = walk->vma; >>>> bool locked = !!(vma->vm_flags & VM_LOCKED); >>>> - struct page *page; >>>> + struct page *page = NULL; >>>> >>>> - /* FOLL_DUMP will return -EFAULT on huge zero page */ >>>> - page = follow_trans_huge_pmd(vma, addr, pmd, FOLL_DUMP); >>>> + if (pmd_present(*pmd)) { >>>> + /* FOLL_DUMP will return -EFAULT on huge zero page */ >>>> + page = follow_trans_huge_pmd(vma, addr, pmd, FOLL_DUMP); >>>> + } else if (unlikely(thp_migration_supported() && is_swap_pmd(*pmd))) { >>>> + swp_entry_t entry = pmd_to_swp_entry(*pmd); >>>> + >>>> + if (is_migration_entry(entry)) >>>> + page = migration_entry_to_page(entry); >>>> + else >>>> + VM_WARN_ON_ONCE(1); >>> Could you explain why do we need this WARN_ON? I haven't really checked >>> the swap support for THP but cannot we have normal swap pmd entries? >> I have some patches to add the swap pmd entry support, but they haven't >> been merged yet. >> >> Similar checks are for all THP migration code paths, so I follow the >> same style. > I haven't checked other migration code paths but what is the reason to > add the warning here? Even if this shouldn't happen, smaps is perfectly > fine to ignore that situation, no?