From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72B32C4BA1E for ; Wed, 26 Feb 2020 18:01:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3151620732 for ; Wed, 26 Feb 2020 18:01:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3151620732 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CFF056B000D; Wed, 26 Feb 2020 13:01:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CB06F6B000E; Wed, 26 Feb 2020 13:01:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC63B6B0010; Wed, 26 Feb 2020 13:01:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0122.hostedemail.com [216.40.44.122]) by kanga.kvack.org (Postfix) with ESMTP id A23346B000D for ; Wed, 26 Feb 2020 13:01:06 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 721B14DA3 for ; Wed, 26 Feb 2020 18:01:06 +0000 (UTC) X-FDA: 76533044532.13.time79_97e72f7e9135 X-HE-Tag: time79_97e72f7e9135 X-Filterd-Recvd-Size: 8051 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Wed, 26 Feb 2020 18:01:04 +0000 (UTC) X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04426;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=9;SR=0;TI=SMTPD_---0Tr-Vlmd_1582740057; Received: from US-143344MP.local(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0Tr-Vlmd_1582740057) by smtp.aliyun-inc.com(127.0.0.1); Thu, 27 Feb 2020 02:01:01 +0800 Subject: Re: [v2 PATCH] mm: shmem: allow split THP when truncating THP partially To: David Hildenbrand , Alexander Duyck Cc: "Michael S. Tsirkin" , Hugh Dickins , "Kirill A. Shutemov" , Andrea Arcangeli , Andrew Morton , linux-mm , LKML References: <1575420174-19171-1-git-send-email-yang.shi@linux.alibaba.com> <9b8ff9ca-75b0-c256-cf37-885ccd786de7@linux.alibaba.com> <9c30a891-011b-e041-2647-444d09fa7b8a@linux.alibaba.com> <84492260-726c-dda1-3ec8-b445fe269cad@redhat.com> From: Yang Shi Message-ID: <00338ec9-76fc-bc0e-39af-495ff3b5c7e9@linux.alibaba.com> Date: Wed, 26 Feb 2020 10:00:55 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <84492260-726c-dda1-3ec8-b445fe269cad@redhat.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2/26/20 9:45 AM, David Hildenbrand wrote: > On 26.02.20 18:31, Yang Shi wrote: >> >> On 2/21/20 4:24 PM, Alexander Duyck wrote: >>> On Fri, Feb 21, 2020 at 10:24 AM Yang Shi wrote: >>>> >>>> On 2/20/20 10:16 AM, Alexander Duyck wrote: >>>>> On Tue, Dec 3, 2019 at 4:43 PM Yang Shi wrote: >>>>>> Currently when truncating shmem file, if the range is partial of THP >>>>>> (start or end is in the middle of THP), the pages actually will just get >>>>>> cleared rather than being freed unless the range cover the whole THP. >>>>>> Even though all the subpages are truncated (randomly or sequentially), >>>>>> the THP may still be kept in page cache. This might be fine for some >>>>>> usecases which prefer preserving THP. >>>>>> >>>>>> But, when doing balloon inflation in QEMU, QEMU actually does hole punch >>>>>> or MADV_DONTNEED in base page size granulairty if hugetlbfs is not used. >>>>>> So, when using shmem THP as memory backend QEMU inflation actually doesn't >>>>>> work as expected since it doesn't free memory. But, the inflation >>>>>> usecase really needs get the memory freed. Anonymous THP will not get >>>>>> freed right away too but it will be freed eventually when all subpages are >>>>>> unmapped, but shmem THP would still stay in page cache. >>>>>> >>>>>> Split THP right away when doing partial hole punch, and if split fails >>>>>> just clear the page so that read to the hole punched area would return >>>>>> zero. >>>>>> >>>>>> Cc: Hugh Dickins >>>>>> Cc: Kirill A. Shutemov >>>>>> Cc: Andrea Arcangeli >>>>>> Signed-off-by: Yang Shi >>>>> One question I would have is if this is really the desired behavior we >>>>> are looking for? >>>>> >>>>> By proactively splitting the THP you are likely going to see a >>>>> performance regression with the virtio-balloon driver enabled in QEMU. >>>>> I would suspect the response to that would be to update the QEMU code >>>>> to identify the page size of the shared memory ramblock. At that >>>>> point I suspect it would start behaving the same as how it currently >>>>> handles anonymous memory, and the work done here would essentially >>>>> have been wasted other than triggering the desire to resolve this in >>>>> QEMU to avoid a performance regression. >>>>> >>>>> The code for inflating a the balloon in virtio-balloon in QEMU can be >>>>> found here: >>>>> https://github.com/qemu/qemu/blob/master/hw/virtio/virtio-balloon.c#L66 >>>>> >>>>> If there is a way for us to just populate the value obtained via >>>>> qemu_ram_pagesize with the THP page size instead of leaving it at 4K, >>>>> which is the size I am assuming it is at since you indicated that it >>>>> is just freeing the base page size, then we could address the same >>>>> issue and likely get the desired outcome of freeing the entire THP >>>>> page when it is no longer used. >>>> If qemu could punch hole (this is how qemu free file-backed memory) in >>>> THP unit, either w/ or w/o the patch the THP won't get split since the >>>> whole THP will get truncated. But, if qemu has to free memory in sub-THP >>>> size due to whatever reason (for example, 1MB for every 2MB section), >>>> then we have to split THP otherwise no memory will be freed actually >>>> with the current code. It is not about performance, it is about really >>>> giving memory back to host. >>> I get that, but at the same time I am not sure if everyone will be >>> happy with the trade-off. That is my concern. >>> >>> You may want to change the patch description above if that is the >>> case. Based on the description above it makes it sound as if the issue >>> is that QEMU is using hole punch or MADV_DONTNEED with the wrong >>> granularity. Based on your comment here it sounds like you want to >>> have the ability to break up the larger THP page as soon as you want >>> to push out a single 4K page from it. >> Yes, you are right. The commit log may be confusing. What I wanted to >> convey is QEMU has no idea if THP is used or not so it treats memory >> with base size unless hugetlbfs is used since QEMU is aware huge page is >> used in this case. >> This may sounds irrelevant to the problem, I would just remove that. >> >>> I am not sure the description for the behavior of anonymous THP with >>> respect to QEMU makes sense either. Based on the description you made >>> it sound like it was somehow using the same process used for huge >>> pages. That isn't the case right? My understanding is that in the case >>> of an anonymous THP it is getting broken into 4K subpages and then >>> those are freed individually. That should leave you with the same >>> performance regression that I had brought up earlier. >> No, anonymous THP won't get split immediately and those memory also >> won't get freed immediately if QEMU does MADV_DONTNEED on sub THP range >> (for example, 1MB range in THP). The THP will get freed when: >> 1. Host has memory pressure. The THP will get split and unmapped pages >> will be freed. >> 2. Other sub pages in the same THP are MADV_DONTNEED'ed (eventually the >> whole THP get unmapped). >> >> The difference between shmem and anonymous page is shmem will not get >> freed unless hole punch the whole THP, anonymous page will get freed >> sooner or later. >> > As far as I understood Hugh, the "page size" we'll see in QEMU via > fstatfs() is 4k, not 2MB. IMHO, that's the block size of the "device", > and breaking up THP is the right think to to obey the documentation of > "FALLOC_FL_PUNCH_HOLE". This is what the patch attempts to accomplish. > > IMHO THP is called "transparent" because it shouldn't have any such > visible side effects. AFAICT, the lazy split is due to locking issue in partial unmap paths. Please refer to "Partial unmap and deferred_split_huge_page()" section in Documentation/vm/transhuge.rst. > > As always, anybody correct me if I am wrong here. >