From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6790BC433E1 for ; Mon, 22 Jun 2020 23:02:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2C0D420738 for ; Mon, 22 Jun 2020 23:02:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="N01y1zIP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2C0D420738 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nvidia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9AB856B0002; Mon, 22 Jun 2020 19:02:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 95AFB6B0003; Mon, 22 Jun 2020 19:02:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8703D6B0005; Mon, 22 Jun 2020 19:02:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0204.hostedemail.com [216.40.44.204]) by kanga.kvack.org (Postfix) with ESMTP id 6DE0F6B0002 for ; Mon, 22 Jun 2020 19:02:09 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A7924180AD815 for ; Mon, 22 Jun 2020 23:02:08 +0000 (UTC) X-FDA: 76958372736.17.pet94_4510fe826e36 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id 5EC3E180D0185 for ; Mon, 22 Jun 2020 23:02:08 +0000 (UTC) X-HE-Tag: pet94_4510fe826e36 X-Filterd-Recvd-Size: 5487 Received: from hqnvemgate24.nvidia.com (hqnvemgate24.nvidia.com [216.228.121.143]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Mon, 22 Jun 2020 23:02:07 +0000 (UTC) Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate24.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Mon, 22 Jun 2020 16:00:35 -0700 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate101.nvidia.com (PGP Universal service); Mon, 22 Jun 2020 16:02:06 -0700 X-PGP-Universal: processed; by hqpgpgate101.nvidia.com on Mon, 22 Jun 2020 16:02:06 -0700 Received: from [10.2.59.206] (10.124.1.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 22 Jun 2020 23:01:51 +0000 Subject: Re: [PATCH 13/16] mm: support THP migration to device private memory To: Yang Shi , Zi Yan CC: Ralph Campbell , , , Linux MM , , Linux Kernel Mailing List , Jerome Glisse , Christoph Hellwig , Jason Gunthorpe , "Ben Skeggs" , Andrew Morton , "Shuah Khan" , "Huang, Ying" References: <20200619215649.32297-1-rcampbell@nvidia.com> <20200619215649.32297-14-rcampbell@nvidia.com> <4C364E23-0716-4D59-85A1-0C293B86BC2C@nvidia.com> From: John Hubbard Message-ID: Date: Mon, 22 Jun 2020 16:01:50 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: X-Originating-IP: [10.124.1.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) Content-Type: text/plain; charset="utf-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1592866835; bh=Y6NBr0D1yM6UcnMk8vnlqkMWMVjsve14PP2JOfM1rUE=; h=X-PGP-Universal:Subject:To:CC:References:From:Message-ID:Date: User-Agent:MIME-Version:In-Reply-To:X-Originating-IP: X-ClientProxiedBy:Content-Type:Content-Language: Content-Transfer-Encoding; b=N01y1zIPkJH/Lms5VPCw+1hdpKCS6PQZaY+D44RrVIXwNweSuXmYr3z3AfacpMrRd G6+/fqzWMSPGkgXP5CHVQpYsuLjMkY+7UKSYaWnH1EkZLLsUN4HnVHQvpRTB4ETFdv mhmHUu3UNo7z2CHs8x9IfXei5XwR3Y9Zs7z/4nGOSZHwHXZyStlyAfAisuSqTF6kJk 6gCLh23dt5mh+yBb98fGlamGj9DI3thkQ6Su+m8vH3eU9D0J67Zac+QkklzP31wVYD lHOctZ9rMoZOScVyrBhik3PE1youRWx4x2Ase8LtWQUNfGk29QlDgJu+UXoKCypz1b XQu2ultXjtJaw== X-Rspamd-Queue-Id: 5EC3E180D0185 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2020-06-22 15:33, Yang Shi wrote: > On Mon, Jun 22, 2020 at 3:30 PM Yang Shi wrote: >> On Mon, Jun 22, 2020 at 2:53 PM Zi Yan wrote: >>> On 22 Jun 2020, at 17:31, Ralph Campbell wrote: >>>> On 6/22/20 1:10 PM, Zi Yan wrote: >>>>> On 22 Jun 2020, at 15:36, Ralph Campbell wrote: >>>>>> On 6/21/20 4:20 PM, Zi Yan wrote: >>>>>>> On 19 Jun 2020, at 17:56, Ralph Campbell wrote: ... >>> Ying(cc=E2=80=99d) developed the code to swapout and swapin THP in one = piece: https://lore.kernel.org/linux-mm/20181207054122.27822-1-ying.huang@i= ntel.com/. >>> I am not sure whether the patchset makes into mainstream or not. It cou= ld be a good technical reference >>> for swapping in device private pages, although swapping in pages from d= isk and from device private >>> memory are two different scenarios. >>> >>> Since the device private memory swapin impacts core mm performance, we = might want to discuss your patches >>> with more people, like the ones from Ying=E2=80=99s patchset, in the ne= xt version. >> >> I believe Ying will give you more insights about how THP swap works. >> >> But, IMHO device memory migration (migrate to system memory) seems >> like THP CoW more than swap. A fine point: overall, the desired behavior is "migrate", not CoW. That's important. Migrate means that you don't leave a page behind, even a read-only one. And that's exactly how device private migration is specified. We should try to avoid any erosion of clarity here. Even if somehow (really?) the underlying implementation calls this THP CoW, the actual goal is to migrate pages over to the device (and back). >> >> When migrating in: >=20 > Sorry for my fat finger, hit sent button inadvertently, let me finish her= e. >=20 > When migrating in: >=20 > - if THP is enabled: allocate THP, but need handle allocation > failure by falling back to base page > - if THP is disabled: fallback to base page >=20 OK, but *all* page entries (base and huge/large pages) need to be cleared, when migrating to device memory, unless I'm really confused here. So: not CoW. thanks, --=20 John Hubbard NVIDIA