From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E42D9C4741F for ; Thu, 1 Oct 2020 18:45:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 53BC321481 for ; Thu, 1 Oct 2020 18:45:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="lgJSB/tV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 53BC321481 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nvidia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 958766B005D; Thu, 1 Oct 2020 14:45:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 90ABE6B0062; Thu, 1 Oct 2020 14:45:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F8C28E0001; Thu, 1 Oct 2020 14:45:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0156.hostedemail.com [216.40.44.156]) by kanga.kvack.org (Postfix) with ESMTP id 519D36B005D for ; Thu, 1 Oct 2020 14:45:26 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id E306E1F1A for ; Thu, 1 Oct 2020 18:45:25 +0000 (UTC) X-FDA: 77324234610.10.swing08_2415c0b2719d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id BB02F16A0D2 for ; Thu, 1 Oct 2020 18:45:25 +0000 (UTC) X-HE-Tag: swing08_2415c0b2719d X-Filterd-Recvd-Size: 6511 Received: from hqnvemgate25.nvidia.com (hqnvemgate25.nvidia.com [216.228.121.64]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Thu, 1 Oct 2020 18:45:24 +0000 (UTC) Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate25.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Thu, 01 Oct 2020 11:44:32 -0700 Received: from [10.20.170.18] (10.124.1.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Thu, 1 Oct 2020 18:45:18 +0000 Date: Thu, 1 Oct 2020 13:45:15 -0500 From: Alex Goins X-X-Sender: agoins@agoins-DiGiTS To: =?ISO-8859-15?Q?Christian_K=F6nig?= CC: , , , John Hubbard , Zi Yan Subject: Re: [PATCH RFC 0/1] drm/ttm: Allocate transparent huge pages without clearing __GFP_COMP In-Reply-To: <4566cf03-1c9e-1626-6c92-7b5fa29d6b75@amd.com> Message-ID: References: <20200930221821.13719-1-agoins@nvidia.com> <4566cf03-1c9e-1626-6c92-7b5fa29d6b75@amd.com> User-Agent: Alpine 2.20 (DEB 67 2015-01-07) X-NVConfidentiality: public MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="8323329-272512797-1601577919=:18933" X-Originating-IP: [10.124.1.5] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1601577872; bh=4VSo1s79PcEYekovpQD/6gkaFNEUkYfYmrnwO4ysQQI=; h=Date:From:X-X-Sender:To:CC:Subject:In-Reply-To:Message-ID: References:User-Agent:X-NVConfidentiality:MIME-Version: Content-Type:X-Originating-IP:X-ClientProxiedBy; b=lgJSB/tVznCMIDm1+XsYCy4ZyMFy7XqgHuwHhhwb6m0K//3jxlwXK2+ORzcvUkT3m ZcPv7zcft0i8AOxNR4HEaKBq+mSI7GoeMpw/jTq3N3sDgqTnrWPqBQ4AeFR7NFJ3qO WgIumC901uUs8lAQeX+Nu1WyVhj4TWXBBhwhd/yhs3yjOVeRL1AFafOfwdj+dqaK4+ M/dqFFiBiyqOGsp9RTQMN2yqpjlwF4mmzSgL/qAhCxnNdkd6l5tLW9K5maUBpM7ly8 nNWhhVToBDbk3VT6pdd9IDc+9MqYiKNFe+xLg8dbbA8veoGU7bODloNP1W+NRl8hIA pfhXcdDyrW/FA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --8323329-272512797-1601577919=:18933 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Christian, On Thu, 1 Oct 2020, Christian K=C3=B6nig wrote: > Hi Alex, >=20 > first of all accessing the underlying page of an exported DMA-buf is > illegal! So I'm not 100% sure what you're intentions are here, please > explain further. We have some mapping requirements that I was hoping I could address by ma= pping these pages manually. Are you sure that it's illegal to access the underlying pages of an expor= ted DMA-BUF? There appears to be quite a few usages of this already. See the = usage of drm_prime_sg_to_page_addr_arrays() in vgem, vkms, msm, xen, and etnavi= v. drm_gem_prime_import_dev() uses driver->gem_prime_import_sg_table() when importing a DMA-BUF from another driver, and the listed drivers then extr= act the pages from the given SGT using drm_prime_sg_to_page_addr_arrays(). These = pages can then be mapped and faulted in. See commit af33a9190d02 ('drm/vgem: Enable dmabuf import interfaces'). Af= ter importing the pages from the SGT, vgem can fault them in, taking a refcou= nt with get_page() first. get_page() throws a BUG if the refcount is zero, which = it will hit on each of the 'tail' pages from TTM THP allocations.=20 All of this currently works fine with TTM DMA-BUFs when the kernel is bui= lt with !CONFIG_TRANSPARENT_HUGEPAGE. However, 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' doesn't change how TTM alloc= ates pages. > Then the reason for TTM not using compound pages is that we can't > guarantee that they are mapped as a whole to userspace. >=20 > The result is that the kernel sometimes tried to de-compound them which > created a bunch of problems. >=20 > So yes this is completely intentional. Understood, I figured something like that was the case, so I wanted to ge= t your input first. Do you know what the problems were, exactly? Practical issue= s aside, it seems strange to call something a transparent huge page if it's non-compound. Besides making these pages compound, would it be reasonable to split them= before sharing them, in e.g. amdgpu_dma_buf_map (and in other drivers that use T= TM)? That's where it's supposed to make sure that the shared DMA-BUF is access= ible by the target device. Thanks, Alex > Regards, > Christian. >=20 > Am 01.10.20 um 00:18 schrieb Alex Goins: > > Hi Christian, > >=20 > > I've been looking into the DMA-BUFs exported from AMDGPU / TTM. Would > > you mind giving some input on this? > >=20 > > I noticed that your changes implementing transparent huge page suppor= t > > in TTM are allocating them as non-compound. I understand that using > > multiorder non-compound pages is common in device drivers, but I thin= k > > this can cause a problem when these pages are exported to other drive= rs. > >=20 > > It's possible for other drivers to access the DMA-BUF's pages via > > gem_prime_import_sg_table(), but without context from TTM, it's > > impossible for the importing driver to make sense of them; they simpl= y > > appear as individual pages, with only the first page having a non-zer= o > > refcount. Making TTM's THP allocations compound puts them more in lin= e > > with the standard definition of a THP, and allows DMA-BUF-importing > > drivers to make sense of the pages within. > >=20 > > I would like to propose making these allocations compound, but based = on > > patch history, it looks like the decision to make them non-compound w= as > > intentional, as there were difficulties figuring out how to map them > > into CPU page tables. I did some cursory testing with compound THPs, = and > > nothing seems obviously broken. I was also able to map compound THP > > DMA-BUFs into userspace without issue, and access their contents. Are > > you aware of any other potential consequences? > >=20 > > Commit 5c42c64f7d54 ("drm/ttm: fix the fix for huge compound pages") = should > > probably also be reverted if this is applied. > >=20 > > Thanks, > > Alex > >=20 > > Alex Goins (1): > > drm-ttm: Allocate compound transparent huge pages > >=20 > > drivers/gpu/drm/ttm/ttm_page_alloc.c | 5 ++--- > > 1 file changed, 2 insertions(+), 3 deletions(-) > >=20 >=20 >=20 --8323329-272512797-1601577919=:18933--