From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4E70C4363D for ; Fri, 2 Oct 2020 06:41:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2AFFD206E3 for ; Fri, 2 Oct 2020 06:41:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="dYcz01kK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2AFFD206E3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8BBF96B006E; Fri, 2 Oct 2020 02:41:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 86D9B6B0070; Fri, 2 Oct 2020 02:41:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6C2C66B0071; Fri, 2 Oct 2020 02:41:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0060.hostedemail.com [216.40.44.60]) by kanga.kvack.org (Postfix) with ESMTP id 415186B006E for ; Fri, 2 Oct 2020 02:41:30 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DABAE1EFD for ; Fri, 2 Oct 2020 06:41:29 +0000 (UTC) X-FDA: 77326039098.04.skirt04_0d12bf0271a2 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id C34D3800887C for ; Fri, 2 Oct 2020 06:41:29 +0000 (UTC) X-HE-Tag: skirt04_0d12bf0271a2 X-Filterd-Recvd-Size: 11554 Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-eopbgr760088.outbound.protection.outlook.com [40.107.76.88]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Fri, 2 Oct 2020 06:41:29 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CxgG9ABAyVaH0BorybXBcqWiQW4WKBHs+z/GnP9kLLMUhvStzDBhoReKwuKdyYj4hd9pS3hQt0n08IuJ7+uVEbBEGwSqfJEEJgIWfmJDu55aFt44Ml41i3ZExcP02EJKqNo1IuJ70oMcKuBqeMwIfPOq1/pDYRxxfXwSkGZwk2OVaPrdSgrIF3xd5fMQvJPkb0eGksfKORxFuEfSNfRW2p4CXMNJPiuyjZ5fEZDAFLNiDOWJryu2sT6YTGOOe5O0fBWtR5PhyGKMkwwsH2ezcf1VVgz472GusU6yViCk+voPqMRszv2FMNJw3UHozAC34z70oGKIVRg1uFbIW+26VQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZMIUvTNbAA85kJyF8p7k66fVrQGDnd2MDAB/zeyblAQ=; b=PFMYEL8rHEQxwJp/7bUOYa01g3A1jtekKE7bxH4ESmoUWvI4ygYBDBrPu6FClFkgYW7mv8Dp4AfCkXITERMyV0Ak7pl8/uMLto+dTwM26UtiyI4xHezppw+1drQhIDbAJ9k9KMfMKy4MYeuFHs98PtKrIQsOYnAfUGidAsrlgT5eUkbBzPMwe7YbduJBQBvggd/oRjAjohPTVut5TBmeblzh9reJrn1TFNoQfn8EQ0JUEpDx5FeDs5wdw2GDJh9eDugHrYCBzPDkctBdIXtrMZ49AoP6naAfUZn4tgxpmqxmvbzts+z/brDGxRXAJF+WfgPbj8/CP+sNszW4g3oQEQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZMIUvTNbAA85kJyF8p7k66fVrQGDnd2MDAB/zeyblAQ=; b=dYcz01kKVEYDKJN6pnIUi56WbgTyKMBQDHfbXxMiSSWZ8lDV/8mrvv1oPGTlmWcEUJ2dTucGX+0LMRcars2rAqlj9nH19/eBF1tivMYpBd8eIFxY3QMLTGRHlrg7jLUxRuQ4WuNKP36WmN7fiDl/mkYsEPMR6QhJHFHFji8Xa/I= Authentication-Results: nvidia.com; dkim=none (message not signed) header.d=none;nvidia.com; dmarc=none action=none header.from=amd.com; Received: from MN2PR12MB3775.namprd12.prod.outlook.com (2603:10b6:208:159::19) by MN2PR12MB4815.namprd12.prod.outlook.com (2603:10b6:208:1bf::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3433.32; Fri, 2 Oct 2020 06:41:22 +0000 Received: from MN2PR12MB3775.namprd12.prod.outlook.com ([fe80::f8f7:7403:1c92:3a60]) by MN2PR12MB3775.namprd12.prod.outlook.com ([fe80::f8f7:7403:1c92:3a60%6]) with mapi id 15.20.3433.032; Fri, 2 Oct 2020 06:41:22 +0000 Subject: Re: [PATCH RFC 0/1] drm/ttm: Allocate transparent huge pages without clearing __GFP_COMP To: Alex Goins , Daniel Vetter Cc: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, linux-mm@kvack.org, John Hubbard , Zi Yan References: <20200930221821.13719-1-agoins@nvidia.com> <4566cf03-1c9e-1626-6c92-7b5fa29d6b75@amd.com> From: =?UTF-8?Q?Christian_K=c3=b6nig?= Message-ID: Date: Fri, 2 Oct 2020 08:41:15 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US X-Originating-IP: [2a02:908:1252:fb60:be8a:bd56:1f94:86e7] X-ClientProxiedBy: AM3PR05CA0094.eurprd05.prod.outlook.com (2603:10a6:207:1::20) To MN2PR12MB3775.namprd12.prod.outlook.com (2603:10b6:208:159::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [IPv6:2a02:908:1252:fb60:be8a:bd56:1f94:86e7] (2a02:908:1252:fb60:be8a:bd56:1f94:86e7) by AM3PR05CA0094.eurprd05.prod.outlook.com (2603:10a6:207:1::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.22 via Frontend Transport; Fri, 2 Oct 2020 06:41:20 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: d17e1cb2-84b1-4487-e09e-08d8669e2cde X-MS-TrafficTypeDiagnostic: MN2PR12MB4815: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Of2TGoXawIQ2jhPCIK/01X99G0D9OIrAxsMwCjNZRIQcdZchGBBFmRBIuhnciia5Psp2rwsrBRoNr/NhICR1j3CZ2pj2D77Pdb9cTlnDRq7baqzbB3R/W+m0YYmCBoyFlE2AOWNjrM8i48XgEQ0sBairEiu724tW5H5Ie0aBDioztTV6D5p/5zvqzVm2Sl18GQQz3w3oKViIT1KyS61ukRWfuRfCmcGY9Lm+Q0MkE67FtGJPr2GMUXlAdiIsG917xqVrwzJ5n5u8Kn5IM/ajewuCjmOIKonoKtJSd3B8dOX8sCB5v1zAMcHn0J78+eeMbvdHv8DnmMO6NZglbzaIa9mcPvQxhMhuXY9bGrJdYimS+yuC1yaYU/to4HU8KQmRAG90BM3VOJspUjLFo24FeA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB3775.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(396003)(136003)(376002)(366004)(346002)(6486002)(66574015)(83380400001)(186003)(16526019)(478600001)(8936002)(36756003)(86362001)(31696002)(316002)(2616005)(8676002)(31686004)(110136005)(54906003)(52116002)(4326008)(2906002)(5660300002)(6666004)(66476007)(66946007)(66556008)(14583001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: lLAKk3XMv4/EumvcrWIFE89nR+eNOdZi73Q3cEVBUeX1v308ClMDcTAhad8R5gOsSfRoeLngs+D1qcId2yLUI7w7S+c8LhFXjkAHPoxnsrW59Zrt20VqfucYCKlQlYyEDU6m43qO48t3OQ+DwD6x8lBW0ToQ5AhS0b22jhsKo0bSykN0J7Pc/lLCMJ+TLe9DmDulyGIpthCtrM0ZpFVmg8L97IevxCjL1yktKrk70DHZTh5j7uThvK6aHnS1pH95pjtg5Xwtxz0kZSDYeUkqi26O1GywXUdrnBTe/cTNagL43aaaCu77hV/NeGzG2UlPBcaUdQpubEuI3X30GDS3ahvVL+M6gxOwztkE6ryST2K2YG6Z94YRtS+uTBAaKgb+bZddw3VBQNJQDi+w7PyJ/7nsQaVdkpuVIy0OmNHqCAyMYF2k45DY6QmcbpDINqM7DiX5YWqYia/b6oECQMFPXncBm4qukXL9PZxk2UcS9ug9j9igsgyBMpQ5ZhAnOAc+w39TmCHirQ3CDZ2PiOpL36AyrFICTZ1mTT4Nw/E/IZm8Y68wqsUGyGFDujJzGp3ARtQ8uebfaal2dulsSSjGDS0kPLoF1CPU2lDv6Ww7MeUM/LE45Mg3yoBfdAmt2EAdFw5eeyuPSHsdAKSJSxrpV83iuN2VocnOJ7slmevgnBEhXdrTtmkevF6DgnCj26xj6iCrOKdKLppOksbinsqRvA== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: d17e1cb2-84b1-4487-e09e-08d8669e2cde X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB3775.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Oct 2020 06:41:21.9313 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fdDEPUHqOtZKnwKR4DNDhhcfkiZMh1pQpVQfi8UKYDrsvhPo0iJitFn2HYg+62J5 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4815 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Alex, adding Daniel as well. Am 01.10.20 um 20:45 schrieb Alex Goins: > Hi Christian, > > On Thu, 1 Oct 2020, Christian K=C3=B6nig wrote: > >> Hi Alex, >> >> first of all accessing the underlying page of an exported DMA-buf is >> illegal! So I'm not 100% sure what you're intentions are here, please >> explain further. > We have some mapping requirements that I was hoping I could address by = mapping > these pages manually. > > Are you sure that it's illegal to access the underlying pages of an exp= orted > DMA-BUF? yes, I'm 100% sure of that. This was discussed multiple times now on the=20 mailing list. > There appears to be quite a few usages of this already. See the usage > of drm_prime_sg_to_page_addr_arrays() in vgem, vkms, msm, xen, and etna= viv. > drm_gem_prime_import_dev() uses driver->gem_prime_import_sg_table() whe= n > importing a DMA-BUF from another driver, and the listed drivers then ex= tract the > pages from the given SGT using drm_prime_sg_to_page_addr_arrays(). Thes= e pages > can then be mapped and faulted in. No, exactly that doesn't work correctly. You are corrupting internal state in struct page while doing so and risk=20 that userspace is accessing freed up memory. We really need to find a way to fix the few drivers already doing this. > See commit af33a9190d02 ('drm/vgem: Enable dmabuf import interfaces'). = After > importing the pages from the SGT, vgem can fault them in, taking a refc= ount with > get_page() first. get_page() throws a BUG if the refcount is zero, whic= h it will > hit on each of the 'tail' pages from TTM THP allocations. > > All of this currently works fine with TTM DMA-BUFs when the kernel is b= uilt with > !CONFIG_TRANSPARENT_HUGEPAGE. However, 'echo never > > /sys/kernel/mm/transparent_hugepage/enabled' doesn't change how TTM all= ocates > pages. You need to redirect the mapping to dma_buf_mmap() instead. Regards, Christian. > >> Then the reason for TTM not using compound pages is that we can't >> guarantee that they are mapped as a whole to userspace. >> >> The result is that the kernel sometimes tried to de-compound them whic= h >> created a bunch of problems. >> >> So yes this is completely intentional. > Understood, I figured something like that was the case, so I wanted to = get your > input first. Do you know what the problems were, exactly? Practical iss= ues > aside, it seems strange to call something a transparent huge page if it= 's > non-compound. > > Besides making these pages compound, would it be reasonable to split th= em before > sharing them, in e.g. amdgpu_dma_buf_map (and in other drivers that use= TTM)? > That's where it's supposed to make sure that the shared DMA-BUF is acce= ssible by > the target device. > > Thanks, > Alex > >> Regards, >> Christian. >> >> Am 01.10.20 um 00:18 schrieb Alex Goins: >>> Hi Christian, >>> >>> I've been looking into the DMA-BUFs exported from AMDGPU / TTM. Would >>> you mind giving some input on this? >>> >>> I noticed that your changes implementing transparent huge page suppor= t >>> in TTM are allocating them as non-compound. I understand that using >>> multiorder non-compound pages is common in device drivers, but I thin= k >>> this can cause a problem when these pages are exported to other drive= rs. >>> >>> It's possible for other drivers to access the DMA-BUF's pages via >>> gem_prime_import_sg_table(), but without context from TTM, it's >>> impossible for the importing driver to make sense of them; they simpl= y >>> appear as individual pages, with only the first page having a non-zer= o >>> refcount. Making TTM's THP allocations compound puts them more in lin= e >>> with the standard definition of a THP, and allows DMA-BUF-importing >>> drivers to make sense of the pages within. >>> >>> I would like to propose making these allocations compound, but based = on >>> patch history, it looks like the decision to make them non-compound w= as >>> intentional, as there were difficulties figuring out how to map them >>> into CPU page tables. I did some cursory testing with compound THPs, = and >>> nothing seems obviously broken. I was also able to map compound THP >>> DMA-BUFs into userspace without issue, and access their contents. Are >>> you aware of any other potential consequences? >>> >>> Commit 5c42c64f7d54 ("drm/ttm: fix the fix for huge compound pages") = should >>> probably also be reverted if this is applied. >>> >>> Thanks, >>> Alex >>> >>> Alex Goins (1): >>> drm-ttm: Allocate compound transparent huge pages >>> >>> drivers/gpu/drm/ttm/ttm_page_alloc.c | 5 ++--- >>> 1 file changed, 2 insertions(+), 3 deletions(-) >>>