From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 684A0C433E0 for ; Mon, 4 Jan 2021 20:00:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DC7312085B for ; Mon, 4 Jan 2021 20:00:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DC7312085B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0D9208D0026; Mon, 4 Jan 2021 15:00:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0885A8D001C; Mon, 4 Jan 2021 15:00:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6B7A8D0026; Mon, 4 Jan 2021 15:00:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0245.hostedemail.com [216.40.44.245]) by kanga.kvack.org (Postfix) with ESMTP id 9FA578D001C for ; Mon, 4 Jan 2021 15:00:48 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 60FAA1EE6 for ; Mon, 4 Jan 2021 20:00:48 +0000 (UTC) X-FDA: 77669160576.27.milk48_1217de9274d3 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 43D603D668 for ; Mon, 4 Jan 2021 20:00:48 +0000 (UTC) X-HE-Tag: milk48_1217de9274d3 X-Filterd-Recvd-Size: 9959 Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-eopbgr750047.outbound.protection.outlook.com [40.107.75.47]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Mon, 4 Jan 2021 20:00:47 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CqMdtX3UdqQr4VQHjKiSO6IIHnYtOyCDuHCzzYxHBLUEe5wWOjOAc6xCcYe7vfmp7eTIVCMWj1yKl96hsr9SjnQc7Ud+6tsYJb6OmH57TwilOiHcOegFKTRaAZkqInxpkNDqFXpfnyFqsh5WIyvN1LxXxxfJflCgYI6uXc1DhBH4y3lSjdyetKz5G41aPNWqFGq4axWaCQON3cgYl0Y3GkbZEuBluNE0LfP83z1495BKSbkj0rXXduGm7QYhE1iwB4SSS/9TklMev6fy6DdfveNOuYf6ft6RWnM9XvHbjxyl6JFyMkR6iQ89ZG1k5TJ6SHjk5QSrKHYUbImLRlNu2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9HV6BXZTqrgPjnLG9+76jdGMhmFlNXzwvPqrZMlEYFc=; b=f7T6dcMhYDpwXNK5SP5ck+ZkSUZZCiwru272/nmweoE5mMTo9SyzU6/eyF7ObBdVGYphQxC7ibOHpFX0Z9zkBPcnLzuAw9jqrGn7oWsYKGGoQaADUzIdnbY9KuvZseBMiLzaZtpxXYbxTR6w7QjZTNFA0HMN8SaoY4IY3h+8QNfc5AaA4foqgumE645lIonJQXtog+asOlYHpciLTMUQHk4sIG1lmC7XOSqaFBmDMejwoW1MeeJ0O68xQoRbHkBgtdfNfrAS7ACShRmqy+qOdcUmbHzrySd0UW8srU4qrtlzxb/jo5b2Gkvd6T4uXqQLFRjEjP6lVAY36S/yZfyIuA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9HV6BXZTqrgPjnLG9+76jdGMhmFlNXzwvPqrZMlEYFc=; b=YclvT7qCj/+Xs1b0xWlzWB5MIyruPOz0y67Ka/mp2cVlq0A1njDl9qjc/NAcJp5MS5Kx7D+J4Xr+iD+Vgq/8xCSuD5yXWMdDqy6UVAEIo3cllu5TG7eI/MtQf1V16ImuntsSmaxd5KN0dXb64mEFmz8ccC4kltk9JOQMKMCuHVI= Authentication-Results: linux-foundation.org; dkim=none (message not signed) header.d=none;linux-foundation.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2752.namprd12.prod.outlook.com (2603:10b6:805:78::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3721.20; Mon, 4 Jan 2021 20:00:46 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::6d32:940b:f630:b37d]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::6d32:940b:f630:b37d%4]) with mapi id 15.20.3721.024; Mon, 4 Jan 2021 20:00:46 +0000 Subject: Re: Question regarding page fault handlers in kernel mappings To: Matthew Wilcox Cc: linux-mm@kvack.org, linux-pci@vger.kernel.org, Andrew Morton References: <20210104165628.GB22407@casper.infradead.org> From: Andrey Grodzovsky Message-ID: <654b6d02-9ea6-0e36-d736-b529982f09f7@amd.com> Date: Mon, 4 Jan 2021 15:00:42 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 In-Reply-To: <20210104165628.GB22407@casper.infradead.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US X-Originating-IP: [2607:fea8:3edf:49b0:4465:d1cb:357e:d8b0] X-ClientProxiedBy: YTOPR0101CA0021.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::34) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [IPv6:2607:fea8:3edf:49b0:4465:d1cb:357e:d8b0] (2607:fea8:3edf:49b0:4465:d1cb:357e:d8b0) by YTOPR0101CA0021.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3721.20 via Frontend Transport; Mon, 4 Jan 2021 20:00:45 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: d266d9b5-aaa3-4a98-a68c-08d8b0eb6cb1 X-MS-TrafficTypeDiagnostic: SN6PR12MB2752: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kIAmCbhVLl5xQtk4f8k1nSTfhk1lZhUBMgbLg+gWn3/3fo6hmATRocew0ovlCAbdLdVs4ez1aoKt5wbWo6Ws+Rd52YyuowU3YCMQ0Uu0gkQC45UozZuO+rfkKiI6CVkO4fGoR7hPwoPJOI+zDZ3HnINzW37F+HhEiupUT3hrewS3K30GscABbA9tyaG7NF5WySnRubi6b+/QFnHXF32WvJKViSu8Dy/YV8TLMgSO0KyK3ytQWnrMcuQ6/ftgQHHxqrjPypxE5UybM89p2Jimi71///WMqFstTD44/xVnisSo+A2pM8zSYFXg1aQFB15CsHOl6P97cZE9MFyIuA8h9I6knE6ugcyOlHbpQYM31wU0N7f7bI5SybfFSHiBbAtaM6IHUjXnBOM09b1o66NuTq71HGali+9SwiDBV0IpAetJBgKO1W0KyuqZXsvLy50JTTOAb9ywiDpp9NECGW6j0qr9hzsSl6fOc0pmPTC/kh4= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR12MB4623.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(396003)(346002)(366004)(39860400002)(136003)(478600001)(316002)(86362001)(6916009)(66556008)(66476007)(66946007)(31696002)(6666004)(8676002)(5660300002)(31686004)(186003)(16526019)(2616005)(4326008)(83380400001)(36756003)(53546011)(52116002)(6486002)(8936002)(2906002)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?SnZOS0RXM2RSZ0lDZTdvd3RBbjd4SEg2UUY2aGlZdWR1Szk0eWpkR0pKVkFm?= =?utf-8?B?TTJBUktDUmVPUjZ3RXhRUjdEYUpwb1U4TG9kSHFCQVlybFMvVGsyMDJwVDJh?= =?utf-8?B?N1J3dTBHWmt2L0VBZVpjS1lFWDc5b0NIa1ZiZXQ4aUZFcXFtM2x3UDd2dE51?= =?utf-8?B?VkxwTnM4eTNuR081TWVCZ0E5UHFDOGwvUG9FbStSdkdveFBHcUp3RG5Bbktz?= =?utf-8?B?RXZnMWlzRHdYbUxxR3R4TnQrOWliMWEvT0tFdVFmTmpCSUluVkF0ZmFyUHN4?= =?utf-8?B?R1IyaHF4UG5TZDZ5REd5emJXVVBuZWRIdm9vU3U0L0ZvMHM4K1ZYRlZDa2xD?= =?utf-8?B?M1ltYktSelQ3RklQcUV5SXF6SVozK0R4VTNCTVcvdDRBdjI0QlpnT3gzcGV5?= =?utf-8?B?TVpoajN2TGx2am1ZNGNYWXh1emRwRmpsaytxcGNqdzNCUEtIZjFPaVN5bUVS?= =?utf-8?B?L1h5T015MWZjckpLcENHR0Z2dDYvYVlkRnlscFVUZ0RIRUtsSTc5OUxveTVP?= =?utf-8?B?T2dSR1hZU01DZktER0JzdDdHU3AzNW44Z09oYlVDOE1uN1lMSTRLa3NtbXkz?= =?utf-8?B?ZGJQVjZ3WE00aG02ZThpVll0MUZLeDZhNEVYMzAyRmxNcE9lTk9SblZhS2Fu?= =?utf-8?B?KzdHejk4S25GUkozUmo4b1NpVFcxUnFBVXo3RVlIcDBtb0ordFNoN0NieUtF?= =?utf-8?B?eVVhNFF1WkJMZlpMajdacE1qSFkwbkdEUlIya3hTVW9xbVhhUlRVQXR0WG0x?= =?utf-8?B?bVdDejlYdFhOZXF2VU8rUC9lOWxUN2kySS9UY1hPV0N3bXlWcXVNdjJIK1Fu?= =?utf-8?B?UUpLR0t5ZjJDeWpLTXA2eTE0Q29xempaSzVxWU1BY0FyS215WGp3TWxKMzE0?= =?utf-8?B?cDNqdyt5M1c2d3lIZlkwdUN0ZHEvTWVab1lIQ0wvclMwdERRUXNRUzZPcUd4?= =?utf-8?B?aEZOcWRVNU4yWG9nYnl0ejBYa2R3WnVNZCtFUVYwTnVSTERzK3JIOFNFaG9n?= =?utf-8?B?U1Btb2QvVzdrNlNCRnc2SmZqUU1rUVpZSHMrZFZrSEVRZGxuaTczeS85d0ho?= =?utf-8?B?R2hmTTIvQXFGdUJYTDFWblI1RVNjUVpEdmhhSzZoNHpzbnBiWDJvdmN2b3dr?= =?utf-8?B?MFUrMTk2YVhFNHFoWkowbGxLS252RGNUZHEyeGlCUnZXQWdvV2l5UlhJbEdC?= =?utf-8?B?cGZaN3R0R3FmMmFZUmt0bVorMkwrcFZPd2pvMitKTUhSQ01NVHNCWjFYZkJR?= =?utf-8?B?c0tlMStpSTk5bHlnbjZOb05MQlZ3Z01QRnFYRk9QWVhQOURoeFJZSzdUM3Bp?= =?utf-8?B?SWd6QU9DUVJxVHZma0VtWC82QXFUc21WMGZvVm1SZEQ2bEVndVZXYUpRbDVa?= =?utf-8?B?Wlk2NjU5Y1B5bkxGc3hPdHNTZ1g4ZE1XOUY0UEx1NlNTQnlIUnFYczVqbG5p?= =?utf-8?Q?wLv42jG8?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jan 2021 20:00:45.9390 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-Network-Message-Id: d266d9b5-aaa3-4a98-a68c-08d8b0eb6cb1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fQ9uqaX/tm8kD93a/aEwcmKCyx0ibo0I+5dqE6i+no9BVyCYc3+ZQmyDECvX/ri1ih2jthIp43yFn0wVwuHJlw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2752 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 1/4/21 11:56 AM, Matthew Wilcox wrote: > On Mon, Jan 04, 2021 at 11:38:38AM -0500, Andrey Grodzovsky wrote: >> Hello, I am AMD developer and I am trying to implement support for on = the >> fly graceful graphic card extraction. > Are you talking about surprise removal (eg card on the other end of > a Thunderbolt connector where there is no possibility for software > locking), or are you talking about an orderly removal (where the user > requests removal and there is time to tear everything down gracefully)? Surprise removal > >> One issue I am facing is how to avoid >> accesses to physical addresses both in RAM and MMIO from user mode and >> kernel after device is gone. For user accesses (mmap) I use the page f= ault >> handler to route all RW accesses to dummy zero page. I would like to d= o the >> same for kernel side mappings both form RAM (kmap) and device IO >> (ioremap) but it looks like there is no same mechanism of page fault >> handlers for kernel side mappings. > ioremap() is done through the vmalloc space. It would, in theory, be > possible to reprogram the page tables used for vmalloc to point to your > magic page. I don't think we have such a mechanism today, and there ar= e > lots of problems with things like TLB flushes. It's probably going to > be harder than you think. > > I'm adding the linux-pci mailing list so you can be helped with the > logistics of device hot-remove. Thanks, that makes sense as I couldn't find any clear documentation on ho= w to handle page faults for kernel page table while there is a clear mechanism and=20 documenting on how it's done for user processes=C2=A0 page tables (implementing the=20 vm_operations_struct.fault callback) It indeed the would be useful if some one (maybe on PCI side) could give = me an=20 advise on how best to avoid accessing MMIO mappings made through ioremap=C2=A0 once I i= dentify the=20 device is gone. The best we came up with until now is to explicitly test for device being= preset=20 before doing any MMIO r/w access. Andrey