From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D3CDC00140 for ; Tue, 2 Aug 2022 17:57:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E985E8E0001; Tue, 2 Aug 2022 13:57:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E47EA6B0072; Tue, 2 Aug 2022 13:57:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC1C58E0001; Tue, 2 Aug 2022 13:57:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B6F866B0071 for ; Tue, 2 Aug 2022 13:57:45 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7058E1A06C2 for ; Tue, 2 Aug 2022 17:57:45 +0000 (UTC) X-FDA: 79755410490.08.50DFD07 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1anam02on2049.outbound.protection.outlook.com [40.107.96.49]) by imf30.hostedemail.com (Postfix) with ESMTP id B3C2A8011A for ; Tue, 2 Aug 2022 17:57:44 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=B7gZSRaT9KyVhz/2Cpigsqf2qskZbeBk9woeaZ8sB4POTnxCMYmnFUKk5V6mg9nFAQvld6ZhlvkMDaWU/w2urubVBeSmPrxBPeaLd49KamvwBzzHefkF9brH65cLjHRnq3PZwwRRpgZeTaNOFYJlI30IVzR0BJqw/hlVUnWZ3/6mDlHDOSHL/40bOY6YpybNRvOM4G3ycYlf4xO8xh+yORhv/MwqYCLwSgi0moybHOuHgHT8VK6Qc0g2pp/ln8Cd8k3GauQpJG3EsqpZorka4Pw8sgIF86Uu7FKolhaWHAD7CspdepdeUKLlHu1QPKVBfl15fkPe9EUip/T5F3n9xA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=u9UJUkH/MOHFZqR6JChM7h77fuWqRX1QfSAEplvLSxM=; b=OuDYtyvUDDjgrhSTKKFq3/rFVJemE8zWxfCbcJTmmDCaRP00pmAWAHtLUXWq1E4+t4/zIBrti2IbEJBOp4U/4LZdCR6OjKxzZOpqzfrXV3K3Hb17pBDhOQXgjdhbnIoQ0ebU4IBT/BTjhjOAHyQ2ANyhhBy/L1wl/Psk7zsXdprjvZCyW/f+n1pMRlJmj9sNvN673ThWqLV0OXaK0RvZyWykSuqEGt7Sm9evixUj/89qozU9QuE0agSalaWtcWq8do6Hwfsm+YayXkWqpIpcNRC6i8oD6K9MzglwJwuk1mIEmCRM8tU8yGmQpnoMRd6TO+zuqEfT5XTahVVVbxdE+Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=memverge.com; dmarc=pass action=none header.from=memverge.com; dkim=pass header.d=memverge.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=memverge.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=u9UJUkH/MOHFZqR6JChM7h77fuWqRX1QfSAEplvLSxM=; b=Q6zOyvZKIOd7SX3i3/BuZJ8hGx5GtP9iOIBGZOi+dzKshivofdM+N8OwHfIp7Y4yEmCCeAWI2CpcWEvsXSjs6nfi7exMPgJgZk850Hb0wpIhT8ixE+Q6rvaLXuDz4FyZyOT/idhg1JhcBgprhF/a2GRTpHGZS1A4z67RlaiDSJk= Received: from PH0PR17MB4922.namprd17.prod.outlook.com (2603:10b6:510:d7::22) by BYAPR17MB2471.namprd17.prod.outlook.com (2603:10b6:a03:81::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.14; Tue, 2 Aug 2022 17:57:40 +0000 Received: from PH0PR17MB4922.namprd17.prod.outlook.com ([fe80::5ca7:2b89:7f14:b6fb]) by PH0PR17MB4922.namprd17.prod.outlook.com ([fe80::5ca7:2b89:7f14:b6fb%8]) with mapi id 15.20.5482.016; Tue, 2 Aug 2022 17:57:40 +0000 Date: Tue, 2 Aug 2022 23:27:26 +0530 From: Srinivas Aji To: linux-nvdimm , Linux MM Cc: Dan Williams , David Hildenbrand , Vivek Goyal , David Woodhouse , "Gowans, James" , Yue Li , Beau Beauchamp Subject: [RFC PATCH 0/4] Allow persistent data on DAX device being used as KMEM Message-ID: Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-ClientProxiedBy: MA0PR01CA0044.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a01:81::17) To PH0PR17MB4922.namprd17.prod.outlook.com (2603:10b6:510:d7::22) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4f00cde2-912b-46f3-fe4a-08da74b07dd8 X-MS-TrafficTypeDiagnostic: BYAPR17MB2471:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0x9/TBV2G8xfm/KoqMhsK7uwqHXj7pM8zkPhbTpRwqhUEh0SS5iaj3REsHnlHxDDkukS3pmOnDHO1SouXkOt5SeLklJpkcZ95IYLLvxhwZ5nKT4NAQ5YUGqSxP+z54kgNDiTQRiN/TT7TyL5snhloxpkwQvBuGWVUl7osQTsyQKBMNEzH7kT3jLL44pkC3JLNA/es/6pchfpsOBbQfgeQ0C5VAtpL9hbWRyCrMb3Ute0WthOpDLJxEBSp8ABh5eUHmQ7imkA7l+6SliDYUwmNpd1gmb/vLX8IhYFoqDQ/z9yY0f4LO+8oXDwApFN4gPssUvZtDbirVTttltmB8IfBM6WQCsm/MgOO8JLjS2VdUx1pclFronzT+oOoVg/9r2hAQIW+qTlN6wfIduf023uJVfNVdRW/afoHpNOzZRSeDi67GjOwztnJeaLdocz1Bn1ZY0mAXPrZI9FijNCwGqfCWhADTIcRTt2tJfgzR9b92mR2P4XWkEuY14NBsddOXiuxH9vjCr9ZcAUC26wBiEAh2+fsPOF7SnddzlvA3TRixUkZmmly7ntOLbNnFdJVxkktMoQvVMpCGKZXqXhduVrhcK1QKZ/+CRyuVCfujQu2vTbdUIfqkZSwt/d0x0+VO59Un1IZz7j10vkXceo2VHM1zcJBybdVHw/7o/pw9Tp7POzCyy+ZlSHgFEN5YFDAuXNI+Sc8bSR82XqRKwTA2hc8eo1TnXIrDeum8Kx9FRy+pv9JhHUwqbifpMhOA8DYYxnEnWk1uEygzyCxHZgh0WcvFpLMVls6/4I+zifqorsbDo= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR17MB4922.namprd17.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(39830400003)(396003)(376002)(366004)(346002)(136003)(6666004)(6506007)(6512007)(41300700001)(26005)(107886003)(478600001)(2616005)(186003)(54906003)(110136005)(6486002)(83380400001)(36756003)(52116002)(2906002)(44832011)(86362001)(316002)(4326008)(66556008)(8676002)(66476007)(5660300002)(38100700002)(66946007)(38350700002)(8936002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?BdJdJQls0JnSxl8h2CSfXfrhVTWqJnTIrGdpCMDK+BCn6KbAW3GrETf2l5NH?= =?us-ascii?Q?g5LibrQ5UZj6a8rWgzRV6PQD5wIeahPZ0NnNyekFpOd0WYEs7U2kfkFFFxE3?= =?us-ascii?Q?kKXP0gRa1JquSzYyavfehcMSw2gA/XUWCOG66vFkPGV+iUpfx3nagX731Pv0?= =?us-ascii?Q?TCKBsnBhJwvkabQ5wYgQj5mW6ZUQhlGbUaIe31iSoDrEFktpuxBp+k/gcoIA?= =?us-ascii?Q?jSAnQgDOtFTQlLgkHz4ezFFsbONcPyp28GbsR74t5qNrhc0PjJWlygTnguXJ?= =?us-ascii?Q?Zcd0Y86nJpRdbDUBadbp3GbSUjvzh2DpkBBRlqbtlSy+RqSEVkUY/OyniwUY?= =?us-ascii?Q?xt1P4eQhkRruoiR0sr41WLeWDHohyqfdEmLci0CMCAZU1bC50LMtspXY/vUl?= =?us-ascii?Q?cmOARXa+crqEj51zUcAGahmh9z2FxKmpq7TiTQ83GE+FMcmbVVj3gBIN9OXL?= =?us-ascii?Q?Dy/w6voOgqIs11UpNoV6YtozkXwsr+VSZEHWmMPTLOWp+WYj5PqYV5ZFet0i?= =?us-ascii?Q?0PahtynU8zrJdOyzP4wqpqrFur18kdxnFTl7q8rVqYMLtRHM9j0fmdeSEuKR?= =?us-ascii?Q?A3JrtEy440SdPNLLQd9oPMhhvYJ3S0gJq24WOJb8EG/LTyWAYq/xUs/f7gQz?= =?us-ascii?Q?G8gMYi7JIRDfMAizG1ipKOZ71nDak9AQuxWHw4I5to+AUynwK7zUUHd7JivN?= =?us-ascii?Q?jUJzklrKGGOFaCPhwOUV76UBLKjW4eUr3UfrPOCbotdOioWrerCWYWHfItVW?= =?us-ascii?Q?R7WuY4pbhxGtrarFD27NikeSEbMVwZGtd5OIRP/4wYjYxwukOjCiJ/ic/KMs?= =?us-ascii?Q?5R2NWYxczyGdybZBc09O3dL0yX0+Cg4vsFEQ9631uBK+e/zJOvgiwFBrqMa4?= =?us-ascii?Q?EItrpXP1DAlaL7ZBelVMPErO3ml4DKE8YvComzWWEDqn+6ObRof5FlFEOvsN?= =?us-ascii?Q?ip2xolcDq69M2jtaZ6QU2kihJIbsWvORJW1xjm+qpIa1guTGqM676ql1b3iK?= =?us-ascii?Q?gm/FZbIVjICqDQgwvoU1Ezsl0MDml1SP7/iPcxPE2yFOqgyprr6kdA0heOyZ?= =?us-ascii?Q?AIGfKwMwbyQ+gceBLLuaJdv8HlhYnIoL96oKSkQRp3bgoUVGyG3iEMvXcUId?= =?us-ascii?Q?NvY3Cg77eOgcNIZo+kp2cOccHsuLwjDY7LmSoKUrKUACYdj7FFanZ3mkiLuS?= =?us-ascii?Q?8cWWKcxUGC5C8F9Uqykaus/gt0loUbseSEZpMtTNS5ZKk3YEFJZp+4Lyu1/3?= =?us-ascii?Q?UzCTG5ZHU2hZ7SP/wdrmuOXv0ppeEwpQo0TI4eGgZvC47t8AsRemWccqnagm?= =?us-ascii?Q?hgmbqgnggbuOwjppd4wIQeey4aEB3Sg3HjSl7TfhPESTBxWOtbvNtR3cR8PB?= =?us-ascii?Q?bB2CaPCCgLV3nI4D8FUw1uvXcYYfLe34VnURoOCTNS+Qw+P3eK+qVHzTnBsz?= =?us-ascii?Q?J7G946bMDuZOI0/dL98IrQsaVO/7cdKrHuA3LdkizozHnDO246ACoGWGoN6R?= =?us-ascii?Q?4Axw1YwrtwNQne8ylu9bivzrWOJ3CDMoAfdQQPBTp5WlkyygWfrnEHRHA+uD?= =?us-ascii?Q?bbjf44KXd9ZYJu3cl7uBs1+eyLcWreD8pTFu8KtNI2pegJck2sLbLdfIJahd?= =?us-ascii?Q?YQ=3D=3D?= X-OriginatorOrg: memverge.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4f00cde2-912b-46f3-fe4a-08da74b07dd8 X-MS-Exchange-CrossTenant-AuthSource: PH0PR17MB4922.namprd17.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Aug 2022 17:57:40.2504 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2txKfWUPBV6cNXRUN8/ueB3vb6YTQZ1FBqnzpZPIVVgBZH41VvnIEsP8xdHio64ZFzTdJNU0bsp6cKyycFj+bAXjZs7wLL3rml+fXJkNWk8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR17MB2471 ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1659463065; a=rsa-sha256; cv=pass; b=TvRSAH/dJYYWnxG36Jih1MG5oznhp2CRMhwFPf5nrGQ7/i33XL1u8jv9qivCFG/7WsD5Sm +qlWMqU5LspOmx/77vCzdvA5/EdOFcOtRS5wwQ99XR3nmA6SqFMsztc3TV2IdOtVqsU7B5 jV8mjtZwrqzvU3fEiUnjMg8K9fVfsIs= ARC-Authentication-Results: i=2; imf30.hostedemail.com; dkim=pass header.d=memverge.com header.s=selector2 header.b=Q6zOyvZK; spf=pass (imf30.hostedemail.com: domain of srinivas.aji@memverge.com designates 40.107.96.49 as permitted sender) smtp.mailfrom=srinivas.aji@memverge.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=memverge.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659463064; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=u9UJUkH/MOHFZqR6JChM7h77fuWqRX1QfSAEplvLSxM=; b=w0jOpy8XjesL6sxdxKY7aHEXAg/cg2V8Y1kilFqHRFRsaXNXLM3gZvhdPg5XviIsaUdRkV 7pW4TZAEvYAEaykGrScWHzdcxDHWM0Sxxml+QCFGTTTYh4lAQfBRJtEtLAuBZqla5XxPRF x3QWD0zAfWdkmxClGpIijsk+2FkrpgM= X-Rspam-User: X-Stat-Signature: sm53waeqtn5q3utwgtietmpywje8arq5 X-Rspamd-Queue-Id: B3C2A8011A Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=memverge.com header.s=selector2 header.b=Q6zOyvZK; spf=pass (imf30.hostedemail.com: domain of srinivas.aji@memverge.com designates 40.107.96.49 as permitted sender) smtp.mailfrom=srinivas.aji@memverge.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=memverge.com X-Rspamd-Server: rspam02 X-HE-Tag: 1659463064-303667 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Linux supports adding a DAX driver managed memory region as system memory using the KMEM driver (from version 5.1). We would like to use a persistent addressable memory segment as system memory and simultaneously for storing some persistent data. Motivation: It is already possible to partition an NVDIMM device for RAM and storage by creating separate regions on the device and using one of them with KMEM and another as fsdax. This patch set is a start to trying to get zero copy snapshots of processes which are using the DAX device as RAM. That requires dynamically sharing pages between process RAM and the storage within a single NVDIMM region. To do this, we add a layer for handling the persistent data which does the following: 1. When a DAX device is added as KMEM, mark all the memory as allocated and pass it up to a module which is aware of the storage layout. 2. This module scans the memory, identifies the unused parts, and frees those memory pages. 3. Further memory from this device is allocated using the kernel memory allocation API. The memory allocation API currently allows the allocation to be limited only based on NUMA node. So this feature works only when the DAX device used as KMEM is the only memory from its NUMA node. 4. Discarding of blocks previously used for persistent data results in those blocks being freed to system memory. As an example, we implement a simple persistence module using the above framework to provide a block device. A block device assumes all blocks are always available, but in this case we have to get the blocks through the memory allocation API, at an offset not under our control. To provide block device semantics, we maintain an array which maps the logical block number to the real physical page, if one exists. Block device Trim/Discard support is used to mark blocks as unused. While we have the block device here as an example, a memory filesystem might be a more useful implementation. I am not sure if any of the existing in-memory filesystem structures are suited for persistence. Any suggestions for this are appreciated. Srinivas Aji (4): mm/memory_hotplug: Add MHP_ALLOCATE flag which treats hotplugged memory as allocated device-dax: Add framework for keeping persistent data in DAX KMEM device-dax: Add a NONE type for DAX KMEM persistence device-dax: Add a block device persistent type, BLK, for DAX KMEM drivers/dax/Kconfig | 13 + drivers/dax/Makefile | 1 + drivers/dax/kmem.c | 312 +++++++++++++++++- drivers/dax/kmem_blk.c | 573 +++++++++++++++++++++++++++++++++ drivers/dax/kmem_persist.h | 47 +++ include/linux/memory_hotplug.h | 4 + mm/internal.h | 3 +- mm/memory_hotplug.c | 15 +- mm/page_alloc.c | 19 +- 9 files changed, 975 insertions(+), 12 deletions(-) create mode 100644 drivers/dax/kmem_blk.c create mode 100644 drivers/dax/kmem_persist.h -- 2.30.2