From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84760C3064D for ; Wed, 26 Jun 2024 07:21:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB4116B008A; Wed, 26 Jun 2024 03:21:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D3CFC6B008C; Wed, 26 Jun 2024 03:21:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC2186B0092; Wed, 26 Jun 2024 03:21:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9C1A26B008A for ; Wed, 26 Jun 2024 03:21:06 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 1AA3FC0B76 for ; Wed, 26 Jun 2024 07:21:06 +0000 (UTC) X-FDA: 82272193332.24.CB26233 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by imf02.hostedemail.com (Postfix) with ESMTP id 50E7C8000E for ; Wed, 26 Jun 2024 07:21:04 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=linux.microsoft.com header.s=default header.b=G9EeXrw+; dmarc=pass (policy=none) header.from=linux.microsoft.com; spf=pass (imf02.hostedemail.com: domain of shyamsaini@linux.microsoft.com designates 13.77.154.182 as permitted sender) smtp.mailfrom=shyamsaini@linux.microsoft.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719386451; a=rsa-sha256; cv=none; b=Al3uISq8fxabzy5J5KjlUPMGaHmYlcBEhOIW3JJeK9c9Pqs6LQAVQxYpnro6tL86aC11JA XU902Q0q1ijXsgUUWD+6BZtmv51vNYZkWQquQN5RfGkO0zmzGd0mrsgMhUmluvSdWfifNI DEIk+q4+EX92oQ0TU4VrNM/v5mlwoWM= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=linux.microsoft.com header.s=default header.b=G9EeXrw+; dmarc=pass (policy=none) header.from=linux.microsoft.com; spf=pass (imf02.hostedemail.com: domain of shyamsaini@linux.microsoft.com designates 13.77.154.182 as permitted sender) smtp.mailfrom=shyamsaini@linux.microsoft.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719386451; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=moCiPaIn6NwSfydLRZyrKy3vd1OTZl2jw6RofluiaGs=; b=aMH+WwZN6PBU67jMMHY76voYZV7IiF1c14ZSY3hceBBJGROgVQ62zYCwi2AbcbXmG7HjP9 lD8B7qp545NKHJiImggpxMOYlPj0T2q3UUJuvJBOkf8x8ZUevSSPZY7zHn36M1mEiqcnbl 9XyKOzO6wP0Zut3Q/kVhqdjErs8m7lQ= Received: from thinkpad-p16sg1.. (S010600cb7a0d6c8b.vs.shawcable.net [96.55.224.203]) by linux.microsoft.com (Postfix) with ESMTPSA id 3DE0D20B7009; Wed, 26 Jun 2024 00:21:02 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 3DE0D20B7009 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1719386462; bh=moCiPaIn6NwSfydLRZyrKy3vd1OTZl2jw6RofluiaGs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G9EeXrw+/0mJx1pij6BPmT+TnZdwksClZtLPrzVUXgEKvyh9KLmAN+bY+kJ5kgNom MFyu/2j/wa+nzn9EqTks0cn7GZH27tnPkawPFxkWRlLsV9rPpRfOLVMYG4x0WX87hS dE97G7JmI5FAdMeJR770S7ATzYaaz4MdKCiwCrW8= From: Shyam Saini To: dan.j.williams@intel.com Cc: akpm@linux-foundation.org, david@redhat.com, iamjoonsoo.kim@lge.com, james.morse@arm.com, jgg@ziepe.ca, jmorris@namei.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, maz@kernel.org, mhocko@suse.com, osalvador@suse.de, pasha.tatashin@soleen.com, sashal@kernel.org, tyhicks@linux.microsoft.com, vbabka@suse.cz, will.deacon@arm.com, code@tyhicks.com, srivatsa@csail.mit.edu, apais@linux.microsoft.com, vijayb@linux.microsoft.com, tballasi@linux.microsoft.com, bboscaccy@linux.microsoft.com Subject: dax alignment problem on arm64 (and other achitectures) Date: Wed, 26 Jun 2024 00:20:38 -0700 Message-Id: <20240626072038.1419889-1-shyamsaini@linux.microsoft.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 50E7C8000E X-Stat-Signature: exnwkbf5xgewnqojs7omyiztypqcmtjn X-Rspam-User: X-HE-Tag: 1719386464-946520 X-HE-Meta: U2FsdGVkX1/sRKm1qtKwMnVuOuPy5wC7uSMfSkLWMPsjXM4zSFI//Gh5+dRZk917m18/58bD3xy8+E2hsU/xObmTUZp6bjRtIf0il8M86x0PzB2/WTPK31ExqlaghazWoxnY9VOrfRPlYaG0rEqng95KwbVWvDWMoLUoEH/dqbklHGmlUSoa3ioJPXb8oqhyjifXvoeftNN8sNXajTvdrQIeqQfseBaW3FFTnmmj7OfJ4UGEAKShRgOsqsPRGeWSP0bqA633NEcIJ8SmlBB0Y/wn9oGe6dhVh0zjkQyJvJhH4zAkEI74wwBc5S5qTfdmRFigajjj45tuvYpltQu+I80HjAKOwA8Y/dVwf8haMtEEQm0jn4uF+7QGUAW80xoZGTPECEIlVQaM3Dpad1S55NOw4Qm59BTU1+k+US5KzftmMe5jTY3ugMWh1N+vo6+47o+bOY79qa0AYcj5YnrwOYJNv072pzpklr3D85j+koWXBd0DGpNQvKHilZz5qYxZxEkve5VZ5bP01d4IXunyuTdHM16J0rJ5ChwdexAOw1wpAz1vn0OP2npuUt+7WxKX6rXqcjzE6xhUIxlQ3egYY+/4iMVg1bu/EOps1U95Ko2+uX/sOFmqRRefghAt1akqpWj5FNVQxKQCfVbV1bL+T/KWIPG125vIYwvORwI76NfsSUSeqNBg0p5xt2DcIE6TUFPb7PiNz/m3cvFSWE6s2+BXo0tvxn4dTluExf3WpBe3iw6HPC+yzpgJpk98iMOwgrNr8fF7AZabwMpMtV6VpIgzCmx+xKKXxFh6ydkqDkWyQFLa4BcRGOLuNj4EltI0eMzsFLs+4OArkXP80PK81NvdNplpm3IUSonPcYGyk+3Nr2L4v9PUC31ZwNdVV5CmjBygvQ83ijic/qkI8846lvbSmUhXBEsRkUN7/C5pEqparN2e5iE+3I5UtfuTsrR+mVpb7pRp8+2mBvrpLZ0 FG79o8PV ICBEkLoJd1O8jxbsAaeBlsIAcixcB4R/G7v0ldToFqOlhFY87uoO3YPFreDCNKqRWa0LOppFD5cV2yHI/JrfXti2ftQkG99NoculZHAX7JN9RJsAGgbaRyJzxCgU5sWGkXrQaSqJOY2ZlH4RHH8cwrBsDGmQtoD898LG2cobeCflyDWCwqXysjCwyqLotqZlD7knUv5N+u7mQr0YtM2euYBbevLbxTD87CFUONVwK97OVh+mEN3+LiM/bXJpYLWLGjWLCHmou3oUovzdx5ANxS5qllxZ/IsrGGTyYbdv0k8f/Ih8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Dan, Restarting this thread to get more insights about dax alignment problem. So having a devdax pmem of size 128M is [1] not usable and entire memory is wasted? For 256M size devdax pmem, again 126M seems to be wasted and only 128M can be hot added/removed. This was observed on ARM64 platform. do we have any potential or existing solution for this problem ? > > > > > Since we last talked about this the enabling for EFI "Special Purpose" > > > / Soft Reserved Memory has gone upstream and instantiates device-dax > > > instances for address ranges marked with EFI_MEMORY_SP attribute. > > > Critically this way of declaring device-dax removes the consideration > > > of it as persistent memory and as such no metadata reservation. So, if > > > you are willing to maintain the metadata external to the device (which > > > seems reasonable for your environment) and have your platform firmware > > > / kernel command line mark it as EFI_CONVENTIONAL_MEMORY + > > > EFI_MEMORY_SP, then these reserve-free dax-devices will surface. > > > > Hi Dan, > > > > This is cool. Does it allow conversion between devdax and fsdax so DAX > > aware filesystem can be installed and data can be put there to be > > preserved across the reboot? > > > > It does not because it's not "pmem" by this designation. > > Instead if you want fsdax, zero metadata on the device, and the > ability to switch from fsdax to devdax I think that could be achieved > with a new sysfs attribute at the region-device level. Currently the > mode of a namespace with no metadata on it defaults to "raw" mode > where "raw" treats the pmem as a persistent memory block device with > no DAX capability. There's no reason the default could instead be > devdax with pages mapped. > > Something like: > ndctl disable-region region0 > echo 1 > /sys/bus/nd/devices/region0/pagemap > echo devdax > /sys/bus/nd/devices/region0/raw_default this interface file seems to be not available can we use sub-section hotplug feature here, there aren't much details available about using that, is it via sysfs ? I appreciate your help and guidance on this. Thanks, Shyam [1] https://elixir.bootlin.com/linux/v6.10-rc5/source/drivers/dax/kmem.c#L102