From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA0FEEB64DB for ; Thu, 22 Jun 2023 13:55:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5A23B8D0002; Thu, 22 Jun 2023 09:55:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5525C8D0001; Thu, 22 Jun 2023 09:55:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41ADC8D0002; Thu, 22 Jun 2023 09:55:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3316E8D0001 for ; Thu, 22 Jun 2023 09:55:48 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 06869140443 for ; Thu, 22 Jun 2023 13:55:48 +0000 (UTC) X-FDA: 80930531976.04.8C63991 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf07.hostedemail.com (Postfix) with ESMTP id 8A98D40003 for ; Thu, 22 Jun 2023 13:55:45 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AQZgWWdM; spf=pass (imf07.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687442145; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=p6G7WiAZOcQ9lIcGNb7i76CvK3NP16OpxiaWi+YzEms=; b=DTd9nkUcQGrSR+i6AtGqv0IgLhNBG4ssn+fHVJ/VX500gPneLiy72pR9FEJG1/A/+VnGrP 3o33h7vWKaUyIOJH53Yo+cB3VUqsptDQSDqeC6G8bMV4HNp6KVNbbeFEFkK9Nmc7w/MOYG ZYNIFXOVwQsdwFNoHQydoTVTl9ZoKkk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687442145; a=rsa-sha256; cv=none; b=mQpMXrXqLr3LBOEdeS8Cdeq12ZnT5icVrTYhNmA0CCqG2cxr8i6s7e62hIPJauA5zjpYfr w1PKU+FBbDwzIxLbiObJqRSP4guN8YZDC1SnSWLvyctf1gTTQ6f7sGZ7r4SvMcDzQLKiVA SAWVjyD1a8Y60VGgyeQewVhTRSY1kQc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AQZgWWdM; spf=pass (imf07.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687442144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p6G7WiAZOcQ9lIcGNb7i76CvK3NP16OpxiaWi+YzEms=; b=AQZgWWdMEnBTlrd7B7AcDbx5NvfPRHw33tvKh9nH5ZQPVpHA6VTIkC3e0GN1IgKFxPoyus 9w0r0MXg1RTO2T1922AcH1A/F0PSg6n5qPAMmIi1BYIyejpD6txY3u2vkpWFPcE5FDpOa2 BpLXR5GQkde6vfHsU12KmfAO3oxXhtY= Received: from mail-lf1-f69.google.com (mail-lf1-f69.google.com [209.85.167.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-307-y3_XI6LyPh21lS2_XtMi6Q-1; Thu, 22 Jun 2023 09:55:43 -0400 X-MC-Unique: y3_XI6LyPh21lS2_XtMi6Q-1 Received: by mail-lf1-f69.google.com with SMTP id 2adb3069b0e04-4f863fcc56aso4128411e87.1 for ; Thu, 22 Jun 2023 06:55:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687442141; x=1690034141; h=content-transfer-encoding:in-reply-to:organization:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=p6G7WiAZOcQ9lIcGNb7i76CvK3NP16OpxiaWi+YzEms=; b=MmV7TsEbcVeeQAMdEXwZxdh+oIc95/jAF2chkl8F3EkVeij66jrmZ8GhQbb2GrjgHB /2DRauSRkSF6k3d8t+ttpkmOVFqC7huwy6xvDCmMs+2QYTNyZk0mMuj4WkzN2jdXohvz OJt+8hoeWSfYs7Ok7HPYCiEIjXhTan1tCLZlJZ17aECov3rWnMqK8DfYL0wQXUtW6E6k Ll20RH13Nm0Q4/M1KzhZdFmCizLsrrY8wlLetyyWmAToklIsIOY5TeXPNGeceQCBWHtd ulbAOLvXbdkuLTF0mAUNaEYUtK+eMnTQ2TXSBTwU9BOVr5cP8TmAa6EIPW51mtx4DGSN +WDw== X-Gm-Message-State: AC+VfDxm7NKIlhw3+FRjypgwxRC5XhS3lApEFABWM4jWotmUxCm4E9oX U0SaXwNUW9tS/wafJcqQvueB4kIw8jr3G/+jGZrkT7eIENlVgu83JCUkPWbPUKpzlp0DBQK1Ae7 KTOLHS9/H7MA= X-Received: by 2002:a05:6512:2e7:b0:4f8:71ca:bb15 with SMTP id m7-20020a05651202e700b004f871cabb15mr9177803lfq.48.1687442141566; Thu, 22 Jun 2023 06:55:41 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4B3PJA6vAcVuODvAd1+uWbP6euZQ43Yz3zVopBeuZdQAZiCsBeOr4t+6/ZZr44BUky2qiISA== X-Received: by 2002:a05:6512:2e7:b0:4f8:71ca:bb15 with SMTP id m7-20020a05651202e700b004f871cabb15mr9177762lfq.48.1687442140970; Thu, 22 Jun 2023 06:55:40 -0700 (PDT) Received: from [192.168.3.108] (p4ff23922.dip0.t-ipconnect.de. [79.242.57.34]) by smtp.gmail.com with ESMTPSA id z5-20020a1c4c05000000b003fa78d1055esm399654wmf.21.2023.06.22.06.55.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 22 Jun 2023 06:55:40 -0700 (PDT) Message-ID: <39fe2152-c4fc-6b8e-90e8-1060234a1943@redhat.com> Date: Thu, 22 Jun 2023 15:55:38 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [PATCH 0/3] mm: use memmap_on_memory semantics for dax/kmem To: "Verma, Vishal L" , "Williams, Dan J" , "osalvador@suse.de" , "Jiang, Dave" , "rafael@kernel.org" , "lenb@kernel.org" , "akpm@linux-foundation.org" Cc: "Huang, Ying" , "linux-mm@kvack.org" , "linux-cxl@vger.kernel.org" , "nvdimm@lists.linux.dev" , "linux-kernel@vger.kernel.org" , "linux-acpi@vger.kernel.org" , "dave.hansen@linux.intel.com" References: <20230613-vv-kmem_memmap-v1-0-f6de9c6af2c6@intel.com> <29c9b998-f453-59f2-5084-9b4482b489cf@redhat.com> From: David Hildenbrand Organization: Red Hat In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 8A98D40003 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: wj4tos8mma17q6ndejo9btudzjbsidw1 X-HE-Tag: 1687442145-706096 X-HE-Meta: U2FsdGVkX18lN5u0O74ax+6l4DTKg6wPe9tFvK+u7fmVRhWMomSlC/eqH1lOZly21+ZaDn35qz9sy+umUoOTT+bD2tvQmAZP2aQ6p6ZD50l9eLSgwi2PyIydC7GjmtxCYJG/y/djO8u1/GM/aKtslVlnVt0HonN+nz+LUrvUL5z5KHESDrTQ/q5eQWAZ2D/hUKYoNjyq4/L/+iisSOQf9tYJuAXRAdNR5dE+6iQy9g5ra7DrB/pVrWpfQQoNXlGXw5JiZc5f9fr3vLM12X0TyILgjR8q8zmKhX9d+O8bwkjTYNdRMtHfYWVNYVtcBhhm73cJ6llOrbpm7XFU0Tmx6P2e0TpeY2ghkkgMO4YVfwzUzcHf1IPfSLPyIY0dbcGJvgIn4BUjz064Pdq2IteZ+jlO2Lf+I9vAbUZtx/aabsinSOoQRyUV4m0HPDIUYWPdfFK6r6lsYmHB03J7R+tuCpSrolIV71/1xukrWModR3Adm0a5lTLGH9VWH3kg+4NF0PmpyAx0YMRreCie361krrEgGZmrmmSAYU2TDfnVsmBJ83FsouWmA02WszJvqDobfrWzoguuxK/ij/6k0AKLnnotW9DQrFzps5uU3q7/dl9cuX8JEwar9WhtgYbkmMnaLmRKX1WuLsiIGOpukXsuD5R4fCaru+2mzpPq9EMxWAqe5lvcdzOW39Int36C3k0FX0Awf9buzZqSmX4GXo9tyUrjFLySqlvXNrSOX9aD8JYz7K9kCIMk/1kz9yfw85Kml/lwg5RtLes7BeuWrkpe/ynyLvgFifyGMUY241TwsJ1NUQUrSvmUQlpuZk2PFQfCSHs3c6b1vjXNfZyD5IH4f+hBBDWPD5OlVi3oGFo1tczlEsThaC3isyr0DlxEQ/wv+7lBdsFYPeRmGAirplApNfKp5FNvG4aD88rfPvp8Ojw2bM2r7aAqb25gNBqZq1WMMQO+s161ylzED8CkeDh 6pyCEmkM lfzoug4YUgWbyXDjdr/Vt7+YC1AUaaTRfdaMTZIKJSZlwnw89AzOvcSREon31GJx5Y2qwTSWz+P7y/YB87TTijFGT11hXjzJdmCXYO3zpzuc5GSng3LHvzcBuf/hMCqQVBcottnoxg8UCC+dg4VrrGUmefu5mq1uO/NAV9qRgYioZZCmZgrj90fFgw4XC8W1vLM8mdiq1KVoYnp/WcNLt0vGpvUTV3vaEU2CiOLAZmB+orYMjkXWho4xATwAAQXEMWjKeq5iwH45kWj8mteNQAuoZwMZ5jqxmKsvrtPIZjeU/pPYjXzPAQiRcWXEl/cFHiP78bFe1q6d02cm7WH3Ue/klwrBYpICDQowwvUcgC1rhOsvQeL7h+HCPAn5R94+HIeNzPKqxlbIToff/AqPN2/dILlMMEqWhorl6WgEuze6b2R4GGUZKU6KdgZtlNQiEgmm4uCFX5dtE632cYf+wKnJalvu81ib1Mm4hOhooTCQgNAU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 21.06.23 21:32, Verma, Vishal L wrote: > On Fri, 2023-06-16 at 09:44 +0200, David Hildenbrand wrote: >> On 16.06.23 00:00, Vishal Verma wrote: >>> The dax/kmem driver can potentially hot-add large amounts of memory >>> originating from CXL memory expanders, or NVDIMMs, or other 'device >>> memories'. There is a chance there isn't enough regular system memory >>> available to fit ythe memmap for this new memory. It's therefore >>> desirable, if all other conditions are met, for the kmem managed memory >>> to place its memmap on the newly added memory itself. >>> >>> Arrange for this by first allowing for a module parameter override for >>> the mhp_supports_memmap_on_memory() test using a flag, adjusting the >>> only other caller of this interface in dirvers/acpi/acpi_memoryhotplug.c, >>> exporting the symbol so it can be called by kmem.c, and finally changing >>> the kmem driver to add_memory() in chunks of memory_block_size_bytes(). >> >> 1) Why is the override a requirement here? Just let the admin configure >> it then then add conditional support for kmem. > > Configure it in the current way using the module parameter to > memory_hotplug? The whole module param check feels a bit awkward, > especially since memory_hotplug is builtin, the only way to supply the > param is on the kernel command line as opposed to a modprobe config. Yes, and that's nothing special. Runtime toggling is not implemented. > > The goal with extending mhp_supports_memmap_on_memory() to check for > support with or without consideration for the module param is that it > makes it a bit more flexible for callers like kmem. Not convinced yet that the global parameter should be bypassed TBH. And if so, this should be a separate patch on top that is completely optional for the remainder of the series. In any case, there has to be some admin control over that, because 1) You usually don't want vmemmap on potentially slow memory 2) Using memmap-on-memory prohibits gigantic pages from forming on that memory (when runtime-allocating them). So "just doing that" without any config knob is problematic. -- Cheers, David / dhildenb