From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FCCCC6FD26 for ; Wed, 2 Nov 2022 00:40:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B54526B0071; Tue, 1 Nov 2022 20:40:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B045C6B0072; Tue, 1 Nov 2022 20:40:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CBE96B0073; Tue, 1 Nov 2022 20:40:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8A71D6B0071 for ; Tue, 1 Nov 2022 20:40:41 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5B6D31610E0 for ; Wed, 2 Nov 2022 00:40:41 +0000 (UTC) X-FDA: 80086646682.12.AE90F73 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf08.hostedemail.com (Postfix) with ESMTP id 17D2F160003 for ; Wed, 2 Nov 2022 00:40:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667349640; x=1698885640; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=fw+qL17DCZK+sk+sa8WqB43dA7mfh93h7iE5JgG3lx4=; b=jnV3ycyu4GqdgtcSqa1ub+0XcqBb15IgDO4hFj+Gj9RkXwldSS/DJP5D UMyfaaP6z11FB3C67956DAWb0XZtk70P5OQa3s0ZcZ11zgwb/AxIPcpCz 2Whw7AvRacO10/F+6PQc8tWW3Ko0XctSCeQy7GHZbI6bUXrGVOWPDepTo WsC6kphnbNqW4sG6wQhttQekYK/941kAy8+S2pu8V3OtC0rvf/VMudUgB YU2E/qq3FMAJrE1W0xoAUC2DBLdYRYCMmTz7mfn004RXLyKE+tI4eHbnc 3H+t0+MX2fLdKiQ93+S7wOl/CLE+LqqJMqhaHF3kLjP1bTfrTVtVAfbqD Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10518"; a="292587144" X-IronPort-AV: E=Sophos;i="5.95,232,1661842800"; d="scan'208";a="292587144" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Nov 2022 17:40:38 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10518"; a="879281950" X-IronPort-AV: E=Sophos;i="5.95,232,1661842800"; d="scan'208";a="879281950" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Nov 2022 17:40:33 -0700 From: "Huang, Ying" To: Michal Hocko Cc: Bharata B Rao , Aneesh Kumar K V , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Morton , Alistair Popple , Dan Williams , Dave Hansen , Davidlohr Bueso , Hesham Almatary , Jagdish Gediya , Johannes Weiner , Jonathan Cameron , Tim Chen , Wei Xu , Yang Shi Subject: Re: [RFC] memory tiering: use small chunk size and more tiers References: <20221027065925.476955-1-ying.huang@intel.com> <578c9b89-10eb-1e23-8868-cdd6685d8d4e@linux.ibm.com> <877d0kk5uf.fsf@yhuang6-desk2.ccr.corp.intel.com> <59291b98-6907-0acf-df11-6d87681027cc@linux.ibm.com> <8735b8jy9k.fsf@yhuang6-desk2.ccr.corp.intel.com> <0d938c9f-c810-b10a-e489-c2b312475c52@amd.com> <87tu3oibyr.fsf@yhuang6-desk2.ccr.corp.intel.com> <07912a0d-eb91-a6ef-2b9d-74593805f29e@amd.com> <87leowepz6.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Wed, 02 Nov 2022 08:39:49 +0800 In-Reply-To: (Michal Hocko's message of "Tue, 1 Nov 2022 15:34:51 +0100") Message-ID: <878rkuchpm.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667349641; a=rsa-sha256; cv=none; b=uopfUoUHl6ceGsUVEi19jhfMucHiAHUbzdoPWx1XyMHHSTaZDQSljmwi21wLTG7vcIJ+TM e1z39t0h2qI9fB/SnlnnAkk7g5OAVgBsQ1JVlkry5KPLECex9Foc+PBMKnyST78DAt6Syp a8m/GU5cbn7PpVKOZgHKxT9YYq1zR+A= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=jnV3ycyu; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667349641; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZhjdZWvmQ+O5UXCpS8XvXDqiUKOvXcypZ0CSgIraCY4=; b=QB1mjH/aZQjkehGXCs5YOqfU2sItIQmm0Xj+jZKSwDGFtFcM7gaKJ34Iq9x4Is82iCMsX1 io9EwJgWJvUOwgv+tZ8d9SgZaVDUgLAN+SQudj0wTYSOSKLuk2DbblxP5TN7KU5vXj0NLZ CS9XCYlfgcQ3iWTumV3JopwO26RLxGg= X-Stat-Signature: jsieiedsxsam5wy4djxtk98751prw3fs X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 17D2F160003 X-Rspam-User: Authentication-Results: imf08.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=jnV3ycyu; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com X-HE-Tag: 1667349639-201916 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Michal Hocko writes: > On Mon 31-10-22 09:33:49, Huang, Ying wrote: > [...] >> In the upstream implementation, 4 tiers are possible below DRAM. That's >> enough for now. But in the long run, it may be better to define more. >> 100 possible tiers below DRAM may be too extreme. > > I am just curious. Is any configurations with more than couple of tiers > even manageable? I mean applications have been struggling even with > regular NUMA systems for years and vast majority of them is largerly > NUMA unaware. How are they going to configure for a more complex system > when a) there is no resource access control so whatever you aim for > might not be available and b) in which situations there is going to be a > demand only for subset of tears (GPU memory?) ? Sorry for confusing. I think that there are only several (less than 10) tiers in a system in practice. Yes, here, I suggested to define 100 (10 in the later text) POSSIBLE tiers below DRAM. My intention isn't to manage a system with tens memory tiers. Instead, my intention is to avoid to put 2 memory types into one memory tier by accident via make the abstract distance range of each memory tier as small as possible. More possible memory tiers, smaller abstract distance range of each memory tier. Best Regards, Huang, Ying