From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90512EB64DD for ; Thu, 27 Jul 2023 05:43:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F21546B0072; Thu, 27 Jul 2023 01:43:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ED20D6B0074; Thu, 27 Jul 2023 01:43:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D99E88D0001; Thu, 27 Jul 2023 01:43:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CA4726B0072 for ; Thu, 27 Jul 2023 01:43:09 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 96E661A0486 for ; Thu, 27 Jul 2023 05:43:09 +0000 (UTC) X-FDA: 81056298498.23.EBB3927 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf19.hostedemail.com (Postfix) with ESMTP id BDF8A1A0003 for ; Thu, 27 Jul 2023 05:43:06 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=dgnCenAc; spf=pass (imf19.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690436587; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=H+o7ojkD19X4r02K/qZKy7WGDYSeXjh4Ic1tWKBxpOI=; b=yZX3trxSd1y23GRcMWYxA6DAA7Rc0L7Vb6BUuoE2oJ6c5FX6RRG3w7u/+OvZ+66ZgAamm7 s49Ji4esVyotVFDT1F7lSmEDBPFY/7key5IM6G2qSmoo5iT0ZiX7vdPct1iR4AJ8UD05sY kk7aKU1167NyupL33I6LLI9EGxj3Ryo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690436587; a=rsa-sha256; cv=none; b=iqGEzAj/xE5qKFfFHdq3gVGsYR5cTKntah+Ro4EwWj6gB2rS7jrRoOa0tcHyPMAXNHIbjT PcyiHbRcRzoPpdhsWsrk5NlFGLPWNZDOr7/5BF1OA93FGI9qDRmeHPeuuQtQbfIxgjoFhL Kul8+4vyLBgJyjhaBN9XQeUCsKqpnII= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=dgnCenAc; spf=pass (imf19.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690436587; x=1721972587; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=CZIXORpK/pKO7gqM7gZSvX2ogA1eMyUczHdXGcg8arI=; b=dgnCenAcKSg8qyRgq6bhBWX6Bv1rWYgEfcMRdHlXheX9f7wbN/RhYE9/ isvkFu5JjE5P2/E1zPd3OT+lDSDUUou6WQ8psCbA5WRyMydT0Hiw1iKcA Udi94WkMc+LqqGJNQz/s6wYVEZah95fCXRLUQzO3d1yRgfMAYAQeA0bLY EagjYDPoAeT3xsRlzpWw/BxY82UH6wgMkDH4OhV9Fbp0tbtdDDFcz+aWR KYkmuaVemOyXt3ZBSGEpcDdlsZBBuWk9vEFFFUk64ysnO2Rvrn8zgoLnl 5e+0qduGzePOGrE2pkG256Yl+FfGy0a1zbuZFiTv4t2Yj1iYzyOOgmrFR w==; X-IronPort-AV: E=McAfee;i="6600,9927,10783"; a="371821342" X-IronPort-AV: E=Sophos;i="6.01,234,1684825200"; d="scan'208";a="371821342" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jul 2023 22:43:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10783"; a="756522804" X-IronPort-AV: E=Sophos;i="6.01,234,1684825200"; d="scan'208";a="756522804" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jul 2023 22:43:00 -0700 From: "Huang, Ying" To: Alistair Popple Cc: Andrew Morton , , , , , , "Aneesh Kumar K . V" , Wei Xu , Dan Williams , Dave Hansen , "Davidlohr Bueso" , Johannes Weiner , "Jonathan Cameron" , Michal Hocko , Yang Shi , Rafael J Wysocki , Dave Jiang Subject: Re: [PATCH RESEND 1/4] memory tiering: add abstract distance calculation algorithms management References: <20230721012932.190742-1-ying.huang@intel.com> <20230721012932.190742-2-ying.huang@intel.com> <87r0owzqdc.fsf@nvdebian.thelocal> <87r0owy95t.fsf@yhuang6-desk2.ccr.corp.intel.com> <87sf9cxupz.fsf@nvdebian.thelocal> <878rb3xh2x.fsf@yhuang6-desk2.ccr.corp.intel.com> <87351axbk6.fsf@nvdebian.thelocal> <87edkuvw6m.fsf@yhuang6-desk2.ccr.corp.intel.com> <87y1j2vvqw.fsf@nvdebian.thelocal> Date: Thu, 27 Jul 2023 13:41:23 +0800 In-Reply-To: <87y1j2vvqw.fsf@nvdebian.thelocal> (Alistair Popple's message of "Thu, 27 Jul 2023 14:07:31 +1000") Message-ID: <87a5vhx664.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Queue-Id: BDF8A1A0003 X-Rspam-User: X-Stat-Signature: tf5dngmiwia8ofy8adpwqrsgjiu6taus X-Rspamd-Server: rspam03 X-HE-Tag: 1690436586-445574 X-HE-Meta: U2FsdGVkX19ozDI0vXN8t1s7OwbljiKbOh2M1FJTnrGzGKPo46R7zNUsZPtwzRFJI4iJZHRO+JP1fGOsxHixsztQ9pj82KkkE3d7HzlTjkFyDTgDqusI+aGCVlDSmU4oTbtVzMzwL3FFUh83klQjBM4oYf/8rf6Np5ddx3bMcr3y3zT6lwPCmeb7MCe9w1sEADp5/H9zzzzYbKsCm+9AtN5r/R75WWcHCHomy5T43SLXNEWu9weHdL20VufErJwuCegPrySPQbX3g8Hh9LmuqGH8GpHpIF3SYMLdkcvuANTncND2usiFG8DFfKYJQ+mwYErWjQxwKfztLTcKTsYOVZcUsn+R+7Vb0UzgfabO+qg0rs38Qhu4CowDbZtKOp+KLKKKAMLeJTolrlMEwn4Q8QWj5JHrR23PZYJG6uQjMNVH1hoopq7XGvwzitvCLLeeHuVpDlqxDQPxzEW9YziPI2vgTr+2T4vhhuTIocNRUOMx9ybxgjg3oMVnrSV2qVAWUgMT34MjdQziEkfkp0CyLag83Rz/hIDoaXLxj70ESuJSeOgnw3n7ihzS4bAhWx7ld2a1yWDbGGytCFlLw/+rwADUMCfCRN23RcFWha6hkL/Btl+K9lz1fGk0OO8BF90AwZJJsdQLDvExX/XJQTnKrcHDke25b74EivdCscEt+F/gVp4HXYqkiUXHjhkiEhcLWFOCJ6jvIndtkWWb7TaH/qWOGnC542fJ7P3sbyOeJVHDaJBWT1WLIrbqqwENaWVrRrDZA142NMlNP9gTRDLj2NFs/IgoazKI7ckVfwEWi73V7UjqEVvUhcfCcV75y9peMdX47q4+Avm2OmdiUxefP7OSlgc8xigwmujsvZm8keCKONDZm70j5+2WeKwS0UBjGuR6thwkSPeqenhIUO7KExLzjYolw1JbULLDgrjriHgbA9CdUctEY/dyXinluSGUoRCGzzSeN0WpjMrlTBM ZXadtAwk FjeLSNBDD36M8moNzeiGX0Hz7T2K+JULTClaV3tEXvhWH4BHLEMZmLzYmLFA4h9kRbSZXoi13BWS9pwePdmxyQJ4JUnLpDNHJCxZL13lAT32tKWAy/E0QXZH2J1AeXnTOrYUd/4o9qrAf/vQif3HRDenOJMugJL+wHbJ3IXBPY1DXD9wjoGAtmBfJbjQmjD/EIvFlSRNwgaXkr/l0Jywdi1ERIfcEG39QjyG3eAqvyQsP5YszbXPl1U1GSg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Alistair Popple writes: > "Huang, Ying" writes: > >> Alistair Popple writes: >> >>> "Huang, Ying" writes: >>> >>>>>> And, I don't think that we are forced to use the general notifier >>>>>> chain interface in all memory device drivers. If the memory device >>>>>> driver has better understanding of the memory device, it can use other >>>>>> way to determine abstract distance. For example, a CXL memory device >>>>>> driver can identify abstract distance by itself. While other memory >>>>>> device drivers can use the general notifier chain interface at the >>>>>> same time. >>>>> >>>>> Whilst I think personally I would find that flexibility useful I am >>>>> concerned it means every driver will just end up divining it's own >>>>> distance rather than ensuring data in HMAT/CDAT/etc. is correct. That >>>>> would kind of defeat the purpose of it all then. >>>> >>>> But we have no way to enforce that too. >>> >>> Enforce that HMAT/CDAT/etc. is correct? Agree we can't enforce it, but >>> we can influence it. If drivers can easily ignore the notifier chain and >>> do their own thing that's what will happen. >> >> IMHO, both enforce HMAT/CDAT/etc is correct and enforce drivers to use >> general interface we provided. Anyway, we should try to make HMAT/CDAT >> works well, so drivers want to use them :-) > > Exactly :-) > >>>>>> While other memory device drivers can use the general notifier chain >>>>>> interface at the same time. >>> >>> How would that work in practice though? The abstract distance as far as >>> I can tell doesn't have any meaning other than establishing preferences >>> for memory demotion order. Therefore all calculations are relative to >>> the rest of the calculations on the system. So if a driver does it's own >>> thing how does it choose a sensible distance? IHMO the value here is in >>> coordinating all that through a standard interface, whether that is HMAT >>> or something else. >> >> Only if different algorithms follow the same basic principle. For >> example, the abstract distance of default DRAM nodes are fixed >> (MEMTIER_ADISTANCE_DRAM). The abstract distance of the memory device is >> in linear direct proportion to the memory latency and inversely >> proportional to the memory bandwidth. Use the memory latency and >> bandwidth of default DRAM nodes as base. >> >> HMAT and CDAT report the raw memory latency and bandwidth. If there are >> some other methods to report the raw memory latency and bandwidth, we >> can use them too. > > Argh! So we could address my concerns by having drivers feed > latency/bandwidth numbers into a standard calculation algorithm right? > Ie. Rather than having drivers calculate abstract distance themselves we > have the notifier chains return the raw performance data from which the > abstract distance is derived. Now, memory device drivers only need a general interface to get the abstract distance from the NUMA node ID. In the future, if they need more interfaces, we can add them. For example, the interface you suggested above. -- Best Regards, Huang, Ying