From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.7 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22FC2C43215 for ; Wed, 13 Nov 2019 16:53:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8E9B3206E6 for ; Wed, 13 Nov 2019 16:52:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=intel-com.20150623.gappssmtp.com header.i=@intel-com.20150623.gappssmtp.com header.b="ssIxIeeI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E9B3206E6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 607F06B0007; Wed, 13 Nov 2019 11:52:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B7446B0008; Wed, 13 Nov 2019 11:52:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4CE026B000A; Wed, 13 Nov 2019 11:52:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0170.hostedemail.com [216.40.44.170]) by kanga.kvack.org (Postfix) with ESMTP id 369F46B0007 for ; Wed, 13 Nov 2019 11:52:59 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id DDA84824999B for ; Wed, 13 Nov 2019 16:52:58 +0000 (UTC) X-FDA: 76151848836.17.month08_8e59e23300f47 X-HE-Tag: month08_8e59e23300f47 X-Filterd-Recvd-Size: 6414 Received: from mail-ot1-f65.google.com (mail-ot1-f65.google.com [209.85.210.65]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Wed, 13 Nov 2019 16:52:58 +0000 (UTC) Received: by mail-ot1-f65.google.com with SMTP id b16so2202527otk.9 for ; Wed, 13 Nov 2019 08:52:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=bVs75VZBcY9S0Y5srgopU4beZRbRSaSe1fH2IBEEzVk=; b=ssIxIeeIslCiIJXffnM/puPFNeNmqyOWWUWtDbvqe4d/BOtTR/zgDVzI5AzNkOXmek d5nc5yKCwdpjNhJbj9X5bsPU3H+lpjXv+S41FpkAjafu/1ILKgYSNIw2YJGDDibDOGDm iTXa5j1XEcQEaGqtyExlnhGu7ZABwiMrrBp/NEh42TS5RELZjjcY2rG8aq+EO8xPu0/7 YjBvENhMkWS/hqSYnuOJuNFQgWDohADDi8v8UzWFS7ovBAkCbFzxXr5d9EYLgL/55pis 97SdNjAs5pm7As0sJuK/MBKxP83CWf52Z8Zcsl/4mp4LBamQZO7kYC3mpyycgVGUjrX5 B1vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=bVs75VZBcY9S0Y5srgopU4beZRbRSaSe1fH2IBEEzVk=; b=QmkacW+i2T1qVKYxOHsQF+ZQKl1YNUV7+qxdcFJtGJRDodSJt6bUzc5v6GPfTUN6Ls w23LMlOqwnSAMXZcI29Bp5WbFBJe7rYUopw4euYBPF5iPKJ+XRyNNC9X+/6e5C5n0B9V TyhQkf8eUNVAB3bM8ah++sGoSYLoPQgO4BszpawVmqh+YKKjt1PAwe8X58sPyDL6VFRM Rs7N+PeOrhennvwcfAek9jzv6oArCbHiSJzAy0JAIy74vGXG2oBmZOHPOf9jxJPQ2nZZ U+XX/uObNDj4C52cbTMGzMnW3stJ38g+9u4f6chwg+hPFzt1C7ncUJOrC25hp2cAXZCt pi/g== X-Gm-Message-State: APjAAAUeLAhHrBNjNH7u7B1C/pnKf0A9jBpHPq2W3V9iWEY6sCvippTP fE+bl1Ow+X87Ah6ievQ5kD2LmPDXFs4j2qXT4P2jGQ== X-Google-Smtp-Source: APXvYqzqgVb0Dn3nghpgE0JOdTaxD1sSLXcYeBJ5SaBwI3JLusskqPp89fKtEBxxBB3bkmRTm/7hKGIUPIpcjOs3bko= X-Received: by 2002:a9d:30c8:: with SMTP id r8mr4241297otg.363.1573663976870; Wed, 13 Nov 2019 08:52:56 -0800 (PST) MIME-Version: 1.0 References: <20191004114330.104746-1-Jonathan.Cameron@huawei.com> <20191004114330.104746-2-Jonathan.Cameron@huawei.com> <20191113094742.00000dc4@huawei.com> <77b6a6e8-9d44-1e1c-3bf0-a8d04833598d@intel.com> In-Reply-To: <77b6a6e8-9d44-1e1c-3bf0-a8d04833598d@intel.com> From: Dan Williams Date: Wed, 13 Nov 2019 08:52:46 -0800 Message-ID: Subject: Re: [PATCH V5 1/4] ACPI: Support Generic Initiator only domains To: Tao Xu Cc: Jonathan Cameron , Linux MM , Linux ACPI , Linux Kernel Mailing List , Linux ARM , X86 ML , Keith Busch , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , "Rafael J . Wysocki" , Linuxarm , Andrew Morton Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Nov 13, 2019 at 5:57 AM Tao Xu wrote: > > On 11/13/2019 5:47 PM, Jonathan Cameron wrote: > > On Tue, 12 Nov 2019 09:55:17 -0800 > > Dan Williams wrote: > > > >> [ add Tao Xu ] > >> > >> On Fri, Oct 4, 2019 at 4:45 AM Jonathan Cameron > >> wrote: > >>> > >>> Generic Initiators are a new ACPI concept that allows for the > >>> description of proximity domains that contain a device which > >>> performs memory access (such as a network card) but neither > >>> host CPU nor Memory. > >>> > >>> This patch has the parsing code and provides the infrastructure > >>> for an architecture to associate these new domains with their > >>> nearest memory processing node. > >> > >> Thanks for this Jonathan. May I ask how this was tested? Tao has been > >> working on qemu support for HMAT [1]. I have not checked if it already > >> supports generic initiator entries, but it would be helpful to include > >> an example of how the kernel sees these configurations in practice. > >> > >> [1]: http://patchwork.ozlabs.org/cover/1096737/ > > > > Tested against qemu with SRAT and SLIT table overrides from an > > initrd to actually create the node and give it distances > > (those all turn up correctly in the normal places). DSDT override > > used to move an emulated network card into the GI numa node. That > > currently requires the PCI patch referred to in the cover letter. > > On arm64 tested both on qemu and real hardware (overrides on tables > > even for real hardware as I can't persuade our BIOS team to implement > > Generic Initiators until an OS is actually using them.) > > > > Main real requirement is memory allocations then occur from one of > > the nodes at the minimal distance when you are do a devm_ allocation > > from a device assigned. Also need to be able to query the distances > > to allow load balancing etc. All that works as expected. > > > > It only has a fairly tangential connection to HMAT in that HMAT > > can provide information on GI nodes. Given HMAT code is quite happy > > with memoryless nodes anyway it should work. QEMU doesn't currently > > have support to create GI SRAT entries let alone HMAT using them. > > > > Whilst I could look at adding such support to QEMU, it's not > > exactly high priority to emulate something we can test easily > > by overriding the tables before the kernel reads them. > > > > I'll look at how hard it is to build an HMAT tables for my test > > configs based on the ones I used to test your HMAT patches a while > > back. Should be easy if tedious. > > > > Jonathan > > > Indeed, HMAT can support Generic Initiator, but as far as I know, QEMU > only can emulate a node with cpu and memory, or memory-only. Even if we > assign a node with cpu only, qemu will raise error. Considering > compatibility, there are lots of work to do for QEMU if we change NUMA > or SRAT table. Thanks for the background. It would still be a useful feature to be able to define a memory + generic-initiator node in qemu. That will mirror real world accelerators with local memory configurations.