From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91DE6C54EBE for ; Mon, 9 Jan 2023 14:00:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BE0E38E0002; Mon, 9 Jan 2023 09:00:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B6A128E0001; Mon, 9 Jan 2023 09:00:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0C3B8E0002; Mon, 9 Jan 2023 09:00:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 8B12D8E0001 for ; Mon, 9 Jan 2023 09:00:45 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3B78A1C6214 for ; Mon, 9 Jan 2023 14:00:45 +0000 (UTC) X-FDA: 80335421250.08.F3E9430 Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) by imf19.hostedemail.com (Postfix) with ESMTP id 2F16B1A0035 for ; Mon, 9 Jan 2023 14:00:42 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=nkbYfZjy; spf=pass (imf19.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.221.52 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673272843; a=rsa-sha256; cv=none; b=6jl+KxX7Azz3U+ZNenAQzpYVGvz0iuJ2O3EtOhH8Ov9s4WA/9TofB6LvKuoIViSOhuBV+K GZZbJPLvYQJU+A4aP+/7/h0VDITyaM3Ov2jfoguA2MvRxZ2j1h55/BtyWhfMNZbT7KwrzZ w+gMT6qf497HI9ytp/npOtEw8ERY/kw= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=nkbYfZjy; spf=pass (imf19.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.221.52 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673272843; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6spJSBkMNBIx4KLHawdyLw2aO1QaVfFr4NtHTqtzCbE=; b=1wf+vdKq5Wqn6YnXx8PwZfIuRq1uWBTitTAvHgDIzEv5JxzV7AhhztKodEqAClDvCAgwED 9owGL3bxcDiCZsO5Om3/lw/wrhJQrgnyDVHEajGthH6UlYuYmZ53Gyu6H7w7FyF0p/ss1v 1TQl9JFUyL8iHpTlZBIuIRrpIEraRH4= Received: by mail-wr1-f52.google.com with SMTP id d17so8259549wrs.2 for ; Mon, 09 Jan 2023 06:00:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=6spJSBkMNBIx4KLHawdyLw2aO1QaVfFr4NtHTqtzCbE=; b=nkbYfZjyGvvRuFvxDCyjPlGYpkB6tABKLC5RkpfMPARFPnxqaKS1Eut4diploFKW0H 4/0NMSkAWlKiyYRO7IJm2irGCh6pk+rZRDzwr6sRII7mO/TFDeOmLVzrwvAqdgFWct/u bBq9vKpFDVZSWI5HyMoPJQldKMMX21zJp44Oz1dyQE5zqaUVoZIIqrqBMBKrHwGFKYqA /KtCGJEvUSxhbqT9jXgwqegtYf28MX6zvdA5pAbKfs4PP8keCdiVSUepCnaSPtnxx2o3 oceDw9YdMxOn0ytwGHLFIRgetKeoAT3oi/ouQ0Hf5D5DPUAG1t2JayHQe/+Lj0626zLe ru3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=6spJSBkMNBIx4KLHawdyLw2aO1QaVfFr4NtHTqtzCbE=; b=z0IShKZhu9PIWkelLbt78PxS/QD77c9OhnymaB4On+DheMsQ5cG3LE3jfGBD8hntr5 Yvog+T3S9VKpMt63wIlq/cahhRwEZEBH2b6l0FOv4+/V7IYYEyrsAPcBHcYmgw2hf3z/ aRYLMnV8H6eP4nUF134YrugoXRn/IrD0n42NDWpGKmNuwbM2BZ97aKfFEW0SMmti8vIB wrkvdwNxvtofMQec5s/f+UVn5LDg+CMkhuJfrzi6VRh/1dgdhIuH2yXbpNnnbTPM5tEs l9y5ZKsIWNcmA7l2onAxdwu5IuDHJZpZjXl7cVb8cZfEo4blHVbGHDfAEx3fVWjLbnJC 8RNQ== X-Gm-Message-State: AFqh2kpk3fx5LtwWcKi6y0icTlkAOIIe+rugTwHMFxIjJ0/PKYWrLunl J+qCYPuS8UiXUgJf+Zjwc+M= X-Google-Smtp-Source: AMrXdXvGu2ASXCfdXWvXzwAEeGjzYTgMzEXlB4bWjfNqar20I5zAnR0hWG1MoWde0sg/fCYFFzh19w== X-Received: by 2002:adf:fd0b:0:b0:29b:2ce6:9f2e with SMTP id e11-20020adffd0b000000b0029b2ce69f2emr16530171wrr.27.1673272841486; Mon, 09 Jan 2023 06:00:41 -0800 (PST) Received: from localhost (host81-157-153-247.range81-157.btcentralplus.com. [81.157.153.247]) by smtp.gmail.com with ESMTPSA id y7-20020a5d4707000000b00291f1a5ced6sm8526567wrq.53.2023.01.09.06.00.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 06:00:40 -0800 (PST) Date: Mon, 9 Jan 2023 14:00:39 +0000 From: Lorenzo Stoakes To: Mike Rapoport Cc: Jonathan Corbet , Andrew Morton , David Hildenbrand , Johannes Weiner , "Matthew Wilcox (Oracle)" , Mel Gorman , Michal Hocko , Vlastimil Babka , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH 2/2] docs/mm: Physical Memory: add structure, introduction and nodes description Message-ID: References: <20230101094523.1522109-1-rppt@kernel.org> <20230101094523.1522109-3-rppt@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 2F16B1A0035 X-Stat-Signature: ugy73zuw3qzgap3dtk1py71rprpjcpju X-HE-Tag: 1673272842-623754 X-HE-Meta: U2FsdGVkX1+TUbfFutjXdkBzKDfLh+5KFKnPDDtOxFfMKsX1irCTxlJh90XIZgsnci4SVzGQNrh0GCFVZrl0aFSB9c0SF27l3A+IYSzjuacVOM9zOxm1VEdEol0pjJHRQSyDeL2lXU8/6nKZzBt3FKtCygp1XPUN18arS9a+vz4hEa4541AKlha/ndOpzyWaO17eoqsIhXFTpyyOsy5OHdFnSJnwb1YbiPS++fGDuRDLBoRdMci2w7xBqRE0tHjrXs4S40y8QE8JTjnujmqPyGGlgmiEZ1UOSg/eA8EGVCGPamua7Q9s1DDcOWLK6EPoF6DLCdG4ukEToq/gdqDaC6KQfNuB8pvG4fpz8zXSHh2QkolAKnsnqcpKQmNUnq2CM9fObRMUMaYUcUe9tYQ/GPArwdnsB2NaaPrZregWibv6VIKmHtz5cTNhohwaq51n0gtKSAd2jcaPMUSdXwuCqx6ARdnn+GoZb8ZtFXqcrJxLt8THv0pHx+UkXv3t3hfKP8CTg8Ybc9HEeL3nVlAypZcAhpKV+q9MdK1en6azNFEmF0kC51iqlIczSNA4DderscluzMPZ0CxwBIi2kDBYvgA2mlxCLbnkbZo1ScakKOW1Dy+PCpswkT6kdh9bhSEggt3VdUoWhi4Q0Rqq+bp4FdJYfjHLHg0P7mrh6k8EcgEFtnKzW1cp3W8cnpIKuOC33fh3amQLteN2KKapHRgNjncP81nQK8bGg1f6u+s24Zoupd9sRDECePQOP6mv438WBAhiHNKRXK8pgu71bloAvpjXnuPrmHIstyZCpjUdlODSHZCuQGGZcdGiNFgjgt8cF+fDXehPGp1ca13IZ5/pp6pJt+ZMQdhox4ZJcFWsEm9XJOIUs+L98NOQPA9Ss83IJ9GnuNCyEwkqXIQkHSIIcDK1eFqqQqkCW3RJKssPgW2W2yDzpGLq3eElBbpXFltmD0dOpb8dhN+LmmrvCzL jLAy4x8s ajvN/m0u2GJeBve1XS1GXuYw1K0jZPT32rLs7N+Dr2qx+iRvtJlgo8FwbZTTvFKS7RO2qGxQZsUhkWkrPNB+en79Rfw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jan 09, 2023 at 03:33:15PM +0200, Mike Rapoport wrote: > > Absolutely wonderfully written. > > Thanks to Mel :) > I should have known :) > > > +Each node may be divided up into a number of blocks called zones which > > > +represent ranges within memory. These ranges are usually determined by > > > +architectural constraints for accessing the physical memory. A zone is > > > +described by a ``struct zone_struct``, typedeffed to ``zone_t`` and each zone > > > +has one of the types described below. > > > > I don't think it's quite right to say 'may' be divided up into zones, as they > > absolutely will be so (and the entire phsyical memory allocator hinges on being > > zoned, even if trivially in UMA/single zone cases). > > Not necessarily. ZONE_DMA or ZONE_NORMAL may span the entire memory. I see what you mean, here again we get the confusion around zones as a term (And Willy has yet to propose a 'zolio' :), what I meant to say is that every byte of memory is in a zone, though a zone may span a node, multiple nodes or all nodes. > > > + > > > +`ZONE_DMA` and `ZONE_DMA32` > > > + represent memory suitable for DMA by peripheral devices that cannot > > > + access all of the addressable memory. Depending on the architecture, > > > + either of these zone types or even they both can be disabled at build > > > + time using ``CONFIG_ZONE_DMA`` and ``CONFIG_ZONE_DMA32`` configuration > > > + options. Some 64-bit platforms may need both zones as they support > > > + peripherals with different DMA addressing limitations. > > > > It might be worth pointing out ZONE_DMA spans an incredibly little range that > > probably won't matter for any peripherals this side of the cretaceous period, > > On RPi4 ZONE_DMA spans 1G, which is quite some part of the memory ;-) > Ah yeah that's another weirdness, my asahi laptop actually puts everything into ZONE_DMA so fair point. Arches do complicate things... (hence why I limit my scope to only one) > > > + > > > +`ZONE_NORMAL` > > > + is for normal memory that can be accessed by the kernel all the time. DMA > > > + operations can be performed on pages in this zone if the DMA devices support > > > + transfers to all addressable memory. ZONE_NORMAL is always enabled. > > > + > > > > Might be worth saying 'this is where memory ends up if not otherwise in another > > zone'. > > This may not be the case on !x86. Yeah again, I am being a fool because I keep burying in my mind the fact that my Asahi laptop literally doesn't do this... :) I think in 'principle' though it still is where things should go unless you just decide to have the first zone only? But in any case, I think then the original explanation is better. > > > > +`ZONE_HIGHMEM` > > > + is the part of the physical memory that is not covered by a permanent mapping > > > + in the kernel page tables. The memory in this zone is only accessible to the > > > + kernel using temporary mappings. This zone is available only some 32-bit > > > + architectures and is enabled with ``CONFIG_HIGHMEM``. > > > + > > > > I comment here only to say 'wow I am so glad I chose to only focus on 64-bit so > > I could side-step all the awkward discussion of high pages' :) > > > > > +The relation between node and zone extents is determined by the physical memory > > > +map reported by the firmware, architectural constraints for memory addressing > > > +and certain parameters in the kernel command line. > > > > Perhaps worth mentioning device tree here? Though perhaps encapsulated in the > > 'firmware' reference. > > It is :) Ack, and that makes sense > > > > +Node structure > > > +-------------- > > > + > > > +The struct pglist_data is declared in `include/linux/mmzone.h > > > +`_. > > > +Here we briefly describe fields of this structure: > > > > Perhaps worth saying 'The node structure' just to reiterate. > > Not sure I follow, can you phrase the entire sentence? > Sorry I wasn't clear here, I meant to say simply reiterate that the pglist_data struct is the one describing a node. > > > + > > > +General > > > +~~~~~~~ > > > + > > > +`node_zones` > > > + The zones for this node. Not all of the zones may be populated, but it is > > > + the full list. It is referenced by this node's node_zonelists as well as > > > + other node's node_zonelists. > > > > Perhaps worth describing what zonelists (and equally zonerefs) are here or > > above, and that this is the canonical place where zones reside. Maybe reference > > populated_zone() and for_each_populated_zone() in reference to the fact that not > > all here may be populated? > > I'd prefer to start simple and than add more content on top. > Absolutely, makes sense! > > > + > > > +`node_zonelists` The list of all zones in all nodes. This list defines the > > > + order of zones that allocations are preferred from. The `node_zonelists` is > > > + set up by build_zonelists() in mm/page_alloc.c during the initialization of > > > + core memory management structures. > > > + > > > +`nr_zones` > > > + Number of populated zones in this node. > > > + > > > +`node_mem_map` > > > + For UMA systems that use FLATMEM memory model the 0's node (and the only) > > > + `node_mem_map` is array of struct pages representing each physical frame. > > > + > > > +`node_page_ext` > > > + For UMA systems that use FLATMEM memory model the 0's (and the only) node > > > + `node_mem_map` is array of extensions of struct pages. Available only in the > > > + kernels built with ``CONFIG_PAGE_EXTENTION`` enabled. > > > + > > > +`node_start_pfn` > > > + The page frame number of the starting page frame in this node. > > > + > > > +`node_present_pages` > > > + Total number of physical pages present in this node. > > > + > > > +`node_spanned_pages` > > > + Total size of physical page range, including holes. > > > + > > > > I think it'd be useful to discuss briefly the meaning of managed, spanned and > > present pages in the context of zones. > > This will be a part of the Zones section. Makes sense again! Overall it's very good. Nitpicking here really!