From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63DACC4167B for ; Wed, 29 Nov 2023 06:14:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A265D6B03A8; Wed, 29 Nov 2023 01:14:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9D6176B03A9; Wed, 29 Nov 2023 01:14:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C5826B03AA; Wed, 29 Nov 2023 01:14:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7DA4B6B03A8 for ; Wed, 29 Nov 2023 01:14:20 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2BA2480427 for ; Wed, 29 Nov 2023 06:14:20 +0000 (UTC) X-FDA: 81509977080.11.B54583B Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf26.hostedemail.com (Postfix) with ESMTP id 20D21140005 for ; Wed, 29 Nov 2023 06:14:17 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="Ua/fyBsY"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf26.hostedemail.com: domain of rppt@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701238458; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LdcsS0vhn6bvTwsGjMaGPYjLuIuqItleEr76bGZhHfk=; b=Krz2t76DERQpFiQcdbJCR8WobHU8ALaEhVZzfQJAIOz4zb1abmZcshjvX0M4hbL/uCo6xp XmltzU8eWpnmnlxil2P3OZEsYq2l/hnyLxIOYKQvorHvx9U3B82Ew6NPKqLCQdR1EJKCiW 6OsnhtgPRA3jj7TJ2eBgAGDIKKJk0bo= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="Ua/fyBsY"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf26.hostedemail.com: domain of rppt@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701238458; a=rsa-sha256; cv=none; b=BUoHHX/iiniWNqoKQoW+D1U11o5Lhl/pqVlzJ9lZv5USu6+7IMhqhccn3rD0Vr5BHdRzpi PvGw6b1hRetC6Nf/xJqR845Zl75NUNirK+53I8HY46K1gSCcXpcH+QivXEYiDlPsBRmlIK 7fXep5cJLAQHkyqOT+QNqhSS0k4FNIY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id 5F6D4B8390D; Wed, 29 Nov 2023 06:14:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F0297C433C8; Wed, 29 Nov 2023 06:14:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1701238455; bh=mbLTAPRYp1W0J/xE4UKIoyZfhh5JrrZ9nLLWrbqDiA4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=Ua/fyBsYPFCKRU+n9gjojfDPPjZolZkL02LLa3ytNKIjFIkedHYw6ztgGaRgfSbQp bd4jeL9XWjrHuz5oP2XwJOvq+/jaVEESUMnno0wWNxF/qWY+DKY3+OIjl2mzTAPhQC BNo/MwzRd7g7F4Yj4R0sg28dD+532FAS/7fxleaiJK42hw8yaeWGFddIyb2p2RSdnw tWscpglgRN2p1s4mjGQKCGgMq34Vw5CVS3itPdDAwP6OSxGl3bX0WPNsOvmuGUdjGu wn92Acg55YqavKhcOdB7lQYkqGFFjVHIdQR5el2X/Jb+mYKiNtYPCue5w658OP2e8b FGSihS5oElpnw== Date: Wed, 29 Nov 2023 08:14:00 +0200 From: Mike Rapoport To: Serge Semin Cc: Thomas Bogendoerfer , Andrew Morton , Matthew Wilcox , Alexey Malahov , Arnd Bergmann , Aleksandar Rikalo , Aleksandar Rikalo , Dragan Mladjenovic , Chao-ying Fu , Jiaxun Yang , Yinglu Yang , Tiezhu Yang , Marc Zyngier , linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 5/7] mm/mm_init.c: Extend init unavailable range doc info Message-ID: <20231129061400.GK636165@kernel.org> References: <20231122182419.30633-1-fancer.lancer@gmail.com> <20231122182419.30633-6-fancer.lancer@gmail.com> <20231123101854.GF636165@kernel.org> <20231124081900.GG636165@kernel.org> <20231128071339.GJ636165@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 20D21140005 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: c1paq7y3tpot5fduq6fhjm84j34pxmr4 X-HE-Tag: 1701238457-23676 X-HE-Meta: U2FsdGVkX19N2X3Vp3t+JTxKP8nNsg2dgy0XEfD4338b93RouNmoonRykUttUCMetJ2AL1u7CN8/5l2kOwZzk66UzbW4Tz5EC4WVH7amXeSocEZjrpp8tLQdTMbA9vx+q9UVQ0MSexkvpbybs31Jrtx/epXkiqcn/QR2NZ0cIy5/6lKU+jNMsDN2bsg7tm4eXbHdTBTKPzEAZ1Ss8SgPWbN+F1yL+tjWHzU8RMavmolC8LTstBC7qPTPS4voYczQlnxmuSJCEgIYLweEj5RpMfF8omvAj5sZT+WvK4CAV9wh4OiKazWJB+J/TIoHud4YCDTSgnaycVishKRPM2PNQfdOs+t7/cH6nDMHz62CYhaj/KDUoTBsdFQ8jYw9XtJNF29RdaLqMpYFIkUytSjFDd5J+EXn8yCs3ZEJhqSYz1+zqo6EIfgM0D6TC3OkeyG83HspkTdUtTbjd4j464XpJ+FmfTPUMrbRQb7jafshW0jyKLBrh13wynN2ouLp0PzFP16riK4WrGvczFiGc8MKX92t1rfGt2ZT+LCt8gwt56rCP5uBdA29nn5N5xqgwAGyEO9a6mCaHkk1FXasFajSuNRVdRdZ0kXbdvhdAOVETCJYJbCDMh/av3bvL+E6k6C30ANHzNUVSBm43EIvg9ivrnf1PaTL6clqQ8bErr0Zm9rvTnCkBLZ2kuhVm+joYyRJWwv+Ip3vgNeDna93HI2mvzg5XqdPVVPkJCky2yCDGRs0KpqrQxnoer3ylhngGm2ByGo1qsl4MRDzSzBhlojl4W6DbTlAeWGud9nJFAbXC4E+pNlrGmNcLa46kXDAB/Q6PxOW1TZFUDBjeM+KLq20TDYlFey/K+fAWH/VSCgE5ceQSTlT+YCiKsYOIcibVi4H0pv7jJa0g2BKiIWYFpzGR+OJl3wihaUtzDucWfW1d00GvUeWisgSkXU7NEGXGHPTddYn55ebtNhqxQB+2XL nbeiw+an GcjNlDFNbXoNepizqRd3sUAqwHtJfa6MprRZBpRg1CF9TSd+h7uPKqK1FYAyPCILGbcc9GEaQsDCU8nfOW1NV341+rcXbY2NsHzp5PzQWpaHoB1t2xG5F77/zNXa8nsYgv6ZLM3HlK7imVAA3/aOs4VW6IDRSk9JQwja8IynTwvZywOpx8kmg1qTlMu4orzdRqOEyPHchxqunYQhHkX4ak+Y2bPL1rtzvp8GVlj8/BnqLn7Ea5cl2i0jz5vaDVcq8GObFU7nkRlWV6tIcw8jRia7g+Tmz1XcST3VDLjzk6nmmCghH08L7kEMZPfokYW7KE6XF32usv5Q7h7Rt1fp0V9A1VFRSfuWSveV1Of384mxTyD4gkahS5dMR91BElqXl0kvrDrPVCltmnY0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Nov 28, 2023 at 01:51:32PM +0300, Serge Semin wrote: > On Tue, Nov 28, 2023 at 09:13:39AM +0200, Mike Rapoport wrote: > > On Fri, Nov 24, 2023 at 02:18:44PM +0300, Serge Semin wrote: > > > Do you mind posting your physical memory layout? > > I actually already did in response to the last part of your previous > message. You must have missed it. Here is the copy of the message: Sorry, for some reason I didn't scroll down your previous mail :) > > On Fri, Nov 24, 2023 at 02:18:44PM +0300, Serge Semin wrote: > > > On Fri, Nov 24, 2023 at 10:19:00AM +0200, Mike Rapoport wrote: > > > ... > > > > > > > > My guess is that your system has a hole in the physical memory mappings and > > > > with FLATMEM that hole will have essentially unused struct pages, which are > > > > initialized by init_unavailable_range(). But from mm perspective this is > > > > still a hole even though there's some MMIO ranges in that hole. > > > > > > Absolutely right. Here is the physical memory layout in my system. > > > 0 - 128MB: RAM > > > 128MB - 512MB: Memory mapped IO > > > 512MB - 768MB..8.256GB: RAM > > > > > > > > > > > Now, if that hole is large you are wasting memory for unused memory map and > > > > it maybe worth considering using SPARSEMEM. > > > > > > Do you think it's worth to move to the sparse memory configuration in > > > order to save the 384MB of mapping with the 16K page model? AFAIU flat > > > memory config is more performant. Performance is critical on the most > > > of the SoC applications especially when using the 10G ethernet or > > > the high-speed PCIe devices. > > Could you also answer to my question above regarding using the > sparsemem instead on my hw memory layout? Currently MIPS defines section size to 256MB, so with your memory layout with SPARSMEM there will be two sections of 256MB, at 0 and at 512MB, so you'll save memory map for 256M which is roughly 1M with 16k pages. It's possible With SPARSEMEM the pfn_to_page() and page_to_pfn() are a bit longer in terms of assembly instructions, but I really doubt you'll notice any performance difference in real world applications. > > With FLATMEM the memory map exists for that > > hole and hence pfn_valid() returns 1 for the MMIO range as well. That makes > > __update_cache() to check folio state and that check would fail if the memory > > map contained garbage. But since the hole in the memory map is initialized > > with init_unavailable_range() you get a valid struct page/struct folio and > > everything is fine. > > Right. That's what currently happens on MIPS32 and that's what I had > to fix in the framework of this series by the next patch: > Link: https://lore.kernel.org/linux-mips/20231122182419.30633-4-fancer.lancer@gmail.com/ > flatmem version of the pfn_valid() method has been broken due to > max_mapnr being uninitialized before mem_init() is called. So > init_unavailable_range() didn't initialize the pages on the early > bootup stage. Thus afterwards, when max_mapnr has finally got a valid > value any attempts to call the __update_cache() method on the MMIO > memory hole caused the unaligned access crash. The fix for max_mapnr makes pfn_valid()==1 for the entire memory map and this fixes up the struct pages in the hole. > > > > With that, the init_unavailable_range() docs need not mention IO space at > > all, they should mention holes within FLATMEM memory map. > > Ok. I'll resend the patch with mentioning flatmem holes instead of > mentioning the IO-spaces. > > > > > As for SPARSEMEM, if the hole does not belong to any section, pfn_valid() > > will be false for it and __update_cache() won't try to access memory map. > > Ah, I see. In case of the SPARSEMEM config an another version of > pfn_valid() will be called. It's defined in the include/linux/mmzone.h > header file. Right? If so then no problem there indeed. Yes, SPARSMEM uses pfn_valid() defined in include/linux/mmzone.h > -Serge(y) -- Sincerely yours, Mike.