From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFB3EC27C54 for ; Thu, 6 Jun 2024 23:25:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 628B76B009F; Thu, 6 Jun 2024 19:25:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D8586B00A1; Thu, 6 Jun 2024 19:25:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4794A6B00A2; Thu, 6 Jun 2024 19:25:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2496E6B009F for ; Thu, 6 Jun 2024 19:25:37 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C2B781209CD for ; Thu, 6 Jun 2024 23:25:36 +0000 (UTC) X-FDA: 82202047872.17.443B8F3 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by imf09.hostedemail.com (Postfix) with ESMTP id C3C75140003 for ; Thu, 6 Jun 2024 23:25:34 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=LkBm34Fc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf09.hostedemail.com: domain of richard.weiyang@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=richard.weiyang@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717716334; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UI9UI2R5SSWpUiLNHTwod6WmQwS+SKWbcbf3ru1VGG0=; b=E8z2OLP6kufSbQIlKoSTNDyZG863+YzH1/pB2mpi580pQ4dBetEEpdm5Vt+FLeDne7PfnJ /+rJ8SLjPuHfeh7QEmud5mn6a/pv0geI2ee5e6o+ugdyW9/2Oh0LgDrN5yWYb7uLlStuFn chFGNe4ZfnyFP58j380mT6kLXuqLSqo= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=LkBm34Fc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf09.hostedemail.com: domain of richard.weiyang@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=richard.weiyang@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717716334; a=rsa-sha256; cv=none; b=Hu44wjE/4MHyPicL4IM7s0ZjDNBF575A92LRInejrHNJAmBK8YX8h/8B9R+mTKwoMAGAqE CiIpf5pmJ5SVlXU2LhkxS3JycL8I1enencneeeD56N0m2Z4RwTWknowHZll5+5wHlU/QGC lRj5AEyD//yrFAF2atHJ4E69wqtittU= Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-a690161d3ccso160885166b.2 for ; Thu, 06 Jun 2024 16:25:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717716333; x=1718321133; darn=kvack.org; h=user-agent:in-reply-to:content-disposition:mime-version:references :reply-to:message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=UI9UI2R5SSWpUiLNHTwod6WmQwS+SKWbcbf3ru1VGG0=; b=LkBm34FcK3eIMrRjUb0s6KdG5dE6oRBGWTEFDm6SwEwK+6FW9qzswJ5LfaeyL0Lsrn cuKXEb3DSazBflCPg4cr1XFQkzbGytZta0y/29KGlwgx7BynukhD2WLgEeM20nyMCUAi Yiy5qq5eKFyuaMakL9rJj3Y2Tn6RgDiUR+FGBv7NYU6B8WlyriXhse2JD/Ad7mH12EqG F8+rMKN9b+bkJ1BkZYA00Wi78m24ugoWzVPRPXmIaeaqS3S6+li280Qepo9r4h+9yOc6 g7ptoptRtoQ+/2rmBx8oeqCPqkgyDIJ6dOk/X2nbP29qdxMIGxrgSvPlHnly10vYN867 McQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717716333; x=1718321133; h=user-agent:in-reply-to:content-disposition:mime-version:references :reply-to:message-id:subject:cc:to:from:date:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=UI9UI2R5SSWpUiLNHTwod6WmQwS+SKWbcbf3ru1VGG0=; b=LzUQJQDiCA4WF0PZel+KJ4E3n8mh6MHrnEPSYGnaUgFhMmF81MCiY+gw0brnYI85Lf Vo3y70B/s0lY9QrkXNIVj9oV3+stqNT5dpzn+sXO3VNQU0ZkYzYCQYWqfz3rcuX0fZEA DqrbSt2v1nuRqejAPD8SOmXA9lgKGtTFh8QeDOVwHqf5RDBDyNjN9AQwaYq0ooG3saYt lLBAh+c0HcZ05mO3qy4eHDb8hJWS4fkS7z/Hw3NIZkKRvXsyYDpVEXvJMy/67pezCHlO tH5UVLKO3TBxnrwChngTBzAcLB/iN5Xves7HrjNpnz4mDNtKh7Pn8ZFf7STBfuknzRyX BGhg== X-Forwarded-Encrypted: i=1; AJvYcCVjJCFoKuX7ThW0cp3qVFOdJbkLBGBwIz1s6UQ1PZsYYKSUb7uzk2omad0hKH3y//hudO0UZPBmLTB9Lzz8QFpUth4= X-Gm-Message-State: AOJu0YymOaQ8DaNid95RqfRq3ujKFzO2o9DfXUgHpwJDYW/zLd94LJQe us9YbV09q05kNWYuD0Oegon4Phnz4J4uJ74nuEQ//xlqDCgTq8dB X-Google-Smtp-Source: AGHT+IH76dVMh8P2qFOrosEjGzoNHBPaUgVrj55aptnVhpsivr+NkRmN8h80xNCEsFKsQZlra3OxTg== X-Received: by 2002:a17:906:2413:b0:a68:e2f5:68b1 with SMTP id a640c23a62f3a-a6cdb1ee266mr57350366b.49.1717716333001; Thu, 06 Jun 2024 16:25:33 -0700 (PDT) Received: from localhost ([185.92.221.13]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a6c805cd687sm152930966b.82.2024.06.06.16.25.32 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Jun 2024 16:25:32 -0700 (PDT) Date: Thu, 6 Jun 2024 23:25:31 +0000 From: Wei Yang To: David Hildenbrand Cc: Wei Yang , rppt@kernel.org, akpm@linux-foundation.org, osalvador@suse.de, linux-mm@kvack.org Subject: Re: [PATCH] mm: increase totalram_pages on freeing to buddy system Message-ID: <20240606232531.qnmxhaaytlh2h4tk@master> Reply-To: Wei Yang References: <20240601133402.2675-1-richard.weiyang@gmail.com> <0316a276-a0d8-4fc2-ad67-0d4732b6d89b@redhat.com> <20240602005820.2uk23ot4mskfl5sl@master> <8297c4d3-f97b-4923-9b27-19294fa130eb@redhat.com> <20240603200123.bvkttf2yqutecjtv@master> <9fa4f1be-790c-4823-aff2-f864807759f1@redhat.com> <20240605224401.re54hhnlkar5akfm@master> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20170113 (1.7.2) X-Rspamd-Queue-Id: C3C75140003 X-Stat-Signature: a7fpw4uddytjy9j8r7hrgeyr5ee5jpcj X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1717716334-591704 X-HE-Meta: U2FsdGVkX18Z0w01iG46VDVel7QT0o9OHzrEANx2cnlhTg79Ulax8J1Qv0+mw56nVp3vZeLAWDmhB3qmAu3XFltM0VJ8LJDWWSPXUs1wTGbovDY/px1TNy5RBL3ZaA9OutXPLdmF4ONF3bPI1ef4s5aRVmVFtVbcfp3ALpzZG8mhYAZgg7VtaDjpQaytS1nVNoxXf33SJIE2tvwBGHsV9FWdYy8HEK9vnOAHzyykHjGbqIFOd5Z0wEbOvQ7jiCsPTB8ES3Rg0DEn1OtCb1dNHyugxtYW1UQz50FbM2DUEukSXF+4XydRxwkrQ2th219fH4jRDh2UamZbeGG/xcOykkDkiCodgcmWCT/vwoCMZpOmmdqnpI88YXdJVojEJAJU4l/LPHK/TUvTHhXZJSzC46gFUvRGFloxcgc2FBBRg85m1cnOdX6+y+6ocTv5EI6ZWYEgAilFWwi5XsE0EaSeYpX+LwyZsRuwEWUtNhz/3k6sCCZZGM+KJJUQWwtUktjC6YSBIe9Dg7/vpyBjL1UCj4szi6bI7GDfZGvvLM7bn+TTAtzKtMXcr5foXBccerdLRrUxg1HcrvV3DsUP+S/IDbpyzr9gGShD0k4AmbJkVfzy6XN/Wcr/knzoOCX8OtFmtd+OcfCvVVsL2FWS/N7bbIfUxLGCaLA7GMFFyu1xH08J9c0oRV3MJzWPYBWCGGB3xASXO+CQPDFm2vAxF1CXDcYYgus5e8z8uXxhBnsBswkNQuxcnPwBZ3uUjY7kCycA/2jtmgZnaiR/cCFrO6D1RD3HoCXNizo5PqDCgAds22O2sbuJqRNIZe+ZzO61pB7eV0qLf9YuGL1LKgoCNLdcWdymrtOqhXYgAeu9PV4i5kBEOtJr+6ZoiJsB3i9Vd98ueSAd8K/UWHqPggf4QEayfr55WBG/Z/aw9CBiNHophmo+8ezfXteNWepQ09BOMsOTFc3C7wgBD3pgzoPDQp9 FjWZzQpE ym01YeI+Yt6LR8ZnDCx4sXKdOB9ElndE/92NilmJB2aRgt8Kc8kdWzSECdfVMQQktKFlL8j5qdUCDG3TPZSAD7tN+iMlo2SayLYtOQQFVIJ1MVuE6rpqM5MrPoY+aBUjmjLRVv6g9ymVhFz+97gbq/qSgG5VsQgqpMtZ3+oHmEjSslTOGHif5Ynj7KzFfS7xPzRt7yAwTNx8gc3ZEopX/fpVBaFKx13HHEn/3vOQApYmnxah7OYEgl87j+GJh+nSdDwlgXcZ9FoU+Nh1/dtfxMjvwOHdwYx8FHZGurQFhL6X0ZOhGM1SYzI/x3nhm2xg/bfqwhDOj+RqvTZxidddWqzgjNK6AmOrfNz3ERhpFuJzpJxD7Hii12Jt6Bm8ceBVSe0mlkhWBn/fqlhZzJJJFZxnd+oKbNdCMr0nttfVJHYVJ8M+JN8noUdqSRaBoKnihktamSPdpXBR1r17qPJGkq8KXpf6EusLKhdD/BDagLs/FqypAjM38qb5zRn2F+Ld5wxWnbpOZH100HogkJR1NgYicb+2XmxByez2O4S+y2LKWRz69mGyl1APDES+n7jyTnD3VhCT1KRWIeypGtJL/pt5bgyuqfMoe1RDvw9/d06v+vJYAb1GxWY7hK848d6B1Jvu2WoFTkR7lrfvONUMG3R2Q7/FWrN4yTrGQesqRTPDbKKSARiZgvwmbGIC2ZgkgkPmKwNbt3Ozre8IRSL6/lXAcguLteAs1wIsm+q1R4w4yhsA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jun 06, 2024 at 09:14:25AM +0200, David Hildenbrand wrote: >On 06.06.24 00:44, Wei Yang wrote: >> On Mon, Jun 03, 2024 at 10:43:51PM +0200, David Hildenbrand wrote: >> > On 03.06.24 22:01, Wei Yang wrote: >> > > On Mon, Jun 03, 2024 at 10:55:10AM +0200, David Hildenbrand wrote: >> > > > On 02.06.24 02:58, Wei Yang wrote: >> > > > > On Sat, Jun 01, 2024 at 06:15:33PM +0200, David Hildenbrand wrote: >> > > > > > On 01.06.24 17:32, David Hildenbrand wrote: >> > > > > > > On 01.06.24 15:34, Wei Yang wrote: >> > > > > > > > Total memory represents pages managed by buddy system. >> > > > > > > >> > > > > > > No, that's managed pages. >> > > > > > > >> > > > > > > > After the >> > > > > > > > introduction of DEFERRED_STRUCT_PAGE_INIT, it may count the pages before >> > > > > > > > being managed. >> > > > > > > > >> > > > > > > >> > > > > > > I recall one reason that is done, so other subsystem know the total >> > > > > > > memory size even before deferred init is done. >> > > > > > > >> > > > > > > > free_low_memory_core_early() returns number of pages for all free pages, >> > > > > > > > even at this moment only early initialized pages are freed to buddy >> > > > > > > > system. This means the total memory at this moment is not correct. >> > > > > > > > >> > > > > > > > Let's increase it when pages are freed to buddy system. >> > > > > > > >> > > > > > > I'm missing the "why", and the very first sentence of this patch is wrong. >> > > > > > >> > > > > > Correction: your statement was correct :) That's why >> > > > > > adjust_managed_page_count() adjusts that as well. >> > > > > > >> > > > > > __free_pages_core() only adjusts managed page count, because it assumes >> > > > > > totalram has already been adjusted early during boot. >> > > > > > >> > > > > > The reason we have this split for now, I think, is because of subsystems that >> > > > > > call totalram_pages() during init. >> > > > > > >> > > > > > So the "why" question remains, because this change has the potential to break >> > > > > > other stuff. >> > > > > > >> > > > > >> > > > > Thanks, I didn't notice this. >> > > > >> > > > I think having your cleanup would be very nice, as I have patches in the >> > > > works that would benefit from being able to move the totalram update from >> > > > memory hotplug code to __free_pages_core(). >> > > > >> > > >> > > I got the same feeling. >> > > >> > > > We'd have to make sure that no code relies on totalram being sane/fixed >> > > > during boot for the initial memory. I think right now we might have such >> > > > code. >> > > > >> > > >> > > One concern is totalram would change when hotplug is enabled. That sounds >> > > those codes should do some re-calculation after totalram changes? >> > >> > We don't have such code in place -- there were discussions regarding that >> > recently. >> > >> > It would be reasonable to take a look at all totalram_pages() users and >> > determine if they could be affected by deferring updating it. >> > >> > At least page_alloc_init_late()->deferred_init_memmap() happens before >> > do_basic_setup()->do_initcalls(), which is good. >> > >> >> But deferred_init_memmap() will spawn threads to do the work. I am afraid >> do_initcalls() won't wait for the completion of defer_init? Do I miss >> something? > >Don't we wait for them to finish? > >/* Block until all are initialised */ >wait_for_completion(&pgdat_init_all_done_comp); You are right. I missed this. So we still need to wait for mm fully initialized and then to initialise others. I thought everything would start in parallel. > >-- >Cheers, > >David / dhildenb -- Wei Yang Help you, Help me