From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07AE0ECE587 for ; Mon, 14 Oct 2019 13:08:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BA63C21882 for ; Mon, 14 Oct 2019 13:08:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BA63C21882 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5FE608E0006; Mon, 14 Oct 2019 09:08:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 587918E0001; Mon, 14 Oct 2019 09:08:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 475F98E0006; Mon, 14 Oct 2019 09:08:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0175.hostedemail.com [216.40.44.175]) by kanga.kvack.org (Postfix) with ESMTP id 2498C8E0001 for ; Mon, 14 Oct 2019 09:08:41 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id B6304181AEF10 for ; Mon, 14 Oct 2019 13:08:40 +0000 (UTC) X-FDA: 76042419600.05.sign57_4b6e0740c2363 X-HE-Tag: sign57_4b6e0740c2363 X-Filterd-Recvd-Size: 5489 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Mon, 14 Oct 2019 13:08:38 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 62208337; Mon, 14 Oct 2019 06:08:37 -0700 (PDT) Received: from [10.162.42.133] (p8cg001049571a15.blr.arm.com [10.162.42.133]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2319A3F68E; Mon, 14 Oct 2019 06:08:26 -0700 (PDT) Subject: Re: + mm-hugetlb-make-alloc_gigantic_page-available-for-general-use.patch added to -mm tree To: Michal Hocko Cc: akpm@linux-foundation.org, ard.biesheuvel@linaro.org, broonie@kernel.org, christophe.leroy@c-s.fr, dan.j.williams@intel.com, dave.hansen@intel.com, davem@davemloft.net, gerald.schaefer@de.ibm.com, gregkh@linuxfoundation.org, heiko.carstens@de.ibm.com, jgg@ziepe.ca, jhogan@kernel.org, keescook@chromium.org, kirill@shutemov.name, linux@armlinux.org.uk, mark.rutland@arm.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mpe@ellerman.id.au, paul.burton@mips.com, paulus@samba.org, penguin-kernel@i-love.sakura.ne.jp, peterz@infradead.org, ralf@linux-mips.org, rppt@linux.vnet.ibm.com, schowdary@nvidia.com, schwidefsky@de.ibm.com, Steven.Price@arm.com, tglx@linutronix.de, vbabka@suse.cz, vgupta@synopsys.com, willy@infradead.org, yamada.masahiro@socionext.com, linux-mm@kvack.org References: <20191011202932.GZoUOoURm%akpm@linux-foundation.org> <20191014121730.GE317@dhcp22.suse.cz> <89f328f0-ca00-a9b0-8df6-808a53bfcdd4@arm.com> <20191014130023.GF317@dhcp22.suse.cz> From: Anshuman Khandual Message-ID: Date: Mon, 14 Oct 2019 18:38:52 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20191014130023.GF317@dhcp22.suse.cz> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 10/14/2019 06:30 PM, Michal Hocko wrote: > On Mon 14-10-19 18:23:00, Anshuman Khandual wrote: >> >> >> On 10/14/2019 05:47 PM, Michal Hocko wrote: >>> On Fri 11-10-19 13:29:32, Andrew Morton wrote: >>>> alloc_gigantic_page() implements an allocation method where it scans over >>>> various zones looking for a large contiguous memory block which could not >>>> have been allocated through the buddy allocator. A subsequent patch which >>>> tests arch page table helpers needs such a method to allocate PUD_SIZE >>>> sized memory block. In the future such methods might have other use cases >>>> as well. So alloc_gigantic_page() has been split carving out actual >>>> memory allocation method and made available via new >>>> alloc_gigantic_page_order(). >>> >>> You are exporting a helper used for hugetlb internally. Is this really >> >> Right, because the helper i.e alloc_gigantic_page() is generic enough which >> scans over various zones to allocate a page order which could not be allocated >> through the buddy. Only thing which is HugeTLB specific in there, is struct >> hstate from where the order is derived with huge_page_order(). Otherwise it >> is very generic. >> >>> what is needed? I haven't followed this patchset but don't you simply >> >> Originally I had just implemented similar allocator inside the test itself >> but then figured that alloc_gigantic_page() can be factored out to create >> a generic enough allocator helper. >> >>> need a generic 1GB allocator? If yes then you should be looking at >> >> The test needs a PUD_SIZE allocator. >> >>> alloc_contig_range. >> >> IIUC alloc_contig_range() requires (start pfn, end pfn) for the region to be >> allocated. But before that all applicable zones need to be scanned to figure >> out any available and suitable pfn range for alloc_contig_range() to try. In >> this case pfn_range_valid_gigantic() check seemed reasonable while scanning >> the zones. >> >> If pfn_range_valid_gigantic() is good enough or could be made more generic, >> then the new factored alloc_gigantic_page_order() could be made a helper in >> mm/page_alloc.c > > OK, thanks for the clarification. This all means that this patch is not > the right approach. If you need a more generic alloc_contig_range then > add it to page_alloc.c and make it completely independent on the hugetlb > config and the code. Hugetlb allocator can reuse that helper. Sure. Just to confirm again, the checks on pfns in pfn_range_valid_gigantic() while looking for contiguous memory is generic enough in it's current form ? static bool pfn_range_valid_gigantic(struct zone *z, unsigned long start_pfn, unsigned long nr_pages) { unsigned long i, end_pfn = start_pfn + nr_pages; struct page *page; for (i = start_pfn; i < end_pfn; i++) { if (!pfn_valid(i)) return false; page = pfn_to_page(i); if (page_zone(page) != z) return false; if (PageReserved(page)) return false; if (page_count(page) > 0) return false; if (PageHuge(page)) return false; } return true; }