From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13A7CC433E0 for ; Fri, 7 Aug 2020 07:09:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BD2D120866 for ; Fri, 7 Aug 2020 07:09:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TmSBvJ3d" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BD2D120866 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 40D308D0090; Fri, 7 Aug 2020 03:09:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3BD2E8D0026; Fri, 7 Aug 2020 03:09:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D34A8D0090; Fri, 7 Aug 2020 03:09:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0150.hostedemail.com [216.40.44.150]) by kanga.kvack.org (Postfix) with ESMTP id 196278D0026 for ; Fri, 7 Aug 2020 03:09:25 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A2DBD180ACF6C for ; Fri, 7 Aug 2020 07:09:24 +0000 (UTC) X-FDA: 77122896648.27.chain87_4f1044226fbe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 78A7E3D663 for ; Fri, 7 Aug 2020 07:09:24 +0000 (UTC) X-HE-Tag: chain87_4f1044226fbe X-Filterd-Recvd-Size: 4996 Received: from mail-oi1-f196.google.com (mail-oi1-f196.google.com [209.85.167.196]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Fri, 7 Aug 2020 07:09:23 +0000 (UTC) Received: by mail-oi1-f196.google.com with SMTP id j7so1066656oij.9 for ; Fri, 07 Aug 2020 00:09:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=uLQHBFTmBmwUqDoaiiE130WzMyrApr6UvOSb4IUTTB8=; b=TmSBvJ3duCrYdmyJEMPijGgyzrsxf4Wsp0DSFhOtD6F+n69OQjKaYTBX+nmXYpsOhZ gsTTpuRxsgbNl9n81S8XZx8geuSicmXGyklUHmZ4oLofkRedfWazPB8v6w0yqdsmFAY8 o8Pg10anUkIByOne6YFJJVmsG0ZX0pgGpdGn1O/H62wI335Bv/hVJvDsqqDGIzoXT17W iAJrDkyVjmqGVlvmcilO59TFZqXSxmd1rUz92L/+5tPfFC+uNpOk1XNHBNZ1w0K8+eQY Q3fObk3+JE8uisNWIkHu5OyZFLANXbjTOWjQDnn0r3L2+zBpKdjeEjgEE2hD+wpF/eJh m5iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uLQHBFTmBmwUqDoaiiE130WzMyrApr6UvOSb4IUTTB8=; b=lw4aepSpLzgqD8yNsleGvcyp3NmgbQDMTLVQUjmXDoKGSKocIhKTE7kFJyr5Qm1qpT MS7j0uBg6Edn1yQUH+sg3JVIZmqNTaPMES/e3TWIxXanficLDVChBaSpyd5bZl8ncy5c sI1qn+MeQZQ7gt6MvTwcnumS0Tm4IC6F/8MHfCpKFI9+mMtyEZZ2bnI8blE2EcxQW12g HLGp1CsKIeavXO5eml9tGKz/wp7y5mCCtklAi4zqkhcy4FEM1ISpOteuH6+AXyI6cGYS lnJ+fdfgCr4I5TzFwXsOz1z3pbX1NRvYtQ5NtFlpPE8RcWnTexI5oaCzkeavr8EV+3/8 f6YQ== X-Gm-Message-State: AOAM531MMftS9KggQkScTIAdEWoc2pH0ci/2iwLRPLTZ15Rknm5714nu LzGhdW5R7VgbKChjwzwhCVwkkJ9XhM1Y261m8Wo= X-Google-Smtp-Source: ABdhPJxUoUC9D1qb/TSkAtYOZ8P2AojnvVZYFEvhk1z4B1ubehX59+bxtrZtMZqnwBJfHPii3A/W7qOJg1fB8ab3phE= X-Received: by 2002:aca:f38b:: with SMTP id r133mr10348803oih.81.1596784163233; Fri, 07 Aug 2020 00:09:23 -0700 (PDT) MIME-Version: 1.0 References: <1596435031-41837-1-git-send-email-pullip.cho@samsung.com> <5f41af0f-4593-3441-12f4-5b0f7e6999ac@redhat.com> In-Reply-To: <5f41af0f-4593-3441-12f4-5b0f7e6999ac@redhat.com> From: Pekka Enberg Date: Fri, 7 Aug 2020 10:08:41 +0300 Message-ID: Subject: Re: [PATCH] mm: sort freelist by rank number To: David Hildenbrand Cc: pullip.cho@samsung.com, Andrew Morton , "linux-mm@kvack.org" , LKML , hyesoo.yu@samsung.com, janghyuck.kim@samsung.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 78A7E3D663 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Cho and David, On Mon, Aug 3, 2020 at 10:57 AM David Hildenbrand wrote: > > On 03.08.20 08:10, pullip.cho@samsung.com wrote: > > From: Cho KyongHo > > > > LPDDR5 introduces rank switch delay. If three successive DRAM accesses > > happens and the first and the second ones access one rank and the last > > access happens on the other rank, the latency of the last access will > > be longer than the second one. > > To address this panelty, we can sort the freelist so that a specific > > rank is allocated prior to another rank. We expect the page allocator > > can allocate the pages from the same rank successively with this > > change. It will hopefully improves the proportion of the consecutive > > memory accesses to the same rank. > > This certainly needs performance numbers to justify ... and I am sorry, > "hopefully improves" is not a valid justification :) > > I can imagine that this works well initially, when there hasn't been a > lot of memory fragmentation going on. But quickly after your system is > under stress, I doubt this will be very useful. Proof me wrong. ;) > > ... I dislike this manual setting of "dram_rank_granule". Yet another mm > feature that can only be enabled by a magic command line parameter where > users have to guess the right values. > > (side note, there have been similar research approaches to improve > energy consumption by switching off ranks when not needed). I was thinking of the exact same thing. PALLOC [1] comes to mind, but perhaps there are more recent ones? I also dislike the manual knob, but is there a way for the OS to detect this by itself? My (perhaps outdated) understanding was that the DRAM address mapping scheme, for example, is not exposed to the OS. I think having more knowledge of DRAM controller details in the OS would be potentially beneficial for better page allocation policy, so maybe try come up with something more generic, even if the fallback to providing this information is a kernel command line option. [1] http://cs-people.bu.edu/rmancuso/files/papers/palloc-rtas2014.pdf - Pekka