From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C97AC433DF for ; Wed, 19 Aug 2020 16:50:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CF31720674 for ; Wed, 19 Aug 2020 16:50:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="W9P7Q8MB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF31720674 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 55BC28D0013; Wed, 19 Aug 2020 12:50:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E4238D0002; Wed, 19 Aug 2020 12:50:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3ABF18D0013; Wed, 19 Aug 2020 12:50:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0066.hostedemail.com [216.40.44.66]) by kanga.kvack.org (Postfix) with ESMTP id 1F6CB8D0002 for ; Wed, 19 Aug 2020 12:50:55 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DB0771EF3 for ; Wed, 19 Aug 2020 16:50:54 +0000 (UTC) X-FDA: 77167907628.05.snail24_5c1854127029 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id A7FA6180260CB for ; Wed, 19 Aug 2020 16:50:54 +0000 (UTC) X-HE-Tag: snail24_5c1854127029 X-Filterd-Recvd-Size: 5431 Received: from mail-il1-f193.google.com (mail-il1-f193.google.com [209.85.166.193]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Wed, 19 Aug 2020 16:50:54 +0000 (UTC) Received: by mail-il1-f193.google.com with SMTP id z3so21254525ilh.3 for ; Wed, 19 Aug 2020 09:50:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=5MrhWijunOxFmEpPBfxi3nnE6h7nM/ESqf0MycrK1bc=; b=W9P7Q8MBYzAc+VHyoKcnaxdruKcc9ZSyiScd2J9v6cmWw9V53iCYL9xm13GYojtTkR O6m5ZuZ9sEndatn2ReAANkq9jfawHew8GkHaxJ7eCnNbboi2Rrr2dEhAGzjGMmiU0eoT Z8cmT1nhszZ5Ckk1v697ZJR/Q+V4CZtXY3FE3dU+Z7xU5ZX8wl9HYV/a6xtCQYb4uP5P mnIleeiKLLzpUzagPV4LUSGt3TjxNf950c8/KRfjdkpt8dxuVU9b+dxFWW9IRB3oVPjc EYqkKspt3S6ncXjxD8QGNeqVZ1flPjVKbPN35R7csKRwUoZQ6YrDEP4I5Q/DzmPV//qt zAJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=5MrhWijunOxFmEpPBfxi3nnE6h7nM/ESqf0MycrK1bc=; b=i+EbPBQH325/3crJVGqyNVcMRIuRD4oxRWbcWvMkyCfeDoWbH6CEYNnG8eKUeo8GTK dEQIoU/j+BHTx8AZ/YDJRaHBpmfiCg+J+bSIElvwyrGWjNG3McI0VQ5GoGMob7/AILiQ K+SR0RZuwJBbr2/T2UGTTCEZX1/hU0ks54o6FRoZ9LmCsYjVFFnQKHZCnqB+v1/ye1MK mbbCjRRRD4kekPDjpQcxwP7rXxS63N8R4DGXntFLNFnoQE2gIfQnImG+OlmSH0TghzCL EQdN2lTg0/UURwp/jYPfb8IL2cno/sSqB7WdNXF4YMFhYpMVCJDLE6oGb+AcoD2WDu4n VJ2g== X-Gm-Message-State: AOAM530JF2p84SyxfOhDZWlHRMlOwJSpDc7QV7egKxizXwXnjjk0d33L s02vxKfHXK5PCkcBNHgzjYxvneCl6gvbC8ten9Y= X-Google-Smtp-Source: ABdhPJzDxPAeWvgDmCjOAXYKMfFt0BmzaZvdHp9BeoKGX7wn8WeyKXgPImYKn2S0snSZja2ODcihfVN2edVUz2wHw/o= X-Received: by 2002:a92:c8c1:: with SMTP id c1mr23266694ilq.42.1597855853425; Wed, 19 Aug 2020 09:50:53 -0700 (PDT) MIME-Version: 1.0 References: <1597816075-61091-1-git-send-email-alex.shi@linux.alibaba.com> <1597816075-61091-2-git-send-email-alex.shi@linux.alibaba.com> <715f1588-9cd5-b845-51a5-ca58549c4d28@arm.com> In-Reply-To: From: Alexander Duyck Date: Wed, 19 Aug 2020 09:50:42 -0700 Message-ID: Subject: Re: [PATCH v2 2/2] mm/pageblock: remove false sharing in pageblock_flags To: Alex Shi Cc: Anshuman Khandual , Matthew Wilcox , David Hildenbrand , Andrew Morton , Hugh Dickins , Alexander Duyck , LKML , linux-mm Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: A7FA6180260CB X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Aug 19, 2020 at 1:11 AM Alex Shi wrote= : > > > > =E5=9C=A8 2020/8/19 =E4=B8=8B=E5=8D=883:57, Anshuman Khandual =E5=86=99= =E9=81=93: > > > > > > On 08/19/2020 11:17 AM, Alex Shi wrote: > >> Current pageblock_flags is only 4 bits, so it has to share a char size > >> in cmpxchg when get set, the false sharing cause perf drop. > >> > >> If we incrase the bits up to 8, false sharing would gone in cmpxchg. a= nd > >> the only cost is half char per pageblock, which is half char per 128MB > >> on x86, 4 chars in 1 GB. > > > > Agreed that increase in memory utilization is negligible here but does > > this really improve performance ? > > > > It's no doubt in theory. and it would had a bad impact according to > commit e380bebe4771548 mm, compaction: keep migration source private to = a single > > but I do have some problem in running thpscale/mmtest. I'd like to see if= anyone > could give a try. > > BTW, I naturally hate the false sharing even it's in theory. Anyone who d= oesn't? :) You keep bringing up false sharing but you don't fix the false sharing by doing this. You are still allowing the flags for multiple pageblocks per cacheline so you still have false sharing even after this. What I believe you are attempting to address is the fact that multiple pageblocks share a single long value and that long is being used with a cmpxchg so you end up with multiple threads potentially all banging on the same value and watching it change. However the field currently consists of only 4 bits, 3 of them for migratetype and 1 for the skip bit. In the case of the 3 bit portion a cmpxchg makes sense and is usually protected by the zone lock so you would only have one thread accessing it in most cases with the possible exception of a section that spans multiple zones. For the case such as the skip bit and MIGRATE_UNMOVABLE (0x0) where we would be clearing or setting the entire mask maybe it would make more sense to simply use an atomic_or or atomic_and depending on if you are setting or clearing the flag? It would allow you to avoid the spinning or having to read the word before performing the operation since you would just be directly applying an AND or OR via a mask value.