From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5578C433E2 for ; Tue, 15 Sep 2020 21:22:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 473E92078E for ; Tue, 15 Sep 2020 21:22:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 473E92078E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ruivo.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C5F28900070; Tue, 15 Sep 2020 17:22:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C0F5F90006B; Tue, 15 Sep 2020 17:22:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AFE43900070; Tue, 15 Sep 2020 17:22:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0240.hostedemail.com [216.40.44.240]) by kanga.kvack.org (Postfix) with ESMTP id 9B64590006B for ; Tue, 15 Sep 2020 17:22:44 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 5C07E8249980 for ; Tue, 15 Sep 2020 21:22:44 +0000 (UTC) X-FDA: 77266570248.27.clock74_31083a527114 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 33F263D668 for ; Tue, 15 Sep 2020 21:22:44 +0000 (UTC) X-HE-Tag: clock74_31083a527114 X-Filterd-Recvd-Size: 3829 Received: from lobo.ruivo.org (lobo.ruivo.org [173.14.175.98]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Tue, 15 Sep 2020 21:22:43 +0000 (UTC) Received: by lobo.ruivo.org (Postfix, from userid 1011) id D79E75315F; Tue, 15 Sep 2020 17:22:42 -0400 (EDT) Received: from jake.ruivo.org (bob.qemu.ruivo [192.168.72.19]) by lobo.ruivo.org (Postfix) with ESMTPA id 0861E52B01; Tue, 15 Sep 2020 17:22:22 -0400 (EDT) Received: by jake.ruivo.org (Postfix, from userid 1000) id B66B91A18EC; Tue, 15 Sep 2020 17:22:22 -0400 (EDT) Date: Tue, 15 Sep 2020 17:22:22 -0400 From: Aristeu Rozanski To: Oscar Salvador , naoya.horiguchi@nec.com Cc: akpm@linux-foundation.org, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v3 0/5] HWpoison: further fixes and cleanups Message-ID: <20200915212222.GA18315@cathedrallabs.org> References: <20200914101559.17103-1-osalvador@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200914101559.17103-1-osalvador@suse.de> User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspamd-Queue-Id: 33F263D668 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Oscar, Naoya, On Mon, Sep 14, 2020 at 12:15:54PM +0200, Oscar Salvador wrote: > The important bit of this patchset is patch#1, which is a fix to take off > HWPoison pages off a buddy freelist since it can lead us to having HWPoison > pages back in the game without no one noticing it. > So fix it (we did that already for soft_offline_page [1]). > > The other patches are clean-ups and not that important, so if anything, > consider patch#1 for inclusion. > > [1] https://patchwork.kernel.org/cover/11704083/ I found something strange with your and Naoya's hwpoison rework. We have a customer with a testcase that basically does: p1 = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); p2 = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); madvise(p1, size, MADV_MERGEABLE); madvise(p2, size, MADV_MERGEABLE); memset(p1, 'a', size); memset(p2, 'a', size); madvise(p1, size, MADV_SOFT_OFFLINE); madvise(p1, size, MADV_UNMERGEABLE); madvise(p2, size, MADV_UNMERGEABLE); where size is about 200,000 pages. It works on a x86_64 box (with and without the hwpoison rework). On ppc64 boxes (tested 3 different ones with at least 250GB memory) it fails to take a page off the buddy list (page_handle_poison()/take_page_off_buddy()) (madvise MADV_SOFT_OFFLINE returns -EBUSY). Without the hwpoison rework the test passes. Possibly related is that ppc64 takes a long time to run this test and according perf, it spends most of the time clearing pages: 17.15% ksm_poison [kernel.kallsyms] [k] copypage_power7 13.39% ksm_poison [kernel.kallsyms] [k] clear_user_page 8.70% ksm_poison libc-2.28.so [.] __memset_power8 8.63% ksm_poison [kernel.kallsyms] [k] opal_return 6.04% ksm_poison [kernel.kallsyms] [k] __opal_call 2.67% ksm_poison [kernel.kallsyms] [k] opal_call 1.52% ksm_poison [kernel.kallsyms] [k] _raw_spin_lock 1.45% ksm_poison [kernel.kallsyms] [k] opal_flush_console 1.43% ksm_poison [unknown] [k] 0x0000000030005138 1.43% ksm_poison [kernel.kallsyms] [k] opal_console_write_buffer_space 1.26% ksm_poison [kernel.kallsyms] [k] hvc_console_print (...) I've run these tests using mmotm and mmotm with this patchset on top. Do you know what might be happening here? -- Aristeu