From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2A71C00A89 for ; Thu, 5 Nov 2020 08:29:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1506820729 for ; Thu, 5 Nov 2020 08:29:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="X8RLCUcr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1506820729 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 112C26B009F; Thu, 5 Nov 2020 03:29:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C4356B00A0; Thu, 5 Nov 2020 03:29:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF45E6B00A1; Thu, 5 Nov 2020 03:29:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0145.hostedemail.com [216.40.44.145]) by kanga.kvack.org (Postfix) with ESMTP id B57FF6B009F for ; Thu, 5 Nov 2020 03:29:35 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 57B8B3625 for ; Thu, 5 Nov 2020 08:29:35 +0000 (UTC) X-FDA: 77449690710.10.nose49_4403196272c8 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id 3322616A4A1 for ; Thu, 5 Nov 2020 08:29:35 +0000 (UTC) X-HE-Tag: nose49_4403196272c8 X-Filterd-Recvd-Size: 8640 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Thu, 5 Nov 2020 08:29:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1604564974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oCDxxImzZ+LzgA8fVQGOhq1vCKWuV/eegehVAiEkD7I=; b=X8RLCUcrDIdWn8KNkG0Cv8npw0547XBuv8BB4+hZiyAMI/qkQKYYMQo254PNB6SbcF1M4m l7YyTDjysoqgYH7olIRMCft4edSyUcUT5N5u8MhP5ZSYbS5w3BDQgSqNpFFsiRZAcbilmU lmXsKgOxOztCOoChwuVlJTnEjjygtr4= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-97-tsFjCXNmNLW_Q3SAPTiidA-1; Thu, 05 Nov 2020 03:29:32 -0500 X-MC-Unique: tsFjCXNmNLW_Q3SAPTiidA-1 Received: by mail-wm1-f71.google.com with SMTP id y1so310905wma.5 for ; Thu, 05 Nov 2020 00:29:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:mime-version:subject :from:in-reply-to:cc:date:message-id:references:to; bh=adPaZhsg9rSI30Nv3OJs5JJH153rGjOkRFJFWWaQFmA=; b=gHIxv1LI7qJSkuTmQybc1byRzqdR/b2FaN90Q2r/ThgaI6aayec5BPF1rao++837We mLGE6/8he5wvs0UGWYPdVe24gBpEatbbjgAulP/cwHU2HtnpWyyRwPkUz2mi23IG0zCS OATSHvX9Psg6ngDFiMULEEJoWx08RWdl5G5GMzNCOzfe8YCeGPgSNzCEFOVfWkfEx1Ws bpDlxGMnyvKGpk09hg19BEJkiFcmM6vOxWudbZhEDpXNPO94DM1fWjD2Pkztwj4Q3btK WEgYvP7a2tTUiDovFGINxLFIvns9X0EvjwSa//hqpMDlliKxc7N9BsCGaa/RbKggcKTd Wqow== X-Gm-Message-State: AOAM533U6jRBfMlIhoHkLuakRkpVdNg3mrCcAlfTF3OmWIfdqVpFLs0a VvVTlewaGv9bq1jsBw/Lkh4t2/1sLn2/hM+2VqKer5/X0sNohn3P3FXL8mIJBy35NYpQ7+x+ULk MtVXwitnLBZo= X-Received: by 2002:a5d:688c:: with SMTP id h12mr1528854wru.92.1604564970848; Thu, 05 Nov 2020 00:29:30 -0800 (PST) X-Google-Smtp-Source: ABdhPJwa2txjVKqbSfl58jHdphu7Qv2zDJxMqnz9YHvcTy6jiI+C7kFhAWNcUEsLj0a4rpiZiHQbAQ== X-Received: by 2002:a5d:688c:: with SMTP id h12mr1528817wru.92.1604564970537; Thu, 05 Nov 2020 00:29:30 -0800 (PST) Received: from ?IPv6:2003:d8:2f34:7900:7495:eb9e:985d:aaa0? (p200300d82f3479007495eb9e985daaa0.dip0.t-ipconnect.de. [2003:d8:2f34:7900:7495:eb9e:985d:aaa0]) by smtp.gmail.com with ESMTPSA id e25sm1381379wrc.76.2020.11.05.00.29.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 05 Nov 2020 00:29:30 -0800 (PST) Mime-Version: 1.0 (1.0) Subject: Re: [PATCH v1 4/4] powernv/memtrace: don't abuse memory hot(un)plug infrastructure for memory allocations From: David Hildenbrand In-Reply-To: <87o8kcttjp.fsf@mpe.ellerman.id.au> Cc: David Hildenbrand , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, Michal Hocko , Benjamin Herrenschmidt , Paul Mackerras , Rashmica Gupta , Andrew Morton , Mike Rapoport , Michal Hocko , Oscar Salvador , Wei Yang Date: Thu, 5 Nov 2020 09:29:29 +0100 Message-Id: <1D39DC0E-C07A-4B9E-B811-67684A4A0FE9@redhat.com> References: <87o8kcttjp.fsf@mpe.ellerman.id.au> To: Michael Ellerman X-Mailer: iPhone Mail (18A8395) Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > Am 05.11.2020 um 03:53 schrieb Michael Ellerman : >=20 > =EF=BB=BFDavid Hildenbrand writes: >> Let's use alloc_contig_pages() for allocating memory and remove the >> linear mapping manually via arch_remove_linear_mapping(). Mark all pages >> PG_offline, such that they will definitely not get touched - e.g., >> when hibernating. When freeing memory, try to revert what we did. >> The original idea was discussed in: >> https://lkml.kernel.org/r/48340e96-7e6b-736f-9e23-d3111b915b6e@redhat.co= m >> This is similar to CONFIG_DEBUG_PAGEALLOC handling on other >> architectures, whereby only single pages are unmapped from the linear >> mapping. Let's mimic what memory hot(un)plug would do with the linear >> mapping. >> We now need MEMORY_HOTPLUG and CONTIG_ALLOC as dependencies. >> Simple test under QEMU TCG (10GB RAM, single NUMA node): >> sh-5.0# mount -t debugfs none /sys/kernel/debug/ >> sh-5.0# cat /sys/devices/system/memory/block_size_bytes >> 40000000 >> sh-5.0# echo 0x40000000 > /sys/kernel/debug/powerpc/memtrace/enable >> [ 71.052836][ T356] memtrace: Allocated trace memory on node 0 at 0x0= 000000080000000 >> sh-5.0# echo 0x80000000 > /sys/kernel/debug/powerpc/memtrace/enable >> [ 75.424302][ T356] radix-mmu: Mapped 0x0000000080000000-0x00000000c0= 000000 with 64.0 KiB pages >> [ 75.430549][ T356] memtrace: Freed trace memory back on node 0 >> [ 75.604520][ T356] memtrace: Allocated trace memory on node 0 at 0x0= 000000080000000 >> sh-5.0# echo 0x100000000 > /sys/kernel/debug/powerpc/memtrace/enable >> [ 80.418835][ T356] radix-mmu: Mapped 0x0000000080000000-0x0000000100= 000000 with 64.0 KiB pages >> [ 80.430493][ T356] memtrace: Freed trace memory back on node 0 >> [ 80.433882][ T356] memtrace: Failed to allocate trace memory on node= 0 >> sh-5.0# echo 0x40000000 > /sys/kernel/debug/powerpc/memtrace/enable >> [ 91.920158][ T356] memtrace: Allocated trace memory on node 0 at 0x0= 000000080000000 >=20 > I gave this a quick spin on a real machine, seems to work OK. >=20 > I don't have the actual memtrace tools setup to do an actual trace, will > try and get someone to test that also. >=20 > One observation is that previously the memory was zeroed when enabling > the memtrace, whereas now it's not. >=20 > eg, before: >=20 > # hexdump -C /sys/kernel/debug/powerpc/memtrace/00000000/trace=20 > 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |............= ....| > * > 10000000 >=20 > whereas after: >=20 > # hexdump -C /sys/kernel/debug/powerpc/memtrace/00000000/trace > 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |............= ....| > * > 00000080 e0 fd 43 00 00 00 00 00 e0 fd 43 00 00 00 00 00 |..C.......C.= ....| > 00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |............= ....| > * > 00000830 98 bf 39 00 00 00 00 00 98 bf 39 00 00 00 00 00 |..9.......9.= ....| > 00000840 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |............= ....| > * > 000008a0 b0 c8 47 00 00 00 00 00 b0 c8 47 00 00 00 00 00 |..G.......G.= ....| > 000008b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |............= ....| > ... > 0fffff70 78 53 49 7d 00 00 29 2e 88 00 92 41 01 00 49 39 |xSI}..)....A= ..I9| > 0fffff80 b4 07 4a 7d 28 f8 00 7d 00 48 08 7c 0c 00 c2 40 |..J}(..}.H.|= ...@| > 0fffff90 2d f9 40 7d f0 ff c2 40 b4 07 0a 7d 00 48 8a 7f |-.@}...@...}= .H..| > 0fffffa0 70 fe 9e 41 cc ff ff 4b 00 00 00 60 00 00 00 60 |p..A...K...`= ...`| > 0fffffb0 01 00 00 48 00 00 00 60 00 00 a3 2f 0c fd 9e 40 |...H...`.../= ...@| > 0fffffc0 00 00 a2 3c 00 00 a5 e8 00 00 62 3c 00 00 63 e8 |...<......b<= ..c.| > 0fffffd0 01 00 20 39 83 02 80 38 00 00 3c 99 01 00 00 48 |.. 9...8..<.= ...H| > 0fffffe0 00 00 00 60 e4 fc ff 4b 00 00 80 38 78 fb e3 7f |...`...K...8= x...| > 0ffffff0 01 00 00 48 00 00 00 60 2c fe ff 4b 00 00 00 60 |...H...`,..K= ...`| > 10000000 >=20 >=20 > That's a nice way for root to read kernel memory, so we should probably > add a __GFP_ZERO or memset in there somewhere. Thanks for catching that! Will have a look on Monday if alloc_contig_pages(= ) already properly handled __GFP_ZERO so we can use it, otherwise I=E2=80= =98ll fix that. I don=E2=80=98t recall that memory hotunplug does any zeroing - that=E2=80= =98s why I didn=E2=80=98t add any explicit zeroing. Could be you were just = lucky in your experiment - I assume we=E2=80=98ll leak kernel memory alread= y. Thank! > cheers