From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5E9FC32754 for ; Tue, 6 Aug 2019 16:15:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6E688216F4 for ; Tue, 6 Aug 2019 16:15:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=intel-com.20150623.gappssmtp.com header.i=@intel-com.20150623.gappssmtp.com header.b="irgws0PH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6E688216F4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 05D966B0008; Tue, 6 Aug 2019 12:15:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F29576B000C; Tue, 6 Aug 2019 12:15:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DC9526B0279; Tue, 6 Aug 2019 12:15:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from mail-oi1-f198.google.com (mail-oi1-f198.google.com [209.85.167.198]) by kanga.kvack.org (Postfix) with ESMTP id AAA2C6B0008 for ; Tue, 6 Aug 2019 12:15:39 -0400 (EDT) Received: by mail-oi1-f198.google.com with SMTP id e11so35315968oiy.0 for ; Tue, 06 Aug 2019 09:15:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:mime-version:references :in-reply-to:from:date:message-id:subject:to:cc; bh=XmBWrF/l706tTfdjqSR6o57eiT2cKuc01B3zqvw2hyQ=; b=R3PQAYPnLoRh5qTW9/FDjsr/yNU4frKRymZdOryWeD1w6uzGj/YnUzBqShHaQB7JIF pEHZrw6rfHtVHt3WeKjnL+BBpYBDDDHE5SGA6IjhtCI1bq6ut6idaPW7zBHpLCIAvy9p IekgzQLwrRTjOgTaIVlecgBynHdGweqjZe109vF18jZNnWK38oFhfFiBbhMroEKzg7LC RGLNvzp6fLLEe35WhlMZpf35w5UoTguDOe90LWksqIobNHLJQDbRSwVXN9uLmuSXRZHz VIj4N8RnnMSx4xkqIy5mpFX0+6m6GIwTCoWH9Gvyud1YQ5DtpeyHUYy2jK06A35SfaZw ThpQ== X-Gm-Message-State: APjAAAWa8fEeuJHCTu4gRMh12Qp/y/m8mFqSeUaoDPz0gLEKO3H+buhC lGttOn2Op43slLooQKVw/jNYuv91tVAgM/JX2ToPY8iXaFNkkfoIGNC9cSNMLC9N6dM2b7wABhu Q71fo0lCVjGsyMuQEQG/2csCkHCOUsB3MPnoce+J5DjutlfXIWPLtQgl+OeuxZHwrIQ== X-Received: by 2002:aca:31c5:: with SMTP id x188mr2116249oix.161.1565108139230; Tue, 06 Aug 2019 09:15:39 -0700 (PDT) X-Received: by 2002:aca:31c5:: with SMTP id x188mr2116190oix.161.1565108138234; Tue, 06 Aug 2019 09:15:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1565108138; cv=none; d=google.com; s=arc-20160816; b=iPbAz4mXb4HapNz3CQxtGSEp5YN5PNn8BKPOfXwSwk4inFCcvOHVGFXvcZ1pVcY/h4 6vNsiRJK3b+/Beu4JKbd6FbGTNJDq4Vm8/HH69SfctFb59CBFfUTyIwVfQl5ebdDsl+2 3hx6wWpDlPh8K7LhO8QBTaW1ieKmWZrPO2Rqr64FB9s9OaATXO8pkOdhHCK062xwji8s t5n78c93Pvl7Q4BWQicoqKQTuwTQrC5e8xP9b5NKcmdNG/sTs9igOD5UdTur9VHsfQUi myfPRQREhuNVoXpymGRnxnYqUt7HIY/BSqjnDE4rn4vr9ILGZG90DMhO9OVYy/NA/9Zl wwqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=XmBWrF/l706tTfdjqSR6o57eiT2cKuc01B3zqvw2hyQ=; b=pXo1zP7Ema2QNZU/C1xeP3X1nGC32TE2YigjpNqYVRwJ0X9WxZwEwq/HxjFluGo0Bw ROHPvfjXS9IC0x9MYIbgbSfnr8y8/BqUMjvW1/M6qYrZzapX8+TFqD1a40NQoWME2WF4 GqBLxF3i2gkSKLOVkzPwLUlwznf2gS5Sn10wbb5XAk6A76GkssoWoCwdKHe6KGQ8dtfW vXWnWJLI2WlC44jgNCQ/qzEy2TZIYEj7BzgaPQjNBQWxP5w/MYPFgeuMILuaoZMNixq0 eU5QxVv9bCxKZCQhYu7Wav96Wk3fOSHzRRZBwvOPae7NxVVYqN6KTblNVlrNPX7/kHF8 HjCg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=irgws0PH; spf=pass (google.com: domain of dan.j.williams@intel.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id k16sor45723560otf.68.2019.08.06.09.15.38 for (Google Transport Security); Tue, 06 Aug 2019 09:15:38 -0700 (PDT) Received-SPF: pass (google.com: domain of dan.j.williams@intel.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=irgws0PH; spf=pass (google.com: domain of dan.j.williams@intel.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=XmBWrF/l706tTfdjqSR6o57eiT2cKuc01B3zqvw2hyQ=; b=irgws0PHUZn4pJRAqbDrBGoE91h4tKEf578kGsM+bhYBlN6zwuD6eCoEh9iXCZaWgX jj7qvYikZUNeXPLxtdUYy81ZnldK54ER7B463rLo2I40Xv+1Elh0MRjmaliKPrSqg1BZ 7Ar/Dl6ahyj1hhsKqwS5AkMp9ZYEqKFoEEjR0L9FJd5StW8e9YahlfT3s3GuWOQadX+a Q7YL8GSIzyttc14Y6B8Pdnrjx0OeGfTv51MuwzhtZRkBZNsuigWjWXh9XmyoWLwL5QDL WekEBBpKKvxaNdUJBsRNR7qdSORyORQv1GHPQvJVpWEeR7ZwNUPns20p+q+Ns+yixgKz 2azw== X-Google-Smtp-Source: APXvYqwyeSIhx1y4ui2pMFiAgfRnKRGa17W9CpGizynyyEF79cuw/8LleWldT3a5dd0HmWWGVR2QiMroX2nDRghDgeI= X-Received: by 2002:a9d:7248:: with SMTP id a8mr4080253otk.363.1565108137806; Tue, 06 Aug 2019 09:15:37 -0700 (PDT) MIME-Version: 1.0 References: <20190725023100.31141-1-t-fukasawa@vx.jp.nec.com> <20190725023100.31141-3-t-fukasawa@vx.jp.nec.com> <20190725090341.GC13855@dhcp22.suse.cz> <40b3078e-fb8b-87ef-5c4e-6321956cc940@vx.jp.nec.com> <20190726070615.GB6142@dhcp22.suse.cz> <3a926ce5-75b9-ea94-d6e4-6888872e0dc4@vx.jp.nec.com> <20190806064636.GU7597@dhcp22.suse.cz> In-Reply-To: <20190806064636.GU7597@dhcp22.suse.cz> From: Dan Williams Date: Tue, 6 Aug 2019 09:15:25 -0700 Message-ID: Subject: Re: [PATCH 2/2] /proc/kpageflags: do not use uninitialized struct pages To: Michal Hocko Cc: Toshiki Fukasawa , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , "akpm@linux-foundation.org" , "adobriyan@gmail.com" , "hch@lst.de" , Naoya Horiguchi , Junichi Nomura , "stable@vger.kernel.org" Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Aug 5, 2019 at 11:47 PM Michal Hocko wrote: > > On Mon 05-08-19 20:27:03, Dan Williams wrote: > > On Sun, Aug 4, 2019 at 10:31 PM Toshiki Fukasawa > > wrote: > > > > > > On 2019/07/26 16:06, Michal Hocko wrote: > > > > On Fri 26-07-19 06:25:49, Toshiki Fukasawa wrote: > > > >> > > > >> > > > >> On 2019/07/25 18:03, Michal Hocko wrote: > > > >>> On Thu 25-07-19 02:31:18, Toshiki Fukasawa wrote: > > > >>>> A kernel panic was observed during reading /proc/kpageflags for > > > >>>> first few pfns allocated by pmem namespace: > > > >>>> > > > >>>> BUG: unable to handle page fault for address: fffffffffffffffe > > > >>>> [ 114.495280] #PF: supervisor read access in kernel mode > > > >>>> [ 114.495738] #PF: error_code(0x0000) - not-present page > > > >>>> [ 114.496203] PGD 17120e067 P4D 17120e067 PUD 171210067 PMD 0 > > > >>>> [ 114.496713] Oops: 0000 [#1] SMP PTI > > > >>>> [ 114.497037] CPU: 9 PID: 1202 Comm: page-types Not tainted 5.3.0-rc1 #1 > > > >>>> [ 114.497621] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.0-0-g63451fca13-prebuilt.qemu-project.org 04/01/2014 > > > >>>> [ 114.498706] RIP: 0010:stable_page_flags+0x27/0x3f0 > > > >>>> [ 114.499142] Code: 82 66 90 66 66 66 66 90 48 85 ff 0f 84 d1 03 00 00 41 54 55 48 89 fd 53 48 8b 57 08 48 8b 1f 48 8d 42 ff 83 e2 01 48 0f 44 c7 <48> 8b 00 f6 c4 02 0f 84 57 03 00 00 45 31 e4 48 8b 55 08 48 89 ef > > > >>>> [ 114.500788] RSP: 0018:ffffa5e601a0fe60 EFLAGS: 00010202 > > > >>>> [ 114.501373] RAX: fffffffffffffffe RBX: ffffffffffffffff RCX: 0000000000000000 > > > >>>> [ 114.502009] RDX: 0000000000000001 RSI: 00007ffca13a7310 RDI: ffffd07489000000 > > > >>>> [ 114.502637] RBP: ffffd07489000000 R08: 0000000000000001 R09: 0000000000000000 > > > >>>> [ 114.503270] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000240000 > > > >>>> [ 114.503896] R13: 0000000000080000 R14: 00007ffca13a7310 R15: ffffa5e601a0ff08 > > > >>>> [ 114.504530] FS: 00007f0266c7f540(0000) GS:ffff962dbbac0000(0000) knlGS:0000000000000000 > > > >>>> [ 114.505245] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > > >>>> [ 114.505754] CR2: fffffffffffffffe CR3: 000000023a204000 CR4: 00000000000006e0 > > > >>>> [ 114.506401] Call Trace: > > > >>>> [ 114.506660] kpageflags_read+0xb1/0x130 > > > >>>> [ 114.507051] proc_reg_read+0x39/0x60 > > > >>>> [ 114.507387] vfs_read+0x8a/0x140 > > > >>>> [ 114.507686] ksys_pread64+0x61/0xa0 > > > >>>> [ 114.508021] do_syscall_64+0x5f/0x1a0 > > > >>>> [ 114.508372] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > >>>> [ 114.508844] RIP: 0033:0x7f0266ba426b > > > >>>> > > > >>>> The reason for the panic is that stable_page_flags() which parses > > > >>>> the page flags uses uninitialized struct pages reserved by the > > > >>>> ZONE_DEVICE driver. > > > >>> > > > >>> Why pmem hasn't initialized struct pages? > > > >> > > > >> We proposed to initialize in previous approach but that wasn't merged. > > > >> (See https://marc.info/?l=linux-mm&m=152964792500739&w=2) > > > >> > > > >>> Isn't that a bug that should be addressed rather than paper over it like this? > > > >> > > > >> I'm not sure. What do you think, Dan? > > > > > > > > Yeah, I am really curious about details. Why do we keep uninitialized > > > > struct pages at all? What is a random pfn walker supposed to do? What > > > > kind of metadata would be clobbered? In other words much more details > > > > please. > > > > > > > I also want to know. I do not think that initializing struct pages will > > > clobber any metadata. > > > > The nvdimm implementation uses vmem_altmap to arrange for the 'struct > > page' array to be allocated from a reservation of a pmem namespace. A > > namespace in this mode contains an info-block that consumes the first > > 8K of the namespace capacity, capacity designated for page mapping, > > capacity for padding the start of data to optionally 4K, 2MB, or 1GB > > (on x86), and then the namespace data itself. The implementation > > specifies a section aligned (now sub-section aligned) address to > > arch_add_memory() to establish the linear mapping to map the metadata, > > and then vmem_altmap indicates to memmap_init_zone() which pfns > > represent data. The implementation only specifies enough 'struct page' > > capacity for pfn_to_page() to operate on the data space, not the > > namespace metadata space. > > Maybe I am dense but I do not really understand what prevents those > struct pages to be initialized to whatever state nvidimm subsystem > expects them to be? Is that a initialization speed up optimization? No, not an optimization. If anything a regrettable choice in the initial implementation to not reserve struct page space for the metadata area. Certainly the kernel could fix this going forward, and there are some configurations where even the existing allocation could store those pfns, but there are others that need that reservation. So there is a regression risk for some currently working configurations. As always we could try making the reservation change and fail to instantiate old namespaces that don't reserve enough capacity to see who screams. I think the risk is low, but non-zero. That makes my first choice to teach kpageflags_read() about the constraint. > > The proposal to validate ZONE_DEVICE pfns against the altmap seems the > > right approach to me. > > This however means that all pfn walkers have to be aware of these > special struct pages somehow and that is error prone. True, but what other blind pfn walkers do we have besides kpageflags_read()? I expect most other pfn_to_page() code paths are constrained to known pfns and avoid this surprise, but yes I need to go audit those.