From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6305BC3F6B0 for ; Mon, 15 Aug 2022 21:08:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D75568D0002; Mon, 15 Aug 2022 17:08:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D23528D0001; Mon, 15 Aug 2022 17:08:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEAF08D0002; Mon, 15 Aug 2022 17:08:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AF6238D0001 for ; Mon, 15 Aug 2022 17:08:31 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 8499840146 for ; Mon, 15 Aug 2022 21:08:31 +0000 (UTC) X-FDA: 79803065622.29.4166500 Received: from mail-yw1-f177.google.com (mail-yw1-f177.google.com [209.85.128.177]) by imf20.hostedemail.com (Postfix) with ESMTP id 219481C01A7 for ; Mon, 15 Aug 2022 21:08:30 +0000 (UTC) Received: by mail-yw1-f177.google.com with SMTP id 00721157ae682-32fd97c199fso78747627b3.6 for ; Mon, 15 Aug 2022 14:08:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=xtY8G2D96F+0+SkQlxIHRu/YzFBnO7oJKVZE1E2i6CE=; b=T+UlcDuGshW/Wrvb8I2bfsuqi3uLSo/KEj/2dcG1scaCe1BmCEp6nwWKtFOC+RKtke TFymD4X+iW7efPkL0lOEn9zP5UJ1cYBVAryMrp1hgBS6icx840NK8QKskBNX7wbAiEtF dW/QFugI1o7HbO4aznI9osl63Uk6yGlKz73Ib3mAuYRDGpbqXwYWATiQqIwVbsts1jW4 UpMHoVFEmcJwpezxLr2Jm6nrIVF8UkSZ6oQWniKSbNTQFbKZQdLwARSUb5xvZbGS/FTe iNNO0jtAPvVo08Vx2EIZTwNXgjoxoSPyBg2hnXy9SazgSvr8GYOsq+RTnjPCnFbbSeYr Ercg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=xtY8G2D96F+0+SkQlxIHRu/YzFBnO7oJKVZE1E2i6CE=; b=fytQVcIjkucJaphAbiASSEHddyHwlZv2f70jzCyOc79Ssd3+mykhMEJculsjNm5N5y HHUKrJKl/Sb5yKl10kzT5V0I4l1lzarH9SwVLMlq2lgWmLODj1ZK+zZq6LkNjbjy1jOo ke9HDotukoRhd8VlzQDlkzHZmWQ0IewieEFMU3QzWYKenUd9sA18St7uFq2o1A2Jo94k NprWZk+9ghydpEdzctdJI4AjtKsAJfVLdc7J0y5lT/UhLA1Z1KttpqwrI6zr0bB/0ksb Y47AYmgKSiNaYheBEMy8TRwA9f37SVZx2DibX2MrA3YJmkVuBI1/UamOEXUgZeBKaPLk uKww== X-Gm-Message-State: ACgBeo3IwmbSXMbsHYDAxF40dRP7eekk/jsrARgGeXNUiCuciKcU9iM7 izLW1ck/dUck4cGxELuvUVq21fZNoMr0Z3hPlC+WXg== X-Google-Smtp-Source: AA6agR4/slEjKmTYAamIqy5YLABY94pTV9udVVsZlGMG1xpU/FxKXseywg2I31pPZvCw3WMb/rHSq4PunXVanyqOh0E= X-Received: by 2002:a81:1b08:0:b0:31e:5f26:8ae9 with SMTP id b8-20020a811b08000000b0031e5f268ae9mr15210170ywb.155.1660597710276; Mon, 15 Aug 2022 14:08:30 -0700 (PDT) MIME-Version: 1.0 References: <20220614120231.48165-1-kirill.shutemov@linux.intel.com> <20220614120231.48165-3-kirill.shutemov@linux.intel.com> <8cf143e7-2b62-1a1e-de84-e3dcc6c027a4@suse.cz> <20220810141959.ictqchz7josyd7pt@techsingularity.net> In-Reply-To: <20220810141959.ictqchz7josyd7pt@techsingularity.net> From: Dionna Amalie Glaze Date: Mon, 15 Aug 2022 14:08:18 -0700 Message-ID: Subject: Re: [PATCHv7 02/14] mm: Add support for unaccepted memory To: Mel Gorman Cc: Vlastimil Babka , "Kirill A. Shutemov" , Borislav Petkov , Andy Lutomirski , Sean Christopherson , Andrew Morton , Joerg Roedel , Ard Biesheuvel , Andi Kleen , Kuppuswamy Sathyanarayanan , David Rientjes , Tom Lendacky , Thomas Gleixner , Peter Zijlstra , Paolo Bonzini , Ingo Molnar , Dario Faggioli , Dave Hansen , Mike Rapoport , David Hildenbrand , Marcelo Cerri , tim.gardner@canonical.com, Khalid ElMously , philip.cox@canonical.com, "the arch/x86 maintainers" , Linux Memory Management List , linux-coco@lists.linux.dev, linux-efi , LKML , Mike Rapoport Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660597711; a=rsa-sha256; cv=none; b=PGfFE3nsMrLki+3PYMTbvOw9uFmFAQM84aG+s1wigwuoHJguB12qjcWT8kfhGNfCCjYG4G 4AX4CDD6CHihzgIBL9E+iRYMMe4wu8BXdgAGGj3IAfcfxoTq5oJ3VMSHF1Y3x2Ey6mrcPx fwLuuHWS1MGH4YVru1yRpHn+/MFBA3E= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=T+UlcDuG; spf=pass (imf20.hostedemail.com: domain of dionnaglaze@google.com designates 209.85.128.177 as permitted sender) smtp.mailfrom=dionnaglaze@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660597711; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xtY8G2D96F+0+SkQlxIHRu/YzFBnO7oJKVZE1E2i6CE=; b=DPuYEtv03CboSo3/jzm0O10xC0+C6hPzsTqXG13hPJKm8HxYXjIDvQEHe+/zS4sfXAfbjH Bru/3XdPiCjBcvejgf9rAlFDP+6neLoehlFU2mf5znEOVcUiD0p2SCzf2kYblpT5dYnBha c43QVZl9uIm91CBBqemdrzLmDPaXL4c= Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=T+UlcDuG; spf=pass (imf20.hostedemail.com: domain of dionnaglaze@google.com designates 209.85.128.177 as permitted sender) smtp.mailfrom=dionnaglaze@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam05 X-Rspam-User: X-Stat-Signature: 3ui5h71jwmu4bo9xpo4y63cr4wr67toh X-Rspamd-Queue-Id: 219481C01A7 X-HE-Tag: 1660597710-137690 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > > > The unpredictable performance of the application early in boot may be > unacceptable and unavoidable. It might take a long time but it could > eventually generate bug reports about "unpredictable performance early > in boot" that will be hard to track down unless accept_memory is observed > using perf at the right time. Even when that does happen, there will need > to be an option to turn it off if the unpredictable performance cannot > be tolerated. Second, any benchmarking done early in boot is likely to > be disrupted making the series a potential bisection magnet that masks a > performance bug elsewhere in the merge window. I'm doing some boot performance tests now before I run some workload memory acceptance latency tests. Note that this testing is on AMD SEV-SNP, so this patch series on top of the AMD guest patches v12, plus a patch Brijesh Singh wrote to define __accept_memory for SEV-SNP https://github.com/AMDESE/linux/commit/ecae2582666d50ce1e633975d703d2f904183ece I was getting pretty consistent boot times, only going up slightly as the memory size increased, but at 256GB, the VM crashes because it touches some unaccepted memory without first accepting it. 255GB boots fine. The stack track is in mm/page_alloc.c. I've done a little investigation, but I can't account for why there's a hard cutoff of correctness at 256GB [ 0.065563] RIP: 0010:memmap_init_range+0x108/0x173 [ 0.066309] Code: 77 16 f6 42 10 02 74 10 48 03 42 08 48 c1 e8 0c 48 89 c3 e9 3a ff ff ff 48 89 df 48 c1 e7 06 48 03 3d d9 a2 66 ff 48 8d 47 08 47 34 01 00 00 00 48 c7 47 38 00 00 00 00 c7 47 30 ff ff ff ff [ 0.069108] RSP: 0000:ffffffffad603dc8 EFLAGS: 00010082 ORIG_RAX: 0000000000000404 [ 0.070193] RAX: ffffdba740000048 RBX: 0000000000000001 RCX: 0000000000000000 [ 0.071170] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffdba740000040 [ 0.072224] RBP: 0000000000000000 R08: 0000000000001000 R09: 0000000000000000 [ 0.073283] R10: 0000000000000001 R11: ffffffffad645c60 R12: 0000000000000000 [ 0.074304] R13: 00000000000000a0 R14: 0000000000000000 R15: 0000000000000000 [ 0.075285] FS: 0000000000000000(0000) GS:ffffffffadd6c000(0000) knlGS:0000000000000000 [ 0.076365] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 0.077194] CR2: ffffdba740000074 CR3: 0008001ee3a0c000 CR4: 00000000000606b0 [ 0.078209] Call Trace: [ 0.078524] [ 0.078887] ? free_area_init+0x5c1/0x66c [ 0.079417] ? zone_sizes_init+0x52/0x6c [ 0.079934] ? setup_arch+0xa55/0xb6d [ 0.080417] ? start_kernel+0x64/0x65a [ 0.080897] ? secondary_startup_64_no_verify+0xd6/0xdb [ 0.081620] > > -- > Mel Gorman > SUSE Labs -- -Dionna Glaze, PhD (she/her)