From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DB8AC43334 for ; Thu, 9 Jun 2022 02:49:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF8176B0073; Wed, 8 Jun 2022 22:49:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BA65B6B0074; Wed, 8 Jun 2022 22:49:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A20A16B0075; Wed, 8 Jun 2022 22:49:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8F6586B0073 for ; Wed, 8 Jun 2022 22:49:07 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 50D7A30309 for ; Thu, 9 Jun 2022 02:49:07 +0000 (UTC) X-FDA: 79557165534.23.DF5606C Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) by imf23.hostedemail.com (Postfix) with ESMTP id 77358140054 for ; Thu, 9 Jun 2022 02:49:06 +0000 (UTC) Received: by mail-lf1-f45.google.com with SMTP id w20so14271686lfa.11 for ; Wed, 08 Jun 2022 19:49:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz-org.20210112.gappssmtp.com; s=20210112; h=message-id:date:mime-version:user-agent:subject:content-language:to :cc:references:from:in-reply-to:content-transfer-encoding; bh=xAbZi9QDkiuV/tRVVMDCL6/4gqIWbhumm5dxt4PBAPs=; b=mkO6iieR9vohwp1xfYcTplFYTwrNTrfHLcZKf5R3REebDklh4RIYgDqdJZWa0QGSrP 6tXCA6mR0XyW4YI4R8nc/oDsbahxe9NpdpYrOjv9ZkRqR+rigjTTPHtC0Ljslas1edyX T0xrMZSOO0q8Y5I1l0TU8odLEOoKae6tAxxP6wJtELRkMdkoWPE2hkLQSA2Y3KZTn/XS zehFdrrGOqViKQ5Ce0G/+WpU2viDbIADVkdbJ8ZjxB11hk5o3R/R9z7tPkJ2gNJTrTv9 5wAZzPd1Pb9MkaKRPipYauD55cQEeR67bhBywedGMo3pBEt+jggI4Hez+DRU3UmUS+Bu qD6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=xAbZi9QDkiuV/tRVVMDCL6/4gqIWbhumm5dxt4PBAPs=; b=LmhuwgYOzfrypaqihYJS/SfwQdRcCnDFBrNiQK2INqJYsYCMfK6oZk3bQ8W/lWXbpJ 57aaqAgRwFwrWgQGW0FrzvFTsHQVUT4L/T4a/540AO03c9FjjIHY4/JXPHJkcHhAFhoI +A2u2yqq8Ph5TdyMlYgrbMfBDRk+tPp62qUsTet0P84z83cXsHmlp68ya3RlAmXf8kFF cO3nAFLfb0IwsroYfEySmXFGcDQC5ZGblcKjyDTdgYbuWPskq15Y24OA6xRmF0HtE80h Iki2QABBL2DLp+hHJf4oDzL5Kuz2nSv6KzCOoS/7B5DsLVOie/fyf2lGPPB5axAshOQi yvuQ== X-Gm-Message-State: AOAM530j4im3+X7Cd6tPUBhAS8cCN/VE31JVzmiNgxr4xYfT72YsxUPq TfCSdme8C6gj7X+oRqKFnT3R0Q== X-Google-Smtp-Source: ABdhPJynRptJNEYyuq3lMoyZ6zhAFMwwfmmh2PZrONJF64Gu0ggbbHlLz+APd6K22daaYz8evDzC7A== X-Received: by 2002:a05:6512:110e:b0:479:506f:ee04 with SMTP id l14-20020a056512110e00b00479506fee04mr10901344lfg.304.1654742944523; Wed, 08 Jun 2022 19:49:04 -0700 (PDT) Received: from [192.168.1.65] ([46.188.121.129]) by smtp.gmail.com with ESMTPSA id y25-20020a0565123f1900b004786eb19049sm4011264lfa.24.2022.06.08.19.49.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 08 Jun 2022 19:49:04 -0700 (PDT) Message-ID: <2a4cc632-c936-1e42-4fdc-572334c58ee1@openvz.org> Date: Thu, 9 Jun 2022 05:49:02 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: [next] arm64: boot failed - next-20220606 Content-Language: en-US To: Naresh Kamboju , Shakeel Butt , Linux ARM Cc: Stephen Rothwell , Linux-Next Mailing List , open list , regressions@lists.linux.dev, lkft-triage@lists.linaro.org, linux-mm , Andrew Morton , Ard Biesheuvel , Arnd Bergmann , Catalin Marinas , Raghuram Thammiraju , Mark Brown , Will Deacon , Roman Gushchin , Qian Cai References: <20220607162504.7fd5a92a@canb.auug.org.au> From: Vasily Averin In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1654742947; a=rsa-sha256; cv=none; b=nZOQnm1rij+uYm+r4TN9i87wQJdpvtAAsNE6eSttaSLgq2vMeHI2HkhHcdjy9FvC/RIOZ+ E7ScuicZ1p3Vtsw9o1ns1OhCawOCTNAS1sq1Hf9VdVS73sSu84XsOjYA02u1RstsUYy2+X A322g8ICc9Ttco+ck7q/Rmty/zMtPv4= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=openvz-org.20210112.gappssmtp.com header.s=20210112 header.b=mkO6iieR; spf=pass (imf23.hostedemail.com: domain of vvs@openvz.org designates 209.85.167.45 as permitted sender) smtp.mailfrom=vvs@openvz.org; dmarc=pass (policy=none) header.from=openvz.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1654742947; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xAbZi9QDkiuV/tRVVMDCL6/4gqIWbhumm5dxt4PBAPs=; b=pfRUlbxm+ZnoJuLUG5IN61uPzZUfX0entxognpaSP1y0ZFCpctfrPGqcRHJdGBMMp15GsS 9a9FwS5GXrcYWJQf0YaQcwrE9EDYFcMT/4DNG0PI2rhKDtR8PP9CAcmzY8NE7+McJMnQKj mZYkGzULUfJeZ5SNVRrxuDV3fHTrHPU= X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 77358140054 X-Rspam-User: Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=openvz-org.20210112.gappssmtp.com header.s=20210112 header.b=mkO6iieR; spf=pass (imf23.hostedemail.com: domain of vvs@openvz.org designates 209.85.167.45 as permitted sender) smtp.mailfrom=vvs@openvz.org; dmarc=pass (policy=none) header.from=openvz.org X-Stat-Signature: y1zamqdw7agef1ogw1i8bssiuxox97ha X-HE-Tag: 1654742946-841375 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Dear ARM developers, could you please help me to find the reason of this problem? On 6/7/22 18:29, Naresh Kamboju wrote: > On Tue, 7 Jun 2022 at 19:47, Shakeel Butt wrote: >> >> On Tue, Jun 7, 2022 at 3:28 AM Naresh Kamboju wrote: >>> >>> Hi Shakeel, >>> >>>>>> Can you test v5.19-rc1, please? If that does not fail, then you could >>>>>> bisect between that and next-20220606 ... >>>>>> >>>>> >>>>> This is already reported at >>>>> https://lore.kernel.org/all/Yp4F6n2Ie32re7Ed@qian/ and I think we know >>>>> the underlying issue (which is calling virt_to_page() on a vmalloc >>>>> address). >>>> >>>> Sorry, I might be wrong. Just checked the stacktrace again and it >>>> seems like the failure is happening in early boot in this report. >>>> Though the error "Unable to handle kernel paging request at virtual >>>> address" is happening in the function mem_cgroup_from_obj(). >>>> >>>> Naresh, can you repro the issue if you revert the patch "net: set >>>> proper memcg for net_init hooks allocations"? >>> >>> yes. You are right ! >>> 19ee3818b7c6 ("net: set proper memcg for net_init hooks allocations") >>> After reverting this single commit I am able to boot arm64 successfully. >>> >>> Reported-by: Linux Kernel Functional Testing >>> >> >> Can you please run script/faddr2line on "mem_cgroup_from_obj+0x2c/0x120"? > > ./scripts/faddr2line vmlinux mem_cgroup_from_obj+0x2c/0x120 > mem_cgroup_from_obj+0x2c/0x120: > mem_cgroup_from_obj at ??:? > > Please find the following artifacts which are causing kernel crashes. > > vmlinux: https://builds.tuxbuild.com/2ABl8X9kHAAU5MlL3E3xExHFrNy/vmlinux.xz > System.map: https://builds.tuxbuild.com/2ABl8X9kHAAU5MlL3E3xExHFrNy/System.map Dear Naresh, thank you very much mem_cgroup_from_obj(): ffff80000836cf40: d503245f bti c ffff80000836cf44: d503201f nop ffff80000836cf48: d503201f nop ffff80000836cf4c: d503233f paciasp ffff80000836cf50: d503201f nop ffff80000836cf54: d2e00021 mov x1, #0x1000000000000 // #281474976710656 ffff80000836cf58: 8b010001 add x1, x0, x1 ffff80000836cf5c: b25657e4 mov x4, #0xfffffc0000000000 // #-4398046511104 ffff80000836cf60: d34cfc21 lsr x1, x1, #12 ffff80000836cf64: d37ae421 lsl x1, x1, #6 ffff80000836cf68: 8b040022 add x2, x1, x4 ffff80000836cf6c: f9400443 ldr x3, [x2, #8] x5 : ffff80000a96f000 x4 : fffffc0000000000 x3 : ffff80000ad5e680 x2 : fffffe00002bc240 x1 : 00000200002bc240 x0 : ffff80000af09740 x0 = 0xffff80000af09740 is an argument of mem_cgroup_from_obj() according to System.map it is init_net This issue is caused by calling virt_to_page() on address of static variable init_net. Arm64 consider that addresses of static variables are not valid virtual addresses. On x86_64 the same API works without any problem. Unfortunately I do not understand the cause of the problem. I do not see any bugs in my patch. I'm using an existing API, mem_cgroup_from_obj(), to find the memory cgroup used to account for the specified object. In particular, in the current case, I wanted to get the memory cgroup of the specified network namespace by the name taken from for_each_net(). The first object in this list is the static structure unit_net On x86_64 I can translate its address to page: crash> p &init_net $1 = (struct net *) 0xffffffff90c7bdc0 crash> vtop 0xffffffff90c7bdc0 VIRTUAL PHYSICAL ffffffff90c7bdc0 402c7bdc0 PGD DIRECTORY: ffffffff8fe10000 PAGE DIRECTORY: 401e15067 PUD: 401e15ff0 => 401e16063 PMD: 401e16430 => 8000000402c000e3 PAGE: 402c00000 (2MB) PTE PHYSICAL FLAGS 8000000402c000e3 402c00000 (PRESENT|RW|ACCESSED|DIRTY|PSE|NX) PAGE PHYSICAL MAPPING INDEX CNT FLAGS fffff227d00b1ec0 402c7b000 0 0 1 17ffffc0001000 reserved However, as far as I understand this does not work for arm64. Could you please help me to understand what is wrong here? Below are: link to my patch: https://lore.kernel.org/all/20220603182442.63750C385B8@smtp.kernel.org/ and the quote of my investigation of similar report: https://lore.kernel.org/all/Yp4F6n2Ie32re7Ed@qian/ > virt_to_phys used for non-linear address: ffffd8efe2d2fe00 (init_net) > WARNING: CPU: 87 PID: 3170 at arch/arm64/mm/physaddr.c:12 __virt_to_phys ... > Call trace: > __virt_to_phys > mem_cgroup_from_obj > __register_pernet_operations @@ -1143,7 +1144,13 @@ static int __register_pernet_operations(struct list_head *list, * setup_net() and cleanup_net() are not possible. */ for_each_net(net) { + struct mem_cgroup *old, *memcg; + + memcg = mem_cgroup_or_root(get_mem_cgroup_from_obj(net)); <<<< Here + old = set_active_memcg(memcg); error = ops_init(ops, net); + set_active_memcg(old); + mem_cgroup_put(memcg); ... +static inline struct mem_cgroup *get_mem_cgroup_from_obj(void *p) +{ + struct mem_cgroup *memcg; + + rcu_read_lock(); + do { + memcg = mem_cgroup_from_obj(p); <<<< + } while (memcg && !css_tryget(&memcg->css)); ... struct mem_cgroup *mem_cgroup_from_obj(void *p) { struct folio *folio; if (mem_cgroup_disabled()) return NULL; folio = virt_to_folio(p); <<<< here ... static inline struct folio *virt_to_folio(const void *x) { struct page *page = virt_to_page(x); <<< here ... (arm64) #define virt_to_page(x) pfn_to_page(virt_to_pfn(x)) ... #define virt_to_pfn(x) __phys_to_pfn(__virt_to_phys((unsigned long)(x))) ... phys_addr_t __virt_to_phys(unsigned long x) { WARN(!__is_lm_address(__tag_reset(x)), "virt_to_phys used for non-linear address: %pK (%pS)\n", ... virt_to_phys used for non-linear address: ffffd8efe2d2fe00 (init_net) Thank you, Vasily Averin