From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D39CCC2D0CD for ; Sat, 17 May 2025 17:29:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 60F026B0085; Sat, 17 May 2025 13:29:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5BDAB6B0088; Sat, 17 May 2025 13:29:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4838E6B0089; Sat, 17 May 2025 13:29:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 247E06B0085 for ; Sat, 17 May 2025 13:29:49 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A091CE1CAD for ; Sat, 17 May 2025 17:29:50 +0000 (UTC) X-FDA: 83453087340.26.08AE5BC Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf15.hostedemail.com (Postfix) with ESMTP id BDC02A000F for ; Sat, 17 May 2025 17:29:48 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=IZrEqKkp; spf=pass (imf15.hostedemail.com: domain of surenb@google.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747502988; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dDIpbhYmZJI1u8nxmq7YJGWDzmMOT17RA4JmcLEon/s=; b=j+X7WmkA0DfjbQ5aAvQ3qrI/URTffY3LxF0OTkY8RHiFFXA1bm3GUD5KV6j9foyqZq1dyr +9nZr6txp0qSkSshQ/ckpJPXc7R6e5JiVWJDvjlwtmYIOj84DEMW+v9b67vc4237gPOkAF 79nrM+WpuTRE8rj/zRSYYZJ/+u4l1LU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747502988; a=rsa-sha256; cv=none; b=ThLkC2Eah0+adV3D+a9limmqqhgmI4c4gPkOoL5gRaQi+ljc/heZeAtsSy4v2o6/EXV9wF LAsiQHuyN9wkd0kOOOAomNWEfzQ+++MggGysUEkIUz2LUjZdnO2pala/EVH0YzCZOahya1 wiQsQqaOSVsuLRc0mYfReb3ErpwROg8= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=IZrEqKkp; spf=pass (imf15.hostedemail.com: domain of surenb@google.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-47666573242so231931cf.0 for ; Sat, 17 May 2025 10:29:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747502988; x=1748107788; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=dDIpbhYmZJI1u8nxmq7YJGWDzmMOT17RA4JmcLEon/s=; b=IZrEqKkpAGFBZn0myORID6RfQ7BdcwDlF8jOFLM5DVps9gKE4fv3zfYao3D4Ni+NRH o62LZdoQ3uosdTBx5MLxwtprzKYvrfwFPtc8gl5jMi4rt769l0I/qcorP3W8fLIREj/z DSKPURExskFB0W2kzJxatTt2yokt8K/xvWpWeJBr6qUIhdA1i1Q8yQKtqCd4Nz9xZ04d 7yfJ0bebtRWmVzkuFIuFPgvCBil7SdwqqguaYAylF+vw//nG3AHCRGH/V6SDtJqh/9T1 yF6Q7hmCTuEa7b1UhM3HZVj94C0CATeEwlzxglhCc73sZJSyKbUU9gpbGkBLXgbvsl9w ts1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747502988; x=1748107788; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dDIpbhYmZJI1u8nxmq7YJGWDzmMOT17RA4JmcLEon/s=; b=szsu4DDE7HOhjXxlH75k7VrktXB/NVc+HbQkBcx38WmY6GKNBdVRjfd9II5i4uSoTE 6Wyoc08SaGQ2OPybHINPKBmYT+L4RYAMl84dgvXny0mO4m5uEF/UF0tv+9TUWHR00fE6 y/rsGlbYFKePzY6jwsSTCnVHSIVvJe7ddzizUm1xSxj+T4EXRk1Yeugjic2NBx8EiQPe cvB7g/vOYtA/quSZC/oMhMMyJzVF2osbe/8xv+6dvkIQEnBwAGzZLvMkcROFfGWHMvan WSoRrfQCP6chBiQ3TXBxA/4hiJEBMWBQbATS5/VgtTbWT+dOs8kXKq3Peo/J3hONxPO0 LEJQ== X-Forwarded-Encrypted: i=1; AJvYcCWSkPxARuc3XCv61izCRc1pEzIvRgfQ5DMzIoiGTx6GbFH09VcrFJ5AxydCAHz2yphK+ufY1Yf0/Q==@kvack.org X-Gm-Message-State: AOJu0Yyo95GJ3fGOj5j912Si6TMXNj6m69q+UL7i3zgr1p4GHVY/h2gd XjZ5fsdJNeRHcPZhgGXMfCCGYpUCR0AvQRH/KwPu4XzNhjycRvoyv/nQARMa7nps3zyASkSvlnc Yt6VKx61n5g5h4Pmbg9q1adBSMLnZ0TiA3tkU1YVA X-Gm-Gg: ASbGncsPCIUMoD9l9fn53aj8MoayotMN45G0hdo5VWO698D4ILOPzodLo+CgA/ZGITB lNAZNf5ZvxTHkdcLxex4FmNBeByVHbwLJ6+WXKkAMu/lu3795Ab8LKfzM60O1hO9M9bWCEUEH6D BJEbeUIlaXgPkRi2puOKMmEqI/5+F4o7g= X-Google-Smtp-Source: AGHT+IG/0RMF6sYCcNYhqKbRYqIDdDvRi9d1UQvHkFGwsFMLMczDo+w9g3gg2XBtks6eeHJduO99zrU0okM/JVIYtn8= X-Received: by 2002:ac8:7e96:0:b0:48a:ba32:370 with SMTP id d75a77b69052e-4958cd26fc4mr3069691cf.10.1747502987311; Sat, 17 May 2025 10:29:47 -0700 (PDT) MIME-Version: 1.0 References: <20250516131246.6244-1-00107082@163.com> <6646d582.18f8.196dd0d5071.Coremail.00107082@163.com> <233aab47.38f2.196df28812a.Coremail.00107082@163.com> In-Reply-To: <233aab47.38f2.196df28812a.Coremail.00107082@163.com> From: Suren Baghdasaryan Date: Sat, 17 May 2025 10:29:35 -0700 X-Gm-Features: AX0GCFs_zppa4PHMtyVLmbWxhyxVX8jVA1kD9vwokc2_eQ05u0RzT_4X3sHsht0 Message-ID: Subject: Re: BUG: unable to handle page fault for address To: David Wang <00107082@163.com> Cc: kent.overstreet@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: BDC02A000F X-Stat-Signature: kz8t4mp9uqz9qe9icbshq8cmwrzsoscd X-Rspam-User: X-HE-Tag: 1747502988-184583 X-HE-Meta: U2FsdGVkX1+TFCrxEYuRPemdUL3UNOp0u6cck1FjIom994rBWmeX46wEiqhOjsAsWASNp3Koze+Bkjc0n57jGKtRurSS6pN8fuG5xBpkOHERtCyyRJi7xxAJ7a0KWZ7BOvlsDjSXYWnmo3Lq+xzS74m4DHIhjaejL2GmNgK7l2TW1HE7Xytfubc5T3H8p/WoawsNudGsesgmeEQ/S8Vjh1BCNBr66uU44Q8LKUBzN/CJPfru6nE5KGx61lnqvVzsY1TGOJB90WRdh9KPvscmMTQC3hl6ikAs6NgiD9J3+MFlBErYS7Ts6FBJdAzHc4+aCmbieH8SBAn3bObaBAvxLf03G/ISjQgJLHk8GdV/WzSaQq15iXV3bn1LSTIGJgLElo5Zr52UGIxZ2n+NPwPAEmEIrm82rPDTYGaycycZb5ZhcIkJN8lRcc2+xGrK4+OhSn4aIoop8rfmBpjvu7h+wo5ONIAidxevdxsM5QVtdcASQnUakRzleKSEeFmQfgA4f3wDQTgv7kH0xbYZS2CANugQV4G9JREzule18BjlVaCxHkEmWWM2Sw4wRraLwS/so0hS65qnhzieLt2oEMAXoL3fPL/0KVwD0MC6LZUyutdsipGKNH/3hdcZTTb4iDoWG5uXpeT1Rh0kpJaHGH59/RqYMaMZeKDzi5GqtuaCo0rrFY+EKk68t/DIj+hihv2Yr5qHR5A2kXEnJ+RMM8Wdp4xKLvozmZQwkaxAWBiu7ZNgJA4+4d+0sEdKhLO1vDRC/ZBGmBcR+7Hw4W8fe4zbn4mtWG9e32hC+G9H6iQqLmgv425uqC4igue+OugYZUsudzvOiDceGnWNbxKozp4XRlOjoq1l+G+Wpn/Fl435S4r+STdu6vX8nmVKmwyGs8MO2Rm9kYFvrqs/HM3qJWI0PXno2yuVUBJpCOqUuLvp9gf96/CjZrQAJmFzhCpNtGk2l2aGsbbRTULOiHr6N+R T4/TUCFT IVk2kjGu+4R336xHuGoMjD7Cv+1OR2dHcCb2nkqy7o5ZQGIHyidYm+rTwJCydvublApgLdLhtgMFqi0ylwHU5MbpG/JENnKBVag9WvpQl6K90SM09bsSqitpI9C6VYF6YwGXVwMabMnZd+hZb8Ggm365cFCb+kwICGoJzHmAOrw3yPWFesGF+YFXtBs5/J0pVa8ZKR0XTakJHKJu+6KboarFSucvijU/8o6Lv+vDJKT2Ib7Yqka2cs0D+OrGJt+s0OL+F3CcHYTCLb66Ypkv8X61GbxRFrRLKMDp3tpV2ltAuTH+bDsFXys4EwTN4pDuD8pvAnCa/pF7zCoCDCh7EgV0Hve+yVa7KTRv3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, May 17, 2025 at 9:51=E2=80=AFAM David Wang <00107082@163.com> wrote= : > > > At 2025-05-18 00:39:30, "Suren Baghdasaryan" wrote: > >On Sat, May 17, 2025 at 12:02=E2=80=AFAM David Wang <00107082@163.com> w= rote: > >> > >> > >> At 2025-05-17 08:11:24, "Suren Baghdasaryan" wrote= : > >> >On Fri, May 16, 2025 at 10:03=E2=80=AFAM Suren Baghdasaryan wrote: > >> >> > >> >> Hi David, > >> >> > >> >> On Fri, May 16, 2025 at 6:13=E2=80=AFAM David Wang <00107082@163.co= m> wrote: > >> >> > > >> >> > Hi, > >> >> > > >> >> > I caught a page fault when I was changing my nvidia driver: > >> >> > (This happens randomly, I can reproduce it with about 1/3 probabi= lity) > >> >> > > >> >> > [Fri May 16 12:05:41 2025] BUG: unable to handle page fault for a= ddress: ffff9d28984c3000 > >> >> > [Fri May 16 12:05:41 2025] #PF: supervisor read access in kernel = mode > >> >> > [Fri May 16 12:05:41 2025] #PF: error_code(0x0000) - not-present = page > >> >> > ... > >> >> > [Fri May 16 12:05:41 2025] RIP: 0010:release_module_tags+0x103/0x= 1b0 > >> >> > ... > >> >> > [Fri May 16 12:05:41 2025] Call Trace: > >> >> > [Fri May 16 12:05:41 2025] > >> >> > [Fri May 16 12:05:41 2025] codetag_unload_module+0x135/0x160 > >> >> > [Fri May 16 12:05:41 2025] free_module+0x19/0x1a0 > >> >> > ... > >> >> > (full kernel logs are pasted at the end.) > >> >> > > >> >> > Using a image with DEBUG_INFO, the calltrack parses as: > >> >> > > >> >> > RIP: 0010:release_module_tags (./include/linux/alloc_tag.h:134 li= b/alloc_tag.c:352 lib/alloc_tag.c:573) > >> >> > [Fri May 16 12:05:41 2025] codetag_unload_module (lib/codetag.c:3= 55) > >> >> > [Fri May 16 12:05:41 2025] free_module (kernel/module/main.c:1305= ) > >> >> > [Fri May 16 12:05:41 2025] __do_sys_delete_module (kernel/module/= main.c:795) > >> >> > > >> >> > The offending lines in my codebase: > >> >> > 126 static inline struct alloc_tag_counters alloc_tag_rea= d(struct alloc_tag *tag) > >> >> > 127 { > >> >> > ... > >> >> > 131 > >> >> > 132 for_each_possible_cpu(cpu) { > >> >> > 133 counter =3D per_cpu_ptr(tag->counters= , cpu); > >> >> > >>>> 134 v.bytes +=3D counter->bytes; <-----= ---------here > >> >> > 135 v.calls +=3D counter->calls; > >> >> > > >> >> > > >> >> > Nvidia drivers are out-tree... there could be some strange behavi= or in it causes this.. but, > >> >> > when I check the code, I got concerned about lifecycle of tag->co= unters. > >> >> > Based on following defination: > >> >> > 108 #define DEFINE_ALLOC_TAG(_alloc_tag) = \ > >> >> > 109 static DEFINE_PER_CPU(struct alloc_tag_counte= rs, _alloc_tag_cntr); \ > >> >> > 110 static struct alloc_tag _alloc_tag __used __a= ligned(8) \ > >> >> > 111 __section(ALLOC_TAG_SECTION_NAME) =3D { = \ > >> >> > 112 .ct =3D CODE_TAG_INIT, = \ > >> >> > 113 .counters =3D &_alloc_tag_cntr }; > >> >> > 114 > >> >> > _alloc_tag_cntr is the data referenced by tag->counters, but they= are in different section, > >> >> > and alloc_tag only prepare storage for section ALLOC_TAG_SECTION_= NAME. > >> >> > right? > >> >> > Then what happens to those ".data..percpu" section when module is= unloaded? > >> >> > Is it safe to keep using those ".data..percpu" section after modu= le unloaded, > >> >> > or even during module is unloading? > >> >> > >> >> Yes, I think you are right, free_module() calls percpu_modfree() wh= ich > >> >> would free the per-cpu memory allocated for the module. Before > >> >> 0db6f8d7820a ("alloc_tag: load module tags into separate contiguous > >> >> memory") we would not unload the module if there were tags which we= re > >> >> still in use. After that change we load module tags into separate > >> >> memory, so I expected this to work but due to this external referen= ce > >> >> it indeed should lead to UAF. > >> >> I think the simplest way to fix this would be to bypass > >> >> percpu_modfree() inside free_module() when there are module tags st= ill > >> >> referenced, store mod->percpu inside alloc_tag_module_section and f= ree > >> >> it inside clean_unused_module_areas_locked() once we know the count= ers > >> >> are not used anymore. I'll take a stab at it and will send a patch = for > >> >> testing today. > >> > > >> >Ok, I went with another implementation, instead dynamically allocatin= g > >> >percpu memory for modules at the module load time. This has another > >> >advantage of not needing extra PERCPU_MODULE_RESERVE currently > >> >required for memory allocation tagging to work. > >> >David, the patch is posted at [1]. Please give it a try and let me > >> >know if the fix works for you. > >> >Thanks, > >> >Suren. > >> > > >> >[1] https://lore.kernel.org/all/20250517000739.5930-1-surenb@google.c= om/ > >> > > >> > > >> >> Thanks, > >> >> Suren. > >> >> > >> > >> Hi, the patch does fix my issue. > >> I now have another similar concern about modules RO data, > >> The codetag defined as > >> 24 struct codetag { > >> 25 unsigned int flags; /* used in later patches */ > >> 26 unsigned int lineno; > >> 27 const char *modname; > >> 28 const char *function; > >> 29 const char *filename; > >> 30 } __aligned(8); > >> > >> Those modname/function/filename would refer to RO data section, right? > >> When module unloaded, its RO data section would be released at some po= int. > >> My question is is it safe to use RO data during module unload? because= these > >> lines seems to access those data: > >> > >> + pr_info("%s:%u module %s func:%s has %llu allo= cated at module unload\n", > >> + tag->ct.filename, tag->ct.lineno, tag-= >ct.modname, > >> + tag->ct.function, counter.bytes); > > > >These lines are called from release_module_tags() using this call chain: > > > >delete_module() > > free_module() > > codetag_unload_module() > > release_module_tags() > > > >and codetag_unload_module() is called at the very beginning of > >free_module(), when no other module memory has been freed yet. So, > >from what I understand, this should be safe. > >After we unload the module these pointers inside the tags will be > >dandling but we should not be using them anymore since we do not > >report unloaded modules. Do you see some usage that I missed? > > Why data..percpu. is different. The page fault error caught when I reinst= all nvidia drivers is also > raised from release_module_tags(). > > Is data..percpu. section released earlier? No but counters are different because the allocations that still reference these tags from unloaded modules will be decrementing them when they are freed. That's where UAF is coming from. So, the counters might be accessed after the module is unloaded, other fields should not. > > Thanks > David > >