From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2846C43334 for ; Tue, 28 Jun 2022 13:42:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 339C46B0071; Tue, 28 Jun 2022 09:42:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E8FE6B0072; Tue, 28 Jun 2022 09:42:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 163FC8E0001; Tue, 28 Jun 2022 09:42:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 08F5E6B0071 for ; Tue, 28 Jun 2022 09:42:34 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D340B7C3 for ; Tue, 28 Jun 2022 13:42:33 +0000 (UTC) X-FDA: 79627759386.26.1515C14 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by imf03.hostedemail.com (Postfix) with ESMTP id 18C8120030 for ; Tue, 28 Jun 2022 13:42:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1656423753; x=1687959753; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=eU6cDt2dD8mM+1EzptzQ28Y5zLtIBvSjYlse5iTMKDQ=; b=EBxFQcHyHxXpdEZw7FTl/B36fHdsxD4pOV16W1LkLv7aMVSOGqg90p2E gkohv3j/GR/PLyd8buQhEuImXN0cF6e/WbPw7aRcXGBDHQ3/qBeCyn5LW Z4bmQiwpNKIxgyXTFD/u9jrsSe04qxBuEiycrCkhNxjp/KDPNlEKqDYAi Y=; Received: from unknown (HELO ironmsg03-sd.qualcomm.com) ([10.53.140.143]) by alexa-out-sd-01.qualcomm.com with ESMTP; 28 Jun 2022 06:42:31 -0700 X-QCInternal: smtphost Received: from nasanex01c.na.qualcomm.com ([10.47.97.222]) by ironmsg03-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2022 06:42:31 -0700 Received: from nalasex01a.na.qualcomm.com (10.47.209.196) by nasanex01c.na.qualcomm.com (10.47.97.222) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Tue, 28 Jun 2022 06:42:31 -0700 Received: from [10.216.8.122] (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Tue, 28 Jun 2022 06:42:28 -0700 Message-ID: <78bfc1da-0bc4-ea43-474f-c51a84920a5c@quicinc.com> Date: Tue, 28 Jun 2022 19:12:25 +0530 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: Discussion on race between freed page_ext access and memory offline operation Content-Language: en-US To: David Hildenbrand , Andrew Morton , Michal Hocko , Vlastimil Babka , Minchan Kim CC: "linux-mm@kvack.org" , LKML References: <59edde13-4167-8550-86f0-11fc67882107@quicinc.com> <8fefe59d-c893-39f4-3225-65343086c867@redhat.com> From: Charan Teja Kalla In-Reply-To: <8fefe59d-c893-39f4-3225-65343086c867@redhat.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nalasex01a.na.qualcomm.com (10.47.209.196) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656423753; a=rsa-sha256; cv=none; b=NVHXbUv4ERv2rGgjgTIOOy8AgHsrcodZsdiIF8y55MCtA/xM4fwGtUS80cJpOq+1mHWVE1 //0ITEJvqm+mkFopP65Ch/hgI3IuHExXo5lLrN7bcaFrPhG5wXa3eIplwsG4Xh1jG3Qj67 1OHCw6IckVqew6axzaN6/V6uQj6FU8k= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcdkim header.b=EBxFQcHy; dmarc=pass (policy=none) header.from=quicinc.com; spf=pass (imf03.hostedemail.com: domain of quic_charante@quicinc.com designates 199.106.114.38 as permitted sender) smtp.mailfrom=quic_charante@quicinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656423753; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eU6cDt2dD8mM+1EzptzQ28Y5zLtIBvSjYlse5iTMKDQ=; b=P9NKS2aA6qZSjHkZb9gkDfY2OLX2BkNbjcJNKEsksIpq41PvHeNXxD8zKB1xrpYtkCFX/0 VdckEVOE4iuE/THBRDoBKwgTSV2wjkM/ZqLrxWKi0870c0P3sz35tOOlIGWAfiPDatXQVM FhbqiQU7ABfR6Dpqm7dX0c5lnRfwDRA= X-Rspam-User: X-Stat-Signature: ef7eo67fwxof8jx98hx9gmkjc5m8f1t5 X-Rspamd-Queue-Id: 18C8120030 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcdkim header.b=EBxFQcHy; dmarc=pass (policy=none) header.from=quicinc.com; spf=pass (imf03.hostedemail.com: domain of quic_charante@quicinc.com designates 199.106.114.38 as permitted sender) smtp.mailfrom=quic_charante@quicinc.com X-Rspamd-Server: rspam03 X-HE-Tag: 1656423752-37970 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Thanks David for the inputs!! On 6/27/2022 10:05 PM, David Hildenbrand wrote: > On 27.06.22 18:09, Charan Teja Kalla wrote: >> The below race between page_ext and online/offline of the respective >> memory blocks will cause use-after-free on the access of page_ext structure. >> >> process1 process2 >> --------- --------- >> a)doing /proc/page_owner doing memory offline >> through offline_pages >> >> b)PageBuddy check is failed >> thus proceed to get the >> page_owner information >> through page_ext access. >> page_ext = lookup_page_ext(page); >> >> migrate_pages(); >> ................ >> Since all pages are successfully >> migrated as part of the offline >> operation,send MEM_OFFLINE notification >> where for page_ext it calls: >> offline_page_ext()--> >> __free_page_ext()--> >> free_page_ext()--> >> vfree(ms->page_ext) >> mem_section->page_ext = NULL >> >> c) Check for the PAGE_EXT flags >> in the page_ext->flags access >> results into the use-after-free(leading >> to the translation faults). >> >> As mentioned above, there is really no synchronization between page_ext >> access and its freeing in the memory_offline. The above is just one >> example but the problem persists in the other paths too involving >> page_ext->flags access(eg: page_is_idle()). >> >> The memory offline steps(roughly) on a memory block is as below: >> 1) Isolate all the pages >> 2) while(1) >> try free the pages to buddy.(->free_list[MIGRATE_ISOLATE]) >> 3) delete the pages from this buddy list. >> 4) Then free page_ext.(Note: The struct page is still alive as it is >> freed only during hot remove of the memory which frees the memmap, which >> steps the user might not perform). >> >> This design leads to the state where struct page is alive but the struct >> page_ext is freed, where the later is ideally part of the former which >> just representing the page_flags. This seems to be a wrong design where >> 'struct page' as a whole is not accessible(Thanks to Minchan for >> pointing this out). > Accessing the struct page -- including any extensions -- is invalid if > the memory section is marked offline. > > Usual PFN walkers use pfn_to_online_page() to make sure we have PFN with > an actual meaning in it. Is there such enforcement from the kernel side to use the pfn_to_online_page() while doing the pfn walk? Eg: In the same read_page_owner()(Not sure of the other places), it is not used while doing the pfn walk. > > There is no real synchronization between pfn_to_online_page() and memory > offline code. For now it wasn't required because it was never relevant > in practice. > Isn't the race here makes the code to still use the page despite it got offlined parallel there by making the statement 'Accessing the struct page -- including any extensions -- is invalid' applicable here. Eg: In the same read_page_owner(), it can go and try to dump the page_owner of a page(agree that it dumps the proper page_owner) in print_page_owner(), where it accesses the page->flags? > After pfn_to_online_page() it takes quite a long time until memory is > actually offlined and then, the memmap is removed. Maybe it's different > for page_ext. > As you already well aware, the memmap will not be removed as long as we are playing just with the offline/online operation but page_ext is freed even during the offline operation making **part of the struct page is mapped and the other part is not**. > > It smells like page_ext should use some mechanism during MEM_OFFLINE to > synchronize against any users of its metadata. Generic memory offlining > code might be the wrong place for that. > > page_ext needs a mechanism to synchronize against any users of the data > it manages. Maybe RCU can help? Let me give a thought about the feasibility of this. But this requires making code at all the places where moving the page_ext users under rcu_lock.