From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8BC6C4332F for ; Tue, 20 Dec 2022 07:26:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 423EE8E0009; Tue, 20 Dec 2022 02:26:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D48F8E0001; Tue, 20 Dec 2022 02:26:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24E548E0009; Tue, 20 Dec 2022 02:26:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 11A038E0001 for ; Tue, 20 Dec 2022 02:26:07 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D5457805DB for ; Tue, 20 Dec 2022 07:26:06 +0000 (UTC) X-FDA: 80261850732.29.BFF6A44 Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) by imf19.hostedemail.com (Postfix) with ESMTP id 3B7B11A0005 for ; Tue, 20 Dec 2022 07:26:05 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=kxywGUZZ; spf=pass (imf19.hostedemail.com: domain of shiyn.lin@gmail.com designates 209.85.215.173 as permitted sender) smtp.mailfrom=shiyn.lin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671521165; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4qeG6PpkfzCDCCR2FoWrTPHP2/xjfirILLn+aFk9hwU=; b=CqBfW74xYwiO6XcUwlZldh8qY1o7S+Kr+KInR9z19XJ+Fho94+IS4zSKOKHL5iFYXaC/Vs 1NFBywfZWTXgDEQ7hQjAGPm5M5zWq2Pjbv8yaYKbvrMDtAKBrCINeCF4/jIr8p8O4o/UgF tib5RoGFO3F1bMkg0OHdAhdZ4SP7xQc= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=kxywGUZZ; spf=pass (imf19.hostedemail.com: domain of shiyn.lin@gmail.com designates 209.85.215.173 as permitted sender) smtp.mailfrom=shiyn.lin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671521165; a=rsa-sha256; cv=none; b=hzfnY60qaE83+HIiRrBXEALPKNrCSQ9OyyBFB+zDLVFeXTAmxUfojHLe7DXgW8ha9hHe9p /Oda3G186EIvea5k6qQ0dIxAFQYs1b1I3kOS+tDAVXIi78bMNB3vAeYxfpfa0cMnhGA0Ig JxrsNikfaNBHT1Ivgnx9kxhqNWBmRl4= Received: by mail-pg1-f173.google.com with SMTP id e190so3334768pgc.9 for ; Mon, 19 Dec 2022 23:26:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4qeG6PpkfzCDCCR2FoWrTPHP2/xjfirILLn+aFk9hwU=; b=kxywGUZZeAuKFns3+pAcT6BLDXR+/Kz44ORCc9msg+h0jxLyHCxe0TVfC51CCemZiw lPK0mCZDKx2zNcxYQh15+8mErgYabTVvmXneeJbCUW7tTxoe8MZFNs+a3Ct9gkvam6G3 zViHBUPs7vX9GhKjdG/guIzrg+nppnH01Y7h5jt+0NKDrn8vAFPoUQsBP6vsmy7ruptW 2FohIsekZhDiWBPnI91e6jfyZYlAj0OpOaCEYL4Yr5HyN0g9UySXS/bVGVn30RwhUCZ7 vKbq4lJxlog20ykfCsbI7ibllD957ydpe1NRTNKsUPqsdXVY+RS54QMXgP9wNeNT2y0D By/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4qeG6PpkfzCDCCR2FoWrTPHP2/xjfirILLn+aFk9hwU=; b=0oA6wrXlM4AoibbOyT89FmClXtUAXaCgws53SPeC3rk4c6hfPeHq6C4oW0tltH2g2v tEbUf/ShyWJYBi2m+sBfvJfDQ4rPTCOFvoWs629YRlQsFRt/Ljl0RAgnr4TpiQ4sPfp7 qxkHWx9Kma/3a7bKypRbW/KVL0VfVk2LdRjgg5SKAFmDQDsJrKJz+Nxa6cpFethxJcIw pF4A/SzByAtr6hJMeMn4QwXxAQGzwRAMMXtZquPpHNWALz4dbwAqTCtGzl8KHqOr8Ltc UIflBiCWWxGJZWJGn7y8+pRLNPC7LbTgiBcbY1WWS13vRXQtsbqkKkmnSbIpxQqAC7IS JODw== X-Gm-Message-State: ANoB5pnibdSc/fKBsWXPSJaqpRgm/oVvP4He5wnrLxDpkog61K7Bwf2K NQnDKUvlG6I84qShGQlyr3s= X-Google-Smtp-Source: AA0mqf55kHPKQwj+Olza1S2CIKLn4EriHod9gAjefV9IM3bc3ZIZf34hXZGOaqVw0YtOQTGKdsoWyA== X-Received: by 2002:aa7:911a:0:b0:577:5afa:6321 with SMTP id 26-20020aa7911a000000b005775afa6321mr49726979pfh.26.1671521164189; Mon, 19 Dec 2022 23:26:04 -0800 (PST) Received: from archlinux.localdomain ([140.121.198.213]) by smtp.googlemail.com with ESMTPSA id q15-20020aa7982f000000b00576f9773c80sm7865544pfl.206.2022.12.19.23.25.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Dec 2022 23:26:03 -0800 (PST) From: Chih-En Lin To: Andrew Morton , Qi Zheng , David Hildenbrand , Matthew Wilcox , Christophe Leroy , John Hubbard , Nadav Amit Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Steven Rostedt , Masami Hiramatsu , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Yang Shi , Peter Xu , Zach O'Keefe , "Liam R . Howlett" , Alex Sierra , Xianting Tian , Colin Cross , Suren Baghdasaryan , Barry Song , Pasha Tatashin , Suleiman Souhlal , Brian Geffon , Yu Zhao , Tong Tiangen , Liu Shixin , Li kunyu , Anshuman Khandual , Vlastimil Babka , Hugh Dickins , Minchan Kim , Miaohe Lin , Gautam Menghani , Catalin Marinas , Mark Brown , Will Deacon , "Eric W . Biederman" , Thomas Gleixner , Sebastian Andrzej Siewior , Andy Lutomirski , Fenghua Yu , Barret Rhoden , Davidlohr Bueso , "Jason A . Donenfeld" , Dinglan Peng , Pedro Fonseca , Jim Huang , Huichun Feng , Chih-En Lin Subject: [PATCH v3 07/14] mm/madvise: Handle COW-ed PTE with madvise() Date: Tue, 20 Dec 2022 15:27:36 +0800 Message-Id: <20221220072743.3039060-8-shiyn.lin@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221220072743.3039060-1-shiyn.lin@gmail.com> References: <20221220072743.3039060-1-shiyn.lin@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 3B7B11A0005 X-Stat-Signature: rq64hihii9i1smxyufqb1yemtcahfant X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1671521165-666675 X-HE-Meta: U2FsdGVkX1+LYAgQn01Y8lGTmwGxaa3swcoZV2L6cQAQNoblK9G+i0ioJwDUYWie6YvZQtGJNU5zHm5RdEXvjaPbWN3RMSTOIayOUOUMCLx1YloFDTOLqfZFQ+1khbEwXSSBTwhZBT5ZWfma63ujxoAdRa370xCEkgNReLxB7cSgmQN58JPvEkMGsLrmf188jvddlIwQvN9N7meGxXOk4+FfB8VbkV5r2WMyAi1rKpO5/e01pYvn2k3IUC6ft8sc6r3dXNBRTkkzlIjk3J63bA4nnZ9aPuaaGEF6g83dQvcC57AoR+aDMaThLUqssSBAYCKxh4OhqcK56fZMqum404AnKgjDyC3G5wlz6KiObVX0xAKgbqQcp/4ILTAcCXXtx5lREyOk8qmRXvRGwSzlwNLxRx8gUxNO/G2kd9FYkNwD3hwO1VgdmSyykZ6YXygw4nTe7sJWaQDt4yfjy8DH22a/I/2c4h5Ow1nOsnYZT157qqnMyPoa6OnPmXMDaEla9J3k6J9H9E2iOPwb1bpsN8GTSymTZ/dX9FiKvPBVlWSvpXsjmaQK+HM7M5+RDe39DmqN8iOpJwRcpOsFhD04yHpsg04xhkjWOguhAt5q2jRByote8XwoiNnxtNHyAQ/Jp2l3kvR94AYuxvqZv8Nd4y5J1DzzOqRf54mO3ruSxzG042YAuHHy5CDzRpuPAgyV25np7iQMgdZdDJIXltQ4TfydB4mOIfzxU9UN1guB8bP3JrzvsOnXx/We9iOYulxzsjvPcoZQBl92As38RyIOT6+U9YAe5QzLDENHXcVR1zrpxu31aKZzv2F5H3fvDTMxw0UbhXHVXKsQsecB8s8J56Dy7WkAecSl+2T8rcop25hYq3qMCU6WBWje5h7Vu9h326LAofBOJlDcfFDPgKgHHofvfB3s8kByhEqP6UsElsegaDCeHHAOcob0q1FxFfE0h3e67brYV4h+8GFeEDO uXgpuME2 YYXhRjkzKZyRd1YNjQCTbS74clV9SCCljuKOTflfbpk0gHYq8ufaN/6RNhs21tNC8JCSQZNmH6R7e/xqOwFXT1ZthKrcq+kegU84B2iDtoQOTnO0h0H+PrCPXxw11sEJqtheOWKyE1PDnEeqfqthkKWtIBMnhv4ftllvcrzrAqZAxjmMBFihkLp+Bb7+65yxKNhoFJjvDUl52aLMD7DNEoqnWOs90wPRcKCtD4zCTSeeSSE8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Break COW PTE if madvise() modify the pte entry of COW-ed PTE. Following are the list of flags which need to break COW PTE. However, like MADV_HUGEPAGE and MADV_MERGEABLE, we should handle it respectively. - MADV_DONTNEED: It calls to zap_page_range() which already be handled. - MADV_FREE: It uses walk_page_range() with madvise_free_pte_range() to free the page by itself, so add break_cow_pte(). - MADV_REMOVE: Same as MADV_FREE, it remove the page by itself, so add break_cow_pte_range(). - MADV_COLD: Similar to MAD_FREE, break COW PTE before pageout. - MADV_POPULATE: Let GUP deal with it. Signed-off-by: Chih-En Lin --- mm/madvise.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/mm/madvise.c b/mm/madvise.c index c7105ec6d08c0..58bccec7caa88 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -408,6 +408,9 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, if (pmd_trans_unstable(pmd)) return 0; #endif + if (break_cow_pte(vma, pmd, addr) < 0) + return 0; + tlb_change_page_size(tlb, PAGE_SIZE); orig_pte = pte = pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl); flush_tlb_batched_pending(mm); @@ -614,6 +617,10 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, if (pmd_trans_unstable(pmd)) return 0; + /* We should only allocate PTE. */ + if (break_cow_pte(vma, pmd, addr) < 0) + goto next; + tlb_change_page_size(tlb, PAGE_SIZE); orig_pte = pte = pte_offset_map_lock(mm, pmd, addr, &ptl); flush_tlb_batched_pending(mm); @@ -974,6 +981,12 @@ static long madvise_remove(struct vm_area_struct *vma, if ((vma->vm_flags & (VM_SHARED|VM_WRITE)) != (VM_SHARED|VM_WRITE)) return -EACCES; + error = break_cow_pte_range(vma, start, end); + if (error < 0) + return error; + else if (error > 0) + return -ENOMEM; + offset = (loff_t)(start - vma->vm_start) + ((loff_t)vma->vm_pgoff << PAGE_SHIFT); -- 2.37.3