From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 377C9C47258 for ; Sat, 20 Jan 2024 07:10:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BAA096B0071; Sat, 20 Jan 2024 02:10:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B57496B007E; Sat, 20 Jan 2024 02:10:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F6806B0080; Sat, 20 Jan 2024 02:10:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8E4F26B0071 for ; Sat, 20 Jan 2024 02:10:03 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 497F980460 for ; Sat, 20 Jan 2024 07:10:03 +0000 (UTC) X-FDA: 81698815086.23.46FB862 Received: from out203-205-221-209.mail.qq.com (out203-205-221-209.mail.qq.com [203.205.221.209]) by imf01.hostedemail.com (Postfix) with ESMTP id 89BE240005 for ; Sat, 20 Jan 2024 07:09:59 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=qq.com header.s=s201512 header.b="cBzu3/On"; dmarc=none; spf=none (imf01.hostedemail.com: domain of cyy@cyyself.name has no SPF policy when checking 203.205.221.209) smtp.mailfrom=cyy@cyyself.name ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705734601; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xyoV1t8aKEfAQW2cATwsxg7a8BFBl4JU4b4mWpoQqG4=; b=sMnuKOLePw3AlXjeu30ddbVW4uMjUTqmsnVfS8sqEy3ZOqtQbMEpseMYsZpIY1zWxHElRQ xOLONXOrexl/8WtMZpropUB8KnNzLJ87QGK/amaE/BJUE5WPaq/HCn+P58Tt0QXpxQRkzH LBLozIQjImYQ6G9YgYqUrm19MG6e/+k= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=qq.com header.s=s201512 header.b="cBzu3/On"; dmarc=none; spf=none (imf01.hostedemail.com: domain of cyy@cyyself.name has no SPF policy when checking 203.205.221.209) smtp.mailfrom=cyy@cyyself.name ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705734601; a=rsa-sha256; cv=none; b=QtJEfLgjFYRF/fTGNwqAWoDYIjpxuK0qjmdpXbrtdPtcuLnXM6MJYZThphtMsCdPYXAae0 5QvV9cYqe3Mn4trk1R5Y9pY3SBYXtASUfZYWkVAKWLWmBYMOaTvOoFx9i7dcTrFCfCHf+G OnxoQD3vAYMiZ1N4yifyLAulwfMqmbA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qq.com; s=s201512; t=1705734595; bh=xyoV1t8aKEfAQW2cATwsxg7a8BFBl4JU4b4mWpoQqG4=; h=Date:Subject:To:Cc:References:From:In-Reply-To; b=cBzu3/On5l5BXRXlKb/kY3sDz1Uucfj7JhiokVqI/ybGW/kp/GcUgQlSRmArn4p/J Cxvf6VJxt8oycx4Mv37He4GOCNuYVtzfF/EA9iUcMV4BtdZNQNlKLHK/5Kya+fz3EK Ao37+xYa/paKUYjhqExLgdPZNkwzC0EZdOWtQH3E= Received: from [IPV6:240e:379:227b:4f00:cd09:8a0a:600e:c38] ([240e:379:227b:4f00:cd09:8a0a:600e:c38]) by newxmesmtplogicsvrszb9-1.qq.com (NewEsmtp) with SMTP id 273AB8E0; Sat, 20 Jan 2024 15:09:51 +0800 X-QQ-mid: xmsmtpt1705734591tujaje9qv Message-ID: X-QQ-XMAILINFO: NiDupExshEc7/Ce0oCF0ITlo7eRWhrzB6nlMTHy+ibkGPS2vJZZqmzOkIfUhHP lqtmyLmYNWFPhG49RlXuuNrqcch58e7pYQOhLngwkBR+C3HgBzJmj7KjnS3g+Nm7wDty6i03qVYE OGwBF9AClUMjH0K3WM7BRRpI/VZBsqNygbF5/BOfAzarng5X6E9I1IVaxQ8f0atYGkeKel+vxl2Z ly4AxuWjWvYm29z5PAh1biE5dkElDjB0qxwZYrAau3q39p+vPCpQj17OEpU88UuhOIt7o1AIOm5G Ofr5DqHWB8C3n1uk/5WTmohFbGNXo7sUzW/mBOLoj4yYHFvOVl5mMLpaPptiaLYfMUVUGV/Q5uhg NrqmDPzIcbFTxUvN/Dn+l7dfk6mFJ/SrqiEsKe2HlW7V4IOrSDDuuB0tGlgqdrhFEhX73p+wGIr8 LdCocRkul0O7MkYWK40DQ4wJBRL3J8nMMbD4Ua67Vw57KudTKLfD3tvDmMqMeTGO2jDVSCfuKYg3 TWNX1VJCV4tVJtz8eIhutNPubaADxmigGZHnHFnB5cWVyuZ4TCp6d3oq/DrZECDdeGep4N2x+aJc scnVAjHSMUi5rQAESklo6olqPGg578fA9GqvDy7H58yrEDBOPdxPC/d0144mEKxpDWln3N6yiZdF pLZX01AAxkU8VwMAlXSQ1DuNG2EXyoFEi0hCJq0B872G5OkeVmGqSOs6ltp9BzJXD21HI3hJBbHS R514suSvcnwgUENql1sbp6M5y7tpHRU61YXGiDhSI4HmSCMXx1LCKFcp6bW2dl45OwULQhS+uYZl nFXaCDdvnwtk9s1f9f9k8N5hpSHhEEZu90MUbp0tWcKDvZpuixysCq/8+Rmjb1DUpV5zsPEldVPn IihwtZcYWBBLs5WIjq9fSbW8dtiL7J6Shwj44gWe0LFkLYPA7Beu+vQVGNl1QMVcgDmAHgLEg/Sl uEujmMui7BX3CXxUIwb1HdfRQzXm2kpSRpmOv6Lp1kUUQjxW+e+w== X-QQ-XMRINFO: NI4Ajvh11aEj8Xl/2s1/T8w= X-OQ-MSGID: <5b25a10f-b3b1-47ae-b8af-34689d30d908@cyyself.name> Date: Sat, 20 Jan 2024 15:09:51 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v10 0/4] RISC-V: mm: Make SV48 the default address space Content-Language: en-US To: Charlie Jenkins Cc: alexghiti@rivosinc.com, anup@brainfault.org, aou@eecs.berkeley.edu, conor@kernel.org, jrtc27@jrtc27.com, konstantin@linuxfoundation.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, mick@ics.forth.gr, palmer@rivosinc.com, paul.walmsley@sifive.com, rdunlap@infradead.org References: <20230809232218.849726-1-charlie@rivosinc.com> From: Yangyu Chen In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 89BE240005 X-Stat-Signature: qrdjbp3hpoqy9kto6guc6b97zsjai6cn X-HE-Tag: 1705734599-680624 X-HE-Meta: U2FsdGVkX1+uaOL/ZPRF0YHcSg4kavmljCv6xSojEcN7A8s2siOVpo/hFEtE0zuppTeKv8tEzrFZjCz8fkWQiLLfY2p8pjHuA693lP+OJKLjPmxkX63HkRaBhzVCrgpwJ4pZD3V5n4YZOYZK8nhcpeforRVBQxBzSGwxnVUN9/zFIJyxXUZVLX6/KYJamk4ILYPmJDXQtNjHmdrUUgNRVg7p2+1+S2XsAq75TIs4qjGF1/Dq9uGyitrPqP1zMxRZSioqd+WXFb/Xxxv8gngQ0bwQ+HFQuoSfqKCO7KwWY3qHZH55V610/HpMnLaJEO2hxZHWOgGyFdT/CCEAnh9X23iYBUeBraoZVaUCcLwrlZwds8wF2WZlB9fEQrdesxl5uwwWwvbCRTxl29hnu4cEG8oaI2gTybzbPXw8k2wCNZK4EgDFLvX9CrXnJD3Yi2IdmjEw3NeI4y9boafNYWlypay1SrN53lqgLqW5H4f1S4BMe0F+i12OcaB6mR+Y1dOhNAkZmdXfPmNYsEs1Z3GJsjHPkHf9vbTxZ1ipxAyv1KYyDqqHKTnl/2XglUI1cek+2rK/WQxLUFw/1pnPl065iYAur+F01fo3AZdUlcayPIjxr2erZs9PhRB6FtCS3bgWeLxi+atYmIHat89LAhLP0sENUHIAcwF38bWXby26g0iikzJLquP2f9m/qk+HzkjVjZsMrDbJ6Jv//gDHLk/vqEQvpHLJnCRaGIY3GJARsRbNdOv1SgRDJmtOrNjV0Ata9TkAm2vBUr+fyxTtFc9ie24uHqVUxHXXex0wbi7T8yspWyNQZ9kYvAi3CgK7IbSfwxtzI5VGsLa2SzcuhXGPeYY1/B9tnk/9qU3J7hw4LZwbzUTMARrzBvWXOr4HoDX1KzSXxeThRGTkQ0gq1IKeAgIAQquQ+/uqqRU0TaSTTfFqUdPp5ehA0wDIpEV6dE0pOEKKh8ETRDouVxyeGRE BD90Rlv2 +qEfmC8d1G5RSkucaYlD80p7G0Tjst0qUEsSldGUPZeHYZRLa2+F3qDd5ZwdSBsYNI1Bj9t8AA4fSNsm9aNVr3CX+7Y4rS0Ahyh19rcVfiZ2jtoIDWGbz7k03bkgwwf+xlvJkT7r3YpHCRCq4UUOFjzI98VWRKhQTIJ9RhZWp87xvSGkN4XuwvXcWoRm0/BadFysdtCAxiOT6E2gwQEHWSAHf3vt5oBio+UwG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000047, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 1/20/24 14:49, Charlie Jenkins wrote: > On Sat, Jan 20, 2024 at 02:13:14PM +0800, Yangyu Chen wrote: >> Thanks for your reply. >> >> On 1/20/24 09:34, Charlie Jenkins wrote: >>> On Sun, Jan 14, 2024 at 01:26:57AM +0800, Yangyu Chen wrote: >>>> Hi, Charlie >>>> >>>> Although this patchset has been merged I still have some questions about >>>> this patchset. Because it breaks regular mmap if address >= 38 bits on >>>> sv48 / sv57 capable systems like qemu. For example, If a userspace program >>>> wants to mmap an anonymous page to addr=(1<<45) on an sv48 capable system, >>>> it will fail and kernel will mmaped to another sv39 address since it does >>> >>> Thank you for raising this concern. To make sure I am understanding >>> correctly, you are passing a hint address of (1<<45) and expecting mmap >>> to return 1<<45 and if it returns a different address you are describing >>> mmap as failing? If you want an address that is in the sv48 space you >>> can pass in an address that is greater than 1<<47. >>> >>>> not meet the requirement to use sv48 as you wrote: >>>> >>>>> else if ((((_addr) >= VA_USER_SV48)) && (VA_BITS >= VA_BITS_SV48)) \ >>>>> mmap_end = VA_USER_SV48; \ >>>>> else \ >>>>> mmap_end = VA_USER_SV39; \ >>>> >>>> Then, How can a userspace program create a mmap with a hint if the address >>>>> = (1<<38) after your patch without MAP_FIXED? The only way to do this is >>>> to pass a hint >= (1<<47) on mmap syscall then kernel will return a random >>>> address in sv48 address space but the hint address gets lost. I think this >>> >>> In order to force mmap to return the address provided you must use >>> MAP_FIXED. Otherwise, the address is a "hint" and has no guarantees. The >>> hint address on riscv is used to mean "don't give me an address that >>> uses more bits than this". This behavior is not unique to riscv, arm64 >>> and powerpc use a similar scheme. In arch/arm64/include/asm/processor.h >>> there is the following code: >>> >>> #define arch_get_mmap_base(addr, base) ((addr > DEFAULT_MAP_WINDOW) ? \ >>> base + TASK_SIZE - DEFAULT_MAP_WINDOW :\ >>> base) >>> >>> arm64/powerpc are only concerned with a single boundary so the code is simpler. >>> >> >> As you say, this code in arm64/powerpc will not meet the issue I address. >> For example, If the addr here is (1<<50) on arm64, the arch_get_mmap_base >> will return base+TASK_SIZE-DEFAULT_MAP_WINDOW which is (1<> And this behavior on arm64/powerpc/x86 does not break anything since we will >> use a larger address space if the hint address is specified on the address > >> DEFAULT_MAP_WINDOW. The corresponding behavior on RISC-V should be if the >> hint address > BIT(47) then use Sv57 address space and use Sv48 when the >> hint address > BIT(38) if we want Sv39 by default. >> >> However, your patch needs the address >= BIT(47) rather than BIT(38) to use >> Sv48 and address >= BIT(56) to use Sv57, thus breaking existing userspace >> software to create mapping on the hint address without MAP_FIXED set. > > Code that needs mmap to provide a specific address must use MAP_FIXED. > On riscv, it was decided that the address returned from mmap cannot be > greater than the hint address. This is currently implemented by using > the largest address space that can fit into the hint address. It may be > possible that this range can be extended to use all of the addresses > that are less than or equal to the hint address. > So this decision might be wrong. It requires some userspace software to modify their mmap flags to fit with this. For example, a binary translate JIT compiler already probes this platform is capable with Sv48, then want to create mapping on some address specified on the mmap hint to align with foreign binary native address but also provide a fallback path with performance overhead. Your patch here will always let userspace software use a fallback path with performance overhead until the userspace software changes its syscall to use MAP_FIXED. But it is not required in x86, arm64, powerpc. > From reading the code even on arm64 if you pass an address that is > greater than DEFAULT_MAP_WINDOW it is not guaranteed that mmap will > return an address that is greater than DEFAULT_MAP_WINDOW. It may still > be provide an address that is less than DEFAULT_MAP_WINDOW if it fails > to find an address above. This seems like this would also break your use > case. > Yeah. As I said before, this patch will always let userspace software use a fallback path and this only happens in RISC-V. Make default sv48 is right, but RISC-V implementation for this and changing the hint address behavior might be wrong. And x86, arm64, powerpc already use 48-bit address space by default but do not change the meaning of hint address on mmap. >> >>>> violate the principle of mmap syscall as kernel should take the hint and >>>> attempt to create the mapping there. >>> >>> Although the man page for mmap does say "on Linux, the kernel will pick >>> a nearby page boundary" it is still a hint address so there is no strict >>> requirement (and the precedent has already been set by arm64/powerpc). >>> >> >> Yeah. There is no strict requirement. But currently x86/arm64/powerpc works >> in this situation well. The hint address on these ISAs is not used as the >> upper bound to allocating the address. However, on RISC-V, you treat this as >> the upper bound. >> >>>> >>>> I don't think patching in this way is right. However, if we only revert >>>> this patch, some programs relying on mmap to return address with effective >>>> bits <= 48 will still be an issue and it might expand to other ISAs if >>>> they implement larger virtual address space like RISC-V sv57. A better way >>>> to solve this might be adding a MAP_48BIT flag to mmap like MAP_32BIT has >>>> been introduced for decades. >>>> >>>> Thanks, >>>> Yangyu Chen >>>> >>> >>> - Charlie >>> >>