From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7B47C64ED8 for ; Mon, 27 Feb 2023 15:14:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81DA26B0072; Mon, 27 Feb 2023 10:14:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CE9A6B0073; Mon, 27 Feb 2023 10:14:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6965D6B0074; Mon, 27 Feb 2023 10:14:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5A8EE6B0072 for ; Mon, 27 Feb 2023 10:14:40 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 16FAC1402C0 for ; Mon, 27 Feb 2023 15:14:40 +0000 (UTC) X-FDA: 80513418720.22.799A74B Received: from mail-vs1-f42.google.com (mail-vs1-f42.google.com [209.85.217.42]) by imf16.hostedemail.com (Postfix) with ESMTP id 4B8AB18001E for ; Mon, 27 Feb 2023 15:14:38 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=i91GA7ik; spf=pass (imf16.hostedemail.com: domain of jthoughton@google.com designates 209.85.217.42 as permitted sender) smtp.mailfrom=jthoughton@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677510878; a=rsa-sha256; cv=none; b=nlN2nwv1H8jyPxLrWnoJHXFiWuc9g51GwuuSwXFdaDV0MqysR98CeTdQKEJjh5OKCwrQmb /TkcKR+qFGP3neZw/9hD0Fmb8z31T9WID+MdSHDgV9Xx3RzY+1m1eQHvmhfnwzcu4OnC2n TYDnXrQ7QooGC4haiNyy7ZHu+PFjT+c= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=i91GA7ik; spf=pass (imf16.hostedemail.com: domain of jthoughton@google.com designates 209.85.217.42 as permitted sender) smtp.mailfrom=jthoughton@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677510878; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DK29JUcB7eAkqoEsw6GF+QQWOkXrCAiRiZPvb7EnDxI=; b=1iNWEnRrpeVxfhs8rcMoW7SnlXrKwuZ3VNwGJYryiqoCxtfe0RhFFvUVxxIQjfezjiKUyc iejAGrfr+aLL//HpxPJkb8cpUH0nXO89D2JARe6FLfVr/9WBou8KHE9qcCS4p7TA70VAlB AufKvP/axsre+i11oi5gFe9yNPPfUpk= Received: by mail-vs1-f42.google.com with SMTP id o32so11509050vsv.12 for ; Mon, 27 Feb 2023 07:14:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1677510877; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=DK29JUcB7eAkqoEsw6GF+QQWOkXrCAiRiZPvb7EnDxI=; b=i91GA7ikHLkyUXv4H6HxEIuUfgXhAGI9NSZjXdKe3/Qc57p9E7Lcxqq5mrmbfL3LsQ ben+D8y6ZTvdkowVBiEifGz6tNOZux/Re4NMCGhXj2jeAotuRaMtt++jC6T0gIK4PmIu xA5Mbi0muKMS1PdU89IDw+kw9HnXdtzrnRepFIgICf9bqEfTcLpJ68RhGQ/DwcwOAKy+ q6a9/q6Vb6XBrWssFF+WJZFO94qB/9ESRbFcVqV90ZcrqaT0jpuoRKVIO/cBphpwDwGR RxcHRvmJLtzDj45KSImx8wholIA5YYRZ+GKiREDe2R+hVIpS4GCIXQ84aO7NVM7VV6Hr jZwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677510877; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DK29JUcB7eAkqoEsw6GF+QQWOkXrCAiRiZPvb7EnDxI=; b=NPjvXdrhqJqNPaYAhNahUs10/naUskLtlLxbiDyyZZEzqHWstS8IrFzqfDCn6jZNGs SywhDSI747ih+j83D4OD+pZTe75ejvf4/aZBdHj1/IGK9xgp+NE9QNyvBs1PUMhRKJHg gd0rGtZPvgLMTMz328ZtKqt0i/rTVmgcOo/vhvW0MwXQEyXb3vdOkpeUFmzPkPzmP/rt S8qQL5ib590c9/TZ/M/GpRqaiVR1Rz9Rcn/D9hhx+2yOSa1Mm3xHOlohi0eXuxxmL/iC OT6O558xmhzVhnh+UBF09Eq1uYuTOvHKJRf2fCdfil2EuzvFTvlfl65wmfQDLK7Jqodv NjqA== X-Gm-Message-State: AO0yUKVqWImCYE3AG5VbqpYyI8QG5cH3exFLAoe85buStwg8W0KR6o38 4iyHiaBdhz6p+P7YhPeB6BpZL04mk4pEpDWSdrb5nw== X-Google-Smtp-Source: AK7set/tc/TZYbFu12DoUY/3mUSkmyU3RCWmBL58iDnCNGTReanN3wW0hP1sv6GKPSs1Yv7/LKV2j8FdzR6Wy4RU43Y= X-Received: by 2002:a05:6102:31a9:b0:415:48dd:e0b9 with SMTP id d9-20020a05610231a900b0041548dde0b9mr7415429vsh.3.1677510877266; Mon, 27 Feb 2023 07:14:37 -0800 (PST) MIME-Version: 1.0 References: <20230218002819.1486479-1-jthoughton@google.com> <20230218002819.1486479-10-jthoughton@google.com> In-Reply-To: From: James Houghton Date: Mon, 27 Feb 2023 07:14:00 -0800 Message-ID: Subject: Re: [PATCH v2 09/46] mm: add MADV_SPLIT to enable HugeTLB HGM To: Mike Kravetz Cc: Muchun Song , Peter Xu , Andrew Morton , David Hildenbrand , David Rientjes , Axel Rasmussen , Mina Almasry , "Zach O'Keefe" , Manish Mishra , Naoya Horiguchi , "Dr . David Alan Gilbert" , "Matthew Wilcox (Oracle)" , Vlastimil Babka , Baolin Wang , Miaohe Lin , Yang Shi , Frank van der Linden , Jiaqi Yan , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Queue-Id: 4B8AB18001E X-Rspamd-Server: rspam01 X-Stat-Signature: 4azeu85oohe4asfenekjnxeyekcejtnm X-HE-Tag: 1677510878-450463 X-HE-Meta: U2FsdGVkX19LX64k2UT18GiqKQHSfS21xhMLZAoIVuMpuhjFTAH8GBQ3wQvrYsT0Z3TLgZwn91BSTDLRTK3EKX9qE7NeEmPHkFNp1OmKMyJOddChzvhZt4vcAn5/QKd2YllLPfYpkhWTkvTfATDN5Q/e+7+vW9ZltuQ2znvXvAHOu7elRYgucJ4H5QrfoqoY5hRDlod+0mVITF9v6/34250a0DInY4U5D3ZDU8QL6LSblHUnAjF1dR6B5gmEtM6K7ExfGR9Fct1pylp1yJ44JS64X6BFQLjzEiwTYXC689FphEqylYsoWbgANpzA/lnDuFD1+ZCYoCfH526oC0tee9QX0Ecfk47FPh9+3VZpbwsFBtMjMYgVsFSnoLodoX0xGq7DQpNLH8Fy2PhqtsamBEIAaUak/g+UjlfQc5ZUFdAqIpw96Ybv5vguFcJUCQW4Ax+NgBHheTWDgu4aD+J1nwQBubUywfiQVQhsCtPI4ercb9tYX2hc4hTqSVol+PeQu8s/F3aw3DFj4Hjfz1K4+S9VeR2VNMnOacJ2/eKEwKUCer8K57xMc5hPIDV7MB0DhpI+stBaoqjuWc2aKnLgz+ZZh4Fi5jH+5j/I95We3s4+1R3SJzgxO1XBCobyGHHFUVj9sgLF8YYvA2tm7KQpRRdAildyPjJrkhJLqdhE61Womqg2Cxe8DEtN6QiLiByVcyk887P8DhlUabPlRyLRN9qrgTzkSmFAvlFYhvX6xGzb6j2ttUPQ+34jvVOiN2RzjrBIFfzoLk2hqZv4GNy6lRSMe3Ceuvr+6Rmlp0zqEyJJ1msIHw8kG10ADgFubYMWEsuccvaAXWMCdh1S4O+dgZvRJEuARGvBeJ+f/4SLyP/m5uNYCJhsyahGSr6CKu1we8hK7sBK9lt9OQTWAJUOTtHg008esu+ACBXARxoY4C90PWjtsp/DHMb4iJOUVQ03ExS8w0ys1wmHMDGEH39 1UIjxHDM 8UBIGhCA68xX6HEC7555R4Rm7TPGt1Wyoj57rnzN3dDw2RvetbrxE2Jl4XLZAnyN6kUVdUMmVH0ahUMHn8HF64HeEsRLCTESOuiAEhDQUwfDVHxhIxpQWQ0sUp6oZe+kI5vG4ujcVQKXcPNouGRNp9wdGndG4KeKhgFlSs1zxYw7c5pvvEq+pHyl7jVe7g9mSFIARdGTEJdcV4KrZr3D66pm9vVYia2WECrWikQU2qo+UToD7dRVO1xulfYuMtGJ6iYbDUGmmRQv/wDg+AtWQEnxd9pIv2lXq66v3UrY7/ZDQ+PnifDEmntlgqllpfE7meGryIR/3iWLod9tckDCoExh0FA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Feb 24, 2023 at 3:25 PM Mike Kravetz wrote: > > On 02/18/23 00:27, James Houghton wrote: > > Issuing ioctl(MADV_SPLIT) on a HugeTLB address range will enable > > HugeTLB HGM. MADV_SPLIT was chosen for the name so that this API can be > > applied to non-HugeTLB memory in the future, if such an application is > > to arise. > > > > MADV_SPLIT provides several API changes for some syscalls on HugeTLB > > address ranges: > > 1. UFFDIO_CONTINUE is allowed for MAP_SHARED VMAs at PAGE_SIZE > > alignment. > > 2. read()ing a page fault event from a userfaultfd will yield a > > PAGE_SIZE-rounded address, instead of a huge-page-size-rounded > > address (unless UFFD_FEATURE_EXACT_ADDRESS is used). > > > > There is no way to disable the API changes that come with issuing > > MADV_SPLIT. MADV_COLLAPSE can be used to collapse high-granularity page > > table mappings that come from the extended functionality that comes with > > using MADV_SPLIT. > > > > For post-copy live migration, the expected use-case is: > > 1. mmap(MAP_SHARED, some_fd) primary mapping > > 2. mmap(MAP_SHARED, some_fd) alias mapping > > 3. MADV_SPLIT the primary mapping > > 4. UFFDIO_REGISTER/etc. the primary mapping > > 5. Copy memory contents into alias mapping and UFFDIO_CONTINUE the > > corresponding PAGE_SIZE sections in the primary mapping. > > > > More API changes may be added in the future. > > > > Signed-off-by: James Houghton > > > > diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h > > index 763929e814e9..7a26f3648b90 100644 > > --- a/arch/alpha/include/uapi/asm/mman.h > > +++ b/arch/alpha/include/uapi/asm/mman.h > > @@ -78,6 +78,8 @@ > > > > #define MADV_COLLAPSE 25 /* Synchronous hugepage collapse */ > > > > +#define MADV_SPLIT 26 /* Enable hugepage high-granularity APIs */ > > + > > /* compatibility flags */ > > #define MAP_FILE 0 > > > > diff --git a/arch/mips/include/uapi/asm/mman.h b/arch/mips/include/uapi/asm/mman.h > > index c6e1fc77c996..f8a74a3a0928 100644 > > --- a/arch/mips/include/uapi/asm/mman.h > > +++ b/arch/mips/include/uapi/asm/mman.h > > @@ -105,6 +105,8 @@ > > > > #define MADV_COLLAPSE 25 /* Synchronous hugepage collapse */ > > > > +#define MADV_SPLIT 26 /* Enable hugepage high-granularity APIs */ > > + > > /* compatibility flags */ > > #define MAP_FILE 0 > > > > diff --git a/arch/parisc/include/uapi/asm/mman.h b/arch/parisc/include/uapi/asm/mman.h > > index 68c44f99bc93..a6dc6a56c941 100644 > > --- a/arch/parisc/include/uapi/asm/mman.h > > +++ b/arch/parisc/include/uapi/asm/mman.h > > @@ -72,6 +72,8 @@ > > > > #define MADV_COLLAPSE 25 /* Synchronous hugepage collapse */ > > > > +#define MADV_SPLIT 74 /* Enable hugepage high-granularity APIs */ > > + > > #define MADV_HWPOISON 100 /* poison a page for testing */ > > #define MADV_SOFT_OFFLINE 101 /* soft offline page for testing */ > > > > diff --git a/arch/xtensa/include/uapi/asm/mman.h b/arch/xtensa/include/uapi/asm/mman.h > > index 1ff0c858544f..f98a77c430a9 100644 > > --- a/arch/xtensa/include/uapi/asm/mman.h > > +++ b/arch/xtensa/include/uapi/asm/mman.h > > @@ -113,6 +113,8 @@ > > > > #define MADV_COLLAPSE 25 /* Synchronous hugepage collapse */ > > > > +#define MADV_SPLIT 26 /* Enable hugepage high-granularity APIs */ > > + > > /* compatibility flags */ > > #define MAP_FILE 0 > > > > diff --git a/include/uapi/asm-generic/mman-common.h b/include/uapi/asm-generic/mman-common.h > > index 6ce1f1ceb432..996e8ded092f 100644 > > --- a/include/uapi/asm-generic/mman-common.h > > +++ b/include/uapi/asm-generic/mman-common.h > > @@ -79,6 +79,8 @@ > > > > #define MADV_COLLAPSE 25 /* Synchronous hugepage collapse */ > > > > +#define MADV_SPLIT 26 /* Enable hugepage high-granularity APIs */ > > + > > /* compatibility flags */ > > #define MAP_FILE 0 > > > > diff --git a/mm/madvise.c b/mm/madvise.c > > index c2202f51e9dd..8c004c678262 100644 > > --- a/mm/madvise.c > > +++ b/mm/madvise.c > > @@ -1006,6 +1006,28 @@ static long madvise_remove(struct vm_area_struct *vma, > > return error; > > } > > > > +static int madvise_split(struct vm_area_struct *vma, > > + unsigned long *new_flags) > > +{ > > +#ifdef CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING > > + if (!is_vm_hugetlb_page(vma) || !hugetlb_hgm_eligible(vma)) > > + return -EINVAL; > > + > > + /* > > + * PMD sharing doesn't work with HGM. If this MADV_SPLIT is on part > > + * of a VMA, then we will split the VMA. Here, we're unsharing before > > + * splitting because it's simpler, although we may be unsharing more > > + * than we need. > > + */ > > + hugetlb_unshare_all_pmds(vma); > > I think we should just unshare the (appropriately aligned) range within the > vma that is the target of MADV_SPLIT. No need to unshare the entire vma. Right I can do that, and I can check for appropriate alignment here (else fail with -EINVAL). > > > + > > + *new_flags |= VM_HUGETLB_HGM; > > + return 0; > > +#else > > + return -EINVAL; > > +#endif > > +} > > + > > /* > > * Apply an madvise behavior to a region of a vma. madvise_update_vma > > * will handle splitting a vm area into separate areas, each area with its own > > @@ -1084,6 +1106,11 @@ static int madvise_vma_behavior(struct vm_area_struct *vma, > > break; > > case MADV_COLLAPSE: > > return madvise_collapse(vma, prev, start, end); > > + case MADV_SPLIT: > > + error = madvise_split(vma, &new_flags); > > + if (error) > > + goto out; > > Not a huge deal, but if one passes an invalid range (such as not huge page > size aligned) to MADV_SPLIT, then we will not notice the error until > later in madvise_update_vma() when the vma split fails. By then, we will > have unshared all pmds in the entire vma (or just the range if you agree > with my suggestion above). Good point. I'll fix this for v3. :) Thanks Mike.