From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D718BC3DA7D for ; Wed, 4 Jan 2023 01:05:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E34A8E0002; Tue, 3 Jan 2023 20:05:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 471F88E0001; Tue, 3 Jan 2023 20:05:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 30CCE8E0002; Tue, 3 Jan 2023 20:05:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1D1DF8E0001 for ; Tue, 3 Jan 2023 20:05:59 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7F6D7C02E1 for ; Wed, 4 Jan 2023 01:05:58 +0000 (UTC) X-FDA: 80315324796.04.E64C786 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by imf24.hostedemail.com (Postfix) with ESMTP id EE2D1180005 for ; Wed, 4 Jan 2023 01:05:56 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=qGUnY8Ef; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of jthoughton@google.com designates 209.85.221.45 as permitted sender) smtp.mailfrom=jthoughton@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672794357; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lgU5P1L7gNa0tdGaVNRVNmjRoo+ImVqTA7zCp84GB0c=; b=54odG/gh9ohI7OFnVAE86VDyS7zX/tXv76AIlIJX2O+l1sElluKYPaW9ypqhtINbB0VWeQ GPnmQD/aMIglBIP/6hkgbBe20LyakNEj/zgsTk37nbbn3OldajbGTZcOkWcuiCO1ff1tPC C0KMNrem9XXg6RucOHeef/dqcsRhXeA= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=qGUnY8Ef; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of jthoughton@google.com designates 209.85.221.45 as permitted sender) smtp.mailfrom=jthoughton@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672794357; a=rsa-sha256; cv=none; b=QomO2vuCcVcJM89cPBj7AG/csrEepczAWyDK4ryuXxhGJ5GGdofPe5rwX9MEuL1s5DUZim jHDmsUK+H+KCm9H2lmJNJpHy6iel/D55EeCWtYxmWKhg23kix97enhxy18aLjkwGBxTvrm VKeK6umYwKWxPv39Pw4EUD8Xmf2y/Qw= Received: by mail-wr1-f45.google.com with SMTP id az7so7186843wrb.5 for ; Tue, 03 Jan 2023 17:05:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=lgU5P1L7gNa0tdGaVNRVNmjRoo+ImVqTA7zCp84GB0c=; b=qGUnY8Efw2L/MMNwi630Qmj5ZtTbvDdbNsCc9pxjCtL6dq5EoWqLgMppGhj0qsYW0X +SRHbP0RRQ1uBfea18yol+aI/Pv12mk+h6GeP3mUDqVxXh0I5o1xz9TDMYzuN2onN2Vr mlalR8Ncr5FwRc2lx4rly7yJJTdzbafuET8EhbjUEwITKEpYR53+7O5whvUuHYLynjFI 9rzjOUndQipPw8tcRjeolIcQxv2kFXsh6GkOK34d2ZYZITvkYF3KnPGiM9G9jtRVLufD UeUbP5GfKFeiS2ZTdsU/Vswh3JhBBdOxD1HZeNsANzU9REwSAxdk08TyvEVfzlNRex1J vZ5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=lgU5P1L7gNa0tdGaVNRVNmjRoo+ImVqTA7zCp84GB0c=; b=GQX7UrwPLDvXn+4yumes/E7+87jVvLNh4ZGLErsB8k3jSUMlsq08K18VbGua9Mg1fE TDd0oyDx8x6Zt0xKMZU9oTOY7wl0BK6U4yd2FRVs63CxiscT/DqQVhR9KX02jBkYRuMD YlbTjTv+TiqtukMd5qqHqTJHGNjhnrdXj8djfHE/vkdOWS2KdVoViwPvpUdULlPUFMjS nMktCq0Vk1QI8Z9EJXFKppC+v/4EXSXa13KomoSR7B1bQXs4ylX11J/UkpNvPRin0eZN Om8Nico74i5U/k2cfJKgjK5seE4Q+Vk4zTveSivcyX12cmo1xGWzIPgotNlZ8dGFdJUV INIg== X-Gm-Message-State: AFqh2krFCCtE4bHQ1y3ZBsjfv9ARQs7tpoCBT58fmYvqYDSiShK+HEKE 5dOTAIsrE4ozN3eMRKt2jBS+ohjbcoUvQ/CiIoo6Mg== X-Google-Smtp-Source: AMrXdXud6BQyRhsvF5DXlHBM3atTldfPr46GP7x93GUitenY95hZSa0+puECh+8/Q3mvX0ziCjtBlpIpyR5jQ44EQzM= X-Received: by 2002:a5d:6148:0:b0:280:91ea:29b7 with SMTP id y8-20020a5d6148000000b0028091ea29b7mr907851wrt.98.1672794355405; Tue, 03 Jan 2023 17:05:55 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: James Houghton Date: Wed, 4 Jan 2023 01:05:42 +0000 Message-ID: Subject: Re: [RFC] Improving userfaultfd scalability for live migration To: Sean Christopherson Cc: David Matlack , Peter Xu , Andrea Arcangeli , Paolo Bonzini , Axel Rasmussen , Linux MM , kvm , chao.p.peng@linux.intel.com, Oliver Upton Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: EE2D1180005 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: gyttfc969m57mz68iimsz8hsdjkrrdhf X-HE-Tag: 1672794356-551255 X-HE-Meta: U2FsdGVkX1/5rAdbhWRSRSMvyugy0+1OxdtlgCwWgyD4mpjl7zg28bu9APKRvSCj7dwautOB+ueBFmSXmxEyr7yKl4xSs66C9aHwEtUP1RxK1dKd/nTm/icrA8JpmTFYAB726SrymmJk3nYlTifiVQhaMVbgT9ESIme9tBIbbvefDiDyKLuJe6OrwWHH1J8yeNC6t9N6cPamE8A5FVhebDXX5lvE2V6DlFKowbX0SF1QoryfORgHO6pfrjhZxHzNoPbeLcJc6EsPbIDrnRusH4qC8zWBtbQIUTltKpwDwIP0WpEUiIabP8WtaXjmVnDnP0wwZwzEktTJkX+xDEXDGdv+Bzozp+chOAWMrKu3BQS345C05UGtq61ajIgBY5/7aoVy02cBLZKAgNNsQj9ycWPDPxq2VgRtXuqmx8cLeUQltlSOgW4Zza5ojVNE9bLi721CWpd+O7cbb7cb8ZUA7UcgGeeXezIvVxLt2IXPdw6NK163f9rD0Rb2ynj3EhMUJ9r9oPRyaXHTag/jnVsMmZj1JVrPj+AvOfnvXwGBi03Pf/5vEoIvXwsQERwL+l8mpTrduFs5F3htLqyjSlZ63+mzppuAP1KPvV/3FDppRE/ZbRHhxHk2xlFYqYHocjZGNLmYvtPwtBQ8NE8ng0UjgeXzaYXPvzgyePq572XX+uq1jXZsB1dL9IQeT3QxGCPDpotLpkEoshHulWdVs0vY4cl8oZFdrlKLwXmakXPTA+wmoiYI6YkJn5OzCwgeNQNQ0IitJABzS4LxKYHZBnkHMviAbomCdau+7yKUuqiDeVK3wRFXKcrXvuVDDUIHgeYKjnZA0yMHJnGlC2/qk7DbRKkPme+El91DQTPoX1iOrpsJXIu6hTeg7Jk6gW2rpNSfKarhV5I7iuLE04psXb4NkYtWRnm+lt/DNsEVNzhYYqT6iXExnOFv2Nfvv8uOYUWPBRy8M5ENyS7nhFhgbdv RJV73/W1 FdScX+3UaHRuUsYo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000882, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jan 4, 2023 at 12:57 AM Sean Christopherson wrote: > > On Thu, Dec 08, 2022, James Houghton wrote: > > - For the no-slow-GUP choice, if someone MADV_DONTNEEDed memory and we > > didn't know about it, we would get stuck in MADV_POPULATE_WRITE. By > > using UFFD_FEATURE_THREAD_ID, we can tell if we got a userfault for a > > thread that is in the middle of a MADV_POPULATE_WRITE, and we can try > > to unblock the thread by doing an extra UFFDIO_CONTINUE. > > > > - For the PF_NO_UFFD_WAIT choice, if someone MADV_DONTNEEDed memory, > > we would just keep trying to start the vCPU without doing anything (we > > assume some other thread has UFFDIO_CONTINUEd for us). This is > > basically the same as if we were stuck in MADV_POPULATE_WRITE, and we > > can try to unblock the thread in a fashion similar to how we would in > > the other case. > > > > So really these approaches have similar requirements for what > > userspace needs to track. So I think I prefer the no-slow-GUP approach > > then. > > Are you planning on sending a patch (RFC?) for the no-slow-GUP approach? It sounds > like there's a rough consensus that that's a viable, minimally invasive solution. Yes, soon. amoorthy@google.com is working on it. Also a small correction regarding userspace needing to track MADV_DONTNEEDs, userfaultfd already handles that with UFFD_EVENT_REMOVE, so it's a non-issue. Even more reason to take the no-slow-GUP approach. - James