From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D3AEC3DA4A for ; Tue, 6 Aug 2024 02:28:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DC6726B00A7; Mon, 5 Aug 2024 22:28:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D76166B00AC; Mon, 5 Aug 2024 22:28:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C164C6B00AF; Mon, 5 Aug 2024 22:28:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A36AD6B00A7 for ; Mon, 5 Aug 2024 22:28:25 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 53AD3121BA8 for ; Tue, 6 Aug 2024 02:28:25 +0000 (UTC) X-FDA: 82420236570.13.94F5BBE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 258F340004 for ; Tue, 6 Aug 2024 02:28:22 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ITc5KAT7; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of jasowang@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=jasowang@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722911295; a=rsa-sha256; cv=none; b=i4C73OajLtFwMbf7CLky0Nk/yC9fy3UADVXwJ0D21CM0sXRz08sSpgmBKXl1VTml4LWPZW 5pbsrKr9sHNjOrNc1lnXVVvqW1EpeYIfYCF9AapfTQ0FsalqlQLCLogIzoPH+8cWuPkc5k trz7n85CEFX1P/AeFO3N6CSka0HqbjQ= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ITc5KAT7; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of jasowang@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=jasowang@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722911295; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RCtr3tcm3DF2AUgHL7KDfXrJvYv0n7KejT7VoVCN7dA=; b=4S5fKT0zoclFzq5cf1xVoSyvYMPbxYkbWq2E/aVPCT+B44ymbhPbOYdl9DhNfcMmkhHdu6 k3SuF2EmNPklgQizAd83HuTCtW/kXyGAgfXIRnr9JbCokribzz2G93E89YD90dAqb8+xvs AooiO/go2e752q6OE5v5eoPm0Ia9zhY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1722911302; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RCtr3tcm3DF2AUgHL7KDfXrJvYv0n7KejT7VoVCN7dA=; b=ITc5KAT7rW/aY/T9xCoNSdgd7FIOeYhMIfcEwEDOunPZio1zk4xkExA2Dx+nUXwvSQBh2T mX17oS9T73kF7X7tASC/aHgMdRl8Z/0H8ZKEmeOegzCbId15ps8SOUfZVNsiEDvmobwzDW gG2UhwEOett9bETG9IU91sJrmFek0bc= Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-122-Fr0qK5lxPrC_lDNct9a08A-1; Mon, 05 Aug 2024 22:28:20 -0400 X-MC-Unique: Fr0qK5lxPrC_lDNct9a08A-1 Received: by mail-pj1-f71.google.com with SMTP id 98e67ed59e1d1-2cb5d93c529so179252a91.0 for ; Mon, 05 Aug 2024 19:28:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722911300; x=1723516100; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RCtr3tcm3DF2AUgHL7KDfXrJvYv0n7KejT7VoVCN7dA=; b=jUAGzp9AXBBGo6i/UC46sDKnhwynuD952cXM7bAwkM0u4VA7YOtmxfMRWOrP12UicF YIAyccQ0HVx5ta6Hg9i8uAu/O3O/EXEvskdr1usB7slfh3GINWmlLQIfMN/mHcgDteqC UR61UF0Y/ZNaHbePTNM69qSHwPZJ8baM9dudqCP4JoMtvnsSCzT6kGc0K4NwGrjUMbDw nP47KRKPU1bDbe0YYPxW4/OYd7+7ak7NqC8DCgwHZ7ZI1DISGq7pEZ88h6HnkW1X8RVT lgb6zxQVPtev+3IITAarOOP1X/IsgeLGMuE1xc0APLRTIqhrMU9S6kvOCMrZ2FjJYG9m Blrg== X-Forwarded-Encrypted: i=1; AJvYcCVDjzeZFvVPH6l1bmotPpi2QCsk5BGFDXjr39XtaAJVczdZmXUTAbEu9Hqs02uCmNMR9ny4yQcGsWWZzVJaQ8HD+z8= X-Gm-Message-State: AOJu0YwzFa+uj1v/pOhjkzx+4Lfa7+26gCw6gKIvPvmQUm2UOZ1kncn0 GNXLbijhrqGfa2tUH409Xhx6TEaTHacQRy4FQa32nXgMYD78YLOqsZei836IRbpLJhxRIWFlYQu oqI6r0a0M42ARJlLNHP1rlEhXkaazAmyWXZdLyXsZKRkVQ+tjTh4k10+DvN1p/C5+P4G1JbS2dD llZfLWboL4MleLQcrXywfl0JU= X-Received: by 2002:a17:90b:1812:b0:2cd:3445:f87b with SMTP id 98e67ed59e1d1-2cff95298e7mr12474309a91.29.1722911299798; Mon, 05 Aug 2024 19:28:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHCY2l2H8pECfx4W7Kisstu0pFCjpH1L26dAC69h0mCW8/2quAbY6FQIvUicWh3jktk5vb10nn5Wnd3lKIXd3Q= X-Received: by 2002:a17:90b:1812:b0:2cd:3445:f87b with SMTP id 98e67ed59e1d1-2cff95298e7mr12474288a91.29.1722911299240; Mon, 05 Aug 2024 19:28:19 -0700 (PDT) MIME-Version: 1.0 References: <20240805082106.65847-1-jasowang@redhat.com> In-Reply-To: From: Jason Wang Date: Tue, 6 Aug 2024 10:28:08 +0800 Message-ID: Subject: Re: [PATCH] vduse: avoid using __GFP_NOFAIL To: Yongji Xie Cc: Maxime Coquelin , Xuan Zhuo , "Michael S. Tsirkin" , Eugenio Perez Martin , virtualization@lists.linux.dev, linux-kernel , 21cnbao@gmail.com, penguin-kernel@i-love.sakura.ne.jp, linux-mm@kvack.org, Andrew Morton X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Queue-Id: 258F340004 X-Rspamd-Server: rspam01 X-Stat-Signature: cba7iyfc8m3kafc36448tit8ce4wfdus X-HE-Tag: 1722911302-888680 X-HE-Meta: U2FsdGVkX19Tz3voXTQoMjWrKrTQNofrCrIx+nzcUjS5PPOsPv5HJeZX9w+8KmskEO6089gS2HkJfqEeJo4SaVGwxnJgEN+FiTqmFv6MxHR4y1VhAFTaT3irfSfcM6uhqGSkAzE5e429kvcFp+3XI3lV9st69/KdmAnyTeJreXLR7ST0Q4MLFIIBfxS2EXXWD64D6vJqrvss5GuLCvaJ+PoV0B163U0+8k7kGSjPy0yNPqhxWEMuy7AsXDTQ+J6x0lOntPvcq1N+6L5q6Vt2TKaxfMcmCTrniG6ARLcZVJq0PpzV+JzvgxwGOiJvBs9fsjznsBHDX3OqwgyueS5JY7XbOPrYyIuDWolz6HIOmJUP7Xw1GA9jhuqqxPXzeziPhsD3a+AQCHcgulojbi/v3plk0FBr9OMfBl3xj+nnxIb+WC3EcocC3kttRl/MtbcEkhZoqud1Sr5YDlG1OI/4sl9H44BI1HjEXoK3zcyzX2tHEE2uFLX0dw86edP3oZVg88ib3yxolgwlAM+zPgcTPvJCb0bvnP8HtDHXEzMaRUz9XXxLuMllHo4OQg+fFeohMKqU+2xTteNOxT62jybRwQtWIEAT/qLbT2sCdwd7gVgNu7lY1D3+PU8ahnuIuX532yNVkZs1RyPEdV4/RErH9zqUltqlBcRRfEAFE/rYQblqKLHMh2PAvIVjpK16+2aF2JWcjfOMSPbC5RVRVbWO4m9Fmxr9/cGdjPXzJeK847kpXV+fKv4F1VG1B+IgKSWhfk3eQ5yvpAdqIvqPhnEiY+nLCFtc8zFNNMRh9toOAcrMxvRqdZFFt0eu1oZ2CuZMrVkb8icb93Dlaa6DG0y39q0QTNrHNWCHBJEMIPOmYZjMV2xpxWBnPrVkuPMo9ry6qV9sFDAF0C6E4pT9vohUYvFWXRKotrgHOcTTZgohTSCgC2ZAO/JDblCCk8XV7dg9XXetM0tZNgxB9ZNQKY/ Vm7vJlJs VO2uXkDPIPvaNAAqAe5TZRUG2YCPYHP2p24tWhFFhD+6fhlWIXcHv5HJjYgKGPeusT0wdynCsmnh8CalA/Rs+xRvsXQCYjB72op0vfiM5KHYrjXItA7ZxIqRei4Z+w5KPjbMXMrud+5gazFmvtzi4xJxm3eJdGwo0HZNBNKVDAn8J84GqxE+6GtU0h0q9zREIKzQw2rf2tNuuYwrA1uytOmoBlEwITkBGHbcArMvgjT5tbowt3+aYseKmMxFjavm+Kt3PXqQtMG2T1Xp8P5shDQOhsg1Iuw/sh3rg+JV5YqnMbHyANE0YawVetuMHcYgpoVgLrrA1/uoNzKCdFNlNaWC+c8jtfoMK9JMO+7RPvnOUhJJxRdVVAJKNevPBw8QPmTZZLSi9+zVb388poG/bh27Qr03T4ssZJokh3yiC5bbJ4+sWdgjKhgAbsQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Aug 5, 2024 at 6:42=E2=80=AFPM Yongji Xie = wrote: > > On Mon, Aug 5, 2024 at 4:24=E2=80=AFPM Jason Wang w= rote: > > > > On Mon, Aug 5, 2024 at 4:21=E2=80=AFPM Jason Wang = wrote: > > > > > > Barry said [1]: > > > > > > """ > > > mm doesn't support non-blockable __GFP_NOFAIL allocation. Because > > > __GFP_NOFAIL without direct reclamation may just result in a busy > > > loop within non-sleepable contexts. > > > ""=E2=80=9C > > > > > > Unfortuantely, we do that under read lock. A possible way to fix that > > > is to move the pages allocation out of the lock into the caller, but > > > having to allocate a huge number of pages and auxiliary page array > > > seems to be problematic as well per Tetsuon [2]: > > > > > > """ > > > You should implement proper error handling instead of using > > > __GFP_NOFAIL if count can become large. > > > """ > > > > > I think the problem is it's hard to do the error handling in > fops->release() currently. vduse_dev_dereg_umem() should be the same, it's very hard to allow it to fa= il. > > So can we temporarily hold the user page refcount, and release it when > vduse_dev_open()/vduse_domain_release() is executed. The kernel page > allocation and memcpy can be done in vduse_dev_open() which allows > some error handling. Just to make sure I understand this, the free is probably not the big issue but the allocation itself. And if we do the memcpy() in open(), it seems to be a subtle userspace noticeable change? (Or I don't get how copying in vduse_dev_open() can help here). > > > > So I choose another way, which does not release kernel bounce pages > > > when user tries to register usersapce bounce pages. Then we don't nee= d > > > to do allocation in the path which is not expected to be fail (e.g in > > > the release). We pay this for more memory usage but further > > > optimizations could be done on top. > > > > > > [1] https://lore.kernel.org/all/CACGkMEtcOJAA96SF9B8m-nZ1X04-XZr+nq8Z= Q2saLnUdfOGOLg@mail.gmail.com/T/#m3caef86a66ea6318ef94f9976ddb3a0ccfe6fcf8 > > > [2] https://lore.kernel.org/all/CACGkMEtcOJAA96SF9B8m-nZ1X04-XZr+nq8Z= Q2saLnUdfOGOLg@mail.gmail.com/T/#m7ad10eaba48ade5abf2d572f24e185d9fb146480 > > > > > > Fixes: 6c77ed22880d ("vduse: Support using userspace pages as bounce = buffer") > > > Signed-off-by: Jason Wang > > > --- > > > > Note for YongJi: > > > > I can only test it without usersapce bounce pages as neither DPDK nor > > libvduse users use that. Would you want to review and have a test for > > this? > > > > I can do some tests for it. Great. > > Thanks, > Yongji > Thanks