From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B887EB64D9 for ; Mon, 26 Jun 2023 18:21:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A86158D0005; Mon, 26 Jun 2023 14:21:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A36108D0001; Mon, 26 Jun 2023 14:21:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D7958D0005; Mon, 26 Jun 2023 14:21:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7C6AC8D0001 for ; Mon, 26 Jun 2023 14:21:58 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 47F3AAFEC4 for ; Mon, 26 Jun 2023 18:21:58 +0000 (UTC) X-FDA: 80945717916.12.3162470 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 04C4A4001F for ; Mon, 26 Jun 2023 18:21:55 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cylNse77; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf04.hostedemail.com: domain of dakr@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dakr@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687803716; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7d5o216+gZzb7LjG6wT7FRl5pgwrzpiiCyEaun3wVWg=; b=3gPwG6FOluNGp4urdygXeMB8OqTXUujOPhNPLWqkfmzSFFTwHw2XXLLHqbcsAQl3iEDvsz WquZnzZorOF4Yiy0LOpnROVWXe40u/X629WUFfAp55et93enms5MwKSsLcFkBPTkt8/NFg HMRQN833x8ElcJt6iLH8iH4vBl/4vcs= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cylNse77; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf04.hostedemail.com: domain of dakr@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dakr@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687803716; a=rsa-sha256; cv=none; b=uhtkR3vDoPrvmKKBz8yJ4BLkLoxUXhaAD0F7RN5RnSoezjB5drMnwZN/vW9CEzFXX1Wn3N jUe29LuflmeVpe3mbfo0acKeBneh74BVy5AcqmCPMFCMeSZ+OSKrlm39FeveRMiFvcfJRP +Pfg5pxmKsMo4oxJ3Dg7uUgMGw1JEj0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687803715; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7d5o216+gZzb7LjG6wT7FRl5pgwrzpiiCyEaun3wVWg=; b=cylNse77uOyFnKMeb0fMDlPpreErtqnRv9giHa0LoSiF1KAvlwRmZJ0RttaG2xsa1BPEzr gbP9CDXKQE/FiTA+SDRIfqnAl0KYZ2wQtkvNJEOp2PrJcnPV4WiLPfTkYU8XVT8crOXsWk fvZbg0FI04wZuTemX8Q2um8CSO9KNJM= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-151-vvdYj5z2NriF_o6XN2XNSA-1; Mon, 26 Jun 2023 14:21:53 -0400 X-MC-Unique: vvdYj5z2NriF_o6XN2XNSA-1 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-31283f7f2d1so2036908f8f.0 for ; Mon, 26 Jun 2023 11:21:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687803711; x=1690395711; h=content-transfer-encoding:in-reply-to:organization:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7d5o216+gZzb7LjG6wT7FRl5pgwrzpiiCyEaun3wVWg=; b=MRNb8Urk0DFZCBwZ6RIQqvsyey0ZGuFnrGFtHDNZaC9k5syFZO14S/QOSA6K69rAon tk8r1/WTbKNz0gO04IWtQSknKUJa0REXRVjikohWs2p6aTpVyJvbmve8487nfw6K4Lt8 ZHpb7Y7ysypjR8yl8su09BqnHFDDqcduVh9fyMLPvqBP7vdR+/TcCXGiIoWMjG0OPS1m jFgZLesW5RCVcpw3uq+Q5idNJRIW0F7eyb4o+EuOKBsWAfq0GSw3e3kPI7fySEYt0GbZ 6S60+82XKgSC7NoC0qrSTKGJjYm91+OezJjf10N3HoKlZ8EE1C6UaSB0YlPhqPokEPPL 26Vg== X-Gm-Message-State: AC+VfDxgvJ+B9JmVFqK++m/FzTF+83P1jUWapOQ5/MQBsvPUuA4fvdB2 blDlL5erIQwV5967FAFhN+pBoN0U6SxESvz0JKR6Z8lx8qYMD16X1H+OdKWUFl/d4CdhYD+KsXc epSnoh+c9PiM= X-Received: by 2002:adf:f887:0:b0:313:e08e:5599 with SMTP id u7-20020adff887000000b00313e08e5599mr6341892wrp.67.1687803711740; Mon, 26 Jun 2023 11:21:51 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5n+Lk6EWcxrnDxX3B3uow6vJnKDCug7Zp3C4LwW+oyKCc4uz4GdaK6qErbD8yunrefffBgxg== X-Received: by 2002:adf:f887:0:b0:313:e08e:5599 with SMTP id u7-20020adff887000000b00313e08e5599mr6341876wrp.67.1687803711451; Mon, 26 Jun 2023 11:21:51 -0700 (PDT) Received: from ?IPV6:2a02:810d:4b3f:de9c:642:1aff:fe31:a15c? ([2a02:810d:4b3f:de9c:642:1aff:fe31:a15c]) by smtp.gmail.com with ESMTPSA id f14-20020a5d58ee000000b00313f1c5b56dsm3387316wrd.79.2023.06.26.11.21.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 26 Jun 2023 11:21:51 -0700 (PDT) Message-ID: <362330be-1ff1-d2cb-de6a-6ad42cbb9d58@redhat.com> Date: Mon, 26 Jun 2023 20:21:49 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: [PATCH v2 14/16] maple_tree: Refine mas_preallocate() node calculations To: Peng Zhang Cc: maple-tree@lists.infradead.org, linux-mm@kvack.org, Matthew Wilcox , Andrew Morton , "Liam R. Howlett" , linux-kernel@vger.kernel.org, David Airlie , Boris Brezillon References: <20230612203953.2093911-1-Liam.Howlett@oracle.com> <20230612203953.2093911-15-Liam.Howlett@oracle.com> <26d8fbcf-d34f-0a79-9d91-8c60e66f7341@redhat.com> <43ce08db-210a-fec8-51b4-351625b3cdfb@redhat.com> <57527c36-57a1-6699-b6f0-373ba895014c@redhat.com> From: Danilo Krummrich Organization: RedHat In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 04C4A4001F X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 63b4g56zzmd7u1woosuoeky5jx5do1zh X-HE-Tag: 1687803715-327142 X-HE-Meta: U2FsdGVkX18/nnRXDAsd4MyGa+tN2UCqmWvgHb9hGXuK245tDYAIb2CbUkzFWxXYimEsB1YywbM1EwAvtyEBkLGPsV8qwANacXtNwwNr5buwnL4ik3N40jplmwy9eQ4ThijWnYfSYOkHt9wzWHbkcVxgObrIQmIeiIY/nH6XVAgPGnN1fxfLouljxTTxiKY4C3v5zEJnqINafKonFVklbdhwvtqdvimuKWg7LolfBT7nIywpUb52iwhUkeu00fBvvbyjlxzHDvAg4kN4SkmRRWQveBvfn6pkI7dsp/C3FuG7jwxlxU4z6Nh34gOTjynYSy6GJrFHLhG5SSlBhfggd7Ezwrl1FUKdHyYreBqCyPDYTuM0TrhDnftNWqxxEhaeSqIJlQ75syvat3xpkhzmCUmCky8TT3nfH3h1EeyWT3CZHbuk27jnUgsjfH2xxaVH7Z7jKTeEWkpFbuwL9oT9/MnzhkcBZLpyzFNlmTI7E065sk3f4AK4PHspwxjUBe5JKuAvbponkHwWNVH0AbyoYUSBZWDilUZaOICP9M6AZxXTYUqj4HodilVSccYk6lJc/yiKWqvRR49PGIMQ1+zDM1LImlcx5r+hZEuIcxhW/W25bQOEHLFtEfmKKrYwFMbwmnyshMhgbM6WJUsjPn3cvXz2FfEs0rS06lNg+8KfR6n+o8ssTBlw38fJQpM+Pqgs1etUA9ZlKacfDM22z7bBfJ+H5klcEsITHWLAUqlwdWz/NhIIoMOSYT2cN1jtLTPR8f+rJjoGGeTsvYp0s7bmRDR+XcO0HK3QutU9/1M4ZZLtagL9pLtH27ZEpnM8l8Q/z4ETjbuF/I0BGwAvaaXmvtbjBpxwR/zGpmXYQVxhRQA0qSabL1YstvEN4NQ/BRWiUTuDARd8bnbUwonmbb8Xk49BJcesIQHx21Jo5nCXGz7tjaxrUrkSQyF6AVoJHVsq6r5fQTfNVwyMqT9eNeq TDj1WZcb KB1HiR06MgAjaWYQe2pPYoiS0CsTXuenanTxtfC9UIrQ8IUZPL8XkZsowBD3M+NTYq4m8ZwSiqE8pgw0rloba7d6x27840jebqx53ViDP/ahZJvxPz5j3XoEd6TY565shWM8N3IKPytgkKzOsRu0u1x2+KpnQoeG28nkVTDGXWYw1CsbMXACa2IyhsblcQyvmjP3LJjahink6C7nZY/hdLfeqvLSlC6kvytM0zliI6w3Kl5s/UqkG3hdH2wmrSSQIGWy77gVVhSt55xYpKtrdY1TBCSsnVGF+7DNujhXZXvD4V7kEaWSCM/hH9CRBbl4HtnSpS7C7GnIyY6Q18XCpN1SkPf0vE2hKkeXG3qAa6GEUZ3hAN/IhfcRDjv+pl7u/4FuZD85VTG549raysWfy3KGhyCGN+QctWlbz53DKV+y+PlzQvWDAnh/hql3GeE44Rdg2BgSdorHz5UM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 6/26/23 16:49, Peng Zhang wrote: > > > 在 2023/6/26 22:27, Danilo Krummrich 写道: >> On 6/26/23 15:19, Matthew Wilcox wrote: >>> On Mon, Jun 26, 2023 at 02:38:06AM +0200, Danilo Krummrich wrote: >>>> On the other hand, unless I miss something (and if so, please let me >>>> know), >>>> something is bogus with the API then. >>>> >>>> While the documentation of the Advanced API of the maple tree >>>> explicitly >>>> claims that the user of the API is responsible for locking, this >>>> should be >>>> limited to the bounds set by the maple tree implementation. Which >>>> means, the >>>> user must decide for either the internal (spin-) lock or an external >>>> lock >>>> (which possibly goes away in the future) and acquire and release it >>>> according to the rules maple tree enforces through lockdep checks. >>>> >>>> Let's say one picks the internal lock. How is one supposed to ensure >>>> the >>>> tree isn't modified using the internal lock with mas_preallocate()? >>>> >>>> Besides that, I think the documentation should definitely mention this >>>> limitation and give some guidance for the locking. >>>> >>>> Currently, from an API perspective, I can't see how anyone not >>>> familiar with >>>> the implementation details would be able to recognize this limitation. >>>> >>>> In terms of the GPUVA manager, unfortunately, it seems like I need >>>> to drop >>>> the maple tree and go back to using a rb-tree, since it seems there >>>> is no >>>> sane way doing a worst-case pre-allocation that does not suffer from >>>> this >>>> limitation. >>> >>> I haven't been paying much attention here (too many other things going >>> on), but something's wrong. >>> >>> First, you shouldn't need to preallocate.  Preallocation is only there >> >> Unfortunately, I think we really have a case where we have to. >> Typically GPU mappings are created in a dma-fence signalling critical >> path and that is where such mappings need to be added to the maple >> tree. Hence, we can't do any sleeping allocations there. >> >>> for really gnarly cases.  The way this is *supposed* to work is that >>> the store walks down to the leaf, attempts to insert into that leaf >>> and tries to allocate new nodes with __GFP_NOWAIT.  If that fails, >>> it drops the spinlock, allocates with the gfp flags you've specified, >>> then rewalks the tree to retry the store, this time with allocated >>> nodes in its back pocket so that the store will succeed. >> >> You are talking about mas_store_gfp() here, right? And I guess, if the >> tree has changed while the spinlock was dropped and even more nodes >> are needed it just retries until it succeeds? >> >> But what about mas_preallocate()? What happens if the tree changed in >> between mas_preallocate() and mas_store_prealloc()? Does the latter >> one fall back to __GFP_NOWAIT in such a case? I guess not, since >> mas_store_prealloc() has a void return type, and __GFP_NOWAIT could >> fail as well. > mas_store_prealloc() will fallback to __GFP_NOWAIT and issue a warning. > If __GFP_NOWAIT allocation fails, BUG_ON() in mas_store_prealloc() will > be triggered. Ok, so this is an absolute last resort and surely should not be relied on. I think the maple tree should either strictly enforce (through locking policy) that this can never happen or if API wise it is OK not to lock these two is legit, return an error code rather then issue a warning and even worse call BUG_ON() in case it can't fix things up. - Danilo > >> >> So, how to use the internal spinlock for mas_preallocate() and >> mas_store_prealloc() to ensure the tree can't change? >> >