From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28251C02198 for ; Sat, 8 Feb 2025 21:34:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B66A6B0082; Sat, 8 Feb 2025 16:34:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 865626B0083; Sat, 8 Feb 2025 16:34:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 72C656B0088; Sat, 8 Feb 2025 16:34:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 576FE6B0082 for ; Sat, 8 Feb 2025 16:34:22 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 06ED4A024B for ; Sat, 8 Feb 2025 21:34:22 +0000 (UTC) X-FDA: 83098081164.28.3C69BB3 Received: from mail-vs1-f47.google.com (mail-vs1-f47.google.com [209.85.217.47]) by imf22.hostedemail.com (Postfix) with ESMTP id 24A18C0002 for ; Sat, 8 Feb 2025 21:34:19 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=fZ7IHyuX; spf=pass (imf22.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.217.47 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739050460; a=rsa-sha256; cv=none; b=jCII0pUoLfqLWC6U2noDMj9/fe9dQIOV7+bZvuTdFzsELx+EJ2XTVNCbm3J0dAPUDFHWR6 wVEVzwxHP18aLLsCscOtPcFQyZwIIptJleYTnecXKRuQYNlqM1/1urxegb7oD4nkKqNHQU 6lma34r/hYovSJfJAim6FDxJmCQ0wic= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=fZ7IHyuX; spf=pass (imf22.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.217.47 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739050460; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Tr1A4JBrm8DZqdxvHg/4VfN3wYIha/zlu4oSdgbrsgY=; b=Ph8I8qbaKtdto9uctye/N0e9EjrfzbRmLU8C6mc3RBjECJHV0eVZLssAPODW6jtItdM7TQ M3Gr/pCaXIvvubvmuChrrXfHBRXNhiaoSEO1hWHVnVdBwWmpph/PYWM891p65JxSjok8pP fm/+Xpdf/Vr4FfqW60Hzrwn8aCH2RmE= Received: by mail-vs1-f47.google.com with SMTP id ada2fe7eead31-4ba7738fbc7so1756881137.3 for ; Sat, 08 Feb 2025 13:34:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739050459; x=1739655259; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Tr1A4JBrm8DZqdxvHg/4VfN3wYIha/zlu4oSdgbrsgY=; b=fZ7IHyuXw06vtuoWXI0VjdDuY4Ks7OviaEMNZKahW/G5t9ywrPSmd2uQzqduQ1HUXE TETngIITypi4zK45Eh6ZNGoNW/a/xMxZNL/yiqbJn71yh2huM/DZ/TNg7Ed1tOXK1y+B F40eV0V1WosZgz5sS3bHfGaZrKPHXqu2lkssqlbyMQJesLbGhMF7PHrmzwg69Pquq9o3 pKaAu1yOovPw1MEjbcErCXUMKsagSr3MBdXVU2DN81A+MO3f73ojl3HgdpH+4/fdiYGa JOYfkYV0JRQ+SJfbProNRnETOIlkClZC8DdLugS6wp1g1JjHaan0Tqo8rLFpCBS+jLSq dWYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739050459; x=1739655259; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Tr1A4JBrm8DZqdxvHg/4VfN3wYIha/zlu4oSdgbrsgY=; b=WAc2/78SOaCL4kXZ6Wa7EqvWn83LEZGQsK5saEhvHkamVySl0QH0fHYQp9YYNsHQuo TCI90qazuvMK6o5nN5g/3PfBGn29+NTbiHE1jTM7SXGXV2KsMD61wb729PuWHtSDoU3R eABAZ/hH+7QJlAzD/dtM1Ft1I6EUKGw+x9dnh+bRTm6cHtu+Zf7oXfF15lxK4M1Fte2l Us9ZGMPCL5Gxyd6bgkAP24BdRFPYy3kelj7Zp+X4XMfzapI0LNAKxAV5YymXzTOS2bkw swKx9Jkdvvyaaek4vuvRPeAKVGi2INFuMF/OpJATVTE4z++QG2IHuSDgXisk48pKBet8 hj/w== X-Forwarded-Encrypted: i=1; AJvYcCWKRpDHIBCmNH3/F8D2ngrpDeNfTSYZq+GYd+PmxXP6TWtn5qdCaBg09v8xrfZpfWsJl/fM2dL++w==@kvack.org X-Gm-Message-State: AOJu0YwN5wCv2ka8mnBykv9hWohW3KkLTSnsomwnUVJTcr2jix5cX3JW 2YFVIh82qD08IGqt/YUPzx34/XssDLN0Lv8FbR/yWTZvSt2rUI8kPA5cLxNa+pzf6bwjReBnvpE 4pA6qUggeC/1GtgwiY27BtcNk+Kc= X-Gm-Gg: ASbGncuycPRl2F1j1pjpILLhJHhlLqItrPJSJpKUtkpVBEWc8ovr/I+UddMVn6AxAje 0/O05wX43g5VSa9mbVwkn5V91joYpzE3LxRJ6090S8fXWnMFQuAS/kjDh9+P6DeGIKgeeWrxQTE FWH4KncNSi6RCB68PXEFqBfd+OBRTowQ== X-Google-Smtp-Source: AGHT+IGCMVvzBkTONyCiGLzGRx+BYTIKnuFp2mhSuxrcG5tm0TfmfKRTUXaNMu0nFAC/FmnQ5jzwqcE+MIxDGGSNRQM= X-Received: by 2002:a05:6102:6a98:b0:4bb:9b46:3f6f with SMTP id ada2fe7eead31-4bb9b467825mr1105069137.1.1739050459210; Sat, 08 Feb 2025 13:34:19 -0800 (PST) MIME-Version: 1.0 References: <1737717687-16744-1-git-send-email-yangge1116@126.com> <28edc5df-eed5-45b8-ab6d-76e63ef635a9@126.com> In-Reply-To: <28edc5df-eed5-45b8-ab6d-76e63ef635a9@126.com> From: Barry Song <21cnbao@gmail.com> Date: Sun, 9 Feb 2025 10:34:08 +1300 X-Gm-Features: AWEUYZmHeEniBO1KvmZ5SsYHPj_w-1AYYlBIQu74Tt0dYgU0dn8Jppk0fRFDeAs Message-ID: Subject: Re: [PATCH] mm/cma: add an API to enable/disable concurrent memory allocation for the CMA To: Ge Yang Cc: akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, david@redhat.com, baolin.wang@linux.alibaba.com, aisheng.dong@nxp.com, liuzixing@hygon.cn Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: by1cue54a5tqwu8zjnertk4c1iwrxbnu X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 24A18C0002 X-Rspam-User: X-HE-Tag: 1739050459-175473 X-HE-Meta: U2FsdGVkX18jZXeNMWFeQ2txm965VLo/Qqyd8lzKDNVXRx9W0Nx5/QY6Hp2vMVAlM4Bf/slc+N6BrF/KCXy1tBxEO1KqWQIrw4MzL8G2i5fC6Kz6BZjCHNa65C9gmHZzk4atEn3xpMeG4JzCVzKvxY9YTi2nop3P6L/1+IFfqYHcr4STRRgXEtJhPSGx/ApNPlJCZPAx4YFyjpM7tu4Iv1TreRsbkgbaPwmc0brkMYXi6QkkwcaOluKtNalHinBTgXOipMEaOGg5XdaKLhrX0CRb/2u5khj/WkrdK45Okj6Q7ipnTFJibyvcX4+h7mpxV2MTSQVOHdKD7eDU4iN4GNesLlhBkCCqna2e97U80EpRBtfP3oNtWyJ59YlvRAyqgkIn9q6zEixI7TqOzZUxBviFQa0ppmu3McQtSlm9J0F5JYisW/+ngwN25qHvBa9iWqrRBCCcVruJcm9llRfP1DwjnFOZRcO7G+oWRcHmTfT2dbE9OUv0tE2kBfNLZTtsPfSyOz95S1jBOMaaofdF8C4xT6KJEwTfbiN9avP0Rl+eqUS9htD46OOPOCUSeAIxriwXRC6INgujEC6FCedgE8+MVqaJPDsDNSBJzFYazedaPhkbH92vKMMBBllwmq/TFCODSu4BHk2K0YK4ENxCoucLcZV4ZFK8WX1jYWZsqNo4AwG23U0mkpfsAdKlwIULIRpz0LxLBpDEC51gIwe/2L4sUJMJNyrttlkGk58T11qYT2fwImlaoHkU2neuS/nX/uLwTOhDVQKp+09yMUs8bg+SVhuOuX2r6ecZIbb5nQRwwK35yMAVYjKfZlHYo+v+ueWvwBnsBDjapQXdspPvDhkWPpT9J0qaGQOl6nl2ciHE5Am6OGCSFQDeHYB8hfe/Ey8Id1BiNxB/qVEh0giHP5Nzpx8bST9wubfo0NdtvSF42cTtQIAr6Q0LpBk0vKkDVNk61/LAWPtEKTbyH7F +zMa3eRt v0oP+pCNLSBfWzvAP9zP+I9kvy3brcl62h0FmyQW6YTjmYDavg8Y2s2WCq8d2Ejamc5T0jVjht6zbt/U/QWJFav4fBAfWPEXCPPbyhSo7h0XHU4eVX+u/94NI4qy/VRJbhGLiH0HBoi40wLT5NtLf0w2JYglB1bsRFiz9ycNh9ZMd3eZCArt2VnqczDd+Gb5VYrRrkWa2f++k4bbXz33GBvjOM7SrY6a/ibQUXcIT4Lhvvzkjzp8R3VEMloVcoXLT+vk7I+ECuV3lUcECQXjR9Qyuet/gDqkUyDP80xvGgfccJzR2MhkFao2+lDyclmPixBCOKtnmjxbxY23nQazomh2bDrLZ+UwKRFQKyNipXlY2TWa0e+u4N3r9w0svBZ9XwYgyTcpWFccolrdjGKjKx9fSoQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000099, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, Feb 8, 2025 at 9:50=E2=80=AFPM Ge Yang wrote: > > > > =E5=9C=A8 2025/1/28 17:58, Barry Song =E5=86=99=E9=81=93: > > On Sat, Jan 25, 2025 at 12:21=E2=80=AFAM wrote: > >> > >> From: yangge > >> > >> Commit 60a60e32cf91 ("Revert "mm/cma.c: remove redundant cma_mutex loc= k"") > >> simply reverts to the original method of using the cma_mutex to ensure > >> that alloc_contig_range() runs sequentially. This change was made to a= void > >> concurrency allocation failures. However, it can negatively impact > >> performance when concurrent allocation of CMA memory is required. > > > > Do we have some data? > Yes, I will add it in the next version, thanks. > > > >> > >> To address this issue, we could introduce an API for concurrency setti= ngs, > >> allowing users to decide whether their CMA can perform concurrent memo= ry > >> allocations or not. > > > > Who is the intended user of cma_set_concurrency? > We have some drivers that use cma_set_concurrency(), but they have not > yet been merged into the mainline. The cma_alloc_mem() function in the > mainline also supports concurrent allocation of CMA memory. By applying > this patch, we can also achieve significant performance improvements in > certain scenarios. I will provide performance data in the next version. > I also feel it is somewhat > > unsafe since cma->concurr_alloc is not protected by any locks. > Ok, thanks. > > > > Will a user setting cma->concurr_alloc =3D 1 encounter the original iss= ue that > > commit 60a60e32cf91 was attempting to fix? > > > Yes, if a user encounters the issue described in commit 60a60e32cf91, > they will not be able to set cma->concurr_alloc to 1. A user who hasn't encountered a problem yet doesn't mean they won't encounter it; it most likely just means the testing time hasn't been long enough. Is it possible to implement a per-CMA lock or range lock that simultaneousl= y improves performance and prevents the original issue that commit 60a60e32cf91 aimed to fix? I strongly believe that cma->concurr_alloc is not the right approach. Let's not waste our time on this kind of hack or workaround. Instead, we should find a proper fix that remains transparent to users. > >> > >> Fixes: 60a60e32cf91 ("Revert "mm/cma.c: remove redundant cma_mutex loc= k"") > >> Signed-off-by: yangge > >> Cc: > >> --- > >> include/linux/cma.h | 2 ++ > >> mm/cma.c | 22 ++++++++++++++++++++-- > >> mm/cma.h | 1 + > >> 3 files changed, 23 insertions(+), 2 deletions(-) > >> > >> diff --git a/include/linux/cma.h b/include/linux/cma.h > >> index d15b64f..2384624 100644 > >> --- a/include/linux/cma.h > >> +++ b/include/linux/cma.h > >> @@ -53,6 +53,8 @@ extern int cma_for_each_area(int (*it)(struct cma *c= ma, void *data), void *data) > >> > >> extern void cma_reserve_pages_on_error(struct cma *cma); > >> > >> +extern bool cma_set_concurrency(struct cma *cma, bool concurrency); > >> + > >> #ifdef CONFIG_CMA > >> struct folio *cma_alloc_folio(struct cma *cma, int order, gfp_t gfp)= ; > >> bool cma_free_folio(struct cma *cma, const struct folio *folio); > >> diff --git a/mm/cma.c b/mm/cma.c > >> index de5bc0c..49a7186 100644 > >> --- a/mm/cma.c > >> +++ b/mm/cma.c > >> @@ -460,9 +460,17 @@ static struct page *__cma_alloc(struct cma *cma, = unsigned long count, > >> spin_unlock_irq(&cma->lock); > >> > >> pfn =3D cma->base_pfn + (bitmap_no << cma->order_per_= bit); > >> - mutex_lock(&cma_mutex); > >> + > >> + /* > >> + * If the user sets the concurr_alloc of CMA to true, = concurrent > >> + * memory allocation is allowed. If the user sets it t= o false or > >> + * does not set it, concurrent memory allocation is no= t allowed. > >> + */ > >> + if (!cma->concurr_alloc) > >> + mutex_lock(&cma_mutex); > >> ret =3D alloc_contig_range(pfn, pfn + count, MIGRATE_= CMA, gfp); > >> - mutex_unlock(&cma_mutex); > >> + if (!cma->concurr_alloc) > >> + mutex_unlock(&cma_mutex); > >> if (ret =3D=3D 0) { > >> page =3D pfn_to_page(pfn); > >> break; > >> @@ -610,3 +618,13 @@ int cma_for_each_area(int (*it)(struct cma *cma, = void *data), void *data) > >> > >> return 0; > >> } > >> + > >> +bool cma_set_concurrency(struct cma *cma, bool concurrency) > >> +{ > >> + if (!cma) > >> + return false; > >> + > >> + cma->concurr_alloc =3D concurrency; > >> + > >> + return true; > >> +} > >> diff --git a/mm/cma.h b/mm/cma.h > >> index 8485ef8..30f489d 100644 > >> --- a/mm/cma.h > >> +++ b/mm/cma.h > >> @@ -16,6 +16,7 @@ struct cma { > >> unsigned long *bitmap; > >> unsigned int order_per_bit; /* Order of pages represented by = one bit */ > >> spinlock_t lock; > >> + bool concurr_alloc; > >> #ifdef CONFIG_CMA_DEBUGFS > >> struct hlist_head mem_head; > >> spinlock_t mem_head_lock; > >> -- > >> 2.7.4 > >> > >> > > Thanks Barry