From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB82FD59D7F for ; Mon, 25 Nov 2024 21:47:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5CFAC6B0082; Mon, 25 Nov 2024 16:47:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 580B36B0083; Mon, 25 Nov 2024 16:47:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4201F6B0088; Mon, 25 Nov 2024 16:47:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 240846B0082 for ; Mon, 25 Nov 2024 16:47:34 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 703B11605F9 for ; Mon, 25 Nov 2024 21:47:33 +0000 (UTC) X-FDA: 82825954260.24.B5F5062 Received: from mail-qt1-f182.google.com (mail-qt1-f182.google.com [209.85.160.182]) by imf27.hostedemail.com (Postfix) with ESMTP id C7F5740011 for ; Mon, 25 Nov 2024 21:47:24 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=P4j3+Z0T; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf27.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.182 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732571249; a=rsa-sha256; cv=none; b=EZ9w3GI3qvQgUdJ8gI1pSH7tubVBJv3rQ+Qi/zITWYXWy2p+khSMxbNU4ucS98V9etcfL7 tpnwWVclODm6jq7BSGgutoSRfNODPMT0TYj2frr2KjhCtcBdIfmCAZm0BdbPuJPJal+o92 fOQcx30+fPfD2gASZEwN2ODpcftCUUc= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=P4j3+Z0T; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf27.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.182 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732571249; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XqWF7E0ew5gNQ6985xZJ/5/jCKh2abtLrhXPLjbHCyE=; b=ycJbtN0NzuIHKny1mLhMRkdUrwYHqjobJGDqXwCr0/hHQ3nlzXryP1tRympipgu+MUoenq VMFyg9zztSeyHGK8qDcjQN+guWKKlAjmMM/A6XT+mK1FjrYWV4FFkSxEs9jTt4CN3YEfl4 nJQ+vIF6h1ovPjHMr+hslaiC01zeM0M= Received: by mail-qt1-f182.google.com with SMTP id d75a77b69052e-46101120e70so35420451cf.1 for ; Mon, 25 Nov 2024 13:47:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1732571250; x=1733176050; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=XqWF7E0ew5gNQ6985xZJ/5/jCKh2abtLrhXPLjbHCyE=; b=P4j3+Z0T14tX2t5aoASzuJPwNbMpd+FrY+3uWuXTsKSbHX+N4CGcPIa3beupzTRkQz Idz7m8lmBsV9D+RocHeLsTJvtBCr36jUJXpxbDsA5gpNzwTLwNCREy3F/pQwS6SD2zDb ztGuSNjPXIkzaZojrjDzVMrTdf/nX1zPj9UqW1SlEAZfcQtXbOWB23ipwFz+TRxuMriv UvhzbJRBQ7bIRHPLiWJ4XlnR6JyojQqQUAPOYftEjGzwszuDu9kCICKGP+YGfik5IrLO rtXCAIPwkQnNed8XDZf6OTw1rPD1rBz2kIGPEwM19xrABEyY79k3zr75zWB+V1F2q5pA 0yrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732571250; x=1733176050; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XqWF7E0ew5gNQ6985xZJ/5/jCKh2abtLrhXPLjbHCyE=; b=OZT1hhquW31KwaE0JM1XMLlswsL3scxVEOgUIOoU9XAhCehZGYy9faT/7LDJfdLa9k 6YN6o+zd2P6vXq9OsfRoa3Xv7lr4knM5QADPa8oxioYVY0w/0BsVwBOSdFkn2GD1pY22 E7qSOEE75zXn1PLPMUQPJnigJO/y0LbUjm9SnNJxgD3/00B8sTOHyjQtOylPt6zQzCPn gqO2rXTi39vogMt07SaevOC7b4EM8qWyCOr8IJT1Nd/qk2fs9as36A67e4cky03JnBEK 3TTk3A5mTkjZYlHlzkDkZiW6yhv+Sdsu17HPfYoTNEF8zQBF/ajpeC1GS4z5Gn2vSQU2 7pcA== X-Forwarded-Encrypted: i=1; AJvYcCWi1ZsAX6pqKiAhN4CqvvZGYSQyflDSfGzcM/bZqzijbnLBVFqIMCRfhlgf28c1hY1t5wIvLuMZRQ==@kvack.org X-Gm-Message-State: AOJu0YyeuWvQJXLKpd1eCWOA31nKwO7W0Hu4DHwmFALscxq+BX1LRKEC oHnxPt6A94/BQTC7/PraFP8L/QU5FTyWpAdHe3T6mJZBf4pUrpXiLNKR6cJWOC8= X-Gm-Gg: ASbGnctH1FytzlGoA46cZKCal2WZpCbkvPzC3WzUEEv6Pwi8yhEwNkKnDNiEdU5qxgI JxrK3B4tJSqyPxi5qM42ZFSMDejyjRa1BFgCaJ6WyBpuE2VKLZfqgS/G5IMLMZitPPpVurkvGHW MxbsyJMhwCwoPJck8IFiEIFJV010BF1CbVhzYGEbDim81SAqQjJJCQHlzMDTIy8R1IPyxPsEM2x w0nOt0L2G4HupREBPsfAYiufDjE3ov3ME5flSOujmn+cS7/ X-Google-Smtp-Source: AGHT+IGhSINo662ZCmiw+KQ9F9w1c9zeHSlcbXLmFv0aSFwhD9uNu7MMnjUNn7TV9kplOqArqzYj3A== X-Received: by 2002:a05:6214:629:b0:6d4:3c10:5065 with SMTP id 6a1803df08f44-6d451345419mr187969176d6.32.1732571249894; Mon, 25 Nov 2024 13:47:29 -0800 (PST) Received: from localhost ([2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6d451a831bbsm47429646d6.27.2024.11.25.13.47.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Nov 2024 13:47:28 -0800 (PST) Date: Mon, 25 Nov 2024 16:47:24 -0500 From: Johannes Weiner To: Yosry Ahmed Cc: Kanchana P Sridhar , linux-kernel@vger.kernel.org, linux-mm@kvack.org, nphamcs@gmail.com, chengming.zhou@linux.dev, usamaarif642@gmail.com, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org, linux-crypto@vger.kernel.org, herbert@gondor.apana.org.au, davem@davemloft.net, clabbe@baylibre.com, ardb@kernel.org, ebiggers@google.com, surenb@google.com, kristen.c.accardi@intel.com, wajdi.k.feghali@intel.com, vinodh.gopal@intel.com Subject: Re: [PATCH v4 10/10] mm: zswap: Compress batching with Intel IAA in zswap_batch_store() of large folios. Message-ID: <20241125214724.GA2405574@cmpxchg.org> References: <20241123070127.332773-1-kanchana.p.sridhar@intel.com> <20241123070127.332773-11-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspam-User: X-Rspamd-Queue-Id: C7F5740011 X-Rspamd-Server: rspam01 X-Stat-Signature: qojyi4c48mmcrdofewchrf7zgzcyqaos X-HE-Tag: 1732571244-531947 X-HE-Meta: U2FsdGVkX1/63qLYjU+W3aNubrMe1zXWgGzy01Pb3b30oWhgYh/tbgyxfd02SvWXSvaxkdCMWsfAfV9z3jiLDdVpZNfDWMDOdYTQAt5+lIGK95uwxEc/VrwpXHRVltfu4kRat1p5zYWn0z2XaME5OnHjw9cnC7lqpVllTxFyigvYxvz3sbVr2lNYqaCWgcntfrytc1CcLw4nXerbQRQShxrB14RCnoX7kozcpXvNNGjDink1xtySKjdsRDNiey5et+QTZNNMWU4D0yT9yz6dbDktQ7citHBRq8guuzJJwQdqqcca8d6d9NwamSw1yQeRiVqzF1CPdQDCDPTWOjAZJvKhsL8p9S4OtRKxr8DQCIZBO9MXN9opGPlrm0+fIlcU4qdF4MF/2ocCgZNsRwavi5nO3kVTnrDC0xM7ZsIUzK816azvmzeaJlESEAtOrGAH+hlwRBGsenzWoG7lkLgvQaMBMA2nYo+tJULXhnyjEDz3mw5JoigwGg8x2d0BgXVAPgFU/2B7ppGQTHJEVXaOO8PcxaB/0wMEcZdO+hTdvNig5zxG3cq4tVUiT/2/YoqdK7mz0oMD4DPFM2gENeCSDzkSqy2/YVNogiZrji8YZWU7suX0ZyI8hUcgnwCtqrk7bd4w0qc9n+95cElSLfqsEchXJ/+V3//bRtDzqvRtDpIP1xpXIlJ1/5z7je6ePZ8yJwuU0RTr+gLyFLPT/YyFXyzegJeB95c0UW+3bGsrnQ6o4YxIRtv7tBRDLIIJhiA0gJUtVN7PTuWYRmkaR2qW0S2Ef0uRuTOYYsBexhOGaVnxEiGQI2KRjRjDEe86UH9YHpFkXrSjD+BUw7bBhxPsEyJbjVyMBrhdd85e1eBUiKvQOqT3cPkQVLMJsJjhk8s/8vPCDRLr+8eyvZVEzrpek2l6jmbRiKR2fD482NL3E2wogLP+Md2PfUlPo8S3vXNKlPqjy3Qn8g1ZkAW5D3g eXpyQlml WQ3wg1WBRPk3LdI2aWmsa3+GeGmmQQWew26+zwDW5RR5Z2Qlp+Xb0uDGXgkVG+tCD+z0QHdiHnavJJc5+mM1xbIlHbKVWkBkjmH+PwJpbekoSlBhdTvXTarnzeUjxRzaI996nKmXmohOrjsyXh5XebzVWWm2pJPCIw/rRn/UgXbZK4mMXbRqpjVZDa/r3oxP5GZmlFhmP/ME2N+GG75iY6zaPtHOl4t0QTiO3lSoOfRIhjtVU09np/mi64TyUjMewfAPnIDqDb2vwBR0Du+AohxpoOehCckd0G5LWmwRfZgFsAfVbZik41v6gXLx4IJsy5OXAT6381+MDmYBUmBPId64eYn+k5y/EEXm3THyA8WaWqG276TOOPhxdesZtQX33HPokVw+RPq3bFg6RC4tJa2XcV9xoFPE9Lzo7fyN7KuTwiAj0JGyX8bbpcQv67sO+mVcC+wg5ZmA65h8SlOmbdCsAjAu79YwdtSuTnj3f8DQy9dMIpfzX9nfyafiSCDDUCKlUg9o6/42yH1TAQ0Ef/7S4rIeepyimGMmHWOJECmaPa/iLzLbcZvvizFjrPHH44u7qlDwp/Rivshf2d2xXchFecPER0wbMNnIx+FWFCDWCw+WQGkksl4ghyw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Nov 25, 2024 at 12:20:01PM -0800, Yosry Ahmed wrote: > On Fri, Nov 22, 2024 at 11:01 PM Kanchana P Sridhar > wrote: > > > > This patch adds two new zswap API: > > > > 1) bool zswap_can_batch(void); > > 2) void zswap_batch_store(struct folio_batch *batch, int *errors); > > > > Higher level mm code, for instance, swap_writepage(), can query if the > > current zswap pool supports batching, by calling zswap_can_batch(). If so > > it can invoke zswap_batch_store() to swapout a large folio much more > > efficiently to zswap, instead of calling zswap_store(). > > > > Hence, on systems with Intel IAA hardware compress/decompress accelerators, > > swap_writepage() will invoke zswap_batch_store() for large folios. > > > > zswap_batch_store() will call crypto_acomp_batch_compress() to compress up > > to SWAP_CRYPTO_BATCH_SIZE (i.e. 8) pages in large folios in parallel using > > the multiple compress engines available in IAA. > > > > On platforms with multiple IAA devices per package, compress jobs from all > > cores in a package will be distributed among all IAA devices in the package > > by the iaa_crypto driver. > > > > The newly added zswap_batch_store() follows the general structure of > > zswap_store(). Some amount of restructuring and optimization is done to > > minimize failure points for a batch, fail early and maximize the zswap > > store pipeline occupancy with SWAP_CRYPTO_BATCH_SIZE pages, potentially > > from multiple folios in future. This is intended to maximize reclaim > > throughput with the IAA hardware parallel compressions. > > > > Suggested-by: Johannes Weiner > > Suggested-by: Yosry Ahmed > > This is definitely not what I suggested :) > > I won't speak for Johannes here but I suspect it's not quite what he > wanted either. It is not. I suggested having an integrated code path where "legacy" stores of single pages is just the batch_size=1 case. https://lore.kernel.org/linux-mm/20241107185340.GG1172372@cmpxchg.org/ > What we really need to do (and I suppose what Johannes meant, but > please correct me if I am wrong), is to make the existing flow work > with batches. > > For example, most of zswap_store() should remain the same. It is still > getting a folio to compress, the only difference is that we will > parallelize the page compressions. zswap_store_page() is where some > changes need to be made. Instead of a single function that handles the > storage of each page, we need a vectorized function that handles the > storage of N pages in a folio (allocate zswap_entry's, do xarray > insertions, etc). This should be refactoring in a separate patch. > > Once we have that, the logic introduced by this patch should really be > mostly limited to zswap_compress(), where the acomp interfacing would > be different based on whether batching is supported or not. This could > be changes in zswap_compress() itself, or maybe at this point we can > have a completely different path (e.g. zswap_compress_batch()). But > outside of that, I don't see why we should have a completely different > store path for the batching. +1