From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C82FC021A0 for ; Fri, 14 Feb 2025 03:33:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E01256B007B; Thu, 13 Feb 2025 22:33:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D89AA6B0082; Thu, 13 Feb 2025 22:33:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BDC066B0083; Thu, 13 Feb 2025 22:33:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9F85F6B007B for ; Thu, 13 Feb 2025 22:33:43 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5185F4BF4B for ; Fri, 14 Feb 2025 03:33:43 +0000 (UTC) X-FDA: 83117130726.21.767FA5A Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf27.hostedemail.com (Postfix) with ESMTP id 5558F40008 for ; Fri, 14 Feb 2025 03:33:41 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=dZAlbRA8; spf=pass (imf27.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739504021; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZWwj1KR66ci0MimMVTAf+7EryvAhOmVMzbLfi/0Mh9s=; b=2ddommJRTJuW0kTKMGrnJmVugDrXOvaTwUx2IocQi8VnN5fj0bQvASyiHjUSn97AEWHsWJ exFHGscwM6IMKvlX7RnsPyynNWj8ojZOjsPQ3um50Xc9etw486+HIrkjYF5UGKZP3zSRr8 8vjdC7DK2vguepkWJeeCsf3sMDVksSI= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=dZAlbRA8; spf=pass (imf27.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739504021; a=rsa-sha256; cv=none; b=QfYYXPpvnPWyndqUVgZPEJwVBZIx9BAL457bQ0/c2L4jMwQJdMyO+iakQ5EdHoU99eM29f bTltMeRe+BqFPWwp9tHWtmeUckWqYt3YDkYxmDUd/X+gQN3prjTFxiDrZHj3Sn9G95ltsf IxVK9kKTaruO5PsNEr7/+OsGh49W6dI= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-220c8eb195aso30477885ad.0 for ; Thu, 13 Feb 2025 19:33:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739504020; x=1740108820; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ZWwj1KR66ci0MimMVTAf+7EryvAhOmVMzbLfi/0Mh9s=; b=dZAlbRA8RAC75VBU9jFU06hhaCPx9kUtw8sfmAJZITJlqOgDDvsjwovgE9pC7OlyTk a/dRcyAEwPVuHhJrUP/ilyLZ1qgo9foc4PP21CxGZbLLOZJixeMa+HLWZFWVfub1Zv9B AoB+jxmg2jwH+yCWzeCG8WKV7rGCGX4KzqFn4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739504020; x=1740108820; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=ZWwj1KR66ci0MimMVTAf+7EryvAhOmVMzbLfi/0Mh9s=; b=PWeTeL+WY1JDZL3CRY+pQE64snuVl11D5A88kTcAhr1xX7L5xxNTjIyhhMHiBPEFR1 nIIOIRg7Lj8tpYA+7YKfev2CDL0OVgMXMEveofkGD/hSpd8vBqJsEFw5RIhi6lVZtiMT 36fTbmDXiCmIPTVykWJwvklT0A6miSY2m8yiWOrPKn1nCfFo3PQQFtNTd6HgNV0jYHtq HJX/N5hSiHNUI90lyumZGqukd6pSrlO1tx/+hy8zESAGMQMc8/ZtlFz+LUcM50oi08kQ xgj8amjPWa4BG+uY2NcMYCZ72vdCozAwwvsRBLOa4rbZf5yp97tJBQZ51sVPsr5MfBk5 XXUA== X-Forwarded-Encrypted: i=1; AJvYcCWvXUSFnikLRHgy6Jeb1UsT259UckCjinepFpAcJEir/ERAznAAHbCYNsf+Aw0KgdmccN6JtHEsgQ==@kvack.org X-Gm-Message-State: AOJu0YzLyjcKTtTecG5+IsRButdFszgktCUPoW8W9qr2ND0NUToD1krf aFWDWPZdNfEEnVmI0z1QH6NIwSsy0v6NhOCrMEZ9KC/hpGhs4zUD4Fzw8U80tg== X-Gm-Gg: ASbGncsWcvBDW0cKWNKsEqnjBMjkbeXeek+kJ3l4VzPqMTtz9Du5NxD+oyhVtnqax54 UV8ZLG/lS8bDsBApY4EQCm5MflnXHulCqs3APKlYniXvHnFn8yp4MMqcXtJ4aaavjbfBP7vOaDd /vtaXSqKgZB+6R2l84ku8y58bSUsZ6DXkqtYnMHbP0kh1Qvw0phBFJJPYADBAZGZHuoB3mKxwnx ASlpfHLizCMnhAkQZCbJIEZcNN64D4fQfqY/diaUInrNUtybQnUs4+wfB2mgR32/gpC3c7b7LIN bGJKyT6GAdq4AZuKrT8= X-Google-Smtp-Source: AGHT+IGAMjAyYitJ/PuVWBKWSZI7OXnX7FkQeLCk13N2I5HTwIdj4sYm2eua0ju7hcsVjf476FULhg== X-Received: by 2002:a17:902:e74e:b0:20c:9936:f0ab with SMTP id d9443c01a7336-220bbc739cdmr154394835ad.47.1739504020152; Thu, 13 Feb 2025 19:33:40 -0800 (PST) Received: from google.com ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d558528asm19680795ad.216.2025.02.13.19.33.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 19:33:39 -0800 (PST) Date: Fri, 14 Feb 2025 12:33:33 +0900 From: Sergey Senozhatsky To: Yosry Ahmed Cc: Sergey Senozhatsky , Andrew Morton , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kairui Song Subject: Re: [PATCHv4 14/17] zsmalloc: make zspage lock preemptible Message-ID: References: <25268fbedb82a06ba0c13e2c490b164846ef508b@linux.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Queue-Id: 5558F40008 X-Stat-Signature: nmtsmt8jj7jxaw6y76i3oeeued3hctqm X-Rspamd-Server: rspam03 X-HE-Tag: 1739504021-603370 X-HE-Meta: U2FsdGVkX1+/qYkbPGkl5kzxxbO1lqwpyknISnpEv68mP7OPFSH2kn25uyPP7UrPQszQDV1xuJNwe9bSjZY7VgAioI0Sb2Z+A6J1giBTiflqZgncq3KETOumT5JAJD2R6oFzsiPaFCJy+5bdzaWFKhK96Fap3MaCegaycKgk3//FkB2q93iM8iVdeHo6tadtgqMt532iV/P0ERK1CfZwMOSgSsl0C+b3YPErmD3hmZLAzV/VpJkOBN2hkJA3wvBP4fWG2nXZ0S5ZImFurNWxQIZEvQCzcN0WinU42IwDU3rIVSwRnoaXSN0E3RwlcNoIBNYfp9lnMTIdpyroe2KukllUKTS7fF/FPQ+atpa/HAKLCVJNWu8ReADYZxrkBrSK4LDA2WcbXtSujaNsAduT9pDFFensQBoWdxqv7ouYTLNnO5kDhvqGztL2ZuhVV9jJtDkPH08XzZsE6u8N9yFKKjxuiLvkYm8oL65FZkxHH+wd8BlkAhiAImVe6UF6GaQQtJf2etBhNZJizxD3PW8td7JA+lBKEMNiYK9QM2SFjd0WCXGS0GEL7+o+A5SNc3WvjfKdj6fTbUbKjLR73FXy/29M4mALBzsWLWORIJBU6Vzg15XXCWNj4y8HlcR30q//nC4VfuUii1b9Yah0rnWJ68DlGrbOP6LRMXx7Tf+i7b0QMhVQMo1rIjisxL27IKlyggH/WRe6kfSMXYqyLO+3Cr0VGkHBlmlX04wXnRudPkFzoOfUaWUGrF9tEaiSwtLfCJU4AgPdBo7DzMyzj8gB8W9OQnpK5bohfJFMDhnTYfUZ8lbR3I8iIkzT8Y/1mhRtstG6zjqknWimQM0Qk08GeiAHLmu9jSXoGU/PNjA3KCeVKGWM0Ph2dnnc+qcpWKBJ+8a49+N20MKcAkDXfMFnOxl6L4yvlE196gPMPB5vTIHAh1+wWFhhrwFBWN49hchy3YJYT2mloqtRe3l0Tkn iBkOEnrw 7R4DCgY/vU6a6zXWVyErJkPUYUnE8aKLyX7X5k3vf9XrMcK6/Zdp2ypDJHsPyYoLrbVcTI6o09DhbZM1u1MIHAcvt+AJ0vOPlicIxEOK7WJs8f/NFsseBRFfPJS534WOtqjZ0iB+6MFBX0YFPLpTz8NTagaUd2TkWkrKg+ejwRupw82z6rCziDWQUfWfr0enT/3Eac1gteTwfO0BkjzOgxwHdXzsEOxwhFuIdpURj86V8blMLK1Zm12zjYWl0MS6t+ePIXfhM6eZSGOcBmfBz0uI/nqJVwnII/v1LDcgyz7cvH0owopp/+DxKFaMHZ5DXOgEKEWURozOUH0zFClRperyzJV52CZfpPlPkNnXWerYTG7o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.033665, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On (25/02/13 15:25), Yosry Ahmed wrote: > On Thu, Feb 13, 2025 at 05:22:20PM +0900, Sergey Senozhatsky wrote: > > On (25/02/13 16:21), Sergey Senozhatsky wrote: > > > BASE > > > ==== > > > > > > 1363.64user 157.08system 1:30.89elapsed 1673%CPU (0avgtext+0avgdata 825692maxresident)k > > > > > > lock stats > > > > > > class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg > > > &pool->migrate_lock-R: 0 0 0.00 0.00 0.00 0.00 10001 702081 0.14 104.74 125571.64 0.18 > > > &class->lock: 1 1 0.25 0.25 0.25 0.25 6320 840542 0.06 809.72 191214.87 0.23 > > > &zspage->lock-R: 0 0 0.00 0.00 0.00 0.00 6452 664129 0.12 660.24 201888.61 0.30 > > > &zram->table[index].lock: 0 0 0.00 0.00 0.00 0.00 1716362 3096466 0.07 811.10 365551.24 0.12 > > > &zstrm->lock: 0 0 0.00 0.00 0.00 0.00 0 664129 1.68 1004.80 14853571.32 22.37 > > > > > > PATCHED > > > ======= > > > > > > 1366.50user 154.89system 1:30.33elapsed 1684%CPU (0avgtext+0avgdata 825692maxresident)k > > > > > > lock stats > > > > > > class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg > > > &pool->lock#3-R: 0 0 0.00 0.00 0.00 0.00 3648 701979 0.12 44.09 107333.02 0.15 > > > &class->lock: 0 0 0.00 0.00 0.00 0.00 5038 840434 0.06 1245.90 211814.60 0.25 > > > zsmalloc-page-R: 0 0 0.00 0.00 0.00 0.00 0 664078 0.05 699.35 236641.75 0.36 > > > zram-entry->lock: 0 0 0.00 0.00 0.00 0.00 0 3098328 0.06 2987.02 313339.11 0.10 > > > &per_cpu_ptr(comp->stream, cpu)->lock: 0 0 0.00 0.00 0.00 0.00 23 664078 1.77 7071.30 14838397.61 22.34 > > > > So... > > > > I added lock-stat handling to zspage->lock and to zram (in zram it's only > > trylock that we can track, but it doesn't really bother me). I also > > renamed zsmalloc-page-R to old zspage->lock-R and zram-entry->lock to > > old zram->table[index].lock, just in case if anyone cares. > > > > Now bounces stats for zspage->lock and zram->table[index].lock look > > pretty much like in BASE case. > > > > PATCHED > > ======= > > > > class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg > > &pool->lock#3-R: 0 0 0.00 0.00 0.00 0.00 2702 703841 0.22 873.90 197110.49 0.28 > > &class->lock: 0 0 0.00 0.00 0.00 0.00 4590 842336 0.10 3329.63 256595.70 0.30 > > zspage->lock-R: 0 0 0.00 0.00 0.00 0.00 4750 665011 0.08 3360.60 258402.21 0.39 > > zram->table[index].lock: 0 0 0.00 0.00 0.00 0.00 1722291 3099346 0.12 6943.09 721282.34 0.23 > > &per_cpu_ptr(comp->stream, cpu)->lock: 0 0 0.00 0.00 0.00 0.00 23 665011 2.84 7062.18 14896206.16 22.40 > > > > holdtime-max and holdtime-total are higher in the patched kernel. Not > sure if this is just an artifact of lock holders being preemtible. Hmm, pool->lock shouldn't be affected at all, however BASE holds it much longer than PATCHED holdtime-max holdtime-total BASE 104.74 125571.64 PATCHED 44.09 107333.02 Doesn't make sense. I can understand zspage->lock and zram->table[index].lock, but for zram->table[index] things look strange (comparing run #1 and #2) holdtime-total BASE 365551.24 PATCHED 313339.11 And run #3 is in its own league. Very likely just a very very bad way to test things. Re-based on 6.14.0-rc2-next-20250213. BASE ==== PREEMPT_NONE class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg &pool->migrate_lock-R: 0 0 0.00 0.00 0.00 0.00 3624 702276 0.15 35.96 126562.90 0.18 &class->lock: 0 0 0.00 0.00 0.00 0.00 5084 840733 0.06 795.26 183238.22 0.22 &zspage->lock-R: 0 0 0.00 0.00 0.00 0.00 5358 664228 0.12 43.71 192732.71 0.29 &zram->table[index].lock: 0 0 0.00 0.00 0.00 0.00 1528645 3095862 0.07 764.76 370881.23 0.12 &zstrm->lock: 0 0 0.00 0.00 0.00 0.00 0 664228 2.52 2033.81 14605911.45 21.99 PREEMPT_VOLUNTARY class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg &pool->migrate_lock-R: 0 0 0.00 0.00 0.00 0.00 3039 699556 0.14 50.78 125553.59 0.18 &class->lock: 0 0 0.00 0.00 0.00 0.00 5259 838005 0.06 943.43 177108.05 0.21 &zspage->lock-R: 0 0 0.00 0.00 0.00 0.00 5581 664096 0.12 81.56 190235.48 0.29 &zram->table[index].lock: 0 0 0.00 0.00 0.00 0.00 1731706 3098570 0.07 796.87 366934.54 0.12 &zstrm->lock: 0 0 0.00 0.00 0.00 0.00 0 664096 3.38 5074.72 14472697.91 21.79 PREEMPT class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg &pool->migrate_lock-R: 0 0 0.00 0.00 0.00 0.00 2545 701827 0.14 773.56 125463.37 0.18 &class->lock: 0 0 0.00 0.00 0.00 0.00 4697 840281 0.06 1701.18 231657.38 0.28 &zspage->lock-R: 0 0 0.00 0.00 0.00 0.00 4778 664002 0.12 755.62 181215.17 0.27 &zram->table[index].lock: 0 0 0.00 0.00 0.00 0.00 1731737 3096937 0.07 1703.92 384633.29 0.12 &zstrm->lock: 0 0 0.00 0.00 0.00 0.00 0 664002 2.85 3603.20 14586900.58 21.97 So somehow holdtime-max for per-CPU stream is 2.5x higher for PREEMPT_VOLUNTARY than for PREEMPT_NONE. And class->lock holdtime-total is much much higher for PREEMPT than for any other preemption models. And that's BASE kernel, which runs fully atomic zsmalloc and zram. I call this rubbish.