From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 362A0C4332F for ; Tue, 29 Nov 2022 04:01:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C539F6B0071; Mon, 28 Nov 2022 23:01:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C03426B0073; Mon, 28 Nov 2022 23:01:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACB5F6B0074; Mon, 28 Nov 2022 23:01:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9B1136B0071 for ; Mon, 28 Nov 2022 23:01:23 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4000712042E for ; Tue, 29 Nov 2022 04:01:23 +0000 (UTC) X-FDA: 80185130046.24.E09EEF1 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf23.hostedemail.com (Postfix) with ESMTP id BD85E140012 for ; Tue, 29 Nov 2022 04:01:22 +0000 (UTC) Received: by mail-pj1-f52.google.com with SMTP id t11-20020a17090a024b00b0021932afece4so5177761pje.5 for ; Mon, 28 Nov 2022 20:01:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=xoNynUsXcZWRRQTA42Lr8aZhhKDdqzAsGG5E/HJMyjk=; b=L+ggEBnUdDJGpqbNThJ/vIeO3gs8VDddvHdRi3e2qz7IQ4S0zDDeDIZbKvwIpy/08f /+uKWt9P5uP/Gu6ScjZm80hBBD6oSSJQCHVU+NH/WCPOvocEhb575zy1y6cQL9fEFHbe DCx3iUv2XturIqr7QQ50Cw9fwsn6xso0Arp3w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=xoNynUsXcZWRRQTA42Lr8aZhhKDdqzAsGG5E/HJMyjk=; b=QSrCo8/toWGdTLVQw7Um7SxvlfN4pswhucdp1QteTfR4dbPv92BEUJvan4RUUSOtUf peclWYcRPJeUz+xi7jbBOwF9OTemoZm3+XuJjuQrBiUTTukbF3+DYBQcnmp6wZfUK1sX 80a0bzXtwmGETJFNM0ukmgEGnM5NHwfVla7gtctEsmXsWQoSyGnokfapv2HYv2lsTa1E 65TcBBtjlUaQdt1NGANPBg6173VxJKtVS8Te/O0CfDcN9btzwEMVsT+V4xssFDveHPV/ 77wQTPF9hKUBya2bksb6nTBFDAMr1tpQZYsVgVmBlVqr4i5yrmNsKHbA0uZIhKO1mTrg Zmcw== X-Gm-Message-State: ANoB5pkTASDbDb5f8HA4DV6j6IxCIGIXTQtfjWLZZz5VFuGKMceNU7u4 fF2ubwcDj35HPj4C5lyf9y7Rdw== X-Google-Smtp-Source: AA0mqf5cllq8xbDaQrTyt/2TeL7ufjfwuaEUbskEY98DprdDjtJX3d1TQZFhzVs8UlGzzuIZ6lJImA== X-Received: by 2002:a17:90b:2711:b0:213:9b80:ceee with SMTP id px17-20020a17090b271100b002139b80ceeemr63951801pjb.243.1669694481669; Mon, 28 Nov 2022 20:01:21 -0800 (PST) Received: from google.com ([240f:75:7537:3187:2565:b2f5:cacd:a5d9]) by smtp.gmail.com with ESMTPSA id b196-20020a621bcd000000b0056ddd2ac8f1sm8806807pfb.211.2022.11.28.20.01.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Nov 2022 20:01:20 -0800 (PST) Date: Tue, 29 Nov 2022 13:01:15 +0900 From: Sergey Senozhatsky To: Nhat Pham Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: Re: [PATCH v7 3/6] zsmalloc: Consolidate zs_pool's migrate_lock and size_class's locks Message-ID: References: <20221128191616.1261026-1-nphamcs@gmail.com> <20221128191616.1261026-4-nphamcs@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221128191616.1261026-4-nphamcs@gmail.com> ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669694482; a=rsa-sha256; cv=none; b=6ckCjeIu4kNQ98XYMCFAOhRP01QmKkI/f+jJEm0u9/s4bIeAFGxaq7iFyCF/3s8J50/yGn wItb76J3dsXT7P5HsI7wfrRIkg8PDxMYFvCb7UawiBHHEuOXof4Ow0JH5Jywk+qVtwPMpm H3P99YA+W6GfOEn8W9f3p6zWcUynpGA= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=L+ggEBnU; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669694482; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xoNynUsXcZWRRQTA42Lr8aZhhKDdqzAsGG5E/HJMyjk=; b=NYXboY7YTP3cOUsN+58z5lzt1QGtWuQzPSs8PrDb9+S9vsaE9/qn0SCwZ9RrivgOSLJa/T 9eeajv0JysicwZcLcyu27bwcva7ZY4D2kQXqZkbN5Piq8etK0lIpbwCPjJIAsptIy3HS6m v65O2Ij7IRCD0AuyV/OXQ2ZhXfwQ1dM= X-Rspamd-Queue-Id: BD85E140012 X-Rspam-User: Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=L+ggEBnU; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org X-Rspamd-Server: rspam09 X-Stat-Signature: hrsaess9sbdnfb9hnqjfu97ghjkwre6x X-HE-Tag: 1669694482-166474 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On (22/11/28 11:16), Nhat Pham wrote: > Currently, zsmalloc has a hierarchy of locks, which includes a > pool-level migrate_lock, and a lock for each size class. We have to > obtain both locks in the hotpath in most cases anyway, except for > zs_malloc. This exception will no longer exist when we introduce a LRU > into the zs_pool for the new writeback functionality - we will need to > obtain a pool-level lock to synchronize LRU handling even in zs_malloc. > > In preparation for zsmalloc writeback, consolidate these locks into a > single pool-level lock, which drastically reduces the complexity of > synchronization in zsmalloc. > > We have also benchmarked the lock consolidation to see the performance > effect of this change on zram. > > First, we ran a synthetic FS workload on a server machine with 36 cores > (same machine for all runs), using > > fs_mark -d ../zram1mnt -s 100000 -n 2500 -t 32 -k > > before and after for btrfs and ext4 on zram (FS usage is 80%). > > Here is the result (unit is file/second): > > With lock consolidation (btrfs): > Average: 13520.2, Median: 13531.0, Stddev: 137.5961482019028 > > Without lock consolidation (btrfs): > Average: 13487.2, Median: 13575.0, Stddev: 309.08283679298665 > > With lock consolidation (ext4): > Average: 16824.4, Median: 16839.0, Stddev: 89.97388510006668 > > Without lock consolidation (ext4) > Average: 16958.0, Median: 16986.0, Stddev: 194.7370021336469 > > As you can see, we observe a 0.3% regression for btrfs, and a 0.9% > regression for ext4. This is a small, barely measurable difference in my > opinion. > > For a more realistic scenario, we also tries building the kernel on zram. > Here is the time it takes (in seconds): > > With lock consolidation (btrfs): > real > Average: 319.6, Median: 320.0, Stddev: 0.8944271909999159 > user > Average: 6894.2, Median: 6895.0, Stddev: 25.528415540334656 > sys > Average: 521.4, Median: 522.0, Stddev: 1.51657508881031 > > Without lock consolidation (btrfs): > real > Average: 319.8, Median: 320.0, Stddev: 0.8366600265340756 > user > Average: 6896.6, Median: 6899.0, Stddev: 16.04057355583023 > sys > Average: 520.6, Median: 521.0, Stddev: 1.140175425099138 > > With lock consolidation (ext4): > real > Average: 320.0, Median: 319.0, Stddev: 1.4142135623730951 > user > Average: 6896.8, Median: 6878.0, Stddev: 28.621670111997307 > sys > Average: 521.2, Median: 521.0, Stddev: 1.7888543819998317 > > Without lock consolidation (ext4) > real > Average: 319.6, Median: 319.0, Stddev: 0.8944271909999159 > user > Average: 6886.2, Median: 6887.0, Stddev: 16.93221781102523 > sys > Average: 520.4, Median: 520.0, Stddev: 1.140175425099138 > > The difference is entirely within the noise of a typical run on zram. This > hardly justifies the complexity of maintaining both the pool lock and > the class lock. In fact, for writeback, we would need to introduce yet > another lock to prevent data races on the pool's LRU, further > complicating the lock handling logic. IMHO, it is just better to > collapse all of these into a single pool-level lock. > > Suggested-by: Johannes Weiner > Signed-off-by: Nhat Pham > Acked-by: Minchan Kim > Acked-by: Johannes Weiner Reviewed-by: Sergey Senozhatsky