From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8C1BC02193 for ; Wed, 29 Jan 2025 11:25:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 381C1280045; Wed, 29 Jan 2025 06:25:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 330BD280035; Wed, 29 Jan 2025 06:25:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F913280045; Wed, 29 Jan 2025 06:25:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0035D280035 for ; Wed, 29 Jan 2025 06:25:44 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E78C5B0C84 for ; Wed, 29 Jan 2025 11:25:36 +0000 (UTC) X-FDA: 83060259072.14.E745B88 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf14.hostedemail.com (Postfix) with ESMTP id EA1D7100011 for ; Wed, 29 Jan 2025 11:25:34 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=QVMv7Irm; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.180 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738149935; a=rsa-sha256; cv=none; b=lXK2t7gebARLleOOQT/03AMUD1zbM73930C9gkqap8SC70RfAav3Rz3pELIcuUWZ4VjR4q QfDJn7t0rltOt7GoXG/sjp9S1rCZiPbkBUhkpnBnOTWEg19Pyq0c+N6hpUxlB6Off+Yzrh quAptVf4zMxfUUF6ZgqL71m7GHyvmL4= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=QVMv7Irm; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.180 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738149935; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gQsB/B46V0YPini4Cs2ZfKq6VTeUefhKMJmDypXjdDU=; b=orZBY97lyvUj/bAN0LF44VHfu2Cu/omTjHTos+l/ZZqKls8ykSUrTMCU3k4FUi26aMH+th 0b0vlaxcZ7qjWar5+sF+F3kAX9Ms6QXDDATSpECXkUXXEJDVUZjYlKvWSlD4BS/iQiHZX4 YtNt2YN5wB/Y6g+MCRWnKmKRaFl2yL4= Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-21631789fcdso7211505ad.1 for ; Wed, 29 Jan 2025 03:25:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738149933; x=1738754733; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=gQsB/B46V0YPini4Cs2ZfKq6VTeUefhKMJmDypXjdDU=; b=QVMv7IrmkWqBOxeJW4Na3a8RNsIlgaJiWjzIU7icXe1lPQ2O3GsAzcQXhbi6pgGAVP S1DEzUCyECqBgsQYwcdnv53PT+vgs8HWFZotktu3qA9wHCyX8T942Jf/bkkFesUJZQtm I5GHpjXnqVG5rNdGm/7OB7pZtmtoUTYEvPcTQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738149933; x=1738754733; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=gQsB/B46V0YPini4Cs2ZfKq6VTeUefhKMJmDypXjdDU=; b=TZnczDf0AaRPnvbI5zAfsq57qMrZv4Kx7eYkjVaSjOH9yP1FTL36/NckTO5ubTHFY1 JdEhUw6oQ+uR+kmzB9PClWsAfOMKctn3+fsR3GY0njeVjuZ2q1OTJ5aLVCYXvcW9fILt RMtNGxNb6d55UwS22cI6GHqPANLM+rUcI2CiJQ39JtubZJUtcggVPyxayCa9+8Ph/CdW AeZQ+9EssRJoeHyJjji1pQB3RLBeMpNuPK3K6vxwWGGZcGoMiBfr2UVwh3XEObOAcbqP nPKNl8EJ9IwQRZ0L9LvE9s1x/BAHajM/t2fMFAEd/2sdTa73eUPhHXybxmoaep99p7ID v+pQ== X-Forwarded-Encrypted: i=1; AJvYcCVmvT6KH9oNB3sjPSKJMYfZ4v+fCxBFhoOvAhoysh1rAp6Z8mqs0N7e5hqXBv2xtJ+DaZ/KfOPv9Q==@kvack.org X-Gm-Message-State: AOJu0YypT0rtEcmT1EVaTcu5c90WChhDv+IKH0pelranGR2I5KLMGXtC o8LtZUxXdjs8ISCGeCrZxB8+odUPO+WZadeBakB3Q6N78An4JYQEF7PHzl7erw== X-Gm-Gg: ASbGnct++vucByfzA1NzubbvJsbAS13mAeoH5op7SCP5WSo5auGvAHzKdNutMLE/As2 OdvMckuoStSW5riaz3c/OB+ZK+sUy2YmplVmWHog87Q3RxANJl+8Dmyd11funy+mqR3KQ1PBHxY 9YysOaGU0pr4k+RXtZpsfyCEZXifI4nITBFV3T0AMnbAbaSTQXmhmAUSbrF6yU/tm7ltG6QH+WB lDcS/A7ltMg9Dj6CHGBb+kS+E+YBXRhPj9SJbXTLi5/YmXgcJhBTkcUv50YPFkiLz9Ia+jvxeyS PRoC4n33foFytcHPvVY= X-Google-Smtp-Source: AGHT+IGr5n8tF+9smAMzYNMkvdUYJnIKJO7gAbXrl+NnTBL0QJDPU79gqRuZLkrNaOjW9NP0eDy8hw== X-Received: by 2002:a05:6a00:4acc:b0:72f:c0af:7328 with SMTP id d2e1a72fcca58-72fd0856cd4mr4822402b3a.3.1738149933577; Wed, 29 Jan 2025 03:25:33 -0800 (PST) Received: from google.com ([2401:fa00:8f:203:d794:9e7a:5186:857e]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72f8a78f317sm11205252b3a.176.2025.01.29.03.25.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jan 2025 03:25:33 -0800 (PST) Date: Wed, 29 Jan 2025 20:25:27 +0900 From: Sergey Senozhatsky To: Sergey Senozhatsky Cc: Andrew Morton , Minchan Kim , Johannes Weiner , Yosry Ahmed , Nhat Pham , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Uros Bizjak Subject: Re: [PATCHv1 3/6] zsmalloc: make zspage lock preemptible Message-ID: References: <20250129064853.2210753-1-senozhatsky@chromium.org> <20250129064853.2210753-4-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250129064853.2210753-4-senozhatsky@chromium.org> X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: EA1D7100011 X-Stat-Signature: g3mck3bxzwm3in48rmayrafo59i1wktz X-Rspam-User: X-HE-Tag: 1738149934-917034 X-HE-Meta: U2FsdGVkX1+hNzqa2PSom+w9yc6LEXAD+ODKey/Us/v+vKsL0YfeQkFt9ZzuQvUCOyOYt4wRU0Lx49MVntt3wIyofoPGLuPl3zZJe2xoyGwefGmzpFYhGk8q8+46EdGOQpBpqNSXjsBpeI65qUuV9pJGhPzjjkQ8GZQXdZJGq8LQPvO/3/sHwgSv9257AL0MRIX1ibYLsvkjDtpEMIvQJRfrABjBzMLI9VUmEwgXm7WVSECmQgzkBwpJtQZmWV2k8QMv/XeKMvkR0DhGx3sneAJ2n/I0TIfc+AfjqTPahN1H9o868bbk0Ox8pR5GArCModbF0r4BJbAfS8iaBHyAS9eszznqdvatGW0Y7DZdFxDWn1YfVEdTmibbfrjq2H+udYVVyiwRkV+ehAAgRio56ZQhcXQ5OwsP5Fww/pBB7MCkZxuJK1/1cvMbkY6NL/XAvrUvO40bEqOLtZhqP5lfW/Xq1b2kYUhW8Hvgy7vhRLFOj4dFrEp0yoOgFMXuBJL6U3N3d6Sj3VA4tNUqphd5i/KGvyG858KMbJKj2LT0pTsXQ3uREzrMFX6yr/z+7F6X4thK66J9sp6jspIoeqGKu/mO3ULZw5Viy8HoHbJBKV+3MdgjGTmh24BgVxxVRZXohR8RKnUvm/kqlhE3jcJChBMRzbASq3gv6Ifav/lPEUJTEoYzK/koU4S0+RW7Iw4OeMxfzW2roVS8FOTGm8fcYBlMSwS977Tp2Pka7CMTvD0z9ZHXgw9If1MlA3t36ktJ9GnaVPXrFPtPh/bKJmzPm2p4Czt/zzglhKZiIJIrS2LOUYXOvnyh8LI/Its2HrOi5aj6nCqpZ51tuJkc4JjfHUQNcwWqsnGlSkRcPXuRa4YWESUd8C3CSVstH10keN75D5b2zd8Lk1t7ZXIXuv7LUoGMt0goQfZ2Dy03dIaHO+NbUmrYlHi7GK1vAEr7d9fBPD1OgYtTFxHBPHLPhoR QHc60C8M 0AewlepqYqW8A5t7jRGxJqPuDrycCcNi7gv9fEtAKc+Ccnll5wDnvS3EtrQ6dd1J65h0p76w5B7sPosBSGnk3yjYEnG0Bv/9d7mVylhojrU2c0b4qm6MCcfa53GmfGZH1hthhPtjVcMGeUt/+TJtHNic4wLmCtQskZL69t1zqvZxaQj/TIO0yYKZ5jiylJNy1U+2zlOwj5d3t9yzLSWZAEM5ctg94VJyLN5FLYTIB1QtK+h3+a4kIet7C/QWbad7j7VO17YZ2tBsn1n/MzoHcQe06CyBeXeZ+Jgouyv8dXfXekNL451j3LpGnRBhKCLgoUcEUAX8bSdduoYwGNjU5CJd9gPTbutS5KJbPRZ5d+k5lwxI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000297, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: JFI On (25/01/29 15:43), Sergey Senozhatsky wrote: > +static void zspage_lock_init(struct zspage *zspage) > +{ > + atomic_set(&zspage->lock, ZS_PAGE_UNLOCKED); > +} > + > +/* > + * zspage lock permits preemption on the reader-side (there can be multiple > + * readers). Writers (exclusive zspage ownership), on the other hand, are > + * always run in atomic context and cannot spin waiting for a (potentially > + * preempted) reader to unlock zspage. This, basically, means that writers > + * can only call write-try-lock and must bail out if it didn't succeed. > + * > + * At the same time, writers cannot reschedule under zspage write-lock, > + * so readers can spin waiting for the writer to unlock zspage. > + */ > +static void zspage_read_lock(struct zspage *zspage) > +{ > + atomic_t *lock = &zspage->lock; > + int old; > + > + while (1) { > + old = atomic_read(lock); > + if (old == ZS_PAGE_WRLOCKED) { > + cpu_relax(); > + continue; > + } > + > + if (atomic_try_cmpxchg(lock, &old, old + 1)) > + return; > + > + cpu_relax(); > + } > +} > + > +static void zspage_read_unlock(struct zspage *zspage) > +{ > + atomic_dec(&zspage->lock); > +} > + > +static int zspage_try_write_lock(struct zspage *zspage) > +{ > + atomic_t *lock = &zspage->lock; > + int old = ZS_PAGE_UNLOCKED; > + > + preempt_disable(); > + if (atomic_try_cmpxchg(lock, &old, ZS_PAGE_WRLOCKED)) > + return 1; > + > + preempt_enable(); > + return 0; > +} > + > +static void zspage_write_unlock(struct zspage *zspage) > +{ > + atomic_set(&zspage->lock, ZS_PAGE_UNLOCKED); > + preempt_enable(); > +} Below is what I currently have based on a (private) feedback from Uros. No functional changes. I think I'll wait before sending out v2, based on the fact that this is not a fix or a functional change (WR-locked case is relatively uncommon; it's only when we map an object from a page that is currently either under migration or compaction.) --- static void zspage_lock_init(struct zspage *zspage) { atomic_set(&zspage->lock, ZS_PAGE_UNLOCKED); } static void zspage_read_lock(struct zspage *zspage) { atomic_t *lock = &zspage->lock; int old = atomic_read(lock); do { if (old == ZS_PAGE_WRLOCKED) { cpu_relax(); old = atomic_read(lock); continue; } } while (!atomic_try_cmpxchg(lock, &old, old + 1)); } static void zspage_read_unlock(struct zspage *zspage) { atomic_dec(&zspage->lock); } static bool zspage_try_write_lock(struct zspage *zspage) { atomic_t *lock = &zspage->lock; int old = ZS_PAGE_UNLOCKED; preempt_disable(); if (atomic_try_cmpxchg(lock, &old, ZS_PAGE_WRLOCKED)) return true; preempt_enable(); return false; } static void zspage_write_unlock(struct zspage *zspage) { atomic_set(&zspage->lock, ZS_PAGE_UNLOCKED); preempt_enable(); }