From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DE14C43217 for ; Wed, 10 Nov 2021 18:54:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 47313611C9 for ; Wed, 10 Nov 2021 18:54:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 47313611C9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 82C3C6B007D; Wed, 10 Nov 2021 13:54:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6E9796B007E; Wed, 10 Nov 2021 13:54:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53D226B0080; Wed, 10 Nov 2021 13:54:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0245.hostedemail.com [216.40.44.245]) by kanga.kvack.org (Postfix) with ESMTP id 422A16B007D for ; Wed, 10 Nov 2021 13:54:45 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3C9868086F for ; Wed, 10 Nov 2021 18:54:44 +0000 (UTC) X-FDA: 78793922088.23.3C50A1A Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by imf04.hostedemail.com (Postfix) with ESMTP id 730CD50000BE for ; Wed, 10 Nov 2021 18:54:32 +0000 (UTC) Received: by mail-pj1-f48.google.com with SMTP id gf14-20020a17090ac7ce00b001a7a2a0b5c3so2442029pjb.5 for ; Wed, 10 Nov 2021 10:54:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=k939A4eJtAg0Dn5sr7BzkOxLeQ1Yum+0/+a8bmwGq+Q=; b=bBPdoD7696pyZPiApTBXQXkfpbddhSRRYtyndOLvyGqq3nTxSoCdPLIFJEz21j4hl/ RfLXWJd/C+1Oi10aa0PPZi1mVTv0oCGsD5pZ4NYtJjQ76dFy06vct1GOLu4VHoI6GGe4 nmxST4arbt1ooSy+vjfAwUVASmybuhw8PH8u+Ucwvaf+h2edni3gLSBjiwZ9ckDGeh8k IeoauhSF/mvDCHcvvIBVQt7D099OGFTV4lkVc7slTBTfFkSornS5xtNXOJu948hmSdcf RUPRgq6ALD54PX9BpLSROBIeCZa3jrq6OFhCiqzZ+p+IITVDGgnkuh9uwaGwoe/6TApe D+Yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=k939A4eJtAg0Dn5sr7BzkOxLeQ1Yum+0/+a8bmwGq+Q=; b=E7QSirSPS4ifWW9gZQmPJ3kLZp3lz0BOjNaE52biJRCF657d3eV2l4RaenohVcULv6 EW2p3KGC1cL2NXzYCud9PIWiK962wH8FjwyeKQZnEvWIk2w1s7+lyvTNs0yDBvKLF4iI 9VqkytrPl02GbmC35Bpe/vovfWaRbccxI6U2zin5r+jG75H3dqpWVojjJRxMTVZJVrNm xGEsizShpKPeQHJuI2MkOKkjbcmvM0JXRMb5XoenteTLs5mXNDfo5kIcbY4FQG2QNUMr zVXL+kcEa0qPsopLUbgdysSub4yAzxMQBggqW5PEze6gbWSDvN+ErbJvd3MxNv4ZPnPE C7wA== X-Gm-Message-State: AOAM532SapRYf7pWSfb0+55cU4FPtQ3l4bLoVHyQ/JWW5PdnG8xKC/Y/ CQcaJ5VTvrAZ1S37HbQ3DgU= X-Google-Smtp-Source: ABdhPJyVTgRJdGcBZ+NgPKyYkLmk7ITBnOfjABN5xPLVrFdbIGNRodB4ZpOsRGyEvF826X9bAdJjmg== X-Received: by 2002:a17:90b:224f:: with SMTP id hk15mr19449025pjb.173.1636570481825; Wed, 10 Nov 2021 10:54:41 -0800 (PST) Received: from bbox-1.mtv.corp.google.com ([2620:15c:211:201:11d4:2de3:ab82:be64]) by smtp.gmail.com with ESMTPSA id g13sm325253pjc.39.2021.11.10.10.54.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Nov 2021 10:54:41 -0800 (PST) From: Minchan Kim To: Andrew Morton Cc: Sergey Senozhatsky , linux-mm , Minchan Kim Subject: [PATCH 4/8] zsmalloc: introduce obj_allocated Date: Wed, 10 Nov 2021 10:54:29 -0800 Message-Id: <20211110185433.1981097-5-minchan@kernel.org> X-Mailer: git-send-email 2.34.0.rc1.387.gb447b232ab-goog In-Reply-To: <20211110185433.1981097-1-minchan@kernel.org> References: <20211110185433.1981097-1-minchan@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 730CD50000BE X-Stat-Signature: 5jdpj537h47tfr49scdadbyks9mjgn8m Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=bBPdoD76; spf=pass (imf04.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.216.48 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) X-HE-Tag: 1636570472-491003 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The usage pattern for obj_to_head is to check whether the zpage is allocated or not. Thus, introduce obj_allocated. Signed-off-by: Minchan Kim --- mm/zsmalloc.c | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 52c6431ed5c6..8f9cd07033de 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -877,13 +877,21 @@ static unsigned long handle_to_obj(unsigned long ha= ndle) return *(unsigned long *)handle; } =20 -static unsigned long obj_to_head(struct page *page, void *obj) +static bool obj_allocated(struct page *page, void *obj, unsigned long *p= handle) { + unsigned long handle; + if (unlikely(PageHugeObject(page))) { VM_BUG_ON_PAGE(!is_first_page(page), page); - return page->index; + handle =3D page->index; } else - return *(unsigned long *)obj; + handle =3D *(unsigned long *)obj; + + if (!(handle & OBJ_ALLOCATED_TAG)) + return false; + + *phandle =3D handle & ~OBJ_ALLOCATED_TAG; + return true; } =20 static inline int testpin_tag(unsigned long handle) @@ -1606,7 +1614,6 @@ static void zs_object_copy(struct size_class *class= , unsigned long dst, static unsigned long find_alloced_obj(struct size_class *class, struct page *page, int *obj_idx) { - unsigned long head; int offset =3D 0; int index =3D *obj_idx; unsigned long handle =3D 0; @@ -1616,9 +1623,7 @@ static unsigned long find_alloced_obj(struct size_c= lass *class, offset +=3D class->size * index; =20 while (offset < PAGE_SIZE) { - head =3D obj_to_head(page, addr + offset); - if (head & OBJ_ALLOCATED_TAG) { - handle =3D head & ~OBJ_ALLOCATED_TAG; + if (obj_allocated(page, addr + offset, &handle)) { if (trypin_tag(handle)) break; handle =3D 0; @@ -1927,7 +1932,7 @@ static int zs_page_migrate(struct address_space *ma= pping, struct page *newpage, struct page *dummy; void *s_addr, *d_addr, *addr; int offset, pos; - unsigned long handle, head; + unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; int ret =3D -EAGAIN; @@ -1963,9 +1968,7 @@ static int zs_page_migrate(struct address_space *ma= pping, struct page *newpage, pos =3D offset; s_addr =3D kmap_atomic(page); while (pos < PAGE_SIZE) { - head =3D obj_to_head(page, s_addr + pos); - if (head & OBJ_ALLOCATED_TAG) { - handle =3D head & ~OBJ_ALLOCATED_TAG; + if (obj_allocated(page, s_addr + pos, &handle)) { if (!trypin_tag(handle)) goto unpin_objects; } @@ -1981,9 +1984,7 @@ static int zs_page_migrate(struct address_space *ma= pping, struct page *newpage, =20 for (addr =3D s_addr + offset; addr < s_addr + pos; addr +=3D class->size) { - head =3D obj_to_head(page, addr); - if (head & OBJ_ALLOCATED_TAG) { - handle =3D head & ~OBJ_ALLOCATED_TAG; + if (obj_allocated(page, addr, &handle)) { BUG_ON(!testpin_tag(handle)); =20 old_obj =3D handle_to_obj(handle); @@ -2028,9 +2029,7 @@ static int zs_page_migrate(struct address_space *ma= pping, struct page *newpage, unpin_objects: for (addr =3D s_addr + offset; addr < s_addr + pos; addr +=3D class->size) { - head =3D obj_to_head(page, addr); - if (head & OBJ_ALLOCATED_TAG) { - handle =3D head & ~OBJ_ALLOCATED_TAG; + if (obj_allocated(page, addr, &handle)) { BUG_ON(!testpin_tag(handle)); unpin_tag(handle); } --=20 2.34.0.rc1.387.gb447b232ab-goog