From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D8E6C4828D for ; Mon, 5 Feb 2024 06:34:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A88596B0071; Mon, 5 Feb 2024 01:34:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A37E26B0072; Mon, 5 Feb 2024 01:34:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FFDB6B0074; Mon, 5 Feb 2024 01:34:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 7F4F26B0071 for ; Mon, 5 Feb 2024 01:34:15 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 10A2380866 for ; Mon, 5 Feb 2024 06:34:15 +0000 (UTC) X-FDA: 81756785670.07.5F6B1A2 Received: from mail-lj1-f176.google.com (mail-lj1-f176.google.com [209.85.208.176]) by imf08.hostedemail.com (Postfix) with ESMTP id 60154160006 for ; Mon, 5 Feb 2024 06:34:13 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bYOO2iXW; spf=pass (imf08.hostedemail.com: domain of kundanthebest@gmail.com designates 209.85.208.176 as permitted sender) smtp.mailfrom=kundanthebest@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707114853; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=3HPdncqT21gSehRpotmrzWHdvfLvArDs3w3+jgD+uU4=; b=XsWMjMyNAxn/+LvyuMfIJeed4AKuUFKoPWqBdYRQuXqdIOm4WZQdSHbG8uzuA/Af14sHGp UWCJoc0G09VeMLZ8Ko1mMrhGbA54Kfq4qSlTuv3hVzdvuQoIcp4XX9gwSCDh7yzwJTDuKL vHXU/iQUFcM9i47z3wQpW0Ivwzfks/g= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bYOO2iXW; spf=pass (imf08.hostedemail.com: domain of kundanthebest@gmail.com designates 209.85.208.176 as permitted sender) smtp.mailfrom=kundanthebest@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707114853; a=rsa-sha256; cv=none; b=v2k81SM/m55kqbWnkEMNkb3CaJHKtIN9JExNkNixhDgg3UJEcdBT+ItHKhznEmLFZ3gMMB YBubKa1xDT1XG2yDMAObub9w/FqXWep8BgDquKIm6hhIuBD8RmdfY0T3QLAzlr+4l+hSa4 0hNntQHWGDqODPDuOk6sWxi6fAoZYXE= Received: by mail-lj1-f176.google.com with SMTP id 38308e7fff4ca-2d090c83d45so24214621fa.3 for ; Sun, 04 Feb 2024 22:34:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707114851; x=1707719651; darn=kvack.org; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=3HPdncqT21gSehRpotmrzWHdvfLvArDs3w3+jgD+uU4=; b=bYOO2iXWrAnvbYCEVeIsdna0NpIFBAEmMX4rSH4qsiXX2mIiGRWaMTcKCMNL22Ih5O SZWWoH4CBWU1LT0IRDoNv+TPOFSE1SwF5bQGvWxj0QeqOtIHj89T1kfdSQOSJ1ghludw +d6WLrm8J7ypsQGQoAfFcHFMClhDmIhwnh77buxJMf5NhBK3zm7SPDk594DaicG4+n6U kr3cGN9tj6ElYINJmd5TP9DMNjq0uP/44YRtc1mtbFF0y1InZfVgBqNftQf7b+rhWRKn /ePCtT7k4W7NW73luDzNZvm7FmeNWC++6OWDJiIfApYKL/CFQPlOeIsuzDjwa+cGioVS MM2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707114851; x=1707719651; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=3HPdncqT21gSehRpotmrzWHdvfLvArDs3w3+jgD+uU4=; b=KMOVef4NW7Xc28PCm/3Z8Vx3XczAMQAGf1JGadbQ4v0PuDelalaZDH8ZIT38U03rFC Q6C8ZYwtFFqrc8EuCUPn1yMr9cWSjsLeJmdHCAMe1wordPYuBAgZ51TZCHMA2plxxTdU 7HUxHfiT6qBR6KW+Jvgf1Tdbvub55XJN4Kxyg0GCCSRyZMrvr2UJD88fveLF/Ats/RwN lkPvpFEqSGeYmNGp8Fc4soCOJQNPHT9PMMWQHOk4N8RAzJY9chWRtfHvD59pQkSYdWyw C7YfvZPFirvutcHIu0hvJrFT1Fh5lZVFtLHR52Ygwr0naXuIPhBuaPy423q3IPirv21B u3Mw== X-Gm-Message-State: AOJu0Yyk6Cj3Mf4lgmwRWSdwzXOM6H6iU+mKEedDxH+ctLQTct1rgbPZ GFwbo5uTgiYpq77pKZUdZnRFBTY58EibeC7am4NCzttDk1BPb+o7r2rVtlCH3g23EPejSkSYbaQ EZuZGw3qCA535/Y6IVwJX7yH50dEUWb/ObbD90w== X-Google-Smtp-Source: AGHT+IFs8eHPBZ0xsC/7LN33ufUBUzneqGCfjWQf9i0Bq0Ni+TLvEhpAwP6pk3ZFUHGfXw++KliivaGgXMi8E5f+lcc= X-Received: by 2002:a2e:bea0:0:b0:2d0:ac2e:c859 with SMTP id a32-20020a2ebea0000000b002d0ac2ec859mr1150216ljr.25.1707114851120; Sun, 04 Feb 2024 22:34:11 -0800 (PST) MIME-Version: 1.0 From: Kundan Kumar Date: Mon, 5 Feb 2024 12:03:58 +0530 Message-ID: Subject: Pages doesn't belong to same large order folio in block IO path To: "linux-mm@kvack.org" Cc: "ryan.roberts@arm.com" Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 60154160006 X-Rspam-User: X-Stat-Signature: wsftntxuygmwad4shcczf894831dmbf3 X-Rspamd-Server: rspam01 X-HE-Tag: 1707114853-132675 X-HE-Meta: U2FsdGVkX19vTHIYzjns3uH6G2nH0hSKm9BZI7xJgsLFtAhi900bCq7vI4Ly8xprudTRu6Fa5lNzicpj0W6QTDgpbgL0wJVepM49Li1aZqT/J33AVvUUBGbfcOAi57chjoakjUtG9hOODfbf46/k7FxNvjoeeSad8CJju/R1HjffCNN8T5dJkaEsy4Cjl3g0fq+TqweAaV5DkLK7lRFbBwaHmPAgxJj0yDbxl39qRMJDnBWHd/0Zfj5Yjq7pUdRyBwngFoepGcEVY7lG6mdorS2fTzUzi4fF0dGhZECYKx88Mudm/IemAtj9FKSfEg6PMsRX6V4QquHs2ALb+TliJ87yrYPqusTi1bFmTNZFeCKfY23+u7SvSqxMtCSnpfMcAookRWl5QjKuZ/wqQX9r9JLKlBwSYtqGmI4VJm50qVn19k8XJfI4qiG0SpB7r30EV5GSPBjnEu7kTArycH91tKIFXXUpw465zqC/DjQNH3anqlcdQRi9e+Rtt1ZDpeutgJM8iNgauc7upyZ3/MI+pW5n55tScZc/Je91O73nSqgxjpq5BqGSsddr6W0z8vAI3gJba7uRRjfQbSgPHFBMQtWkK9p/9ym6O50k0jkiaDepy2ZYJhcFP51F+06n7HCBiuRrLSjFgD/wBJLQYhvHjkdjBYGt9/aX1PIl32+GUCG+p2n4pDzqiBIxWt493U9LMq9gN+0+tMdaxkKMnkIeT9PD1DIAIYHDediso+EUmIXvhNyBGEZzVw/OpidZnXrx/XYwrV+JChsWI2Eirp6pfCylPhOSqoY2/jatGiPO/18SU7pWQ2I4XDCtq162pQuQchJFog08FVGmwsfeDN13GnAy5EKpCwJ4UTcoe1fFu1rOe6EpaVBzuVAHxDmnOWTLZXjHMJl5QHKw5PU3C3fmspVZqQm3jTxpAo6qBJL/AGF0XpsKOLkKr1S+7l/kvIYa+5zvUjsaNaMS7hsIzxQ 5UxajKCG /vCYufM3ZJx5I8fIs8cEn9GCvxeqo2eHfbVidpaTM3iYtuAluVnp2Mnas00sNrXyOLXnG7DosAK/enGgKDVIO7UeDVFkqq9TRkTYIt+QQes8JDOYB81r6MlRe/HQYCN9I9pLcIxtKeghZT+YTTUwKWnVrhfr9aI4XwTE5w3YGPxNR/0AlRcJILcBeAaIomon4PulzPtrikUPorPPCmzcTZyFSDddFin1dtAO11UgB5nYCLU4Xb9180M/dR1frrKbvzlcauGqLaHD3dbaLNekf+JNJpPUuzOqUcxkM2W7hPubwRYYYGBuiwlebV5zCwuvNlzFFjfzZv9YkY0E3fc19Fyhh79Vyvh9oXiDKlkHlObpqCqKpajh7FaeR8QKVCcf1cUL9i4I2jYqYQzFx88oo99mLd4b5PvQUYGkQRJItnOz2D8P6pH0+eRpozyRdDwH8DIyFwYfju1LN9PoTbHGiLA4oEUIybX51JrcB X-Bogosity: Ham, tests=bogofilter, spamicity=0.134749, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi All, I am using the patch "Multi-size THP for anonymous memory" https://lore.kernel.org/all/20231214160251.3574571-1-ryan.roberts@arm.com/T/#u I enabled the mTHP using the sysfs interface : echo always >/sys/kernel/mm/transparent_hugepage/hugepages-16kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-32kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-64kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-128kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-256kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-512kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-1024kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-2048kB/enabled I can see this patch allocates multi-order folio for anonymous memory. With the large order folios getting allocated I tried direct block IO using fio. fio -iodepth=1 -rw=write -ioengine=io_uring -direct=1 -bs=16K -numjobs=1 -size=16k -group_reporting -filename=/dev/nvme0n1 -name=io_uring_test The fio malloced memory is allocated from a multi-order folio in function alloc_anon_folio(). Block I/O path takes the fio allocated memory and maps it in kernel in function iov_iter_extract_user_pages() As the pages are mapped using large folios, I try to see if the pages belong to same folio using page_folio(page) in function __bio_iov_iter_get_pages(). To my surprise I see that the pages belong to different folios. Feb 5 10:34:33 kernel: [244413.315660] 1603 iov_iter_extract_user_pages addr = 5593b252a000 Feb 5 10:34:33 kernel: [244413.315680] 1610 iov_iter_extract_user_pages nr_pages = 4 Feb 5 10:34:33 kernel: [244413.315700] 1291 __bio_iov_iter_get_pages page = ffffea000d4bb9c0 folio = ffffea000d4bb9c0 Feb 5 10:34:33 kernel: [244413.315749] 1291 __bio_iov_iter_get_pages page = ffffea000d796200 folio = ffffea000d796200 Feb 5 10:34:33 kernel: [244413.315796] 1291 __bio_iov_iter_get_pages page = ffffea000d796240 folio = ffffea000d796240 Feb 5 10:34:33 kernel: [244413.315852] 1291 __bio_iov_iter_get_pages page = ffffea000d7b2b80 folio = ffffea000d7b2b80 I repeat the same experiment with fio using HUGE pages fio -iodepth=1 -iomem=mmaphuge -rw=write -ioengine=io_uring -direct=1 -bs=16K -numjobs=1 -size=16k -group_reporting -filename=/dev/nvme0n1 -name=io_uring_test This time when the memory is mmapped from HUGE pages I see that pages belong to the same folio. Feb 5 10:51:50 kernel: [245450.439817] 1603 iov_iter_extract_user_pages addr = 7f66e4c00000 Feb 5 10:51:50 kernel: [245450.439825] 1610 iov_iter_extract_user_pages nr_pages = 4 Feb 5 10:51:50 kernel: [245450.439834] 1291 __bio_iov_iter_get_pages page = ffffea0005bc8000 folio = ffffea0005bc8000 Feb 5 10:51:50 kernel: [245450.439858] 1291 __bio_iov_iter_get_pages page = ffffea0005bc8040 folio = ffffea0005bc8000 Feb 5 10:51:50 kernel: [245450.439880] 1291 __bio_iov_iter_get_pages page = ffffea0005bc8080 folio = ffffea0005bc8000 Feb 5 10:51:50 kernel: [245450.439903] 1291 __bio_iov_iter_get_pages page = ffffea0005bc80c0 folio = ffffea0005bc8000 Please let me know if you have any clue as to why the pages for malloced memory of fio don't belong to the same folio. -- Kundan Kumar