From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D53C4C77B75 for ; Tue, 16 May 2023 06:11:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 59C7A900003; Tue, 16 May 2023 02:11:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 526BD900002; Tue, 16 May 2023 02:11:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C60D900003; Tue, 16 May 2023 02:11:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 28BA7900002 for ; Tue, 16 May 2023 02:11:38 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E7F7BADE02 for ; Tue, 16 May 2023 06:11:37 +0000 (UTC) X-FDA: 80795096634.28.FD9D20F Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by imf28.hostedemail.com (Postfix) with ESMTP id A24B3C0003 for ; Tue, 16 May 2023 06:11:35 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=DzWE6jWa; spf=pass (imf28.hostedemail.com: domain of quic_charante@quicinc.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=quic_charante@quicinc.com; dmarc=pass (policy=none) header.from=quicinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684217495; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Svy4l+hzxpnZMWtHXBn/4YWN6BCTRpwMkrrJw/0D6dc=; b=v+FtT9wnCDtR5UGWBcRo2xSLnHlGj0c2iUksL8SvyNBmcWX1fKp27raf9Wsxf0dO87AOuL Wnf4MeU9bQHeSkZ6BLNyvAq+3ulgpmGBMZ7pverpRy2ul8KBMhQakHYZp9hkj/yb7/WvjQ JWu2geuTEtfD6IVgtOZJF2sKqQ9f6cg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684217495; a=rsa-sha256; cv=none; b=NdWTUe9eiiFRkx4I74n5A7/UvBCWgbCV3Cir7pixpj2O88ErL1lzX4AXn9GHBHvzYjS4ih ElsIIhf7OsmX/2JgGHaRozJgFrHgygxX/vEbWsmsOrindGgSzTS7REgConEbLdHfZjM5Bi weTOHIct8/ruI6VADt4+vpKSW4ZuKeY= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=DzWE6jWa; spf=pass (imf28.hostedemail.com: domain of quic_charante@quicinc.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=quic_charante@quicinc.com; dmarc=pass (policy=none) header.from=quicinc.com Received: from pps.filterd (m0279873.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34G3PGRM004195; Tue, 16 May 2023 06:11:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=qcppdkim1; bh=Svy4l+hzxpnZMWtHXBn/4YWN6BCTRpwMkrrJw/0D6dc=; b=DzWE6jWaYYkEHal2H1iSz0z0TffpYwbpGnHVrW5TvxPfOAKCfgcFnBiEz3WazGvnC21n QlpeyMTU16SjuDdKjlC8VwuLcQLhl+P5dzdQlXG4XN1wRLCRzcalppB1ib0lUAPHujXJ dDa3WtCAjwLlbKHo03gtgXokPURZ3gWZ8cqvzMmllgvsOsmb7dejj5pX9arLh2GbgSWm ljSihiX261szHZiUtHbTlhtVAT8+WEdSt3Ba3D1nPbunY868G864JZGS9OV8/PeJE43p ICGGH5fxaBquUCgrbcjgRqs5Ncv/mPJTmaXqA6TV8q6bMXCUa6lRHaknnXCTh+oLbWtQ wg== Received: from nalasppmta03.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3qkjsctf21-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 May 2023 06:11:24 +0000 Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 34G6BN3Y003611 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 May 2023 06:11:23 GMT Received: from [10.214.66.119] (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Mon, 15 May 2023 23:11:20 -0700 Message-ID: <0a754ebc-dad5-d300-0239-892b74786764@quicinc.com> Date: Tue, 16 May 2023 11:41:17 +0530 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: [PATCH V8 0/2] mm: shmem: support POSIX_FADV_[WILL|DONT]NEED for shmem files Content-Language: en-US To: , , , , , , , , CC: , References: From: Charan Teja Kalla In-Reply-To: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 9z1wTRIrGOf1WGGS2FCQQDQwjJum_CCB X-Proofpoint-ORIG-GUID: 9z1wTRIrGOf1WGGS2FCQQDQwjJum_CCB X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-16_01,2023-05-05_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 adultscore=0 mlxlogscore=999 clxscore=1011 priorityscore=1501 lowpriorityscore=0 malwarescore=0 spamscore=0 impostorscore=0 suspectscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305160052 X-Stat-Signature: d94fo3bdup7ob87n39e1bo99sss4871z X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: A24B3C0003 X-HE-Tag: 1684217495-841086 X-HE-Meta: U2FsdGVkX185kiWGaVIOQfKly2Ne4G+0aeX/xG7RYuppesR478rk7IuMlza8LP6J1s2k5SXpajevQpC5adixinF6zN842+xW5MCuSLpulKWAoLJPFc88U3Ac4O3HSNIrab5tXUoJlG6WXIum527GU9Y+D/XMg1RZVi09IfqT+BXbjJama5TNBF1B3XfScAAdDh5rZc65XkSM/TGiYmELTaXJY0KaoVGfC14eTyxSNucm0BktOLeB7pHkULCxg3z7FRf5u/mTcYM4Jqguxgjtz5wZx9227YqLRrQ7beC37qNFxuXOXBXNZa4crmyTWjsTeBj8OEYSX1pUga8g3I303zigRR+DhF7D7dUtgGGlF0Mz4iWirxkdUhJnoLYryo8+MLOrfw17rEwuR0FPAkn+mqOcViDJtFYHRc5Zem+pLdgvH0k3WwqM0hvknLgK+vQQs58RIehl87ALXeHU53HdAgnehCtg/4Ogtt0BoXWxDGmcgURyc9VnL8xjZ+oV5JCqvWohbC201fhXQMwV0zQDOzWGzcWz3LQb5wNbtkW/UiRxjZGwZhL1SY5xIezCIajTz3yIDG9Ol2ThQhacJnWK5EFZcLtqOVRO3g/P3cV4OJ2Ncwbv1hFr3PNFGN7rW4xCQa2YFJ8rPoqmRe2ObI/U/DBmEepcwQBU5LBzEZmbv318nQY3ayf88/SRHVb0E+sPz2yGkgc2f5BW6JUaTYMSxGhSxvZQM49oeqUNo42dcdg+D05MiaBovgIqti3JnKv7BMinglsPVDC444U228T1vs7zqOpqpDLruhqTtqaRVzhr+rPw8cj3rUauyFpWE7ub/YECY9Jc/vtyJ7OfUrIl64T7yz4fjj+ePnnCgW8/JNmFCrQkefBAG606ybI07yushnW1EmPCtNGEYokkrfzp5tbou82DqQdOsVUYJeM7zDRdSN2SmTzerkYvqQSY8E9zF4PZoUziHD9X6M9RzUm EmJt7flF ziE5RJotZf6hc0tF2kc6hwX5slB+/yiGK95bwFT1zvni4iJGjorTUvjrAZA5DA5Q2g9ee98DaGnSc+xQG0SJu4uINgoWz1n7CN4AoA3Hn4s2s2c0groxGQAGgntwNnJ6mEr9430AlQZ6qKxED+Lb3ZQqwyqL9C4vRTV4WLEUtvAfkaC/1tkAhaizKeKvWuHarKD7fRDPwlIoFUijrFhNaTkJANJEMFXHLiH9qB8QyIqC5SSIpz3HCN+prgrTk3izfbfok3/ae9NSkVYMYFNl3F9ZZpXPrAFvwkZG7EpI2p5xJ/UKg8oGBXZRGzG95jii9TeB0rB48zLsZGTV1tBlu0quAttXPClTQtOIpVkC8RHtgnYpjfYQZVBmYpyR6wBl/EmF91bA7bJSqFf9NNTzLX5fSGTM2+ZFzgfAM3yATVWWh30JadibMMEyOnw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just a ping to have your valuable inputs here. On 4/28/2023 8:32 PM, Charan Teja Kalla wrote: > This patch aims to implement POSIX_FADV_WILLNEED and POSIX_FADV_DONTNEED > advices to shmem files which can be helpful for the drivers who may want > to manage the pages of shmem files on their own, like, that are created > through shmem_file_setup[_with_mnt](). > > This patchset is unit tested with the below: > (a) Qemu x86_64 with 4 smp cores, 2GB ram, 1Gb swap mounted > on zram block device. > (b) A tmpfs file of size 100MB is created. > (c) Initially this file is filled with a poison value of 0xAA. > (d) POSIX_FADV_[WILL|DONT]NEED is called on this file for a range that > gets generated randomn. This is called for 10K iterations. > (e) Check the poison value at the end of the test. Exit the program if it > is changed. > > use the below script: > val1=$(cat /proc/vmstat | grep pswpout | awk '{print $2}') > str=$(./a.out ) > res=$(echo $str | awk '{print $1}') > val2=$(cat /proc/vmstat | grep pswpout | awk '{print $2}') > > if [[ $res == "FAIL" ]]; then echo $str; > else > if [[ $val1 == $val2 ]]; then echo "FAIL. Does the swap setup is done?"; > else echo $res; > fi > fi > ------------------------------------------------------------------------ > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > > #define NR_ITER (10000) > unsigned int fd; > > void run_tests(int fd, loff_t start, loff_t end) > { > > if (posix_fadvise(fd, start, end, POSIX_FADV_DONTNEED)) { > perror("FAIL : fadvise():dont_need_thread\n"); > exit(6); > } > > if (posix_fadvise(fd, start, end, POSIX_FADV_WILLNEED)) { > perror("FAIL : fadvise():will_need_thread\n"); > exit(6); > } > } > > void get_rand_range(long size, loff_t *start, loff_t *end) > { > *start = rand() % (size >> 1); > *end = *start + (rand() % size); > > if (*end > size) > *start = *end = 0; > } > > void fill_pattern(int fd, long size) > { > char *data; > > data = mmap(0, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); > if (data == MAP_FAILED) { > perror("FAIL : mmap()\n"); > exit(5); > } > memset(data, 0xAA, size); > munmap(data, size); > } > > int check_pattern(int fd, long size) > { > unsigned char *data; > long i; > > data = mmap(0, size, PROT_READ, MAP_SHARED, fd, 0); > if (data == MAP_FAILED) { > perror("FAIL : mmap()\n"); > exit(4); > } > > for (i = 0; i < size; ++i) { > if (data[i] != 0xAA) { > printf("FAIL : Testcase is failed. Seeing %x rather than 0xAA\n", data[i]); > exit(4); > } > } > munmap(data, size); > } > > int main(int argc, char **argv) > { > struct stat buf = {0}; > loff_t start, end, i; > > if (argc < 2) { > printf("FAIL : usage: a.out \n"); > exit(1); > } > > > if (stat(argv[1], &buf)) { > perror("FAIL : stat()\n"); > exit(2); > } > > fd = open(argv[1], O_RDWR); > if (fd < 0) { > perror("FAIL : open()\n"); > exit(3); > } > > fill_pattern(fd, buf.st_size); > for (i = NR_ITER; i > 0; --i) { > get_rand_range(buf.st_size, &start, &end); > run_tests(fd, start, end); > } > check_pattern(fd, buf.st_size); > > close(fd); > > printf("PASS \n"); > > return 0; > } > ------------------------------------------------------------------------ > > Changes in V8: > -- Addressed the comments and fixed the bug caught by Hugh. > -- Updated the commit message for POSIX_FADV_WILLNEED asked by Minchan. > > Changes in V7: > -- Use folio based interface, shmem_read_folio(), for FADV_WILLNEED. > -- Don't swap the SHM_LOCK'ed pages. > -- https://lore.kernel.org/all/cover.1676378702.git.quic_charante@quicinc.com/ > > Changes in V6: > -- Replaced the pages with folio's for shmem changes. > -- https://lore.kernel.org/all/cover.1675690847.git.quic_charante@quicinc.com/ > > Changes in V5: > -- Moved the 'endbyte' calculations to a header function for use by shmem_fadvise(). > -- Addressed comments from suren. > -- No changes in resend. Retested on the latest tip. > -- https://lore.kernel.org/all/cover.1648706231.git.quic_charante@quicinc.com/ > > Changes in V4: > -- Changed the code to use reclaim_pages() to writeout the shmem pages to swap and then reclaim. > -- Addressed comments from Mark Hemment and Matthew. > -- fadvise() on shmem file may even unmap a page. > -- https://patchwork.kernel.org/project/linux-mm/patch/1644572051-24091-1-git-send-email-quic_charante@quicinc.com/ > > Changes in V3: > -- Considered THP pages while doing FADVISE_[DONT|WILL]NEED, identified by Matthew. > -- xarray used properly, as identified by Matthew. > -- Excluded mapped pages as it requires unmapping and the man pages of fadvise don't talk about them. > -- RESEND: Fixed the compilation issue when CONFIG_TMPFS is not defined. > -- https://patchwork.kernel.org/project/linux-mm/patch/1641488717-13865-1-git-send-email-quic_charante@quicinc.com/ > > Changes in V2: > -- Rearranged the code to not to sleep with rcu_lock while using xas_() functionality. > -- Addressed the comments from Suren. > -- https://patchwork.kernel.org/project/linux-mm/patch/1638442253-1591-1-git-send-email-quic_charante@quicinc.com/ > > changes in V1: > -- Created the interface for fadvise(2) to work on shmem files. > -- https://patchwork.kernel.org/project/linux-mm/patch/1633701982-22302-1-git-send-email-charante@codeaurora.org/ > > > Charan Teja Kalla (2): > mm: fadvise: move 'endbyte' calculations to helper function > mm: shmem: implement POSIX_FADV_[WILL|DONT]NEED for shmem > > mm/fadvise.c | 11 +---- > mm/internal.h | 21 ++++++++++ > mm/shmem.c | 127 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > 3 files changed, 149 insertions(+), 10 deletions(-) >