From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28E86C7EE23 for ; Tue, 23 May 2023 19:18:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 89CAD900003; Tue, 23 May 2023 15:18:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 84D16900002; Tue, 23 May 2023 15:18:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6EEA4900003; Tue, 23 May 2023 15:18:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5B465900002 for ; Tue, 23 May 2023 15:18:16 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 0F56E1C6C3A for ; Tue, 23 May 2023 19:18:16 +0000 (UTC) X-FDA: 80822480592.08.A8ED90F Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) by imf14.hostedemail.com (Postfix) with ESMTP id 2684710000E for ; Tue, 23 May 2023 19:18:13 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=DDUUfRIh; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.221.46 as permitted sender) smtp.mailfrom=lstoakes@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684869494; a=rsa-sha256; cv=none; b=rptrBtrZ49VSdSlgfP+2rYp3U7Zssn3N/KWaVKdLwZdofivxPDiyr+S1U+2JpXmijGjVaY ozT3iUSbkiKJ1rwc7XSxI7VsX7r4uvbC1TMqvvDNJ5L/xZIhOFcjCXQsloNtPelWH5DwOV xRpkk/GLuSIIo4Ujvj7doleBBthYGbs= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=DDUUfRIh; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.221.46 as permitted sender) smtp.mailfrom=lstoakes@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684869494; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yxW3MxlFpx/FRHIKsMNTLipCWPh/KLn7GmZFs983h4M=; b=nozEWMMepI6l7/bZDV4VxD62zbjhfWS+itU05ZiMilFXebfKKC/Gn/SJqdXUdvw/Y7QeV/ HHEonfLNxLuGHFLeHUrtPl5DKKHMUwi+UNR8QYHuRxA6//n94jGx0hTRIqzWl5mK9QiPEd +W9ocTxIGNZ+bI3EXqU8cnjQkyDQYMo= Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-3094910b150so6717729f8f.0 for ; Tue, 23 May 2023 12:18:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684869492; x=1687461492; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yxW3MxlFpx/FRHIKsMNTLipCWPh/KLn7GmZFs983h4M=; b=DDUUfRIhJ1Mm9j1zwfKIF2UT2ZLoYuo/VqtwPwLn07z7IKUrtARCi0wzq5PBL0+KCO Wcm8aR6jrI57zZS/BrHe9lEksstBew0ki5BLHsEIg0v7T2MHHxVB9oA9oPntUYrph/AI z+9l13ZSjO8655ZDVFnnEvwuSStes3e7R/qnJOJMj/rbmhwPefDyqCTIx7KrlYJ8QEfZ jHsG6Qj0IN4wbhAGy3deMQHbpeFcKGTC/j2G36tYSvqQaCOWeinfNGlecRo1fpJMJf6f q7neRtn9YNvwNTvrh+Ux5GtpibEzWt+jQj3kNXsl+n0OV7tBcpiNivJpshxey8DZd9Yx P0/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684869492; x=1687461492; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=yxW3MxlFpx/FRHIKsMNTLipCWPh/KLn7GmZFs983h4M=; b=h4PhVtvC5TnHYEyGM94kJDR6dSudQEeP8h+xZujpFSNMtKTXBmzTjUCyeJXxWMvRqa BP67uN7zSbd8rHomonju4W25pONhOnViFWPUMNty4eIL4Y2QZgYSrs333dk1nRfU6ol5 sK7W7RqM6vN+GXLMh4vGuDKDAfQgNxJjKq2Ec3UJOqjW4FOVwH7e6rr06OuihrncHuyX 6ymp+ZxUJs/EfVJjnZxEMeB+HJvt3Dh3q5jJ/jBVUQsd4Ww1GyScNYxYuTMi5Vq6d1un WQdqpiLeCN5ZVS71dmV2chuxmkvb2aKULbbcRq1MCSyd+Ik3buIgsighHG8QDg9AxSkM /y3w== X-Gm-Message-State: AC+VfDxVeF5AHt5ceWNJn6Mtqf74UsLfMvGXwozy4sfq4xQ9nGk2kWmM miJcbHDwdGfk+f6kYmELBtE= X-Google-Smtp-Source: ACHHUZ7XWCdgQy3ajCNp+ADFNmaq/QkZeH3RYMU9ihOihymzO5hdlWNYrnIPM/UxAH33eDaurhGhJg== X-Received: by 2002:a05:6000:1b0e:b0:306:3b78:fe31 with SMTP id f14-20020a0560001b0e00b003063b78fe31mr10050145wrz.69.1684869492236; Tue, 23 May 2023 12:18:12 -0700 (PDT) Received: from localhost (host81-154-179-160.range81-154.btcentralplus.com. [81.154.179.160]) by smtp.gmail.com with ESMTPSA id y11-20020adff6cb000000b003063a1cdaf2sm12009232wrp.48.2023.05.23.12.18.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 12:18:11 -0700 (PDT) Date: Tue, 23 May 2023 20:18:10 +0100 From: Lorenzo Stoakes To: Thomas Gleixner Cc: linux-mm@kvack.org, Andrew Morton , Christoph Hellwig , Uladzislau Rezki , Peter Zijlstra , Baoquan He Subject: Re: [patch 1/6] mm/vmalloc: Prevent stale TLBs in fully utilized blocks Message-ID: <7301efcf-943a-4732-a0d1-215f1606535a@lucifer.local> References: <20230523135902.517032811@linutronix.de> <20230523140002.575854344@linutronix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230523140002.575854344@linutronix.de> X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 2684710000E X-Stat-Signature: qs74gmfmw6fc319q4dgutiu1nqz8i1c1 X-HE-Tag: 1684869493-904778 X-HE-Meta: U2FsdGVkX1+pszRZvqfzXqzh7bVTfWoBWVhy26DztJaWR1KVv56WAek7JabavAZrM4LVicRlrAE6M1KtwnV5XW+BUT04DkRrf5C0vQXyesYbThn13bEEPYFoFBuM6Whq2vhgSHamRvrVuQ1yiaOuc1tzkQUezMcGxPaWVqwQ++h0I6ozYJ5YZ+KlTHJGdL3kz4wFpeN/DHe/6qq56BPxmYtTSXA82VPYYhaakbLUc2YXCAofndC5RoG32klIb97IejVWTS9xPFwc8G6PJ81h8U9T15fkczBhYr5h1lZ+0UDEulc0k0D8YCHzoAEpcvNz0qDI7PJVtm1zUkoHu+wbImgYG7xqV6ukfHL+3q/Sq2QoEaP27uw7rmX96fce7Pfv27U/uq5BME8e4w8d4OLwxWO8JJWVhceEABX10WH+UUES9gwvtXEsG1RX/Z81+IjVsMPorfJ9aGdEVJZXMCnTfA0X3acfSDI4mgWxE1y4yraEXDuh9zfKUZAX5zLLJcaaTLP4FBr1oeWhZOZm56bdqbwR5fcLcB00Kq/itF02N/6HOtPrheIsVnQs4XS/xaryOaQcJJwHHzZaH72RKdJZqFcZnZBkvVw3PsOIUV1+GF2adGSBl40s1hT3DnrQb/y9UwBmmx2P3cs1sYp9RHMTh4P/41w/bmImzew0XpcMRfbILgpdyHCsODLEQcInofITITLr16BNyuuHbysLUe/Ng19BUeDdKDSD77nYeEq65/QOANAX/IQygABpJCrz8Vosuxj32b1bqmn+NDqCExMO1fFHF1p6QLMdK2hI3KiUNUXOkcyCp7e9jZb2c1rvpLq7ZW8fOdhyQ0Sh+uY0JPhzN3l9R8QABMYL20PCB7nZHmLh43yBP820yMUkINuRVM4PvL7jfpQB8CDmk6TT/L9Ub8l1sMSuTBkNWwQmoDzJbupQU6gGATrJdHYyizvEKEySW+AIuXqiFdupGaB8Vm0 1/Wx6piG N3QDMWAFFB1ubwy/yGsSodT4TTgNQXZ+Nav7xKFBNirPcJEcptgta0XJNr/uXWkSCywB4M2o2AFX1ZFk/jO6+vRZ9/iGwLccBHH7jrBJxkyfWB8iJAzhyfBa+96qIyguE1KhT6V41kcV1gCM6slMu0QMW7XDvWTJh/73k40jM1G6CWevAIgmxTnftdgmIcOEm3evvv8U1ot+WoH5+mAIGTJCjmcXiHKrA5i0OOYuRxqtlPFZtoLoj/tD4HrKZwcYTypWD7jiATdEaM41mlnvdsDrji2p7ItNBhpc1HNkL7CW/UlMOSnI8TE7fyU+ugUIwanO7N/X0XP3kzz+q0k1ipxQ8AlnYWKGlVr4wcq0tlKsVYr7ODhMyROemMQUz+Q0er6fpNVBkf/MZ44jpC5xTHLIFWiUYTVyvsSJK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 23, 2023 at 04:02:11PM +0200, Thomas Gleixner wrote: > _vm_unmap_aliases() is used to ensure that no unflushed TLB entries for a > page are left in the system. This is required due to the lazy TLB flush > mechanism in vmalloc. > > This is tried to achieve by walking the per CPU free lists, but those do > not contain fully utilized vmap blocks because they are removed from the > free list once the blocks free space became zero. Oh Lord. > > So the per CPU list iteration does not find the block and if the page was > mapped via such a block and the TLB has not yet been flushed, the guarantee > of _vm_unmap_aliases() that there are no stale TLBs after returning is > broken: > > x = vb_alloc() // Removes vmap_block from free list because vb->free became 0 > vb_free(x) // Unmaps page and marks in dirty_min/max range > > // Page is reused > vm_unmap_aliases() // Can't find vmap block with the dirty space -> FAIL > > So instead of walking the per CPU free lists, walk the per CPU xarrays > which hold pointers to _all_ active blocks in the system including those > removed from the free lists. > > Signed-off-by: Thomas Gleixner > --- > mm/vmalloc.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > --- a/mm/vmalloc.c > +++ b/mm/vmalloc.c > @@ -2236,9 +2236,10 @@ static void _vm_unmap_aliases(unsigned l > for_each_possible_cpu(cpu) { > struct vmap_block_queue *vbq = &per_cpu(vmap_block_queue, cpu); > struct vmap_block *vb; > + unsigned long idx; > > rcu_read_lock(); > - list_for_each_entry_rcu(vb, &vbq->free, free_list) { > + xa_for_each(&vbq->vmap_blocks, idx, vb) { > spin_lock(&vb->lock); > if (vb->dirty && vb->dirty != VMAP_BBMAP_BITS) { > unsigned long va_start = vb->va->va_start; > > LGTM. Reviewed-by: Lorenzo Stoakes