From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97979C77B71 for ; Fri, 21 Apr 2023 15:06:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4E6D6B0071; Fri, 21 Apr 2023 11:06:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CFE416B0075; Fri, 21 Apr 2023 11:06:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC6C46B0078; Fri, 21 Apr 2023 11:06:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AE63F6B0071 for ; Fri, 21 Apr 2023 11:06:53 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7C0C98038D for ; Fri, 21 Apr 2023 15:06:53 +0000 (UTC) X-FDA: 80705725506.03.04477FD Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) by imf14.hostedemail.com (Postfix) with ESMTP id 92B3810001D for ; Fri, 21 Apr 2023 15:06:50 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=iJevz6cE; spf=pass (imf14.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.180 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682089610; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fIAJWB5J2B8K78m9oI7khqky/22dVLaQfbFsBbxztKQ=; b=cJTcXjRZYrA5nupeJqumuJ7Pg+eaRm2gNJ/eAhJftugTAmIQjliIEwhiY9gKCvaocPFNrR fQIsSFtnrKyoL/Tc4NyHenqG2p+lKu+8UOrFLqIQXk3sVJsYumQpAc8hv1hg+9MhBGotjk Gpp6wYCMdyejrWgSgUVu6KCgcKGC0+o= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=iJevz6cE; spf=pass (imf14.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.180 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682089610; a=rsa-sha256; cv=none; b=u9jAITs6q0jcvdsfdF4STdxFYY0Tgh5TY/VlusgTxY6yvIogF8+VYpie5JJ85wpcgO0r4P KsmYpaLhfq/jHbpi5F7uxPvBDnmTSVPnspUWrXeUCIXGK1JUdU6GA6mC1KYQHWfa8zbfoJ Dj/auzgObzWt0U/Rb7jGIFsRUlKAESM= Received: by mail-qk1-f180.google.com with SMTP id af79cd13be357-74ac861476dso109412785a.3 for ; Fri, 21 Apr 2023 08:06:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20221208.gappssmtp.com; s=20221208; t=1682089609; x=1684681609; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=fIAJWB5J2B8K78m9oI7khqky/22dVLaQfbFsBbxztKQ=; b=iJevz6cEXq3yi/2Ws0fGvIgtnolydo04XXjfQDLcp6sC2QIPSBjxB8fOuY3VZ+3C8Y 5myNoUtu9r2OvWThTxM8+M8g9XobkLBd3CEUSpAYLUDR8WHgBBY2UoRlWydMUQZdoxDp izWJcA08AvdEYJnRBaUih6DfAS/X9OQhsFoRIq0hq4DnAcBY3zCEM2tB4Ds8AAUiHXht PeJKfe81CEtMJ35Rhji2ZxdV5zzvPbOUiCWp57GRbs8QEPP9K/CieYhaiFqTz0IYDYma 6LfwJIuPLRErC+nqSqsjMP/O23MSljEDqm9RLS3koXks5B40//c31qAgvqvXyffQmwhP do4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682089609; x=1684681609; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=fIAJWB5J2B8K78m9oI7khqky/22dVLaQfbFsBbxztKQ=; b=XZCmv3WTSG1cNq+kNMgAjBhs5yy15KzAJskOHAVR3v7/O8BuBsoWoxr5kY6OvCawOr +L9gzyOl/jQe/G3sU7huqoyymRrqO3n+XoZIIdIbL95YVi1Ao1p0mgOAZPLjGaHXrlNC 4HI+7pd8eeXsMFLKTw5UNNgaDljlxMRj6iMeuPpd6iCKjUJ1wXdmnUwrv5KvHQT0JP5d BOtV8BF2bv/mzAimMys7OTmUgqv1iEwlc/draLpwp6ExM+VUjLBK3Kh2suyYuOOtWddp SBWHoCqTMVmcOWxwDsWxO6FToUSIXQIC2O3eWinIOYwd4RDb9X11Ea/7y4UtmnymVISO dpyg== X-Gm-Message-State: AAQBX9fp9kC4JPzEaA5a6CPqYhWLglExe9NdziU3tsncnCXgkuREwfv6 fZuuWkjJcP3biq3a7Dl1XPzMRE/tpGEHM3nbsgc= X-Google-Smtp-Source: AKy350b/32igxAJPjM6OJok7UdEMciBdE4rZawPr2m1ECfTAYZx+6ZoGBz/NxB84javPVoPEd7eEfw== X-Received: by 2002:a05:622a:3d4:b0:3e8:1903:ab05 with SMTP id k20-20020a05622a03d400b003e81903ab05mr7372402qtx.64.1682089609592; Fri, 21 Apr 2023 08:06:49 -0700 (PDT) Received: from localhost ([2620:10d:c091:400::5:6f0d]) by smtp.gmail.com with ESMTPSA id j1-20020ac806c1000000b003d65e257f10sm1365634qth.79.2023.04.21.08.06.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Apr 2023 08:06:49 -0700 (PDT) Date: Fri, 21 Apr 2023 11:06:48 -0400 From: Johannes Weiner To: Mel Gorman Cc: linux-mm@kvack.org, Kaiyang Zhao , Vlastimil Babka , David Rientjes , linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: Re: [RFC PATCH 05/26] mm: page_alloc: per-migratetype pcplist for THPs Message-ID: <20230421150648.GB320347@cmpxchg.org> References: <20230418191313.268131-1-hannes@cmpxchg.org> <20230418191313.268131-6-hannes@cmpxchg.org> <20230421124744.skrxvziwg3bx7rgt@techsingularity.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230421124744.skrxvziwg3bx7rgt@techsingularity.net> X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 92B3810001D X-Stat-Signature: ky3kkdfm5xz1iqcjtipp7mkyss8geidx X-HE-Tag: 1682089610-630191 X-HE-Meta: U2FsdGVkX1+RBaMku047SmB8qn3az33/9d77GDot3QCJZv5IE7JZtvKh7VbDBP7mBhOwHZL+QqbgYP43IUVpb2Rj7hJbqvcrJqzNSHnnH7CFgs5XE0TXlTw6iI0RvLYCeSAl7ss/pH/aXmJbiAbmupD4BXXFx4rIulI6HRG0TW1nzMYJAnqu/f5taaWCwiMOut1H0DLsTAMeRBtZ2TTuoSw/g45TjuvLWOlM51qOSEIgABLAOZpVxxhlG3dhJejbnjX9Ig3QF5t9smiDh6ZSftKQmGFFxf630ro0OjYqumrbo+nzpimpfz4NAqgeq/sW4+Jo8nODjH8OIDc68YamJHQGrPMVSHQCIvBYHf6zZLbShTQRYrsIvpVdvcfqfF013SpuCM5EtCSUBxQzdmCCf9P61AobVhS9nhP6w6t40BnDqGrX/+InSn3e0uPRWSvMInMKslB/a/TW0mDM+wUruqYOebk87e9tg6o9NeC2c195Qh6otwoxJuHK7G2NbDIVWAxUDk7EWmXqL1SMSuwdaI99Mqim1ah00F2rrqfY480A3unu5/CzCKN7JBVjtMixzyNRf9kFpvrN8wb+rCCxcRmoI8N0hQeh7SvN8pjwfzln36nUQwNA66WQq64haEttWWGixmRZf7XlDk50Wn7XjNq4+KaMSmzcUtlfWgCL3UBWsyVJe9I4AlLPXDc1ZSzpxaX4o7fbPHCBOZe+bIVfRllpnAe03uxdWuvj7HEfGB1tCquTUjWAWfpGwiEeQYy+/Omp/m7OAdP7b935Wn6hj5gGRVbPsfkMM9ui/OHBuvEspE3MDdxXb25i6OL3PJ/6HEAEZs9KkhDQIpPhVw9CcLVRy0CF9sKoNa4y2iDYpfs8dQeGLPoC17iVOngLme2B5g/ew9w0xVov9jSfRkaPKhkM0ZckXxLFHM02t4WCOzprVJa1W8tvl+SacQ3ooGeIXwibHPLmdhfFMDwly/Y VDfhs9ea F2ebY1rTCK3ChyHa7WsPD+KrrmxFEILS8KK0g2fjf5mGZHUB0CULg8HDP1We6obx1cDxnhWypvkvgB0Zw1vQ1dm4YKFhOqm+3OLKx+tj7hFBBCnowRP5UfU6UKPg9Zjsp8yB6eVwFSkLzEb8bL1Gx52hfiGAFXKqK0W+VvZ0Rfa60qAAOP9Ewz11X+ea5L3RZAR2XQtGZ8QDUVctOT/6nOsqa1lSRcgaWuF6tpTPaBvPzwhjoc/yUK4W5ATRvLaY/m/T8FNHWkWqguD71RHUVb46p63vu7vMX3G8onPnWUldzvo/y7XWnXLbSeWzgmmaQbOHP3QgYx7/hM2rnbNRlz4Ols2zYWGP9mwIh+HFq7h41euN4bwMHJx51n3ASJoDJmg/dIeZKZCiIntfpTx48ffv/U+10Tyj7uLZSQWqpjGnNd8EDWkmcyykPwLR2M2E6TCi2cqWR/uCZubUbwaX+qLsLM37RgMfl+SSWoCU4H8DuzKEIrCtwbOw9c8YoXmGTiBYdFiESkHO9nPICqCN+XNvFUw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Apr 21, 2023 at 01:47:44PM +0100, Mel Gorman wrote: > On Tue, Apr 18, 2023 at 03:12:52PM -0400, Johannes Weiner wrote: > > Right now, there is only one pcplist for THP allocations. However, > > while most THPs are movable, the huge zero page is not. This means a > > movable THP allocation can grab an unmovable block from the pcplist, > > and a subsequent THP split, partial free, and reallocation of the > > remainder will mix movable and unmovable pages in the block. > > > > While this isn't a huge source of block pollution in practice, it > > happens often enough to trigger debug warnings fairly quickly under > > load. In the interest of tightening up pageblock hygiene, make the THP > > pcplists fully migratetype-aware, just like the lower order ones. > > > > Signed-off-by: Johannes Weiner > > Split out :P > > Take special care of this one because, while I didn't check this, I > suspect it'll push the PCP structure size into the next cache line and > increase overhead. > > The changelog makes it unclear why exactly this happens or why the > patch fixes it. Before this, I'd see warnings from the last patch in the series about received migratetype not matching requested mt. The way it happens is that the zero page gets freed and the unmovable block put on the pcplist. A regular THP allocation is subsequently served from an unmovable block. Mental note, I think this can happen the other way around too: a regular THP on the pcp being served to a MIGRATE_UNMOVABLE zero THP. It's not supposed to, but it looks like there is a bug in the code that's meant to prevent that from happening in rmqueue(): if (likely(pcp_allowed_order(order))) { /* * MIGRATE_MOVABLE pcplist could have the pages on CMA area and * we need to skip it when CMA area isn't allowed. */ if (!IS_ENABLED(CONFIG_CMA) || alloc_flags & ALLOC_CMA || migratetype != MIGRATE_MOVABLE) { page = rmqueue_pcplist(preferred_zone, zone, order, migratetype, alloc_flags); if (likely(page)) goto out; } } Surely that last condition should be migratetype == MIGRATE_MOVABLE? > The huge zero page strips GFP_MOVABLE (so unmovable) > but at allocation time, it doesn't really matter what the movable type > is because it's a full pageblock. It doesn't appear to be a hazard until > the split happens. Assuming that's the case, it should be ok to always > set the pageblock movable for THP allocations regardless of GFP flags at > allocation time or else set the pageblock MOVABLE at THP split (always > MOVABLE at allocation time makes more sense). The regular allocator compaction skips over compound pages anyway, so the migratetype should indeed not matter there. The bigger issue is CMA. alloc_contig_range() will try to move THPs to free a larger range. We have to be careful not to place an unmovable zero THP into a CMA region. That means we can not play games with MT - we really do have to physically keep unmovable and movable THPs apart. Another option would be not to use pcp for the zero THP. It's cached anyway in the caller. But it would add branches to the THP alloc and free fast paths (pcp_allowed_order() also checking migratetype).