From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58633C3DA63 for ; Fri, 26 Jul 2024 07:27:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C940A6B009D; Fri, 26 Jul 2024 03:27:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C43DF6B009E; Fri, 26 Jul 2024 03:27:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0B6D6B009F; Fri, 26 Jul 2024 03:27:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 882AB6B009D for ; Fri, 26 Jul 2024 03:27:03 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2F891A6BF9 for ; Fri, 26 Jul 2024 07:27:03 +0000 (UTC) X-FDA: 82381072326.01.08D83C8 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf06.hostedemail.com (Postfix) with ESMTP id 3513A18000C for ; Fri, 26 Jul 2024 07:27:01 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jvDqXa3r; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf06.hostedemail.com: domain of chrisl@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=chrisl@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721978819; a=rsa-sha256; cv=none; b=tQ5ejXy/uD4nJ/RqIl8HPnx4pu3P1lBG0cx28HfK85+LEPh6YZI5Ka9BaJ5K+p7525LZ9L JA60agi2Huz4omva0aImjz+RoiwzYGJI73kBXrwc0gzk2t0QgTzu2P/MVttG1RZXoNa9bv 4pUZoOo+ne0wdsA8qAdjNz8c6Rx91W4= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jvDqXa3r; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf06.hostedemail.com: domain of chrisl@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=chrisl@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721978819; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NJMaLgrtpv/FHoiKNufz9XH9v+cHlZxd4gCuSwAmiOI=; b=rC2WM5/i4LG0pTUlo4IUpZiQOmkKb1R1P29s7SzQjrAtqnLB1VmWK09tpND3t/tTuh/pao BqyU+2Eu/Pmt/Uj2o2aN6h4sDfQ/8u/S/6rel5NJkW+U0gRuIAw4FY1eQnV9liyTD6C3sc gvqtJUbkLUrqYaIS4DvEv5pycrx2qQc= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6CE646170A for ; Fri, 26 Jul 2024 07:27:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 23D0AC4AF0B for ; Fri, 26 Jul 2024 07:26:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1721978820; bh=kPxU1OON3IvpJ9O8NrD8HbE7eAdVGMynVpQbkVljyWY=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=jvDqXa3rAnukdVKp2NiULYcdZRRa5f0V5LH5wU5hvpygXtzYC4Zwyzu/LsTIfxhV6 SZxnGvMyJqEw9YU2Cqbg6BbQnI82t8433eSG6ly48rnMYvOO+pJCAeZxHDiJdACDHP YEA2o6EhjX87X1hWlclli8KmeAFIq2t/8lGqs40ZQmu7EgtxvsXi2H72orWZ26B17f MyhFBC5X/utFHl+rT4vJM299z8Py8btf6HhbzcBVwqEGA+E3NzmPDpoNm4QPL2wtKK S8XUXqJekGobZmZueVmsMcFqeZ6SEqEkOPLAvAxhQlXIAMt3Jz+RgDUGogWmhBELMa bwjKLO0X/7ZPg== Received: by mail-yw1-f170.google.com with SMTP id 00721157ae682-661d7e68e89so13395867b3.0 for ; Fri, 26 Jul 2024 00:26:59 -0700 (PDT) X-Forwarded-Encrypted: i=1; AJvYcCUaWPnvhNhMJsoMTS6LY8SNDHa0cU5BvC9KowQsRmwmCqcT7fClResbLsxzXP01YuH7IafSMX4kIbcKy+76ZxSaIJs= X-Gm-Message-State: AOJu0YxQ6OpN/Wvt3pmaAJlbPVENqTeSe8b6IAVpr6QI7Txuf/hNCYVg 9FKUDPc+XY0+O4SILonUJ5ecOqXT2j/q3cdK+4Tssr6g7fAYjZj/wrqsmBjhYDDaM5LiPrk5ZMH flpNtRZ5s81JRIFRxRuwWD8ZLJaLnGnD6YTClHA== X-Google-Smtp-Source: AGHT+IGvY6UJVtnciFwYJb04nTJUm+NyUa1h/z8v+I1kZ2Jq1rkw45M099OmOdE7bbXs61N6v2oh6oLMSVNL2AS8XcM= X-Received: by 2002:a0d:efc3:0:b0:630:4fab:a090 with SMTP id 00721157ae682-672c0f4420dmr59106467b3.22.1721978818451; Fri, 26 Jul 2024 00:26:58 -0700 (PDT) MIME-Version: 1.0 References: <20240711-swap-allocator-v4-0-0295a4d4c7aa@kernel.org> <20240711-swap-allocator-v4-2-0295a4d4c7aa@kernel.org> <874j8nxhiq.fsf@yhuang6-desk2.ccr.corp.intel.com> <87o76qjhqs.fsf@yhuang6-desk2.ccr.corp.intel.com> <43f73463-af42-4a00-8996-5f63bdf264a3@arm.com> <87jzhdkdzv.fsf@yhuang6-desk2.ccr.corp.intel.com> <87sew0ei84.fsf@yhuang6-desk2.ccr.corp.intel.com> <4ec149fc-7c13-4777-bc97-58ee455a3d7e@arm.com> <87le1q6jyo.fsf@yhuang6-desk2.ccr.corp.intel.com> <87zfq43o4n.fsf@yhuang6-desk2.ccr.corp.intel.com> <87o76k3dkt.fsf@yhuang6-desk2.ccr.corp.intel.com> <87bk2k39lj.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <87bk2k39lj.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Chris Li Date: Fri, 26 Jul 2024 00:26:47 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v4 2/3] mm: swap: mTHP allocate swap entries from nonfull list To: "Huang, Ying" Cc: Ryan Roberts , Andrew Morton , Kairui Song , Hugh Dickins , Kalesh Singh , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Barry Song Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Queue-Id: 3513A18000C X-Rspamd-Server: rspam01 X-Stat-Signature: km47niq1gdufi1ga4fjxz16jnf8yicfa X-HE-Tag: 1721978821-613190 X-HE-Meta: U2FsdGVkX19p+3MtqK29kfGjrUIxuxUqqjmIowNVoQl/TQMcwexdtkz6lk7dflLd+3b4vG62jELeK4reDbLbx6lD9JkiIsp+ixG9Eoy27cKsRz2GOhy2V5kAJIi2ywt8zt6Y+9a1QP5Q4GDiewdggjmFcHp1HIe6P0jFlcmq/MvPX0y0iF631qeNib1FwKiG4jVGenG1gzqUxWNu1vVOO7gPCAtNRsrFuCbdx4prr6tVKwxe/gRs/n6arnHHq6Ru8KYnqVvIPs8z5z9sEj5X2SD3nFHfa6+WR/Wc8CxqMhl9mMJjVKT0eRFEQv3TB7enqeHJZwCY8icZT7kjsehznKuuw8Y8TKQNbe5yFh9o/p035QneOiRYfHSbU5LzqRC9mWqD559eokyJiLemy9Ju5excniwpFmMkrAutiAIwvqbNBVEu19lKIOpbMl3v05zNcs30UZfOngJhEcAF0yapnrVVbYTmkSl3jA2yqZm5K/JeJ5dGaMgpSPD40mvcIxfaC4vaUH6t4fxOcZFjoGvJQM5mIaOsivpTGmAV9FELbMNB0oj1xbnDk0i8vQpteb7CN0BrWuQzgiOHiuto5Y4VQOR81pPSV4CeRtKVbK18NVr+rf77aFISWpKUic6dAlZ8yXUqWV0p0F7WlTWiN0WbhU+y8ieBnSOiM3KzFIsGgWEli7FMxCuRqwOAwvXEd0PZ6hDA16Wlei05wT9QVCoXxG41D0bw0mi1LUa7teE5t0xDrj/DilSqqRmmp6fMvGjwuFxYnL+cr6LmnaGJpx0PohmqCjX1eolIl7WOw6ImEhYF2gdnMlrbxJvFlJvEuYf1rBzmXV3OQJbYD3J3YYGYNpbLKxC+LIlW9NFj3lx9uFFapHhHgj6MO53WWjfEYYmEoFxqI1alqiXdULhnssNvYgQID4Q5FiTmkepOojYZdiAK20L6Qi7DFBkGzB0QOEmDz0os6MfMxa9SnPiR6gb 0vnmY1gi +aMbIozW5Mj8q6NXUxpmpN9Qb5Uxb6wauo/J1/DJh7Vu+fC3QvpTjMU2+cMSNzTAbNGZlQE3aHlAWvSzNWrY77FFw+dLCHCInFjgWsyh1EflywqUQlYgN1xP2mKVH1gLbOxOWWMpbSBWRgX6CBZAUgiWwe3FxtNisH1dKnN0qrUWjJFMeIwJwzJU1mj1bXThOywFpGXD/Gwy4l4cZsiuIxczM/MN2COjDvrSaAX2v3U69YYqTY+yXNqI/nNbng4t2L7lgrCQR5nUIEa8ou+dY37OEtwEX2AhX9Fw7W+pgMSTp0+H92Qo4fkX3e4Zq1ZXkhBtp/8oQ5bPng7w= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jul 26, 2024 at 12:21=E2=80=AFAM Huang, Ying = wrote: > > Chris Li writes: > > > On Thu, Jul 25, 2024 at 10:55=E2=80=AFPM Huang, Ying wrote: > >> > >> Chris Li writes: > >> > >> > On Thu, Jul 25, 2024 at 7:07=E2=80=AFPM Huang, Ying wrote: > >> >> > If the freeing of swap entry is random distribution. You need 16 > >> >> > continuous swap entries free at the same time at aligned 16 base > >> >> > locations. The total number of order 4 free swap space add up tog= ether > >> >> > is much lower than the order 0 allocatable swap space. > >> >> > If having one entry free is 50% probability(swapfile half full), = then > >> >> > having 16 swap entries is continually free is (0.5) EXP 16 =3D 1.= 5 E-5. > >> >> > If the swapfile is 80% full, that number drops to 6.5 E -12. > >> >> > >> >> This depends on workloads. Quite some workloads will show some deg= ree > >> >> of spatial locality. For a workload with no spatial locality at al= l as > >> >> above, mTHP may be not a good choice at the first place. > >> > > >> > The fragmentation comes from the order 0 entry not from the mTHP. mT= HP > >> > have their own valid usage case, and should be separate from how you > >> > use the order 0 entry. That is why I consider this kind of strategy > >> > only works on the lucky case. I would much prefer the strategy that > >> > can guarantee work not depend on luck. > >> > >> It seems that you have some perfect solution. Will learn it when you > >> post it. > > > > No, I don't have perfect solutions. I see puting limit on order 0 swap > > usage and writing out discontinuous swap entries from a folio are more > > deterministic and not depend on luck. Both have their price to pay as > > well. > > > >> > >> >> >> - Order-4 pages need to be swapped out, but no enough order-4 no= n-full > >> >> >> clusters available. > >> >> > > >> >> > Exactly. > >> >> > > >> >> >> > >> >> >> So, we need a way to migrate non-full clusters among orders to a= djust to > >> >> >> the various situations automatically. > >> >> > > >> >> > There is no easy way to migrate swap entries to different locatio= ns. > >> >> > That is why I like to have discontiguous swap entries allocation = for > >> >> > mTHP. > >> >> > >> >> We suggest to migrate non-full swap clsuters among different lists,= not > >> >> swap entries. > >> > > >> > Then you have the down side of reducing the number of total high ord= er > >> > clusters. By chance it is much easier to fragment the cluster than > >> > anti-fragment a cluster. The orders of clusters have a natural > >> > tendency to move down rather than move up, given long enough time of > >> > random access. It will likely run out of high order clusters in the > >> > long run if we don't have any separation of orders. > >> > >> As my example above, you may have almost 0 high-order clusters forever= . > >> So, your solution only works for very specific use cases. It's not a > >> general solution. > > > > One simple solution is having an optional limitation of 0 order swap. > > I understand you don't like that option, but there is no other easy > > solution to achieve the same effectiveness, so far. If there is, I > > like to hear it. > > Just as you said, it's optional, so it's not general solution. This may > trigger OOM in general solution. Agree it is not a general solution. This option is simple and useful. The more general solution is just write out discontiguous swap entries. Chris