From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9A8EC433EF for ; Thu, 25 Nov 2021 11:38:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F7506B0074; Thu, 25 Nov 2021 06:38:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 180A26B0075; Thu, 25 Nov 2021 06:38:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F3C3A6B007B; Thu, 25 Nov 2021 06:38:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0209.hostedemail.com [216.40.44.209]) by kanga.kvack.org (Postfix) with ESMTP id DDB086B0074 for ; Thu, 25 Nov 2021 06:38:16 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E95118DF7C for ; Thu, 25 Nov 2021 11:38:05 +0000 (UTC) X-FDA: 78847253730.28.3356C68 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf02.hostedemail.com (Postfix) with ESMTP id 495F8700170E for ; Thu, 25 Nov 2021 11:38:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1637840284; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pt90fhSz4eU662izl8ce510Rw/gvmtnnCpUlkzOIY8A=; b=hZPrlIVT5adZ/ElQc6iMe/PKPPWI8q4UjnLO+0pfwJ5flwKn5KAXLlcGl/t4vvKGRAAVFd P4KUlzDpnBdkUThDC8CCR3RF8umNCkgJjYiFcI8fho3IZ5vDyK175p5K2aTZJLPwkdZZVY 5v50n/H3p0W3Q8RyKM+Y7dE+VTAtSdk= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-348-lEG49KltPLKld6NYGOGhJA-1; Thu, 25 Nov 2021 06:38:02 -0500 X-MC-Unique: lEG49KltPLKld6NYGOGhJA-1 Received: by mail-wm1-f69.google.com with SMTP id v62-20020a1cac41000000b0033719a1a714so3079522wme.6 for ; Thu, 25 Nov 2021 03:38:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent :content-language:to:cc:references:from:organization:subject :in-reply-to:content-transfer-encoding; bh=pt90fhSz4eU662izl8ce510Rw/gvmtnnCpUlkzOIY8A=; b=rspHnr97uH86lNFqrVRP7kIKhpqCLuSS5hz1luJ4UhqDXfFuMaddALGnZL62feyY+R 2PXuYA4cyElNJXcj+pVHGOFsU2zkJEJeDhnz5CgWm+uPYgAPzmzgPlNI3+RuNqMBOBAI caUM694UdLJ/k8Bfk1lek83pb38tomlJFWnW/+D6zGH4WbnRBorjWQO8B3cBn1wDYrjG kPcbrISNquhg+Xcx+WK7VOXAR246C84f4P+ahic7yfYulL51BqT0A//pMe5VaD5VmuRm 7l0QOb6m+USO3iHq4rGnJJXRlLB9u787a6dT3ZO0pU32NgS7M+5U9SfSNSdjsipKyo0Y 0F5A== X-Gm-Message-State: AOAM533rD7iODwBjFp40LrOwESzPrUGj3TDsgDeRLed4fOrLHcOzb3zH fqWEnW54VTbks4iUj2umIJuWplybTxxeIuy2NJS+k3UkWNWlDR8I7Iv6irIKnnUTmEOaD1Evqh6 /0W7hJUmTpgY= X-Received: by 2002:a5d:5588:: with SMTP id i8mr5574715wrv.552.1637840280696; Thu, 25 Nov 2021 03:38:00 -0800 (PST) X-Google-Smtp-Source: ABdhPJyBvEMK72HxI5F0PDax7zL1kv8LvReOS3/q13lzZYuF1jRcOQMtDVK0QFOE4fkxQXdAuqueJA== X-Received: by 2002:a5d:5588:: with SMTP id i8mr5574680wrv.552.1637840280424; Thu, 25 Nov 2021 03:38:00 -0800 (PST) Received: from [192.168.3.132] (p5b0c679e.dip0.t-ipconnect.de. [91.12.103.158]) by smtp.gmail.com with ESMTPSA id v6sm7606677wmh.8.2021.11.25.03.37.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 25 Nov 2021 03:37:59 -0800 (PST) Message-ID: Date: Thu, 25 Nov 2021 12:37:58 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.2.0 To: Eric Ren Cc: linux-mm@kvack.org References: <7299f769-3dc8-1f77-c4f2-1e3dc7427689@gmail.com> From: David Hildenbrand Organization: Red Hat Subject: =?UTF-8?Q?Re=3a_Question_about_virtio=5fmm=3a_why_plug/unplug_with_?= =?UTF-8?Q?4MB_granularity=ef=bc=9f?= In-Reply-To: <7299f769-3dc8-1f77-c4f2-1e3dc7427689@gmail.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 495F8700170E X-Stat-Signature: cmaxocwfkdwz46to9aju31goxt8b659z Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hZPrlIVT; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf02.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=david@redhat.com X-HE-Tag: 1637840282-68760 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 25.11.21 12:27, Eric Ren wrote: > Hi David, Hi Eric, >=20 > As the subject, I'm wondering if we can make virtio_mm plug/unplug with= =20 > futher more fine granularity like >=20 > 2MB? That's actually very high on my todo list. See "Hot(un)plug Granularity" under: https://virtio-mem.gitlab.io/developer-guide.html >=20 >=20 > I gave it try as below, and it works. >=20 > 1. Revert this patch:=C2=A0=C2=A0 aac65321ba69("mm/memory_hotplug: simp= lify page=20 > onlining"); >=20 > 2. Tell mm hotplug core invoke online page callback with order 9 with=20 > changes as below; >=20 >=20 > ``` >=20 > diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c > index 5873563a3518..22a9636402b1 100644 > --- a/mm/memory_hotplug.c > +++ b/mm/memory_hotplug.c > @@ -622,7 +622,9 @@ static int online_pages_range(unsigned long=20 > start_pfn, unsigned long nr_pages, > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * them as being onlin= e/belonging to this zone ("present"). > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 for (pfn =3D start_pfn; pfn= < end_pfn; pfn +=3D 1ul << order) { > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 order =3D min(MAX_ORDER - 1, get_order(PFN_PHYS(end_pfn -=20 > pfn))); > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 /* 2MB callback */ > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 order =3D min(MAX_ORDER - 2, get_order(PFN_PHYS(end_pfn -=20 > pfn))); > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 /* __free_pages_core() wants pfns to be aligned to the= =20 > order */ > ``` >=20 > 3. in virtio_mem driver, change subblock size to 2MB. >=20 >=20 > Thought the hack testing works, I'm feeling not safe. Yes, it's not safe in general when unplugging memory. When plugging memory, some changes to virtio_mem_online_page_cb() are required. It has to traverse all pageblocks part of the passed range -- so that avoids the mm/memory_hotplug.c change above. But for memory unplug, alloc_contig_range() and pageblock isolation needs changes (below). >=20 >=20 > Could you please help give some insights on these questions? >=20 > 1. Is 4M a hard limitation? No, the target is 2M. It will improve virtio-mem capability on ZONE_NORMAL significantly. >=20 > 2. What troubles we can foresee if using 2MB as granularity? alloc_contig_range() needs to be taught to handle pageblock granularity (2M) correctly. Otherwise unplug on ZONE_NORMAL will be mostly broken. Fortunately, this is WIP, but still needs to fix some things: https://lkml.kernel.org/r/20211115193725.737539-1-zi.yan@sent.com --=20 Thanks, David / dhildenb