From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 777D1C433FE for ; Thu, 20 Jan 2022 08:42:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 93C6F6B0072; Thu, 20 Jan 2022 03:42:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8EBD66B00AB; Thu, 20 Jan 2022 03:42:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B3D06B00AC; Thu, 20 Jan 2022 03:42:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0249.hostedemail.com [216.40.44.249]) by kanga.kvack.org (Postfix) with ESMTP id 6D3D56B0072 for ; Thu, 20 Jan 2022 03:42:29 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 2DAD993684 for ; Thu, 20 Jan 2022 08:42:29 +0000 (UTC) X-FDA: 79050024018.26.F2B64D6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf05.hostedemail.com (Postfix) with ESMTP id 03972100007 for ; Thu, 20 Jan 2022 08:42:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642668147; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XKfas3q2Y/HkFD5WHzkZNGqSak03+yPlc1Wl3ABZ/MA=; b=OtXkM06C9n0fYjvldgVOuWR68eE1zXI0g/SyucFznY+K0ar5BKEeSd4gApI1c0lDEL8W2C mltGciSjOryWY7gQ2aMP4mH0olPKrDmzpm2V/WbDpJRSjNJFAQWc8ZgVzbaHKa9QPwMjvP mr/UQWBhEPAqehmNgC+EtnwYAOUO2TY= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-387-d4etzWCfPMeGZLUscoiyOg-1; Thu, 20 Jan 2022 03:42:26 -0500 X-MC-Unique: d4etzWCfPMeGZLUscoiyOg-1 Received: by mail-ed1-f71.google.com with SMTP id h21-20020aa7c955000000b0040390b2bfc5so5215998edt.15 for ; Thu, 20 Jan 2022 00:42:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent :content-language:to:cc:references:from:organization:subject :in-reply-to:content-transfer-encoding; bh=XKfas3q2Y/HkFD5WHzkZNGqSak03+yPlc1Wl3ABZ/MA=; b=kFDAgGX2RslS5DRZB9OdvQty3ot2ANB8ShJWtTdBdeAGntepm/Ui8Qcg80jr8UsEjX /ERyYDwfG1qpAaLhv6SAwji2GUyr4eYW2JIaNTdFilhE1Y6IIb/M8C+x/MVnJgTk5xIS 4HOsQ2IzPHX9FUq8RUmMOV5Zk1wiwXgKMejBM5pypMB/gKZ+NhgfsZS7+GbNd/bA+8QD sYRsb/aX+ABZVdeSWYp7j482p2cCbkJHUD+I+ne4zm5Qp+oBDhHfn4MHwJ8K/YaFmau8 nEIzids4S7Mnwx0cke9x8bNEItT+xtcmsJKdYNqANZaIDo5xTRmzHByHHZiGaQvYElrV 1pkA== X-Gm-Message-State: AOAM530duob2cMFMo+t7FsmA0+XsnxPa2rXs20UvA/ynd37Ve3CzZHva SP3baNY7FC5t4t47O/3IrJpta/gNOlYxK22t3L8QE2YhergzrxC8I9VSGe0DgxDQ3L9W3rn9dPD 5NnGy6IvEMzI= X-Received: by 2002:aa7:db41:: with SMTP id n1mr34824564edt.307.1642668145198; Thu, 20 Jan 2022 00:42:25 -0800 (PST) X-Google-Smtp-Source: ABdhPJx/ka5nYE9RQxAfTQqOQRXjh4/GUPE9pMARacpmyi60VpyoWD50I04ufeRwvZBALw0xRZjWhQ== X-Received: by 2002:aa7:db41:: with SMTP id n1mr34824547edt.307.1642668144966; Thu, 20 Jan 2022 00:42:24 -0800 (PST) Received: from ?IPV6:2003:cb:c70e:5800:eeb:dae2:b1c0:f5d1? (p200300cbc70e58000eebdae2b1c0f5d1.dip0.t-ipconnect.de. [2003:cb:c70e:5800:eeb:dae2:b1c0:f5d1]) by smtp.gmail.com with ESMTPSA id e15sm984952edy.46.2022.01.20.00.42.24 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 20 Jan 2022 00:42:24 -0800 (PST) Message-ID: <644356e5-2a85-fcea-2280-ff779ae8d38d@redhat.com> Date: Thu, 20 Jan 2022 09:42:23 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.4.0 To: Minchan Kim , Michal Hocko Cc: Andrew Morton , linux-mm , LKML , Suren Baghdasaryan , John Dias References: <20211230193627.495145-1-minchan@kernel.org> From: David Hildenbrand Organization: Red Hat Subject: Re: [RESEND][PATCH v2] mm: don't call lru draining in the nested lru_cache_disable In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 03972100007 X-Stat-Signature: q67nsptxcy36bkxaqb9b5mu1zjc133yh Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OtXkM06C; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf05.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=david@redhat.com X-HE-Tag: 1642668147-659940 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 19.01.22 01:12, Minchan Kim wrote: > On Mon, Jan 17, 2022 at 02:47:06PM +0100, Michal Hocko wrote: >> On Thu 30-12-21 11:36:27, Minchan Kim wrote: >>> lru_cache_disable involves IPIs to drain pagevec of each core, >>> which sometimes takes quite long time to complete depending >>> on cpu's business, which makes allocation too slow up to >>> sveral hundredth milliseconds. Furthermore, the repeated draining >>> in the alloc_contig_range makes thing worse considering caller >>> of alloc_contig_range usually tries multiple times in the loop. >>> >>> This patch makes the lru_cache_disable aware of the fact the >>> pagevec was already disabled. With that, user of alloc_contig_range >>> can disable the lru cache in advance in their context during the >>> repeated trial so they can avoid the multiple costly draining >>> in cma allocation. >> >> Do you have any numbers on any improvements? > > The LRU draining consumed above 50% overhead for the 20M CMA alloc. > >> >> Now to the change. I do not like this much to be honest. LRU cache >> disabling is a complex synchronization scheme implemented in >> __lru_add_drain_all now you are stacking another level on top of that. >> >> More fundamentally though. I am not sure I understand the problem TBH. > > The problem is that kinds of IPI using normal prority workqueue to drain > takes much time depending on the system CPU business. > >> What prevents you from calling lru_cache_disable at the cma level in the >> first place? > > You meant moving the call from alloc_contig_range to caller layer? > So, virtio_mem_fake_online, too? It could and make sense from > performance perspective since upper layer usually calls the > alloc_contig_range multiple times on retrial loop. > ^ I actually do have something like that on my TODO list. The issue is that we have demanding requirements for alloc_contig_range(), discussed in the past for CMA bulk allocations: (1) Fast, unreliable allocations Fail fast and let caller continue with next allocation instead of retrying. Try to not degrade system performance. (2) Slow, reliable allocations Retry as good as possible. Degrading system performance (e.g., disabling lru) is acceptable. virtio-mem is usually (2), although there could be some use cases where we first want to try (1) -- unplug as much memory as we can fast -- to then fallback to (2) -- unplug what remains. CMA bulk allocations are (1). "Ordinary" CMA is mostly (2) I'd assume. -- Thanks, David / dhildenb