From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EBCBC4332F for ; Thu, 8 Dec 2022 15:33:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79D268E0003; Thu, 8 Dec 2022 10:33:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 74D068E0001; Thu, 8 Dec 2022 10:33:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 63CBF8E0003; Thu, 8 Dec 2022 10:33:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 563C28E0001 for ; Thu, 8 Dec 2022 10:33:10 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 233AB1C62C4 for ; Thu, 8 Dec 2022 15:33:10 +0000 (UTC) X-FDA: 80219532540.14.DD5842D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf30.hostedemail.com (Postfix) with ESMTP id 0D45E80017 for ; Thu, 8 Dec 2022 15:33:07 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=fPQiOELN; spf=pass (imf30.hostedemail.com: domain of jbrouer@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=jbrouer@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670513588; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=veFa/Wi52YvjG2I5lacfp9qTDhL6U2v2sBr7NJtbIyE=; b=uek+ID2lqcOiz6/kim5IBidvvj60+NXjgGR7cWCVQWu/PY99qbeRI4C4wyy+j1tVFDAVS+ vEsPgkbp6xdfYp7o3qmPKm2v0Ht7B54FyvqxnjyuEcAZrs46yRUeWS9T1resNwDwkMFW+T p5Od0OUfunhHCv1/JZObJrA23Y+u2p8= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=fPQiOELN; spf=pass (imf30.hostedemail.com: domain of jbrouer@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=jbrouer@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670513588; a=rsa-sha256; cv=none; b=V/rwQHwTZjfesZriXxE4hNPsw7eMYdcMeoKYrUyy6md7QD31pJcLXDIUq+yYcmA/KDU3FB 6AG5Jz/zweesk4n4QHJWUN+SmDfPajbN8ifWCnGNZ2rONk905Vz0UuBQV9oeDPdZRzej/j JYKqkq0b1Yc8xeZp4ID3OT5+i0XY1nc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670513587; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=veFa/Wi52YvjG2I5lacfp9qTDhL6U2v2sBr7NJtbIyE=; b=fPQiOELN1CgpS7apOczXCMDr7Fjv+ZXM7+o9z5gNZxEJ8nZFzJeyoZmiHbwsF/dd3UdAij uB1jETBvlPVmVvLjaw7sdcceiL3oGvEq+Rm7O2MTpy2+PopMGHvrJ50GW8Nl7y23gHQxXk 0Omd8IhxEvvZBR4Y02sd4ruOrusFVK8= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-516-sijSebv9PGmjafhn85HqZw-1; Thu, 08 Dec 2022 10:33:06 -0500 X-MC-Unique: sijSebv9PGmjafhn85HqZw-1 Received: by mail-ej1-f72.google.com with SMTP id hp16-20020a1709073e1000b007adf5a83df7so1391429ejc.1 for ; Thu, 08 Dec 2022 07:33:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:references:to :content-language:subject:cc:user-agent:mime-version:date:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=veFa/Wi52YvjG2I5lacfp9qTDhL6U2v2sBr7NJtbIyE=; b=yU9fSgApCQIkAv+QQ7nr1cJgprElTpIq1G3LXJ8PV1kg/cSdK2wUh0T91Tu0Dl6Mel bl/Kljn9O9dMlaIKgKzH1oany2UeTyvuM9fNmBHC42riZSIi+NESpryDtJWlA9w39kWh /5Jm0N1KSdzOflUM1TwmCoszh4XJgZChetVdEUNRXV1e2lsTUB4H0Co6EPe0tGQl9gWR yfuIhDmjPETwFofaz3pprEmeQqUTcXp980jScZOQi2wgN119HWodJEO8atTLIds2ueCF OFUNNPTa++4iBbBu0Jj79W1b7llzclDCu2fnRE6z/uHH/dmGyyxYnv3KWt5sa6QjOM7o jmhw== X-Gm-Message-State: ANoB5pnfxX6yf5+am+pRTtI5PkyvYe/kQd1uNVAY/wP1FpR4jbRiNBdo MJlvnGxBwrbo2AxfV/I7cfCnOXdiXVKB8Ozke1UC00P/NMCq0enVkaXv1x7ZdIWQ+qhRmQM3aMb WYytVyYURLr4= X-Received: by 2002:a05:6402:3224:b0:46c:d2a3:76b3 with SMTP id g36-20020a056402322400b0046cd2a376b3mr3178653eda.14.1670513584822; Thu, 08 Dec 2022 07:33:04 -0800 (PST) X-Google-Smtp-Source: AA0mqf7q+zecuU1okPpu/XXxrQMV+ikyMSinHIewJwmedRM/A8FTYptz3/OWz8lyC3iQV3SffydjhA== X-Received: by 2002:a05:6402:3224:b0:46c:d2a3:76b3 with SMTP id g36-20020a056402322400b0046cd2a376b3mr3178617eda.14.1670513584409; Thu, 08 Dec 2022 07:33:04 -0800 (PST) Received: from [192.168.41.200] (83-90-141-187-cable.dk.customer.tdc.net. [83.90.141.187]) by smtp.gmail.com with ESMTPSA id f8-20020a056402150800b0046146c730easm3491739edw.75.2022.12.08.07.33.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 08 Dec 2022 07:33:03 -0800 (PST) From: Jesper Dangaard Brouer X-Google-Original-From: Jesper Dangaard Brouer Message-ID: <9e9af4ec-9bd2-8b10-c95a-4272442cb926@redhat.com> Date: Thu, 8 Dec 2022 16:33:02 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.5.1 Cc: brouer@redhat.com, Jesper Dangaard Brouer , Ilias Apalodimas , netdev@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH 00/24] Split page pools from struct page To: Matthew Wilcox , Jesper Dangaard Brouer References: <20221130220803.3657490-1-willy@infradead.org> <7cfbcde0-9d17-0a89-49ae-942a80c63feb@redhat.com> In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 0D45E80017 X-Stat-Signature: efiue6njoi3sdhkb1bezk5xf7j5dhfkd X-HE-Tag: 1670513587-662316 X-HE-Meta: U2FsdGVkX18DNhhNQfSZDyBSkbOw0iVh5gRdJJ0SkIanchHJzpDAvgRH526y6SV12z0vFvd0+pBDqBZZzqB4VjsOkCzDyO43a4r8upOo6hw6Ozyxw0W0G5Vsb3vo9NGe5Agz6RONjBhyw2iY0k6wy1vYpDqRFsSBZWXxLGVugsPz+iFLjXAwLJ5v2GpFDP0LCG8gn/VgDbpOZi2/dwRKV2u2VD8S1+ATBqW3k+mBWpgMvldSnapVhoPmJ1NU4zk4z15wy5Rjq7WpvGw+tcotUis10N6tFPsoeCq/kllWKDkYhe2f2nNVe13fndwPAadWgYVLeA7aGGNuRjf8rfVlYbH2XHvYmTRPnvkPmcz9iKGwIrjq9CyYXTdBMx5o+QBxbnXSDKvLXT+GrNU0RuRZq75o76pSBbvmVHOK1Fj/D6ZHRq0PFiUHSh5pMjcdeHtyERwbK/4sx1IpVHZVQZZuA8Tw73GQZwpeh90Vu7d4pPObTydMU4fJvflxnl3NegPB5j0TyK003A6NZFUUKH7jHDXFjXuv6Hi364HsN+ylGQG8Bvb2cagYYmlgUyXXys5vB4F7KgeHI+PiRtEKNvrrGuJxLO4sQOyqywmmPXVRGSD/kwQWbIW6pIWTX83Dcy4jphZKchFRNLpq8qcr3/xiJMvttZw0PMd823rPo4ITRd5R6yX1A/UtTIoqWHWLmKzqfFi6exdx2A3p2pGe3PYtSFS1AllgUNNgMPjQ/WeG3iBNT+lkPwlLBRIB5fr82R5JIDAjlIEd945Esr1xbS2FF2OCYheA6K5tqwcltd13SsqjhdLPX/KvMdJ8Ln3tK6PmHXup2qwfA/OpSPGwDFMQzoSf5WYnqGAvHoQNFKvntASARfGhvmF8vHhSWPW3IKYYxInpuOQzskeiMqOdyb6kxkNF6VBZN1VpgYcR6rVWZ7fOyfobCwGSjk3zBdBUeDHvXVRkxYbYSPsAJOmDwfU cx7cNPPP B72Wh9Gp57uteS10r7YpJBb+WAGuaUCW7N2DgKEZ/Ak4fdnsyKfEWIv3L8KIwG/nuNBVOsgcpJI6vJygP6HLT6z/TqPXDAboFxXTSrRXJwPc581s= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 06/12/2022 17.08, Matthew Wilcox wrote: > On Tue, Dec 06, 2022 at 10:43:05AM +0100, Jesper Dangaard Brouer wrote: >> >> On 05/12/2022 17.31, Matthew Wilcox wrote: >>> On Mon, Dec 05, 2022 at 04:34:10PM +0100, Jesper Dangaard Brouer wrote: >>>> I have a micro-benchmark [1][2], that I want to run on this patchset. >>>> Reducing the asm code 'text' size is less likely to improve a >>>> microbenchmark. The 100Gbit mlx5 driver uses page_pool, so perhaps I can >>>> run a packet benchmark that can show the (expected) performance improvement. >>>> >>>> [1] https://github.com/netoptimizer/prototype-kernel/blob/master/kernel/lib/bench_page_pool_simple.c >>>> [2] https://github.com/netoptimizer/prototype-kernel/blob/master/kernel/lib/bench_page_pool_cross_cpu.c >>> >>> Appreciate it! I'm not expecting any performance change outside noise, >>> but things do surprise me. I'd appreciate it if you'd test with a >>> "distro" config, ie enabling CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP so >>> we show the most expensive case. I've tested with [1] and [2] and the performance numbers are the same. Microbench [1] is easiest to compare, and numbers below were basically same for both with+without patchset. Type:tasklet_page_pool01_fast_path Per elem: 16 cycles(tsc) 4.484 ns Type:tasklet_page_pool02_ptr_ring Per elem: 47 cycles(tsc) 13.147 ns Type:tasklet_page_pool03_slow Per elem: 173 cycles(tsc) 48.278 ns The last line (with 173 cycles) is then pages are not recycled, but instead returned back into systems page allocator. To related this to something, allocating order-0 pages via normal page allocator API costs approx 282 cycles(tsc) 78.385 ns on this system (with .config). I believe page_pool is faster, because we leverage the bulk page allocator. --Jesper