From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC2B7C4332F for ; Wed, 8 Nov 2023 19:35:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 36C8B80024; Wed, 8 Nov 2023 14:35:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 31C3C8D0073; Wed, 8 Nov 2023 14:35:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E7F180024; Wed, 8 Nov 2023 14:35:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0E4A88D0073 for ; Wed, 8 Nov 2023 14:35:59 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id CFA37A0BCD for ; Wed, 8 Nov 2023 19:35:58 +0000 (UTC) X-FDA: 81435792396.15.A79FF5E Received: from mail-oi1-f177.google.com (mail-oi1-f177.google.com [209.85.167.177]) by imf05.hostedemail.com (Postfix) with ESMTP id 1536010000B for ; Wed, 8 Nov 2023 19:35:56 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=d6w7Ergq; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of mjguzik@gmail.com designates 209.85.167.177 as permitted sender) smtp.mailfrom=mjguzik@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699472157; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=paNOpk393l1aqFflxT4hXJVPVnwtDZPbYHqWDS7Ayz0=; b=gi/4SJTP49UJXWxtguhq+WczksgaTC5WqPJyy04BCvvYFAjxej3igN6LyUp25PWK2MOKqP jZ/KHu3f4DsoEgBGv3jtXWU4V+Y1LqMHTEYnI/Q+5LvEqArtOziXBrJ4qeBHpZLYpYYKkI riv9U8yiG8K9paCgZfhL8TF+e4JFzSw= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=d6w7Ergq; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of mjguzik@gmail.com designates 209.85.167.177 as permitted sender) smtp.mailfrom=mjguzik@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699472157; a=rsa-sha256; cv=none; b=oXt4KxTzt9P5qms2qOxVmqapS2wDoHe53iKx6+aKxXyJvWVXXHPBO5oWpg83g8Fydf8k2g gpR78NcgP0HebX2A5J+7luBSocIHHlgeEThiyoZohA5qqCd2za4E48jFUwPYU7ZmeAC8Vq ohI4B7l1hrYp/nO9/jJcaqd5FiKtzL8= Received: by mail-oi1-f177.google.com with SMTP id 5614622812f47-3b2f507c03cso28370b6e.2 for ; Wed, 08 Nov 2023 11:35:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699472156; x=1700076956; darn=kvack.org; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=paNOpk393l1aqFflxT4hXJVPVnwtDZPbYHqWDS7Ayz0=; b=d6w7ErgqZlgZwEjpZCPyN+xPE2yzGN2xr2UydvJJMITUEN3uQzQINhhx6UuhTbL7cL aum8fuba0QzRxjrobqEEpg3ZDyAkUpuOLA6N0WrHwXiO2uD9NkPx19Ebqml2T93uEUoh DQOwtgYV10N850uEfyJ9rLh2txAhYuBKjwwZNohyUzGWbIPnDf4eKSFlaTLVSaoR7dYf TgnMdqCzYIoGXQ+exPtM5KOfiBMkmYFvNtueLHYEiPqkHAkdKzNPGv9j1LU9SwRuhOVB +mEo6Xad8HJYE36DdJZYSwUBldcUjdLbVRWr7aXJ2g5PgmJMbF4eyeKn29E0AgUICNc8 hkIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699472156; x=1700076956; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=paNOpk393l1aqFflxT4hXJVPVnwtDZPbYHqWDS7Ayz0=; b=Efa+4w+VtZqr4UfHNW3kLv2sEbavpkViLWQcviQ8780uBRbk6oEDmSHG7iYxppQquo SK2aqB8YLnUWDzc33DAURJXPJb7cY65/7ALztQznpdAw9Nwog5I3AOhyoC2vQyYP5bfq jM3erteSx/mVFOheknd15z01KFiaEOiVhLm0QZkK4wHD1VtDGNPxYEQlMRYC37nTKqc0 /nlz2ckv6TSI3S3S8EncbQ5StRsi+ujIWZrcO3bgaCU5qBpM1zb1W2JJ7Rbmc3NymHAu RnuXiC6tuakfxAccnlkOU5cRC8P8sfyhEek9a5aBo2zaU/IGn8Tg1IlaLOf3Q7WMUXx9 qt1Q== X-Gm-Message-State: AOJu0YydsLBXf0CQpjRHpXqtfrF2RNN3buxMx5Pa0v1UvsHIuwA98AYU jXt5zW15DvwV5XNUP+oy3bnu4EBC0S37+jIQDhU= X-Google-Smtp-Source: AGHT+IH6gyK4ACWGImYctZv4/as81sCqw8qfMghC+jEYWgfUdrQXQm8Qvqu7Sa41ug1rQgloLFib/7CUGSZCXCZjHdE= X-Received: by 2002:a05:6808:34b:b0:3b5:2ede:d9c1 with SMTP id j11-20020a056808034b00b003b52eded9c1mr2950312oie.47.1699472156262; Wed, 08 Nov 2023 11:35:56 -0800 (PST) MIME-Version: 1.0 Received: by 2002:a8a:158f:0:b0:4f0:1250:dd51 with HTTP; Wed, 8 Nov 2023 11:35:55 -0800 (PST) In-Reply-To: <202311081129.9E1EC8D34@keescook> References: <5c7333ea4bec2fad1b47a8fa2db7c31e4ffc4f14.1663334978.git.josh@joshtriplett.org> <202311071228.27D22C00@keescook> <20231107205151.qkwlw7aarjvkyrqs@f> <202311071445.53E5D72C@keescook> <202311081129.9E1EC8D34@keescook> From: Mateusz Guzik Date: Wed, 8 Nov 2023 20:35:55 +0100 Message-ID: Subject: Re: [PATCH] fs/exec.c: Add fast path for ENOENT on PATH search before allocating mm To: Kees Cook Cc: Peter Zijlstra , Kees Cook , Josh Triplett , Eric Biederman , Alexander Viro , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 1536010000B X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 77rf11xgsiickbykj8sju8qq99cdgmfr X-HE-Tag: 1699472156-341591 X-HE-Meta: U2FsdGVkX19cFfRI9/4qb+LFXJYvEpbzhAKm2Q8Qf+a+Lzrqj6PCK0SlJn4hX1XDSId5mmHiwQt16V6fJ1QK++EBuCik6JPS9fK2WPmf9+Z5K9Tbf3A0SrzHYB5w2jhCuUKQ4fwDr1nlMzcBhcmW34QiHa6EksL7UxgaWqSn77IPKRQn83hFcMvBYAvQ9x2+EoAJ9v92li6zbgjud2r/KXAnjjfjNH701zrHjlcyDoMmPU3vxxfpD5pk4ES59eMSER9yqVamM5z6rPtl/K7SuQDe7lp6dtOFlYWl1mSKA1Bvg5xkqE/lh0e+FKugiIytONo+7CkOT2J1cSE2hSh9lXBD1oxHN8kd4bwyxyRPAykh9ggHqjUKz/8nyhhz5Sx5R28ZxtzsuZmS1ljvsYHu+n2ZgbgxpNZbhv4wKpMpsJ0ZMttfwIG83R46CJ+0Kij6G/llJF4t9D3ZElEakEqZ4OSNSbQDjYjhuZcULukuW0ePCdTXSWOwheCmITXXO2y98ZVH9uOLBRBpgbJGsHVmxkc7NT8kCY8Nt7kLfJDclgtazgC8Q2goc9MaW0mv1aAdy+YUetJ5NV34yv0PYbR/08Lb5KqrQJajW2+0SlUxS/uokkev5dMW9akYjFFtsPlVaLE4NQwaWWIYlRKSRAiwLBinN4w69CN049nTkh3KMHkLcuwIJK9j6FDjX/Wfwt4CGejYWN63qJgC3a9HMFqGDixM8TlZ3V1w8uTQkJjU2ByMA0Z1OP4chrpgeNCauqIjoD7fLk59IlzphObuNY3bnNK7l0udpL7MYFQc30zjtlKoGMplEV23H3IdWgBC/vQR0BY9w5Z0JaS30rTcDHV172kD/qGOkTSKE9KOzE/iZR8mRHiMQo10z6ZgtsRQA6MIs5XB5iEp5tA7Mdir7bA8JrB0JsLwDoDlgh+49Syz+eTAquwexVKEAEFvmAybXC4NMT+V/ta5ZepaIPE+9Wv 5392MMfz FCPFyiGLrkTRQndk480T1lzW2M2El+HZXPFJdnw94VPW6+qCOZuUQLL8h1eEuv9b352YrmEhyKoQM8A4ul/Osa3/cF2lh4ewZL/ItVSj9VovLr0qVC4pMQ5cwyucU+mSn295C89pu4TVuCHMCxD5Y+tDVBNICdVEYC+Av0D88wAu54+rgH3fa7T7JNQ995JLw9kJZTcnIEU1/K1TmVnEbrLWpbANHaVj9Wqzqp0xQMn8Gp/KqpK/cA4PCQbgientmiW2ro79QgF+D6DXnifam63bsV6p7mV12TOK1lFS9iXUMW/FfLP8TZqU8e3YiiQ74Wvvih/2Q60hai36QFdQUu5gbB9XbJuE69rA7FzanBLQPfDHSNwTgWIx/UValkSvXMEkphcyHHp4mDgqlOIszPAF8QUWQcjORFKJlm155/uU7s8qunzErtODOUi4BgGqv0iAIyyPioHUq1JP7TbC8swIrBDIt+KZTZ1rigI1sG3bARZV2LWCxm4O+z6Sl3mDiaz7kkQoRl9wLJVVJ+QpwJbHb3kgzssXKrsB152fl0k6H7/E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.002081, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 11/8/23, Kees Cook wrote: > On Wed, Nov 08, 2023 at 01:03:33AM +0100, Mateusz Guzik wrote: >> On 11/8/23, Kees Cook wrote: >> > >> > >> > On November 7, 2023 3:08:47 PM PST, Mateusz Guzik >> > wrote: >> >>On 11/7/23, Kees Cook wrote: >> >>> On Tue, Nov 07, 2023 at 10:23:16PM +0100, Mateusz Guzik wrote: >> >>>> If the patch which dodges second lookup still somehow appears slower >> >>>> a >> >>>> flamegraph or other profile would be nice. I can volunteer to take a >> >>>> look at what's going on provided above measurements will be done and >> >>>> show funkyness. >> >>> >> >>> When I looked at this last, it seemed like all the work done in >> >>> do_filp_open() (my patch, which moved the lookup earlier) was heavier >> >>> than the duplicate filename_lookup(). >> >>> >> >>> What I didn't test was moving the sched_exec() before the mm >> >>> creation, >> >>> which Peter confirmed shouldn't be a problem, but I think that might >> >>> be >> >>> only a tiny benefit, if at all. >> >>> >> >>> If you can do some comparisons, that would be great; it always takes >> >>> me >> >>> a fair bit of time to get set up for flame graph generation, etc. :) >> >>> >> >> >> >>So I spawned *one* process executing one statocally linked binary in a >> >>loop, test case from http://apollo.backplane.com/DFlyMisc/doexec.c . >> >> >> >>The profile is definitely not what I expected: >> >> 5.85% [kernel] [k] asm_exc_page_fault >> >> 5.84% [kernel] [k] __pv_queued_spin_lock_slowpath >> >>[snip] >> >> >> >>I'm going to have to recompile with lock profiling, meanwhile >> >>according to bpftrace >> >>(bpftrace -e 'kprobe:__pv_queued_spin_lock_slowpath { @[kstack()] = >> >> count(); }') >> >>top hits would be: >> >> >> >>@[ >> >> __pv_queued_spin_lock_slowpath+1 >> >> _raw_spin_lock+37 >> >> __schedule+192 >> >> schedule_idle+38 >> >> do_idle+366 >> >> cpu_startup_entry+38 >> >> start_secondary+282 >> >> secondary_startup_64_no_verify+381 >> >>]: 181 >> >>@[ >> >> __pv_queued_spin_lock_slowpath+1 >> >> _raw_spin_lock_irq+43 >> >> wait_for_completion+141 >> >> stop_one_cpu+127 >> >> sched_exec+165 >> > >> > There's the suspicious sched_exec() I was talking about! :) >> > >> > I think it needs to be moved, and perhaps _later_ instead of earlier? >> > Hmm... >> > >> >> I'm getting around 3.4k execs/s. However, if I "taskset -c 3 >> ./static-doexec 1" the number goes up to about 9.5k and lock >> contention disappears from the profile. So off hand looks like the >> task is walking around the box when it perhaps could be avoided -- it >> is idle apart from running the test. Again this is going to require a >> serious look instead of ad hoc pokes. > > Peter, is this something you can speak to? It seems like execve() forces > a change in running CPU. Is this really something we want to be doing? > Or is there some better way to keep it on the same CPU unless there is > contention? > sched_exec causes migration only for only few % of execs in the bench, but when it does happen there is tons of overhead elsewhere. I expect real programs which get past execve will be prone to migrating anyway, regardless of what sched_exec is doing. That is to say, while sched_exec buggering off here would be nice, I think for real-world wins the thing to investigate is the overhead which comes from migration to begin with. -- Mateusz Guzik