From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A0BBC4332F for ; Thu, 9 Nov 2023 12:21:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 45E088D00D5; Thu, 9 Nov 2023 07:21:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E6DE8D0073; Thu, 9 Nov 2023 07:21:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2AE958D00D5; Thu, 9 Nov 2023 07:21:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 17D488D0073 for ; Thu, 9 Nov 2023 07:21:08 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id DEC94C065C for ; Thu, 9 Nov 2023 12:21:07 +0000 (UTC) X-FDA: 81438325374.26.004480F Received: from mail-oo1-f52.google.com (mail-oo1-f52.google.com [209.85.161.52]) by imf23.hostedemail.com (Postfix) with ESMTP id E22E1140021 for ; Thu, 9 Nov 2023 12:21:05 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IqDUHbon; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of mjguzik@gmail.com designates 209.85.161.52 as permitted sender) smtp.mailfrom=mjguzik@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699532465; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Cf06u8ZRup48FiodSvGNoaJfH2Amt2aPrTV0aUl6vho=; b=pmxhBn0t40poO4RijVt26vuqPsZUnQNdIrniUPaYXBqL/0lOhUlJqX12IWUvPkQEtMvUdx Zcot8dCUNIOObggcA3dzzhg2/x90hbMOzMUgKp2gCbITRDLxFQhS/amWdttKz/7m3ZPMB1 QyFDD/HGWkvKwRznDWm3LHeh8Xvturs= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IqDUHbon; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of mjguzik@gmail.com designates 209.85.161.52 as permitted sender) smtp.mailfrom=mjguzik@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699532465; a=rsa-sha256; cv=none; b=t4InxuHmqb2orPSYh8PPwqVJyzISOWdstzxY7KUgHdffkTMsowMRtX1XH7s+jos/sObavZ GMg4tzuv7bhMEOD+pHEIpgXMJbJKklB8UGl1lK6Vvb/c/50WkI7DCk2NUc/e58E+9SWoQi JH1H4TdB2vpSPwLPk0YJVtxYxiGeJuA= Received: by mail-oo1-f52.google.com with SMTP id 006d021491bc7-581edcde26cso436644eaf.1 for ; Thu, 09 Nov 2023 04:21:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699532465; x=1700137265; darn=kvack.org; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Cf06u8ZRup48FiodSvGNoaJfH2Amt2aPrTV0aUl6vho=; b=IqDUHbonzY51HVZKw++W0nCqsaDVo19u9RVlzfyw9J0Ldo3hrA9kTeuY6ikkIsZG+U 0Z8Zy3Shj3EGDX9rVTcDaGrThB8HbtCT5bF+7BMgC6mD7EXbM5AQik43VW2XkRCkjf/9 4sGiR2oLDLnc0Lpi2tyv9Se0abcKL/mgnZ81S70h6y4OCrAITFZMZTUqfalb8EVZMDs0 nuzOVPzFbxskBzBT0XUgx5HZT6HivV7TB75UZVRE9XqPN0UAuPnPxG/C1rJxT/FdKEIp 915p75LSwPiC3WBi7KLr6kK/j9rllQZtzvC4ItMtEjhg9ELkxmkktIwRZuS+2Ls+amHi Cltw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699532465; x=1700137265; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Cf06u8ZRup48FiodSvGNoaJfH2Amt2aPrTV0aUl6vho=; b=fN0FJYJRsxuvZG4HXXKqbwvqR+3KIfMBQqMhSQ/UaYn7KRhVjU/pqFeohmgKXkTlMj AZtT0Ht/+rXghpJPIOG2YUjwUJAoJkEHkLro0rNHON7iHwrz4N3SAmGdLn88ADujIPLH NWG8ULRcufgo/9MD5bt4LjmMbP6yEPxH3aaKw0yw9xWqMsSy3LKzLSCaAxkOM46N2LEg fDokUXpa7LkCqyq/XK5RNoXYGOjeMj8n2knVHR8y8/eI+awauoAx1VqJD68klGrbvJBk 9NQy2Zy3bA0hbMBQUlpuprfG+Usc38llNZ4D65XOSVj7kTDna9hqvjP1GooDP3sQ0kvO FDag== X-Gm-Message-State: AOJu0YyCsicuhEDn7U3dFBQn3i3m9kYkZ9/lZhsJewToFDaJjMFb9dma k2jUezZFRY5ozLQjxSl+BAw0Dh7DQJBb6YsvBMQ= X-Google-Smtp-Source: AGHT+IHSCIfAvxAsoOhCmuC/wS5IjQ8yNasMvbesR4csTwSDuItjtE1BiSf3ph5q+6Ne9SxAA+PV13TTUBc5f1h52dg= X-Received: by 2002:a4a:d35a:0:b0:581:ec87:edc0 with SMTP id d26-20020a4ad35a000000b00581ec87edc0mr4683338oos.9.1699532464870; Thu, 09 Nov 2023 04:21:04 -0800 (PST) MIME-Version: 1.0 Received: by 2002:ac9:6696:0:b0:4f0:1250:dd51 with HTTP; Thu, 9 Nov 2023 04:21:04 -0800 (PST) In-Reply-To: <87msvnwzim.fsf@email.froward.int.ebiederm.org> References: <5c7333ea4bec2fad1b47a8fa2db7c31e4ffc4f14.1663334978.git.josh@joshtriplett.org> <202311071228.27D22C00@keescook> <20231107205151.qkwlw7aarjvkyrqs@f> <202311071445.53E5D72C@keescook> <202311081129.9E1EC8D34@keescook> <87msvnwzim.fsf@email.froward.int.ebiederm.org> From: Mateusz Guzik Date: Thu, 9 Nov 2023 13:21:04 +0100 Message-ID: Subject: Re: [PATCH] fs/exec.c: Add fast path for ENOENT on PATH search before allocating mm To: "Eric W. Biederman" , Peter Zijlstra Cc: Kees Cook , Josh Triplett , Alexander Viro , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: E22E1140021 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 5jd4aaymoncc9p3p47qhmjqehdgxj35h X-HE-Tag: 1699532465-986950 X-HE-Meta: U2FsdGVkX18meSsrIjiTeqs92hrhFkL158v3SH8pCAnDsvEmHQnpI/ez4ZirqaJLrClC4cYogm9SNUGzQNMiD33N73d4WeDU2ividSeUbTyDyJRUgBDhqeo2/l3axeBzO14jBBg0LKWJNGXVApFTXWkc0U/jIeha0A7JZ0ZKD0QOoEPVmimOD5nWiQdS/yVjnKaFjWDpgJJc0sXBytrP/5Att38Sj5F778Oj4xFrpx4tQVwFY2eEKJVXes0Iy+8HiFWQF0EfF533iR1LCV07KaqDkh54dCF524DqFYlcUU6RpozJOUBihH9F/eZdb6yLdH+eOPrZKVzPk/b7E24+YzFAqWuo448lRPbpn9P58GLwaAl6bJuGeK2bUBrXcpi4F8PlXDvUVIg+VkYA6d36gNWVplTJf55Dpd5PTb82+fK8PX5LNdIQHNWjmmTrg590ScA99kCRSxbZpyzHil4a7gdLR3d1hpRyeo7z1tnb8FDnsPeMHG75KTfCZFEricfwOafEwLV2blmAboNVmmtt1PVpRTlwz7rIvKiGlGCWHrY6FCZvuEut1QvbGjUBBbffLKIy5e83DyUg4nQ7lHNvS3JOvXlribKmXovixp+DO5ywRd2oX3BKCkj7P8YbctWQB8HL/TveCFI7S62zEtPOCBYOjOXl6ARnf8LyObvpVeSpOgtJ43q7N3jV8s5Vi0HBPSFxCpDvZpIcXCirUFj+ArQY43ighu9Wl7X3tsHTFTDiN/CObLz6mBwbLf4DZzpiEuRy9hsQhjhchTcxrEEIB7caiObCP9itZQjC8m3mLk47DPxNrcDJiF9PHfwnc/O4n53GLGd2k8ta8ucsLRcmrjmfvHWt7xBo1YX6kwzpQNBoGM2k/uRLPt8Of42DegbgaOe3CCutgPgtRWMBP9uIwnbzHS7Te0dGEitcVz2HG1OHYMQsQ+uDrif/lNIv9h/G9bhMeg1GGuNVjesxIsk IafUBM8H 7LLXIQ9aj7+E5KZJUJudnkAUCTcRqzIkWNci4qwJkJAkLSPjEy0WccOgBnUMxy98KLjJukmens7umk4IBh0V49ckJY5HVmin+0qQYNb5mvaSviaYRG3vfBYkrJ9mTz/SVnfYJNx69LxNnUoxmBS/ef0EcVNjhT8bDD8qSmoKfxUxv1QmbADuaZBkL5G3S0JsrCVdneOWGU5UolBS1bhpTVrycqeB8eGe1PXqK3cshRdOiV8KREnAUwDiEL5EKn9Q0s81q55gv33HAJLgEuY8TNAbo6wn3gVtG+CdmTar5jg0KGe+LrytoFyrXk0TSWcQZRLQZAUPWrSo6WQ2aGmYY76vd4Qvm8KyoZOkKhhY20VD+lmpHMWJe1GucRfu0u3iSAvT2deRiSIXlzIBgRla52Fyr/PoUrPJiqTkk280cH5YR8NoEav8xBh1J5dBbmHvcsCq7D7I/1UMi/lEZ3Dm6ulYueGWvVYNoiHfjPlUGb/DULqIrgWCJDwrRZA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 11/9/23, Eric W. Biederman wrote: > Mateusz Guzik writes: >> sched_exec causes migration only for only few % of execs in the bench, >> but when it does happen there is tons of overhead elsewhere. >> >> I expect real programs which get past execve will be prone to >> migrating anyway, regardless of what sched_exec is doing. >> >> That is to say, while sched_exec buggering off here would be nice, I >> think for real-world wins the thing to investigate is the overhead >> which comes from migration to begin with. > > I have a vague memory that the idea is that there is a point during exec > when it should be much less expensive than normal to allow migration > between cpus because all of the old state has gone away. > > Assuming that is the rationale, if we are getting lock contention > then either there is a global lock in there, or there is the potential > to pick a less expensive location within exec. > Given the commit below I think the term "migration cost" is overloaded here. By migration cost in my previous mail I meant the immediate cost (stop_one_cpu and so on), but also the aftermath -- for example tlb flushes on another CPU when tearing down your now-defunct mm after you switched. For testing purposes I verified commenting out sched_exec and not using taskset still gives me about 9.5k ops/s. I 100% agree should the task be moved between NUMA domains, it makes sense to do it when it has the smallest footprint. I don't know what the original patch did, the current code just picks a CPU and migrates to it, regardless of NUMA considerations. I will note that the goal would still be achieved by comparing domains and doing nothing if they match. I think this would be nice to fix, but it is definitely not a big deal. I guess the question is to Peter Zijlstra if this sounds reasonable. > Just to confirm my memory I dug a little deeper and I found the original > commit that added sched_exec (in tglx's git tree of the bit keeper > history). > > commit f01419fd6d4e5b32fef19d206bc3550cc04567a9 > Author: Martin J. Bligh > Date: Wed Jan 15 19:46:10 2003 -0800 > > [PATCH] (2/3) Initial load balancing > > Patch from Michael Hohnbaum > > This adds a hook, sched_balance_exec(), to the exec code, to make it > place the exec'ed task on the least loaded queue. We have less state > to move at exec time than fork time, so this is the cheapest point > to cross-node migrate. Experience in Dynix/PTX and testing on Linux > has confirmed that this is the cheapest time to move tasks between > nodes. > > It also macro-wraps changes to nr_running, to allow us to keep track of > per-node nr_running as well. Again, no impact on non-NUMA machines. > > > Eric > > -- Mateusz Guzik