From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD58AC43334 for ; Wed, 15 Jun 2022 20:56:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1CE9E6B0071; Wed, 15 Jun 2022 16:56:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 17ECD6B0072; Wed, 15 Jun 2022 16:56:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 046256B0074; Wed, 15 Jun 2022 16:56:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E2D276B0071 for ; Wed, 15 Jun 2022 16:56:08 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 4D57621223 for ; Wed, 15 Jun 2022 20:56:08 +0000 (UTC) X-FDA: 79581677616.15.C13022C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf17.hostedemail.com (Postfix) with ESMTP id 822BF40081 for ; Wed, 15 Jun 2022 20:56:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655326567; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PH7XzeH6549kHLQkqVcBn/PNBTAveXUrM8w/yvd/AgM=; b=IgK+1QbR3mQ7r8SW1IVGcHJbzK9g0igIQ+hiqOSqV97c+whpBVeO/boUX+HWmmh4i0UP+U HWI4d/opHo1rf1GXheRBW0u3NzD5+AiEWzQTkNj8oO1+dNkMcvoPTlObOi7sTJpV5/tcNp ru8B+ZTZ9fSuwIHMSVUaj4NARvrYBYg= Received: from mail-io1-f71.google.com (mail-io1-f71.google.com [209.85.166.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-82-cMdvcIxXMT6WeanKMeT24w-1; Wed, 15 Jun 2022 16:56:06 -0400 X-MC-Unique: cMdvcIxXMT6WeanKMeT24w-1 Received: by mail-io1-f71.google.com with SMTP id h4-20020a056602008400b0066a011ac3d6so4567392iob.14 for ; Wed, 15 Jun 2022 13:56:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=PH7XzeH6549kHLQkqVcBn/PNBTAveXUrM8w/yvd/AgM=; b=k1OlCSRdOngxADrzN3gDDwprvUZ3P3axS7OY2EJfvnblIBwRhK6mcuMzXapg+gt8MS rosu+EbhKTEpKTzwqwNgkyukxE/OY3AXWk+TgZm04lKJAJqI4FQSiTqt7rxcInf1+mAF 5FVdSY172wf75iJuFxvGwqqaESwOOCfpbpiuQLz5PQO08A4lDxfIYeGRKfS/iH2vO12D bpF7gOrN55uplcYIM3gl+nrdhLWv2z7GnvxHnyMyah7thqeBibuZ589NwScdLL4zcPry y6gcnHhURpS7hbdl31kT0VrXrQG9O9/fEpdG4IY+xr4/72tN0jGj1h26qstBO/tlysnb a5dQ== X-Gm-Message-State: AJIora81VpofPxX9iiJJbw30np/POeYe+snNwm9bxzOq6pLrQMid4m0B RPSfqj3LVUnaqiOCcrpLwy1FYjn2w7PNqs9t7EQAhdchvV/RJbA2DQzl8NjsEWX3OiqbGwr2mVE vqs5kR3iXpCc= X-Received: by 2002:a05:6638:12c4:b0:332:144f:899b with SMTP id v4-20020a05663812c400b00332144f899bmr901699jas.298.1655326565220; Wed, 15 Jun 2022 13:56:05 -0700 (PDT) X-Google-Smtp-Source: AGRyM1utDdUgcZNBRdEAlq9uqaoxrDpEldilPa4RNIOQFO9Gu+wbYhOXHvvOsTu7Cv+7YjE1Zoi9Ig== X-Received: by 2002:a05:6638:12c4:b0:332:144f:899b with SMTP id v4-20020a05663812c400b00332144f899bmr901691jas.298.1655326564990; Wed, 15 Jun 2022 13:56:04 -0700 (PDT) Received: from xz-m1.local (cpec09435e3e0ee-cmc09435e3e0ec.cpe.net.cable.rogers.com. [99.241.198.116]) by smtp.gmail.com with ESMTPSA id a3-20020a924443000000b002d3b759dc7fsm33197ilm.77.2022.06.15.13.56.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jun 2022 13:56:04 -0700 (PDT) Date: Wed, 15 Jun 2022 16:56:02 -0400 From: Peter Xu To: Nadav Amit Cc: Mike Rapoport , John Hubbard , David Hildenbrand , Linux MM , Mike Kravetz , Hugh Dickins , Andrew Morton , Axel Rasmussen Subject: Re: [PATCH RFC] userfaultfd: introduce UFFDIO_COPY_MODE_YOUNG Message-ID: References: <3eea2e6e-1646-546a-d9ef-d30052c00c7d@redhat.com> <481fc9d0-6122-bf59-9d04-23c10d256764@nvidia.com> <0BB58ACF-2801-4622-BF3B-9913A23AE46C@gmail.com> MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655326567; a=rsa-sha256; cv=none; b=G5MjirnEsVD2RccoV9ZXuvQV2idReObdCy9VCcXJSAiW0JS8FejmmLih3i4V05aeaC5ZUt JDG40/9nhd+Q6qVz9vnqmpratg/NyZI+lZgvRsD6oGtRkFPR6ALThFpZDvF+mjPJ1s+il0 KNj6ISh2xZ2JGB3Qgb0CpIeSUOWRV/4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IgK+1QbR; spf=none (imf17.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655326567; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PH7XzeH6549kHLQkqVcBn/PNBTAveXUrM8w/yvd/AgM=; b=2Gbgi5WYRh+NKnm+7Oejw6QAkmoyuFLgT6H020cVC8QUyeJ6N+XHjmHtIB8OoQdoGgYqNj rzT6XMcZ/8+0Q79NDB86viH2Z8tfkMdL/OR5cDoQgl5SM1kl4YjFgqb6dP3LNNlJFzNmOl 2/DjaQwSU7RueCHtMw86Mf/O2fk3vmU= X-Stat-Signature: m3neurw3hc5wy4mzphdybwb31d9k5uz9 X-Rspamd-Queue-Id: 822BF40081 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IgK+1QbR; spf=none (imf17.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam07 X-Rspam-User: X-HE-Tag: 1655326567-377118 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jun 15, 2022 at 12:42:13PM -0700, Nadav Amit wrote: > >> (3) also has the downside of stack-protector that would be added due to > >> stack-protector strong, which is not-that-bad, but I hate it. > > > > Any short (but further) explanations? > > Sure, but it might not be short. > > So one time Mathew Wilcox tried to convert some function arguments into a > struct that would hold the arguments and transfer it as a single argument, > and he found out that the binary size actually increased. Since I like to > waste my time, I analyzed why. > > IIRC, there were two main reasons. > > The first one is that the kernel by default uses the “strong” > stack-protector. It means that not all functions would have a stack > protector, and actually only quite few would. One main reason that you have > a stack protector is that you provide dereference a local variable. So if > you have a local struct that hold the arguments - you get a stack protector, > and it does introduce slight overhead (~10ns IIRC). There may be some pragma > to prevent the stack protector, but clearly I will be shot if I used it. > > The second reason is that the compiler either reloads data from the struct > you use to hold the arguments or might spill it to the stack if you try to > cache it. > > Consider the following two scenarios: > > A. You access an argument multiple times: > > local1 = args->arg1; > another_fn(); // Or some store to the heap > local2 = args->arg1; > > // You use local1 and local2 > > In this case the compiler would reload args->arg1 from memory, even if there > is a register that holds the value. The compiler is concerned that another_fn() > might have overwritten args->arg1 or - in the case of a store - that the value > was overwritten. The reload might prevent further optimizations of the compiler. > > B. You cache the argument locally (aka, you decided to be “smart”): > > arg1 = args->arg1; > local1 = arg1; > another_fn(); > local2 = arg1; > > You may think that this prevents the reload. But this might even be worse. > The compiler might run out of registers spill arg1 to the stack and then > access it from the stack. Or it might need to spill something else, or > shuffle registers around. > > So what can you do? You can mark another_fn() as pure if it is so, which > does help in very certain cases. There are various limitations though. IIRC, > gcc (or is it clang?) ignores it for inline functions. So if you have an > inline function which does some write that you don’t care about you cannot > force the compiler to ignore it. > > Note that at least gcc (IIRC) regards inline assembly as something that might > write to arbitrary memory address. So having BUG_ON() would require a reload > of the argument from the struct. Ah, I never knew that side of BUG_ON().. > > You may say: “what about the restrict keyword?” Well, this is nice in > theory, but it does not always help and the implementation of gcc and clang > are not even the same in this regard. IIRC, in one of them (I forgot which), > the restrict keyword will prevent the reload if instead another_fn() there > was a store, but once you call a different function, that compiler says “all > bets are off, I ignore the restrict” and would reload/spill arg1 (depending > on A or B). Thanks, I understand now. I did a check-up and my most-used distro (fedora) doesn't really have the strong stack protector enabled.. centos/rhel has. It seems to me besides the two points you mentioned above to increase the obj being generated (and also the performance impact), afaiu it also shrinks the number of vars to push/pop, so even if the lump-sum would be similar to before there's still a valid reason to have it since it provides more safety on stack checks. But I really am not an expert on this so I'd not be able to make any appropriate conclusion or provide any useful input.. It's just that it'll not be a question to answer to uffd code but a more general question, as using a struct pointer to pass over things are really common, afaict, in not only mm but the Linux repository as a whole. -- Peter Xu