From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAEBEC47DA9 for ; Tue, 30 Jan 2024 05:20:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DBDE6B0095; Tue, 30 Jan 2024 00:20:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 78B0F6B0099; Tue, 30 Jan 2024 00:20:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 654FA6B009A; Tue, 30 Jan 2024 00:20:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 568006B0095 for ; Tue, 30 Jan 2024 00:20:37 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id F1A1D1C0EB8 for ; Tue, 30 Jan 2024 05:20:36 +0000 (UTC) X-FDA: 81734827272.22.4B85052 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) by imf05.hostedemail.com (Postfix) with ESMTP id 94277100014 for ; Tue, 30 Jan 2024 05:20:33 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="B/GxCT7t"; spf=pass (imf05.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706592034; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iWiiw0yLlRIv8CS7rk5QZH/NK8+eF5jsc85rziPnGps=; b=HkiwuyiB79P0CAvMUTKCB7wDDleujKSwMNRGw+MtrBxZaJRLCxb0tJrCp60MAQnmRplp5C +FHe4cHp2oO/uSxChZ6nBQsSsGGX95MRbHwnwzdks0SRQLHo+UulzC5JQlEH3b83C1PF2n AzXdoNt++I6T7ELETJkCjVmpfxy9naU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706592034; a=rsa-sha256; cv=none; b=64VBYU39e93UMfsCuB9WLGUpi9+nNxeC+inIYeme6MTsVtsmJA1heZvfQ5cGnWX/n8TxEa kB8P0ACu7vMlim/0DBGYarTBjX5XTyK+9/Hkq7pSrpSz6zYIsS5s4iEvibT0V6lOcSiMO4 WYX0IrQu2iLxvSlZdl/QVVzRayQb2B0= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="B/GxCT7t"; spf=pass (imf05.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706592034; x=1738128034; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=H+kUUwQrQMb1IwmAFXXv1tRw7HnV6M5rIHAXOc0j4jI=; b=B/GxCT7tPpn6+vKJLphjC5WdmrQSQwvyqWmpLvMkDPj0M2v9J4yEAgqU vMjlt1OAEYaNCkd1UTB4bYpHXu6BQNXyqOvuMJnanaT9vuIDDwX3JOwCk rH9wXlGYMpcdjyicy8RKaS9B3QGMtGLwCXaJY4Sgvn4yYgMJlwlk+SyG/ jZb3QUIlMdOhTQqg4IzTGMW8z9bpg2elnYPplQF9nSb9WMnFE+3JDQkKk /B8eLbksSw1mIw+XzJhJBC8Nk08Ulb2BVXvwuEQAJYemm0QJEDKQJYq/e weHOkUKxVgmWmJeG+yddiEsjLROXwVH+WEgH36C2328/Ra5pP1y6ZQ3DQ A==; X-IronPort-AV: E=McAfee;i="6600,9927,10968"; a="16560417" X-IronPort-AV: E=Sophos;i="6.05,707,1701158400"; d="scan'208";a="16560417" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jan 2024 21:20:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10968"; a="737657907" X-IronPort-AV: E=Sophos;i="6.05,707,1701158400"; d="scan'208";a="737657907" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jan 2024 21:20:27 -0800 From: "Huang, Ying" To: Gregory Price Cc: Gregory Price , , , , , , , , , , , , , , , , , , , , Subject: Re: [PATCH v3 4/4] mm/mempolicy: change cur_il_weight to atomic and carry the node with it In-Reply-To: (Gregory Price's message of "Mon, 29 Jan 2024 22:33:57 -0500") References: <20240125184345.47074-1-gregory.price@memverge.com> <20240125184345.47074-5-gregory.price@memverge.com> <87sf2klez8.fsf@yhuang6-desk2.ccr.corp.intel.com> <877cjsk0yd.fsf@yhuang6-desk2.ccr.corp.intel.com> <875xzbika0.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Tue, 30 Jan 2024 13:18:30 +0800 Message-ID: <871q9ziel5.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Queue-Id: 94277100014 X-Rspam-User: X-Stat-Signature: 9iiaxwobbhu48hk5fqo4kzsqiore33aj X-Rspamd-Server: rspam03 X-HE-Tag: 1706592033-360325 X-HE-Meta: U2FsdGVkX1/LSSfAKh3M1MPdZKSmBzSB3fF16ISCQ5crujvJTkcMc358U/YofM4nq+RUVixDH12Jy8uKr4ilK2oCfIGPsp94aFniH/uu/ZTlDMj52Lva345L/osZOJuT2EA2neXCUo1YDbtHMTE7yRp/33X1wLALxerR2LUQN4NAMdXnliQ1VDShopOnSHujAo9xVKak5PEeCYdFZdn1kiU008GoNg5prwRWTXPYfifv18uuCrWhkERjADIUT8ppsjwGy/R0ZNstLfP5A4+Eee6k4voANEfqd0URuvKvlIEk5DEgEV2ZVbtutFMG1egWg1Q9SxYo97QJBsRYdmhvvL8YKkcggLYMI24v4YH3uSOMZFPm1OFY2yD2m9XySil1ujpBXHEA5UAHkXBbOOothiwwpbzOO+TQ3rYLFSO2Mi4msmmV3Kil0V26SNROUTKZK/yzCH1AA0+15291koGs8ZHBZmEpEy9sA021sCR+WEKFXrQW/mbpBds4jiosBfKTzOrrnSeu1xCTjHepMsdLe5IflW5vhUtgFUstMGFXvleUKLHuwC79Q97PlYWiAeAEcV3p4CE2I08BFfyeZf7/NMsS5l45ekUGAfQaxx53Y6CAg1ujQb6s5Er8cKT7ocI1gv3Mb+PzmHmeNzfK+4PbVU6tDB9/cf//+vFNzTgFQv5TqS5HIe0OhQSU8xvVVlUrz0JRWxt9Zuo+KRKJls9fhFsiNu8S39F2/4DKkQB7LTK/RS0IoiS0cW2VL0S/d0eYv3wDqdWmgwD951Tb7ZIACUJ8lXijCtrfnjg+NhCXGhxnrc8xLEgXuL5agjX6ICZUzCCIbb6k2NWnrobohm94ShnhWqbXeiuQlOn5PBqLZH1lolr+HaSpTFPfVoEAMxAVtgZBIpPxfEOUvgCc0qqRUraeQn/9GdBru3wCLoJX/M+gaMJqrOX8tfMUaVCWlirYm2OH7OLvy/qjAXNuu6G nzMUTIKx n2DRLUkiMcLoSqiVIFIcGfgxT/IQv5cfgjIB19iqJAYP7VRNQK78j81SoE5cyJJ6w2/fdtXjqTnbT+Om9Sj0evBqa8meLDiBRj8KEpif/6ZlfKY3kPAbCOnUB5shI5N4swri7T3xMYBAa3lgfuhMbEHdm5EAQ7LFVZlL+uUvbdodY8laQbKw5v/3cPeLuaomVtxFl6jPvsve8UuaTsSwQhLfyj37npUHBd52pdFm/kNHD3oaPMGC+qlWAhLhY0f+Y9SAJrhO/2pxEarYEnFFYNrb4J+r/lg2OYQU2iDJpssoWbSxC+VGxH8Wt5uED0vi0O95KkrK3oSsavtc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Gregory Price writes: > On Tue, Jan 30, 2024 at 11:15:35AM +0800, Huang, Ying wrote: >> Gregory Price writes: >> >> > On Mon, Jan 29, 2024 at 10:48:47AM -0500, Gregory Price wrote: >> >> On Mon, Jan 29, 2024 at 04:17:46PM +0800, Huang, Ying wrote: >> >> > Gregory Price writes: >> >> > >> >> > But, in contrast, it's bad to put task-local "current weight" in >> >> > mempolicy. So, I think that it's better to move cur_il_weight to >> >> > task_struct. And maybe combine it with current->il_prev. >> >> > >> >> Style question: is it preferable add an anonymous union into task_struct: >> >> >> >> union { >> >> short il_prev; >> >> atomic_t wil_node_weight; >> >> }; >> >> >> >> Or should I break out that union explicitly in mempolicy.h? >> >> >> > >> > Having attempted this, it looks like including mempolicy.h into sched.h >> > is a non-starter. There are build issues likely associated from the >> > nested include of uapi/linux/mempolicy.h >> > >> > So I went ahead and did the following. Style-wise If it's better to just >> > integrate this as an anonymous union in task_struct, let me know, but it >> > seemed better to add some documentation here. >> > >> > I also added static get/set functions to mempolicy.c to touch these >> > values accordingly. >> > >> > As suggested, I changed things to allow 0-weight in il_prev.node_weight >> > adjusted the logic accordingly. Will be testing this for a day or so >> > before sending out new patches. >> > >> >> Thanks about this again. It seems that we don't need to touch >> task->il_prev and task->il_weight during rebinding for weighted >> interleave too. >> > > It's not clear to me this is the case. cpusets takes the task_lock to > change mems_allowed and rebind task->mempolicy, but I do not see the > task lock access blocking allocations. > > Comments from cpusets suggest allocations can happen in parallel. > > /* > * cpuset_change_task_nodemask - change task's mems_allowed and mempolicy > * @tsk: the task to change > * @newmems: new nodes that the task will be set > * > * We use the mems_allowed_seq seqlock to safely update both tsk->mems_allowed > * and rebind an eventual tasks' mempolicy. If the task is allocating in > * parallel, it might temporarily see an empty intersection, which results in > * a seqlock check and retry before OOM or allocation failure. > */ > > > For normal interleave, this isn't an issue because it always proceeds to > the next node. The same is not true of weighted interleave, which may > have a hanging weight in task->il_weight. So, I added a check as follows, node_isset(current->il_prev, policy->nodes) If prev node is removed from nodemask, allocation will proceed to the next node. Otherwise, it's safe to use current->il_weight. -- Best Regards, Huang, Ying > That is why I looked to combine the two, so at least node/weight were > carried together. > >> unsigned int weighted_interleave_nodes(struct mempolicy *policy) >> { >> unsigned int nid; >> struct task_struct *me = current; >> >> nid = me->il_prev; >> if (!me->il_weight || !node_isset(nid, policy->nodes)) { >> nid = next_node_in(...); >> me->il_prev = nid; >> me->il_weight = weights[nid]; >> } >> me->il_weight--; >> >> return nid; >> } > > I ended up with this: > > static unsigned int weighted_interleave_nodes(struct mempolicy *policy) > { > unsigned int node; > u8 weight; > > get_wil_prev(&node, &weight); > /* If nodemask was rebound, just fetch the next node */ > if (!weight) { > node = next_node_in(node, policy->nodes); > /* can only happen if nodemask has become invalid */ > if (node == MAX_NUMNODES) > return node; > weight = get_il_weight(node); > } > weight--; > set_wil_prev(node, weight); > return node; > } > > ~Gregory