From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62A3DC47247 for ; Thu, 30 Apr 2020 20:12:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 121E3207DD for ; Thu, 30 Apr 2020 20:12:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="fUyqySXD" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 121E3207DD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6641C8E0001; Thu, 30 Apr 2020 16:12:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 44B6C8E0008; Thu, 30 Apr 2020 16:12:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 276FA8E0005; Thu, 30 Apr 2020 16:12:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0062.hostedemail.com [216.40.44.62]) by kanga.kvack.org (Postfix) with ESMTP id E8DC18E0001 for ; Thu, 30 Apr 2020 16:12:05 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A4D0D8248D7C for ; Thu, 30 Apr 2020 20:12:05 +0000 (UTC) X-FDA: 76765617810.16.mask03_4c6ae7aac2040 X-HE-Tag: mask03_4c6ae7aac2040 X-Filterd-Recvd-Size: 7494 Received: from userp2120.oracle.com (userp2120.oracle.com [156.151.31.85]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Thu, 30 Apr 2020 20:12:04 +0000 (UTC) Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 03UK9RpK005992; Thu, 30 Apr 2020 20:11:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2020-01-29; bh=+I5DK1bpDxPp17bxmkq3TLU7ktjAi9yJhHWfSrepooM=; b=fUyqySXDeLDsctdyoiziddJfcj8mcTOQxSfQWSYjilj7uE73/HfZauZY2u5i3Uvnv5tG KAp31PWGtBoS4N9epLF82hU53mniZOA/7lEKUmGjH9KQ+H8nH1Cv4EpFkW6aOiLQYMoX C+k0eWTaXn4nJZt4GHwStEs5HC9LqEeghzxIIbxp3ejHUWBFXbT7/tq5dR1lXdmCGGJ1 l+UQocYxc5AJd89VOHMDkMnZiIFb1tj2e3zzxxVqrjAbyCo4wTz8IntLONqd/e7Dypen biFbKfiOp/Jdby2w/mf18RluKSLiAmDV3u+b0FlU7mTnkhhI8TC+riHLkuTMqfVSGna9 QQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by userp2120.oracle.com with ESMTP id 30p2p0k2ps-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 30 Apr 2020 20:11:48 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 03UK7Qbg096026; Thu, 30 Apr 2020 20:11:48 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3030.oracle.com with ESMTP id 30qtjy23v9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 30 Apr 2020 20:11:48 +0000 Received: from abhmp0010.oracle.com (abhmp0010.oracle.com [141.146.116.16]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 03UKBlCt024136; Thu, 30 Apr 2020 20:11:47 GMT Received: from localhost.localdomain (/98.229.125.203) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 30 Apr 2020 13:11:45 -0700 From: Daniel Jordan To: Andrew Morton , Herbert Xu , Steffen Klassert Cc: Alex Williamson , Alexander Duyck , Dan Williams , Dave Hansen , David Hildenbrand , Jason Gunthorpe , Jonathan Corbet , Josh Triplett , Kirill Tkhai , Michal Hocko , Pavel Machek , Pavel Tatashin , Peter Zijlstra , Randy Dunlap , Shile Zhang , Tejun Heo , Zi Yan , linux-crypto@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Daniel Jordan Subject: [PATCH 7/7] padata: document multithreaded jobs Date: Thu, 30 Apr 2020 16:11:25 -0400 Message-Id: <20200430201125.532129-8-daniel.m.jordan@oracle.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200430201125.532129-1-daniel.m.jordan@oracle.com> References: <20200430201125.532129-1-daniel.m.jordan@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9607 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 suspectscore=0 malwarescore=0 bulkscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2004300150 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9607 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 phishscore=0 clxscore=1015 bulkscore=0 adultscore=0 lowpriorityscore=0 impostorscore=0 malwarescore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2004300150 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add Documentation for multithreaded jobs. Signed-off-by: Daniel Jordan --- Documentation/core-api/padata.rst | 41 +++++++++++++++++++++++-------- 1 file changed, 31 insertions(+), 10 deletions(-) diff --git a/Documentation/core-api/padata.rst b/Documentation/core-api/p= adata.rst index 9a24c111781d9..b7e047af993e8 100644 --- a/Documentation/core-api/padata.rst +++ b/Documentation/core-api/padata.rst @@ -4,23 +4,26 @@ The padata parallel execution mechanism =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 -:Date: December 2019 +:Date: April 2020 =20 Padata is a mechanism by which the kernel can farm jobs out to be done i= n -parallel on multiple CPUs while retaining their ordering. It was develo= ped for -use with the IPsec code, which needs to be able to perform encryption an= d -decryption on large numbers of packets without reordering those packets.= The -crypto developers made a point of writing padata in a sufficiently gener= al -fashion that it could be put to other uses as well. +parallel on multiple CPUs while optionally retaining their ordering. =20 -Usage -=3D=3D=3D=3D=3D +It was originally developed for IPsec, which needs to perform encryption= and +decryption on large numbers of packets without reordering those packets.= This +is currently the sole consumer of padata's serialized job support. + +Padata also supports multithreaded jobs, splitting up the job evenly whi= le load +balancing and coordinating between threads. + +Running Serialized Jobs +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 Initializing ------------ =20 -The first step in using padata is to set up a padata_instance structure = for -overall control of how jobs are to be run:: +The first step in using padata to run parallel jobs is to set up a +padata_instance structure for overall control of how jobs are to be run:= : =20 #include =20 @@ -162,6 +165,24 @@ functions that correspond to the allocation in rever= se:: It is the user's responsibility to ensure all outstanding jobs are compl= ete before any of the above are called. =20 +Running Multithreaded Jobs +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D + +A multithreaded job has a main thread and zero or more helper threads, w= ith the +main thread participating in the job and then waiting until all helpers = have +finished. padata splits the job into units called chunks, where a chunk= is a +piece of the job that one thread completes in one call to the thread fun= ction. + +A user has to do three things to run a multithreaded job. First, descri= be the +job by defining a padata_mt_job structure, which is explained in the Int= erface +section. This includes a pointer to the thread function, which padata w= ill +call each time it assigns a job chunk to a thread. Then, define the thr= ead +function, which accepts three arguments, ``start``, ``end``, and ``arg``= , where +the first two delimit the range that the thread operates on and the last= is a +pointer to the job's shared state, if any. Prepare the shared state, wh= ich is +typically a stack-allocated structure that wraps the required data. Las= t, call +padata_do_multithreaded(), which will return once the job is finished. + Interface =3D=3D=3D=3D=3D=3D=3D=3D=3D =20 --=20 2.26.2