From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25F01C0650E for ; Wed, 3 Jul 2019 15:08:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B38DD2189E for ; Wed, 3 Jul 2019 15:08:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="kY+5iWQb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B38DD2189E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4355B6B0003; Wed, 3 Jul 2019 11:08:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E57A8E0003; Wed, 3 Jul 2019 11:08:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2AE538E0001; Wed, 3 Jul 2019 11:08:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id D09E46B0003 for ; Wed, 3 Jul 2019 11:08:44 -0400 (EDT) Received: by mail-ed1-f70.google.com with SMTP id b12so1886083ede.23 for ; Wed, 03 Jul 2019 08:08:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:thread-topic :thread-index:date:message-id:references:in-reply-to:accept-language :content-language:content-id:content-transfer-encoding:mime-version; bh=4A4a5FfKWFEl4y6HvDegBvo5QJ9gJlXgx2xduCl6h5Y=; b=ZuxifeCrv50NBOj/gWRS4nNLX5ZiyHKuuR/F5F5Uq/Utpf6XDUrGPzqzJH09ONHWB8 o/vgXx9gEzufEfyOYxiNbjSME4k4/4aYF3hlR+VME7E1i6K8xSeleeD1096ZLGr8uHTA 5cQmlv680B6rbcVGOMDkBb7q/AgkD+K6xtvCdCpgi4sMcNW5WTYqw+MYX3b3KAZ6WQmB uqWtu/Pjfdhaa0BvG81MXCddn53Z1aiZ+3qi7WoKN/2nUScZwaL7qmrMkSlH/5GGl2H/ T3ML6a7KYNjdwq/oWGWodtuhBjskMT9YX63x+kP0LkyN07c8ZMOdoVct1WZPKGFla/co qiJg== X-Gm-Message-State: APjAAAWdZprLDe+z/8XFY/s6GQzfTxQYqluQkzOtPdkDd2UNuGJW8niB GhtrIQou9wwYXKFMFSY1SgCNjEo7NjiOgqiNkW+JuHed64E+7X8j7gKh11x4Tf+oq9IBfT9G13E qdihUdfCyBRncg775QkympSx9fFPoZabDCyLhrFKu/uQFbU8S/TIMUL1ZHMVm8c24cw== X-Received: by 2002:a17:906:d154:: with SMTP id br20mr33860274ejb.76.1562166524307; Wed, 03 Jul 2019 08:08:44 -0700 (PDT) X-Google-Smtp-Source: APXvYqy58D5xgftg2oVaO43OJAI/UUFyU1k+qsf+9MddkU+qthyaVlGMmQq5f/qObFLs47bONZlp X-Received: by 2002:a17:906:d154:: with SMTP id br20mr33860161ejb.76.1562166523177; Wed, 03 Jul 2019 08:08:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562166523; cv=none; d=google.com; s=arc-20160816; b=N89EPHTuZtf1BAuMfwCifACufod1aL0eeEUPkKGTKQk2DIoMQ3/l5FwmNP+aZI2PV1 /VTICTYs8WttBEJEWxAP0h0ym/tc7UVHrqzmjZBzg2yWbkgJBVGL5Chezs7dc4FVLwMP nAE2yC1pm1cZ6caN8TXcq9d1t1L2opaHxUvGnRRB3/GDkCtMOWhaffpxmKIPJZM2Ok4I Rx53PRoEXerckUXN5CFdBi1ps/kNFa7uW7n2H2N4LXfKKdKy4DB6XDykKzx12CY8qOM1 IHzbc4j4lMe0vshDvVE2X2fxARS8vBUVO/7F4+aFmn9T/q/AEi23iWKt8J1dWIBgEWo6 9Xng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:content-id:content-language :accept-language:in-reply-to:references:message-id:date:thread-index :thread-topic:subject:cc:to:from:dkim-signature; bh=4A4a5FfKWFEl4y6HvDegBvo5QJ9gJlXgx2xduCl6h5Y=; b=AoebbJ6lCCFIpDflt13eRI5i8poH3yxY7je7hHnz9lO08Gi703g/Cnb8um65yph+fK ULR+qdpmkqXo9SJgV7ib9N7Qx0aYLocbbUx+fXyzxY9oV9eLwTmVnwdfKWBYlzN7/KMO QmT321BuqTcJZUREAjEGyh8dY5Ke62qt/4xOj2JLeHCW3IVY13JtbIkL6+md4q47HivT GGd985CEOzRufTybfP7Wjgbgu/q2okjLmGA0aYJQB4++2iEHAY6T9gDN5bxgFZ2obF3h 821zJYecbZhF1R48mUL9TftLtlIe6Wz0z6qlpGWZ6XnCXBNxI1XytQYWCDlppafEvQFI TdgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@Mellanox.com header.s=selector2 header.b=kY+5iWQb; spf=pass (google.com: domain of jgg@mellanox.com designates 40.107.8.79 as permitted sender) smtp.mailfrom=jgg@mellanox.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=mellanox.com Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80079.outbound.protection.outlook.com. [40.107.8.79]) by mx.google.com with ESMTPS id va11si2049766ejb.358.2019.07.03.08.08.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 03 Jul 2019 08:08:43 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@mellanox.com designates 40.107.8.79 as permitted sender) client-ip=40.107.8.79; Authentication-Results: mx.google.com; dkim=pass header.i=@Mellanox.com header.s=selector2 header.b=kY+5iWQb; spf=pass (google.com: domain of jgg@mellanox.com designates 40.107.8.79 as permitted sender) smtp.mailfrom=jgg@mellanox.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=mellanox.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4A4a5FfKWFEl4y6HvDegBvo5QJ9gJlXgx2xduCl6h5Y=; b=kY+5iWQbbxLWKp0tqgTmPpt5tRxyIus3ZsKPmVmcLnNrHuDGgMxv59SizeIQiBH0IMeZnOsv/DMoILISYgrqKUs5+ckpiveRZvQjG10Esz6Usy5KLwc+ji4a7aTaWiLSsClhhx0Jeza5fSBi6Q7x+98dp/cSTYBF5fUh7F3LDdI= Received: from VI1PR05MB4141.eurprd05.prod.outlook.com (10.171.182.144) by VI1PR05MB6351.eurprd05.prod.outlook.com (20.179.25.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2032.20; Wed, 3 Jul 2019 15:08:41 +0000 Received: from VI1PR05MB4141.eurprd05.prod.outlook.com ([fe80::f5d8:df9:731:682e]) by VI1PR05MB4141.eurprd05.prod.outlook.com ([fe80::f5d8:df9:731:682e%5]) with mapi id 15.20.2032.019; Wed, 3 Jul 2019 15:08:41 +0000 From: Jason Gunthorpe To: "Kuehling, Felix" CC: Christoph Hellwig , "Deucher, Alexander" , David Airlie , Ralph Campbell , Jerome Glisse , John Hubbard , "linux-rdma@vger.kernel.org" , "linux-mm@kvack.org" , Andrea Arcangeli , "dri-devel@lists.freedesktop.org" , "amd-gfx@lists.freedesktop.org" Subject: Re: [RFC] mm/hmm: pass mmu_notifier_range to sync_cpu_device_pagetables Thread-Topic: [RFC] mm/hmm: pass mmu_notifier_range to sync_cpu_device_pagetables Thread-Index: AQHVHY87cnj6rYaF00uB6DOqwK5J5aa35HaAgAAxJwCAAALKgIAAOioAgADUsAA= Date: Wed, 3 Jul 2019 15:08:41 +0000 Message-ID: <20190703150836.GM18688@mellanox.com> References: <20190608001452.7922-1-rcampbell@nvidia.com> <20190702195317.GT31718@mellanox.com> <20190702224912.GA24043@lst.de> <20190702225911.GA11833@mellanox.com> <1dc82dc8-3e6f-1d6f-b14d-41ae3c1b2709@amd.com> In-Reply-To: <1dc82dc8-3e6f-1d6f-b14d-41ae3c1b2709@amd.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: YTOPR0101CA0030.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::43) To VI1PR05MB4141.eurprd05.prod.outlook.com (2603:10a6:803:4d::16) authentication-results: spf=none (sender IP is ) smtp.mailfrom=jgg@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [156.34.55.100] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: b5b764d1-dc0d-4af0-1bf8-08d6ffc85584 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020);SRVR:VI1PR05MB6351; x-ms-traffictypediagnostic: VI1PR05MB6351: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2733; x-forefront-prvs: 00872B689F x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(4636009)(396003)(136003)(376002)(39860400002)(366004)(346002)(189003)(199004)(386003)(6506007)(25786009)(7416002)(53546011)(316002)(8936002)(26005)(68736007)(76176011)(102836004)(486006)(476003)(2616005)(186003)(5660300002)(14454004)(66476007)(2906002)(4326008)(11346002)(6116002)(3846002)(54906003)(99286004)(52116002)(36756003)(33656002)(7736002)(66446008)(1076003)(8676002)(14444005)(6246003)(64756008)(256004)(86362001)(305945005)(478600001)(53936002)(66946007)(81166006)(71200400001)(66556008)(73956011)(6512007)(6486002)(71190400001)(229853002)(6436002)(81156014)(446003)(6916009)(66066001);DIR:OUT;SFP:1101;SCL:1;SRVR:VI1PR05MB6351;H:VI1PR05MB4141.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: td0S6w4mHfCpfIiwq60twZyFZcsKDyypieDB8FahsjvXOncVtvZwm1IzVFYnIeWYVhnUUAhoGNboHpppDK4NYN5sWo4WY0k/RfzbuXciuXnTm8teFPegBZQJfPq+aA0UM74aKEgAKeaYbx1/PlikKwR5rgu+ZikULKOirR2H5ibffF0p2vSXVQS8TcqW8WkduqizhaS7qXX7A7WAlhuyWd/yHocH7Fj1OEXkuiriDidvL48WjKUejgR4i6pgsmZ6jfTWrQRiKg4rQmb3Lx8vp8pq/bspE06qzMZRGUC2uioevyKu4VPCpryhTmGFrbNzaZ2Me/Si1l4i8Mww5JqYxD7VH/9jCuUR0cFU3MjxKccrFIXn/pmb5o67h3BY3dcId0vINr7DiRUk9a6bhCj9xECu4ASFEntY23RP+eLFvZE= Content-Type: text/plain; charset="us-ascii" Content-ID: <5850BAD4694CCA479493175E5B4779A2@eurprd05.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: b5b764d1-dc0d-4af0-1bf8-08d6ffc85584 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Jul 2019 15:08:41.6325 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: jgg@mellanox.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB6351 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jul 03, 2019 at 02:27:22AM +0000, Kuehling, Felix wrote: > On 2019-07-02 6:59 p.m., Jason Gunthorpe wrote: > > On Wed, Jul 03, 2019 at 12:49:12AM +0200, Christoph Hellwig wrote: > >> On Tue, Jul 02, 2019 at 07:53:23PM +0000, Jason Gunthorpe wrote: > >>>> I'm sending this out now since we are updating many of the HMM APIs > >>>> and I think it will be useful. > >>> This make so much sense, I'd like to apply this in hmm.git, is there > >>> any objection? > >> As this creates a somewhat hairy conflict for amdgpu, wouldn't it be > >> a better idea to wait a bit and apply it first thing for next merge > >> window? > > My thinking is that AMD GPU already has a monster conflict from this: > > > > int hmm_range_register(struct hmm_range *range, > > - struct mm_struct *mm, > > + struct hmm_mirror *mirror, > > unsigned long start, > > unsigned long end, > > unsigned page_shift); > > > > So, depending on how that is resolved we might want to do both API > > changes at once. >=20 > I just sent out a fix for the hmm_mirror API change. I think if you follow my suggestion to apply a prep patch to AMD GPU to make the conflict resolution simple, we should defer this patch until next kernel for the reasons CH gave. > > Or we may have to revert the above change at this late date. > > > > Waiting for AMDGPU team to discuss what process they want to use. >=20 > Yeah, I'm wondering what the process is myself. With HMM and driver=20 > development happening on different branches these kinds of API changes=20 > are painful. There seems to be a built-in assumption in the current=20 > process, that code flows mostly in one direction amd-staging-drm-next ->= =20 > drm-next -> linux-next -> linux. That assumption is broken with HMM code= =20 > evolving rapidly in both amdgpu and mm. It looks to me like AMD GPU uses a pull request model. So a goal as a tree runner should be to work with the other trees (ie hmm.git, etc) to minimize conflicts between the PR you will send and the PR other trees will send. Do not focus on linux-next, that is just an 'early warning system' that conflicts are on the horizon, we knew about this one :) (well, mostly, I was surprised how big it was, my bad) So we must stay in co-ordination with patches in-flight on the list and make the right decision, depending on the situation. Communication here is key :) We have lots of strategies available to deal with these situations. > If we want to continue developing HMM driver changes in > amd-staging-drm-next, we'll need to synchronize with hmm.git more=20 > frequently, both ways. It can't really go both ways. hmm.git has to be only the hmm topic, otherwise it doesn't really work. > I believe part of the problem is, that there is a fairly long > lead-time from getting changes from amd-staging-drm-next into > linux-next, as they are held for one release cycle in drm-next. > Pushing HMM-related changes through drm-fixes may offer a kind of > shortcut. Philip and my latest fixup is just bypassing drm-next > completely and going straight into linux-next, though. I'm not so familiar with the DRM work flow to give you advice on this. Jason